Unsharp Masking, Countershading and Halos: Enhancements or Artifacts?

Countershading is a common technique for local image contrast manipulations, and is widely used both in automatic settings, such as image sharpening and tonemapping, as well as under artistic control, such as in paintings and interactive image processing software. Unfortunately, countershading is a double-edged sword: while correctly chosen parameters for a given viewing condition can significantly improve the image sharpness or trick the human visual system into perceiving a higher contrast than physically present in an image, wrong parameters, or different viewing conditions can result in objectionable halo artifacts.

In this paper we investigate the perception of countershading in the context of a novel mask-based contrast enhancement algorithm and analyze the circumstances under which the resulting profiles turn from image enhancement to artifact for a range of parameters and viewing conditions. Our experimental results can be modeled as a function of the width of the countershading profile. We employ this empirical function in a range of applications such as image resizing, view dependent tone mapping, and countershading analysis in photographs and works of fine art.

Transcript

1. Unsharp Masking, Countershading and Halos: Enhancements or Artifacts? Matthew Trentacoste Rafał Mantiuk Wolfgang Heidrich Florian Dufort University of British Columbia Bangor University
2. Which image has haloes? 2
3. Which image has haloes? 2
4. Which image has haloes? 2
5. Answer: they all do. 3 Blue: original Red: edge enhanced
6. Countershading 4 λ(I - smooth(I,δ)) + I = Iout
7. Countershading in arts 5 G. Seurat, Bathers at Asnieres S. Dalí, Landscape with butterflies
8. Countershading effects 6
9. Countershading effects 6 Unsharp masking
10. Countershading effects 6 Unsharp masking Countershading
11. Countershading effects 6 Unsharp masking Countershading
12. Related work • Unsharp masking [Ramponi 1996][Polesel 2000][Kim 2005] [Nowak 1998][Wang 2001] • Tonemapping operators [Chiu 1993][Durand 2002][Fattal 2002] • Contrast enhancement [Smith 2006, 2008][Luft 2006] [Krawczyk 2007] 7
13. Viewing-distance dependent effect 8
14. Related work • Cornsweet illusion [Georgeson 1977][Campbell 1971,1978] [Burr 1987][Kingdom 1988] • Perceived edge sharpness [Lin 2006] • Perception of 3D countershading [Ritschel 2008][Ihrke 2009] 9
15. Objectives • Why some edge profiles are considered enhancements and others artifacts? • Can this be explained by existing visual models? • Use the data to control tone-mapping, image resizing and image enhancement so that they do not introduce artifacts. 10
16. Experiment 11
17. Psychometrical experiment • Observers shown images with countershaded edges of varying profile widths • Adjust countershading magnitude to just below what was considered objectionable • Profile width varied between .009 and 4.6 deg 12
18. Psychometrical experiment • Observers shown images with countershaded edges of varying profile widths • Adjust countershading magnitude to just below what was considered objectionable • Profile width varied between .009 and 4.6 deg 12
19. Countershading parameters 13 Increase profile width Increase profile amplitude λ(I - smooth(I,δ)) + I = Iout λ δ
20. 14 Perceptual experiment • 6 images: 3 simple edges, 3 natural scenes • Countershading profiles applied to selected edges only • So that observers could focus on judging a particular element of the scene • 15 paid observers participated in 1800 trials
21. 15 Experiment results −2 −1 0 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Profile width σ [log 10 deg] Profile magnitude λ Coast Palm beach Building Model fit Scallop threshold −2 −1 0 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 Profile width σ [log 10 deg] Profile magnitude λ Edge − high Edge − med Edge − low Averaged standard deviation Scallop threshold Complex images Edges
22. 15 Experiment results −2 −1 0 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Profile width σ [log 10 deg] Profile magnitude λ Coast Palm beach Building Model fit Scallop threshold −2 −1 0 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 Profile width σ [log 10 deg] Profile magnitude λ Edge − high Edge − med Edge − low Averaged standard deviation Scallop threshold Complex images Edges medium high low
23. 16 Spatial frequency Indistinguishable countershading Objectionable countershading (halos) Acceptable countershading • Defines regions of acceptable and objectionable contrast • Blue : undetectable countershading Orange : objectionable countershading • Useful contrast enhancement can be achieved in between the two regions Spatial frequency Countershading magnitude
24. Relation to visual models 17 −2 −1 0 1 −1 0 Profile width σ [log 10 deg] Profile magnitude log 10 λ Edge − high − just objectionable Edge − high − just detectable model Edge − high − just detectable measured Edge − med − just objectionable Edge − low − just objectionable Scallop threshold Just detectable thresholds [Krawczyk et al., Dooley & Greenfield] Just objectionable thresholds [our measurements] Just detectable thresholds [our measurements]
25. Applications 18
26. 19 Resizing, different displays
27. 19 Resizing, different displays
28. 20 Tone mapping Original [Durand and Dorsey 2002] Adjust to approximate image scale σs
29. 21 Scale-aware displays • Determine distance of viewer using head-tracking • Present images for specific viewing conditions • Need headset Only works for one viewer
30. 21 Scale-aware displays • Determine distance of viewer using head-tracking • Present images for specific viewing conditions • Need headset Only works for one viewer
31. 22 Countershading estimation
32. Conclusions 23 • Model of the just-objectionable countershading • Function of the profile width • No clear relation to just-noticeable countershading • Several applications for introducing countershading adaptively, depending on the viewing distance −2 −1 0 1 0.2 0.4 0.6 0.8 1 1.2 Profile width m [log 10 deg] Profile magnitude h Coast Palm beach Building Model fit Scallop threshold Countershading magnitude Spatial frequency Indistinguishable countershading Objectionable countershading (halos)
33. ( you and our sponsors ) Thank you Research Chair