Signal Processing Course: Non-linear Approximation and Coding

Transcript

1. Approximation and Coding with Orthogonal Decompositions Gabriel Peyré http://www.ceremade.dauphine.fr/~peyre/numerical-tour/
2. Overview • Approximation and Compression • Decay of Approximation Error • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
3. Sparse Approximation in a Basis
4. Sparse Approximation in a Basis
5. Sparse Approximation in a Basis
6. Hard Thresholding
7. (usually polynomial) Approximation Speed Approximation error decay:
8. (usually polynomial) Approximation Speed Approximation error decay: log 10 (||f fM ||) Log/Log plot: approx. a ne curve log 10 (M/N)
9. Efficiency of Transforms Fourier DCT Local DCT Wavelets log 10 (||f fM ||) log 10 (M/N)
10. Overview • Approximation and Compression • Decay of Approximation Error • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
11. f forward Compression by Transform-coding a[m] = ⇥f, m ⇤ R Image f Zoom on f transform
12. f forward Compression by Transform-coding a[m] = ⇥f, m ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f transform ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z
13. f forward coding Compression by Transform-coding a[m] = ⇥f, m ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z
14. f forward coding Compression by Transform-coding a[m] = ⇥f, m ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f decoding q[m] Z ˜ a[m] dequantization transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z Dequantization:
15. f forward coding Compression by Transform-coding a[m] = ⇥f, m ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f fR , R =0.2 bit/pixel decoding q[m] Z ˜ a[m] dequantization transform backward fR = m IT ˜ a[m] m transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z Dequantization:
16. Thresholding vs. Quantizing
17. Non-linear Approximation and Compression
18. Non-linear Approximation and Compression ˜ a[m] T T 2T 2T a[m] Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z =⇥ |a[m] ˜ a[m]| T 2 Dequantization: ˜ a[m] = sign(q[m]) |q[m]| + 1 2 ⇥ T
19. Non-linear Approximation and Compression ||f fM ||2 # = M Theorem: ||f fR ||2 ||f fM ||2 + MT2/4 where M = # {m \ ˜ a[m] = 0}. ||f fR ||2 = ⇤ m (a[m] ˜ a[m])2 ⇤ |a[m]|<T |a[m]|2 + ⇤ |a[m]| T T 2 ⇥2 ˜ a[m] T T 2T 2T a[m] Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z =⇥ |a[m] ˜ a[m]| T 2 Dequantization: ˜ a[m] = sign(q[m]) |q[m]| + 1 2 ⇥ T
20. A Naive Support Coding Approach Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . =⇤ ||f fM ||2 ⇥ M .
21. A Naive Support Coding Approach Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
22. A Naive Support Coding Approach Simple coding strategy: R = Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
23. A Naive Support Coding Approach Simple coding strategy: R = Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). = Rval = O(M| log 2 (T)|) = O(M log 2 (M)) Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
24. A Naive Support Coding Approach Simple coding strategy: R = Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). Theorem: Under hypotheses (H 1 ) and (H 2 ), ||f fR ||2 = O(R log (R)). = Rval = O(M| log 2 (T)|) = O(M log 2 (M)) Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
25. Entropic Coders
26. Entropic Coders
27. Entropic Coders
28. JPEG-2000 Overview
29. JPEG-2000 Overview
30. JPEG-2000 Overview
31. JPEG-2000 Overview
32. JPEG-2000 Overview
33. Contextual Coding code block width 3 × 3 context window
34. JPEG-2000 vs. JPEG, 0.2bit/pixel
35. Overview • Approximation and Compression • Decay of Approximation Error • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
36. 1D Fourier Approximation
37. 1D Fourier Approximation
38. Sobolev and Fourier
39. Singularities and Fourier 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
40. Sobolev for Images
41. Sobolev for Images
42. Overview • Approximation and Compression • Decay of Approximation Error • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
43. Vanishing moments: p = 3 p = 4 Magnitude of Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 p = 2
44. Vanishing moments: p = 3 p = 4 Magnitude of Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 p = 2 | f, j,n ⇥| Cf || ||1 2j( +d/2) t = x 2jn 2j ⇥f, j,n ⇤ = 1 2j d 2 ⇤ f(x) x 2jn 2j ⇥ dx = 2j d 2 ⇤ R(2jt) (t)dt If f is C on supp(⇥j,n ), p : f(x) = P(x 2jn) + R(x 2jn) = P(2jt) + R(2jt)
45. Vanishing moments: p = 3 p = 4 Magnitude of Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 | f, j,n ⇥| ||f|| || ||1 2j d 2 p = 2 | f, j,n ⇥| Cf || ||1 2j( +d/2) t = x 2jn 2j ⇥f, j,n ⇤ = 1 2j d 2 ⇤ f(x) x 2jn 2j ⇥ dx = 2j d 2 ⇤ R(2jt) (t)dt If f is C on supp(⇥j,n ), p : f(x) = P(x 2jn) + R(x 2jn) = P(2jt) + R(2jt)
46. 1D Wavelet Coefficient Behavior −0.2 −0.1 0 0.1 0.2 −0.2 −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
47. 1D Wavelet Coefficient Behavior If f is C in supp( j,n ), then | f, j,n ⇥| 2j( +1/2)||f||C || ||1 −0.2 −0.1 0 0.1 0.2 −0.2 −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
48. 1D Wavelet Coefficient Behavior If f is C in supp( j,n ), then | f, j,n ⇥| 2j( +1/2)||f||C || ||1 | f, j,n ⇥| 2j/2||f|| || ||1 If f is bounded (e.g. around a singularity), then −0.2 −0.1 0 0.1 0.2 −0.2 −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
49. For Fourier, linear non-linear, sub-optimal. For wavelets, linear non-linear, optimal. Piecewise Regular Functions in 1D Theorem: If f is C outside a finite set of discontinuities: n[M] = ||f fn M ||2 = O(M 1) (Fourier), O(M 2 ) (wavelets).
50. Examples of 1D Approximations 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 −2 −1.8 −1.6 −1.4 −1.2 −1 −0.8 −6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1
51. Large coe cient | f, jn ⇥| < T x1 x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: |Cj | K|S| = constant
52. Large coe cient | f, jn ⇥| < T x1 x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 |Cj | K|S| = constant
53. Large coe cient | f, jn ⇥| < T x1 x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1/2 Regular: 2j2 = (T/C)2 |Cj | K|S| = constant
54. Large coe cient | f, jn ⇥| < T Hand-made approximate: ˜ fM = j j2 n Cj f, j,n ⇥ j,n + j j1 n Cc j f, j,n ⇥ j,n x1 x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1/2 Regular: 2j2 = (T/C)2 |Cj | K|S| = constant
55. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
56. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) M j j2 |Cj | + j j1 |Cc j | j j2 K|S| + j j1 2 j = O(| log(T)| + T 1 +1/2 ) = O(T 1 +1/2 ) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
57. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) M j j2 |Cj | + j j1 |Cc j | j j2 K|S| + j j1 2 j = O(| log(T)| + T 1 +1/2 ) = O(T 1 +1/2 ) ||f fM ||2 = O(M 2 ) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
58. 2D Wavelet Approximation If f is C in supp( j,n ), then | f, j,n ⇥| 2j( +1)||f||C || ||1 | f, j,n ⇥| 2j||f|| || ||1 If f is bounded (e.g. around a singularity), then
59. Fourier Wavelet, both sub-optimal. Wavelets: same result for BV functions (optimal). Piecewise Regular Functions in 2D n[M] = ||f fn M ||2 = O(M 1/2) (Fourier), O(M 1) (wavelets). Theorem: If f is C outside a set of finite length edge curves,
60. Example of 2D Approximations
61. Length(S) = L Localizing the Singular Support in 2D j1 j2 f(x, y) |Cj | LK2 j = constant
62. Length(S) = L Localizing the Singular Support in 2D j1 j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j |Cj | LK2 j = constant
63. Length(S) = L Localizing the Singular Support in 2D j1 j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1 Regular: 2j2 = T/C |Cj | LK2 j = constant
64. Hand-made approximate: ˜ fM = j j2 n Cj f, j,n ⇥ j,n + j j1 n Cc j f, j,n ⇥ j,n Length(S) = L Localizing the Singular Support in 2D j1 j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1 Regular: 2j2 = T/C |Cj | LK2 j = constant
65. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T)
66. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T) M j j2 |Cj | + j j1 |Cc j | j j2 LK2 j + j j1 2 2j = O(T 1 + T 1 +1 ) = O(T 1)
67. ||f fM ||2 = O(M 1) ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T) M j j2 |Cj | + j j1 |Cc j | j j2 LK2 j + j j1 2 2j = O(T 1 + T 1 +1 ) = O(T 1)
68. Overview • Approximation and Compression • Decay of Approximation Error • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
69. Geometric image model: f is C outside a set of C edge curves. BV image: level sets have finite lengths. Geometric image: level sets are regular. Geometry = cartoon image Sharp edges Smoothed edges Geometrically Regular Images | f|
70. Approximation of f, C2 outside C2 edges. Piecewise linear approximation on M triangles: ˜ fM . Geometic Construction : Finite Elements
71. Approximation of f, C2 outside C2 edges. Piecewise linear approximation on M triangles: ˜ fM . Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2
72. Approximation of f, C2 outside C2 edges. Piecewise linear approximation on M triangles: ˜ fM . Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2 M/2 anisotropic triangles. Singular areas:
73. Approximation of f, C2 outside C2 edges. Piecewise linear approximation on M triangles: ˜ fM . Di culties to build e cient approximations. No optimal strategies (greedy solutions). Theorem: If f is C2 outside a set of C2 contours, then one has for an adapted triangulation ||f ˜ fM ||2 = O(M 2). Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2 M/2 anisotropic triangles. Singular areas:
74. Greedy Triangulation Optimization Bougleux, Peyr´ e, Cohen, ECCV’08
75. Curvelet Atoms Parabolic dyadic scaling: Rotation: [Candes, Donoho] [Candes, Demanet, Ying, Donoho] “width length2”
76. Curvelet Tight Frame Spacial sampling: Tight frame of L2(R2): Angular sampling:
77. Discrete curvelets: O(N log(N)) algorithm. Redundancy 5 =⇥ not e cient for compression. M-term curvelet approximation: Curvelet Approximation Theorem: If f is C2 outside a set of C2 edges, ||f fM ||2 = O(M 2(log M)3).
78. Works on elongated edges. Works also on locally parallel textures ! Curvelets Denoising
79. Conclusion
80. Conclusion
81. Conclusion 0 0.2 0.4 0.6 0.8 1
82. Conclusion 0 0.2 0.4 0.6 0.8 1