Signal Processing Course: Non-linear Approximation and Coding

Signal Processing Course: Non-linear Approximation and Coding

E34ded36efe4b7abb12510d4e525fee8?s=128

Gabriel Peyré

January 01, 2012
Tweet

Transcript

  1. Approximation and Coding with Orthogonal Decompositions Gabriel Peyré http://www.ceremade.dauphine.fr/~peyre/numerical-tour/

  2. Overview • Approximation and Compression • Decay of Approximation Error

    • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
  3. Sparse Approximation in a Basis

  4. Sparse Approximation in a Basis

  5. Sparse Approximation in a Basis

  6. Hard Thresholding

  7. (usually polynomial) Approximation Speed Approximation error decay:

  8. (usually polynomial) Approximation Speed Approximation error decay: log 10 (||f

    fM ||) Log/Log plot: approx. a ne curve log 10 (M/N)
  9. Efficiency of Transforms Fourier DCT Local DCT Wavelets log 10

    (||f fM ||) log 10 (M/N)
  10. Overview • Approximation and Compression • Decay of Approximation Error

    • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
  11. f forward Compression by Transform-coding a[m] = ⇥f, m ⇤

    R Image f Zoom on f transform
  12. f forward Compression by Transform-coding a[m] = ⇥f, m ⇤

    R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f transform ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z
  13. f forward coding Compression by Transform-coding a[m] = ⇥f, m

    ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z
  14. f forward coding Compression by Transform-coding a[m] = ⇥f, m

    ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f decoding q[m] Z ˜ a[m] dequantization transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z Dequantization:
  15. f forward coding Compression by Transform-coding a[m] = ⇥f, m

    ⇤ R Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z Image f Zoom on f fR , R =0.2 bit/pixel decoding q[m] Z ˜ a[m] dequantization transform backward fR = m IT ˜ a[m] m transform Entropic coding: use statistical redundancy (many 0’s). ˜ a[m] T T 2T 2T a[m Quantized q[m] bin T q[m] Z Dequantization:
  16. Thresholding vs. Quantizing

  17. Non-linear Approximation and Compression

  18. Non-linear Approximation and Compression ˜ a[m] T T 2T 2T

    a[m] Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z =⇥ |a[m] ˜ a[m]| T 2 Dequantization: ˜ a[m] = sign(q[m]) |q[m]| + 1 2 ⇥ T
  19. Non-linear Approximation and Compression ||f fM ||2 # = M

    Theorem: ||f fR ||2 ||f fM ||2 + MT2/4 where M = # {m \ ˜ a[m] = 0}. ||f fR ||2 = ⇤ m (a[m] ˜ a[m])2 ⇤ |a[m]|<T |a[m]|2 + ⇤ |a[m]| T T 2 ⇥2 ˜ a[m] T T 2T 2T a[m] Quantization: q[m] = sign(a[m]) |a[m]| T ⇥ Z =⇥ |a[m] ˜ a[m]| T 2 Dequantization: ˜ a[m] = sign(q[m]) |q[m]| + 1 2 ⇥ T
  20. A Naive Support Coding Approach Coding: relate # bits R

    to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . =⇤ ||f fM ||2 ⇥ M .
  21. A Naive Support Coding Approach Coding: relate # bits R

    to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
  22. A Naive Support Coding Approach Simple coding strategy: R =

    Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
  23. A Naive Support Coding Approach Simple coding strategy: R =

    Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). = Rval = O(M| log 2 (T)|) = O(M log 2 (M)) Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
  24. A Naive Support Coding Approach Simple coding strategy: R =

    Rval + Rind = Rind log 2 N M ⇥ = O(M log 2 (N/M)) = O(M log 2 (M)). Theorem: Under hypotheses (H 1 ) and (H 2 ), ||f fR ||2 = O(R log (R)). = Rval = O(M| log 2 (T)|) = O(M log 2 (M)) Coding: relate # bits R to # coe cients M. (H 1 ) Ordered coe cients | f, m ⇥| decays like m +1 2 . f RN sampled from f0 , error: (H 2 ) To ensure ||f f0 ||2 ⇥ ||f fM ||2: M ⇥ N⇥/ . ||f f0 ||2 ⇥ N =⇤ ||f fM ||2 ⇥ M .
  25. Entropic Coders

  26. Entropic Coders

  27. Entropic Coders

  28. JPEG-2000 Overview

  29. JPEG-2000 Overview

  30. JPEG-2000 Overview

  31. JPEG-2000 Overview

  32. JPEG-2000 Overview

  33. Contextual Coding code block width 3 × 3 context window

  34. JPEG-2000 vs. JPEG, 0.2bit/pixel

  35. Overview • Approximation and Compression • Decay of Approximation Error

    • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
  36. 1D Fourier Approximation

  37. 1D Fourier Approximation

  38. Sobolev and Fourier

  39. Singularities and Fourier 0 0.2 0.4 0.6 0.8 1 0

    0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
  40. Sobolev for Images

  41. Sobolev for Images

  42. Overview • Approximation and Compression • Decay of Approximation Error

    • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
  43. Vanishing moments: p = 3 p = 4 Magnitude of

    Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 p = 2
  44. Vanishing moments: p = 3 p = 4 Magnitude of

    Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 p = 2 | f, j,n ⇥| Cf || ||1 2j( +d/2) t = x 2jn 2j ⇥f, j,n ⇤ = 1 2j d 2 ⇤ f(x) x 2jn 2j ⇥ dx = 2j d 2 ⇤ R(2jt) (t)dt If f is C on supp(⇥j,n ), p : f(x) = P(x 2jn) + R(x 2jn) = P(2jt) + R(2jt)
  45. Vanishing moments: p = 3 p = 4 Magnitude of

    Wavelet Coefficients f(x) −1 0 1 2 −2 −1 0 1 2 −2 −1 0 1 2 −1 0 1 2 −2 0 2 4 −1 −0.5 0 0.5 1 1.5 k < p, (x)xkdx = 0 | f, j,n ⇥| ||f|| || ||1 2j d 2 p = 2 | f, j,n ⇥| Cf || ||1 2j( +d/2) t = x 2jn 2j ⇥f, j,n ⇤ = 1 2j d 2 ⇤ f(x) x 2jn 2j ⇥ dx = 2j d 2 ⇤ R(2jt) (t)dt If f is C on supp(⇥j,n ), p : f(x) = P(x 2jn) + R(x 2jn) = P(2jt) + R(2jt)
  46. 1D Wavelet Coefficient Behavior −0.2 −0.1 0 0.1 0.2 −0.2

    −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
  47. 1D Wavelet Coefficient Behavior If f is C in supp(

    j,n ), then | f, j,n ⇥| 2j( +1/2)||f||C || ||1 −0.2 −0.1 0 0.1 0.2 −0.2 −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
  48. 1D Wavelet Coefficient Behavior If f is C in supp(

    j,n ), then | f, j,n ⇥| 2j( +1/2)||f||C || ||1 | f, j,n ⇥| 2j/2||f|| || ||1 If f is bounded (e.g. around a singularity), then −0.2 −0.1 0 0.1 0.2 −0.2 −0.1 0 0.1 0.2 −0.5 0 0.5 −0.5 0 0.5 0 0.2 0.4 0.6 0.8 1
  49. For Fourier, linear non-linear, sub-optimal. For wavelets, linear non-linear, optimal.

    Piecewise Regular Functions in 1D Theorem: If f is C outside a finite set of discontinuities: n[M] = ||f fn M ||2 = O(M 1) (Fourier), O(M 2 ) (wavelets).
  50. Examples of 1D Approximations 0 0.2 0.4 0.6 0.8 1

    0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 −2 −1.8 −1.6 −1.4 −1.2 −1 −0.8 −6 −5.5 −5 −4.5 −4 −3.5 −3 −2.5 −2 −1.5 −1
  51. Large coe cient | f, jn ⇥| < T x1

    x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: |Cj | K|S| = constant
  52. Large coe cient | f, jn ⇥| < T x1

    x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 |Cj | K|S| = constant
  53. Large coe cient | f, jn ⇥| < T x1

    x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1/2 Regular: 2j2 = (T/C)2 |Cj | K|S| = constant
  54. Large coe cient | f, jn ⇥| < T Hand-made

    approximate: ˜ fM = j j2 n Cj f, j,n ⇥ j,n + j j1 n Cc j f, j,n ⇥ j,n x1 x2 S = {x1, x2 } Localizing the Singular Support f(x) j1 j2 Small coe cient Singular support: Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1/2) Singular, n Cj : |⇥f, j,n ⇤| C2j/2 Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1/2 Regular: 2j2 = (T/C)2 |Cj | K|S| = constant
  55. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f,

    j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
  56. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f,

    j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) M j j2 |Cj | + j j1 |Cc j | j j2 K|S| + j j1 2 j = O(| log(T)| + T 1 +1/2 ) = O(T 1 +1/2 ) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
  57. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f,

    j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Computing Error and #Coefficients f(x) j1 j2 n Cc j : |⇥f, j,n ⇤| C2j( +1/2) n Cj : |⇥f, j,n ⇤| C2j/2 2j1 = (T/C) 1 +1/2 2j2 = (T/C)2 j<j2 (K|S|) C22j + j<j1 2 j C22j(2 +1) M j j2 |Cj | + j j1 |Cc j | j j2 K|S| + j j1 2 j = O(| log(T)| + T 1 +1/2 ) = O(T 1 +1/2 ) ||f fM ||2 = O(M 2 ) = O(2j2 + 22 j1 ) = O(T2 + T 2 +1/2 ) = O(T 2 +1/2 ) |Cj | K|S| = constant Singulatities regular part
  58. 2D Wavelet Approximation If f is C in supp( j,n

    ), then | f, j,n ⇥| 2j( +1)||f||C || ||1 | f, j,n ⇥| 2j||f|| || ||1 If f is bounded (e.g. around a singularity), then
  59. Fourier Wavelet, both sub-optimal. Wavelets: same result for BV functions

    (optimal). Piecewise Regular Functions in 2D n[M] = ||f fn M ||2 = O(M 1/2) (Fourier), O(M 1) (wavelets). Theorem: If f is C outside a set of finite length edge curves,
  60. Example of 2D Approximations

  61. Length(S) = L Localizing the Singular Support in 2D j1

    j2 f(x, y) |Cj | LK2 j = constant
  62. Length(S) = L Localizing the Singular Support in 2D j1

    j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j |Cj | LK2 j = constant
  63. Length(S) = L Localizing the Singular Support in 2D j1

    j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1 Regular: 2j2 = T/C |Cj | LK2 j = constant
  64. Hand-made approximate: ˜ fM = j j2 n Cj f,

    j,n ⇥ j,n + j j1 n Cc j f, j,n ⇥ j,n Length(S) = L Localizing the Singular Support in 2D j1 j2 f(x, y) Coe cient behavior: Regular, n Cc j : |⇥f, j,n ⇤| C2j( +1) Singular, n Cj : |⇥f, j,n ⇤| C2j Cut-o scales (depends on T): Singular: 2j1 = (T/C) 1 +1 Regular: 2j2 = T/C |Cj | LK2 j = constant
  65. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f,

    j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T)
  66. ||f fM ||2 ||f ˜ fM ||2 j<j2,n Cj |⇥f,

    j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T) M j j2 |Cj | + j j1 |Cc j | j j2 LK2 j + j j1 2 2j = O(T 1 + T 1 +1 ) = O(T 1)
  67. ||f fM ||2 = O(M 1) ||f fM ||2 ||f

    ˜ fM ||2 j<j2,n Cj |⇥f, j,n ⇤|2 + j<j1,n Cc j |⇥f, j,n ⇤|2 Singulatities regular part Computing Error and #Coefficients j1 j2 f(x, y) n Cc j : |⇥f, j,n ⇤| C2j( +1) n Cj : |⇥f, j,n ⇤| C2j 2j2 = T/C 2j1 = (T/C) 1 +1 j<j2 LK2 j C222j + j<j1 2 2j C222j( +1) |Cj | LK2 j = constant = O(2j2 + 22 j1 ) = O(T + T 2 +1 ) = O(T) M j j2 |Cj | + j j1 |Cc j | j j2 LK2 j + j j1 2 2j = O(T 1 + T 1 +1 ) = O(T 1)
  68. Overview • Approximation and Compression • Decay of Approximation Error

    • Fourier for Smooth Functions • Wavelet for Piecewise Smooth Functions • Curvelets and Finite Elements for Cartoons
  69. Geometric image model: f is C outside a set of

    C edge curves. BV image: level sets have finite lengths. Geometric image: level sets are regular. Geometry = cartoon image Sharp edges Smoothed edges Geometrically Regular Images | f|
  70. Approximation of f, C2 outside C2 edges. Piecewise linear approximation

    on M triangles: ˜ fM . Geometic Construction : Finite Elements
  71. Approximation of f, C2 outside C2 edges. Piecewise linear approximation

    on M triangles: ˜ fM . Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2
  72. Approximation of f, C2 outside C2 edges. Piecewise linear approximation

    on M triangles: ˜ fM . Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2 M/2 anisotropic triangles. Singular areas:
  73. Approximation of f, C2 outside C2 edges. Piecewise linear approximation

    on M triangles: ˜ fM . Di culties to build e cient approximations. No optimal strategies (greedy solutions). Theorem: If f is C2 outside a set of C2 contours, then one has for an adapted triangulation ||f ˜ fM ||2 = O(M 2). Geometic Construction : Finite Elements Regular areas: M/2 equilateral triangles. M 1/2 M 1/2 M/2 anisotropic triangles. Singular areas:
  74. Greedy Triangulation Optimization Bougleux, Peyr´ e, Cohen, ECCV’08

  75. Curvelet Atoms Parabolic dyadic scaling: Rotation: [Candes, Donoho] [Candes, Demanet,

    Ying, Donoho] “width length2”
  76. Curvelet Tight Frame Spacial sampling: Tight frame of L2(R2): Angular

    sampling:
  77. Discrete curvelets: O(N log(N)) algorithm. Redundancy 5 =⇥ not e

    cient for compression. M-term curvelet approximation: Curvelet Approximation Theorem: If f is C2 outside a set of C2 edges, ||f fM ||2 = O(M 2(log M)3).
  78. Works on elongated edges. Works also on locally parallel textures

    ! Curvelets Denoising
  79. Conclusion

  80. Conclusion

  81. Conclusion 0 0.2 0.4 0.6 0.8 1

  82. Conclusion 0 0.2 0.4 0.6 0.8 1