Signal and Image Processing
with Orthogonal Decompositions
Gabriel Peyré
www.numerical-tours.com
Slide 2
Slide 2 text
Signals, Images and More
Continuous signal:
f 2
L
2
([0
,
1]).
Slide 3
Slide 3 text
Continuous image:
f 2
L
2
([0
,
1]
2
).
Signals, Images and More
Continuous signal:
f 2
L
2
([0
,
1]).
Slide 4
Slide 4 text
Continuous image:
f 2
L
2
([0
,
1]
2
).
General setting: f : [0, 1]d ! Rs
Videos:
d
= 3
, s
= 1.
Signals, Images and More
Continuous signal:
f 2
L
2
([0
,
1]).
Slide 5
Slide 5 text
Continuous image:
f 2
L
2
([0
,
1]
2
).
General setting: f : [0, 1]d ! Rs
Videos:
d
= 3
, s
= 1.
Color image:
d
= 2
, s
= 3.
Signals, Images and More
Continuous signal:
f 2
L
2
([0
,
1]).
Slide 6
Slide 6 text
Continuous image:
f 2
L
2
([0
,
1]
2
).
General setting: f : [0, 1]d ! Rs
Videos:
d
= 3
, s
= 1.
Color image:
d
= 2
, s
= 3.
Multi-spectral: d = 2, s 3.
Signals, Images and More
Continuous signal:
f 2
L
2
([0
,
1]).
Slide 7
Slide 7 text
Overview
•Orthogonal Representations
•Linear and Non-linear Approximations
•Compression and Denoising
Slide 8
Slide 8 text
Orthogonal basis { m
}m
of L2([0, 1]d)
Continuous signal/image f L2([0, 1]d).
Orthogonal Decompositions
Slide 9
Slide 9 text
Orthogonal basis { m
}m
of L2([0, 1]d)
f =
m
f, m m
||f|| = |f(x)|2dx =
m
| f, m
⇥|2
Continuous signal/image f L2([0, 1]d).
Orthogonal Decompositions
Slide 10
Slide 10 text
Orthogonal basis { m
}m
of L2([0, 1]d)
f =
m
f, m m
||f|| = |f(x)|2dx =
m
| f, m
⇥|2
Continuous signal/image f L2([0, 1]d).
Orthogonal Decompositions
m
Slide 11
Slide 11 text
1-D Wavelet Basis
Wavelets:
j,n
(x) =
1
2j/2
x 2jn
2j
Position n, scale 2j, m = (n, j).
Slide 12
Slide 12 text
1-D Wavelet Basis
Wavelets:
j,n
(x) =
1
2j/2
x 2jn
2j
Position n, scale 2j, m = (n, j).
m1,m2
Basis { m1,m2
(x1, x2
)}m1,m2
of L2([0, 1]2)
m1,m2
(x1, x2
) =
m1
(x1
)
m2
(x2
)
tensor
product
f(x) f, m1,m2
Fourier
transform
2-D Fourier Basis
Basis { m
(x)}m
of L2([0, 1])
m1
m2
x
m
Slide 15
Slide 15 text
3 elementary wavelets { H, V , D}.
Orthogonal basis of L2([0, 1]2):
k
j,n
(x) = 2 j (2 jx n)
k=H,V,D
j<0,2j n [0,1]2
2-D Wavelet Basis
V (x)
H(x) D(x)
Slide 16
Slide 16 text
3 elementary wavelets { H, V , D}.
Orthogonal basis of L2([0, 1]2):
k
j,n
(x) = 2 j (2 jx n)
k=H,V,D
j<0,2j n [0,1]2
2-D Wavelet Basis
V (x)
H(x) D(x)
Slide 17
Slide 17 text
wavelet
f, k
j,n
Example of Wavelet Decomposition
f(x)
transform
x
(j, n, k)
Slide 18
Slide 18 text
Discrete Computations
Discrete orthogonal basis { m
} of CN .
f =
m
f, m m
Slide 19
Slide 19 text
Fast Fourier Transform (FFT), O(N log(N)) operations.
Discrete Computations
Discrete orthogonal basis { m
} of CN .
m
[n] =
1
N
e2i
N
nm
f =
m
f, m m
Slide 20
Slide 20 text
Fast Fourier Transform (FFT), O(N log(N)) operations.
Fast Wavelet Transform, O(N) operations.
Discrete Wavelet basis: no closed-form expression.
Discrete Computations
Discrete orthogonal basis { m
} of CN .
m
[n] =
1
N
e2i
N
nm
f =
m
f, m m
Slide 21
Slide 21 text
Overview
•Orthogonal Representations
•Linear and Non-linear Approximations
•Compression and Denoising
Slide 22
Slide 22 text
Linear Approximation
Slide 23
Slide 23 text
Linear Approximation
Slide 24
Slide 24 text
Linear Approximation
Slide 25
Slide 25 text
Non-Linear Approximation
Slide 26
Slide 26 text
Coe cients
Non-Linear Approximation
Slide 27
Slide 27 text
Coe cients
Non-Linear Approximation
Slide 28
Slide 28 text
Best basis Fastest error decay ||f fM
||2
log(||f fM
||)
log(M)
Efficiency of Transforms
Fourier DCT
Local DCT Wavelets
Slide 29
Slide 29 text
Efficient Approximation
Slide 30
Slide 30 text
Efficient Approximation
Slide 31
Slide 31 text
Efficient Approximation
Slide 32
Slide 32 text
Efficient Approximation
Slide 33
Slide 33 text
Efficient Approximation
Slide 34
Slide 34 text
Efficient Approximation
Slide 35
Slide 35 text
Efficient Approximation
Slide 36
Slide 36 text
Overview
•Orthogonal Representations
•Linear and Non-linear Approximations
•Compression and Denoising
Slide 37
Slide 37 text
JPEG-2000 vs. JPEG, 0.2bit/pixel
Slide 38
Slide 38 text
Compression by Transform-coding
Image f Zoom on f
f
forward
a[m] = ⇥f, m
⇤ R
transform
Slide 39
Slide 39 text
Compression by Transform-coding
Image f Zoom on f
f
forward
a[m] = ⇥f, m
⇤ R
transform
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
Slide 40
Slide 40 text
Compression by Transform-coding
Image f Zoom on f
f
forward
a[m] = ⇥f, m
⇤ R coding
transform
Entropic coding: use statistical redundancy (many 0’s).
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
Slide 41
Slide 41 text
Compression by Transform-coding
Image f Zoom on f
f
forward
a[m] = ⇥f, m
⇤ R coding
decoding
q[m] Z
transform
Entropic coding: use statistical redundancy (many 0’s).
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
Slide 42
Slide 42 text
Compression by Transform-coding
Image f Zoom on f
f
forward
Dequantization: ˜
a[m] = sign(q[m]) |q[m] +
1
2
⇥
T
a[m] = ⇥f, m
⇤ R coding
decoding
q[m] Z
˜
a[m] dequantization
transform
Entropic coding: use statistical redundancy (many 0’s).
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
Slide 43
Slide 43 text
Compression by Transform-coding
Image f Zoom on f f , R =0.2 bit/pixel
f
forward
Dequantization: ˜
a[m] = sign(q[m]) |q[m] +
1
2
⇥
T
a[m] = ⇥f, m
⇤ R coding
decoding
q[m] Z
˜
a[m] dequantization
transform
backward
fR
=
m IT
˜
a[m]
m
transform
Entropic coding: use statistical redundancy (many 0’s).
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
Slide 44
Slide 44 text
Compression by Transform-coding
Image f Zoom on f f , R =0.2 bit/pixel
f
forward
Dequantization: ˜
a[m] = sign(q[m]) |q[m] +
1
2
⇥
T
a[m] = ⇥f, m
⇤ R coding
decoding
q[m] Z
˜
a[m] dequantization
transform
backward
fR
=
m IT
˜
a[m]
m
transform
Entropic coding: use statistical redundancy (many 0’s).
Quantization: q[m] = sign(a[m])
|a[m]|
T
⇥
Z
˜
a[m]
T T 2T
2T a[m]
Quantized q[m]
bin T
q[m] Z
||f fM
||2 = O(M ) =⇥ ||f fR
||2 = O(log (R)R )
Theorem:
Slide 45
Slide 45 text
Noise in Images
Slide 46
Slide 46 text
Denoising
Slide 47
Slide 47 text
Denoising
thresh.
f =
N 1
m=0
f, m
⇥ m
˜
f =
| f, m
⇥|>T
f, m
⇥ m
Slide 48
Slide 48 text
Denoising
thresh.
f =
N 1
m=0
f, m
⇥ m
˜
f =
| f, m
⇥|>T
f, m
⇥ m
In practice:
T 3
for T = 2 log(N)
Theorem: if ||f0 f0,M
||2 = O(M ),
E(|| ˜
f f0
||2) = O( 2
+1 )