Master Mathematical Analysis and Applications Course M2 - S3 Image processing - PDE - Weeks 3-7 Mohammed Hachama [email protected] hachama.github.io/home -Fall 2023-

Heat PM Weickert Morphological Deconv Inpainting Outline 1. Heat equation 2. Perona-Malik PDE 3. Weickert’s PDE 4. Morphological PDE’s 5. Deconvolution 6. Inpainting Image processing (Weeks 2-3) -PDE’s- (2/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Outline 1. Heat equation 2. Perona-Malik PDE 3. Weickert’s PDE 4. Morphological PDE’s 5. Deconvolution 6. Inpainting Image processing (Weeks 2-3) -PDE’s- (3/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter uσ = (gσ ⋆ u0 )(x) where gσ (x) = 1 2πσ2 exp − |x|2 2σ2 ! . Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter uσ = (gσ ⋆ u0 )(x) where gσ (x) = 1 2πσ2 exp − |x|2 2σ2 ! . Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Gaussian filter Fg = 0 1 0 1 2 1 0 1 0 Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Gaussian filter • Convolution Image processing (Weeks 2-3) -PDE’s- (4/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • PDE      ∂u ∂t (x, t) = ∆u(x, t), t ≥ 0, x ∈ R2 u(x, 0) = u0 (x). • Non uniqueness without additional conditions Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 We consider that images are primarily defined on [0, 1]2 and extended to C = [−1, 1]2 via mirror symmetry. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 We consider that images are primarily defined on [0, 1]2 and extended to C = [−1, 1]2 via mirror symmetry. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 We consider that images are primarily defined on [0, 1]2 and extended to C = [−1, 1]2 via mirror symmetry. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 We consider that images are primarily defined on [0, 1]2 and extended to C = [−1, 1]2 via mirror symmetry. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 We consider that images are primarily defined on [0, 1]2 and extended to C = [−1, 1]2 via mirror symmetry. • L1 # (C) = u defined on C, such that Z C u(x) dx < +∞. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness on R2 Then, u(t, .) = g√ 2t ∗ u0 is the unique function in L1 # (C), satisfying ( ∂u ∂t = ∆u, (t, x) ∈ (0, ∞) × R2 ∥u − u0 ∥L1(C) → 0, t → 0. and • ∀t1 > 0, ∃c > 0, ∀t ≥ t1 : sup x∈R2 |u(t, x)| ≤ c(t1 ) ∥u0 ∥L1 # (C) • Si u0 ∈ L∞ # (C) : inf x∈R2 u0 (x) ≤ u(t, x) ≤ sup x∈R2 u0 (x), ∀t > 0 Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Existence and uniqueness • On R2, we need a prior bound on the growth rate of u for |x| → ∞ |u(x, t)| ≤ Aea|x|2 , (x ∈ Rn, 0 ≤ t ≤ T) • On bounded domain Ω: u0 continuous or L2 with Dirichlet or Neumann BC. Image processing (Weeks 2-3) -PDE’s- (5/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Implementation: ∀t > 0, ∂u ∂t = ∆u ∂u ∂t ≈ u(n+1) − u(n) τ ∆u ≈ uk i+1,j + uk i−1,j + uk i,j+1 + uk i,j−1 − 4uk i,j uk+1 i,j = (1 − 4 τ)uk i,j + τuk i+1,j + τuk i−1,j + τuk i,j+1 + τuk i,j−1 F = 0 τ 0 τ 1 − 4 τ τ 0 τ 0 • Numerical stability condition: τ ≤ 0.25 Image processing (Weeks 2-3) -PDE’s- (6/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation • Heat equation: ∀t > 0, ∂u ∂t = ∆u Image processing (Weeks 2-3) -PDE’s- (6/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Heat equation Denoising effect/Blur Image processing (Weeks 2-3) -PDE’s- (6/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Anisotropic diffusion Denoising effect/Blur Image processing (Weeks 2-3) -PDE’s- (6/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Anisotropic diffusion Image processing (Weeks 2-3) -PDE’s- (6/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Outline 1. Heat equation 2. Perona-Malik PDE 3. Weickert’s PDE 4. Morphological PDE’s 5. Deconvolution 6. Inpainting Image processing (Weeks 2-3) -PDE’s- (7/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Perona-Malik PDE • Equation:      ∂u ∂t = div(c(|∇u|)∇u) Ω × ]0, T[ , u(x, 0) = u0 (x) Ω, un = 0 ∂Ω × ]0, T[ where c(s) > 0, non-increasing, c(0) = 1, and lim x→+∞ c(s) = 0: • c1(s) = 1/(1 + s2/λ2), • c2(s) = exp � −s2/(2λ2) , • Charbonnier diffusivity: c3(s) = 1/( q 1 + (s/k)2). • Ill-posed problem: A sol. (classical, weak, . . . ) does not exist. • Good numerical results. Image processing (Weeks 2-3) -PDE’s- (8/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Perona-Malik PDE • Central differences to approximate the gradient ci,j = c(|∇ui,j |) |∇ui,j | ≃ s ui+1,j − ui−1,j 2 2 + ui,j+1 − ui,j−1 2 2 • Discretized divergence and time derivative: ∂u ∂t = ∂ ∂x (c(|∇u|)ux ) + ∂ ∂y (c(|∇u|)uy ) uk+1 i,j −uk+1 i,j ∆t = ck i+1 2 ,j uk i+1,j − uk i,j − ck i−1 2 ,j uk i,j − uk i−1,j + ck i,j+1 2 uk i,j+1 − uk i,j − ck i,j−1 2 uk i,j − uk i,j−1 • Mid-points = averages over neighboring pixels: c i±1 2 ,j = ci±1,j + ci,j 2 , c i,j±1 2 = ci,j±1 + ci,j 2 • ∆t ≤ 0.25 to ensure stability. Image processing (Weeks 2-3) -PDE’s- (8/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Perona-Malik PDE Image processing (Weeks 2-3) -PDE’s- (8/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting PM variant: Catt´ e-Morel-Lions’ PDE • Equation      ∂u ∂t = div(c(|∇Gσ ∗ u|)∇u) Ω × ]0, +∞[ , u(x, 0) = u0 (x) Ω, un = 0 ∂Ω × ]0, +∞[ • Existence and Uniqueness Let c: R+ −→ R+ ∗ be smooth, decreasing with c(0) = 1 , lim s→+∞ c(s) = 0 and s 7−→ c( √ s) smooth. If u0 ∈ L2(Ω), then there exists a unique solution u ∈ C([0, T]; L2(Ω)) ∩ (]0, T[ ; W 1,2(Ω)). Image processing (Weeks 2-3) -PDE’s- (9/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting PM variant: Catt´ e-Morel-Lions’ PDE • Schauder’s fixed-point theorem un+1 = T(un)      ∂un+1 ∂t = div(g(| (∇Gσ ∗ un) |)∇un+1) Ω × ]0, +∞[ , un+1(x, 0) = u0 (x) Ω, ∂un+1 ∂n = 0 ∂Ω × ]0, +∞[ (un)n converges in C([0, T]; L2(Ω)) to the unique solution. Image processing (Weeks 2-3) -PDE’s- (9/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Outline 1. Heat equation 2. Perona-Malik PDE 3. Weickert’s PDE 4. Morphological PDE’s 5. Deconvolution 6. Inpainting Image processing (Weeks 2-3) -PDE’s- (10/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Take direction of the image gradients into account ∂u ∂t = div(D(∇u)∇u) where D represents a matrix-valued diffusion tensor that describes the smoothing directions and the corresponding diffusivities. • D(∇u) = I: constant diffusion coefficient in all directions • D(∇u) = g(|∇uσ |)I: Perona-Malik type nonlinear diffusion • D is a function of the structure tensor ∇u∇uT = " u2 x ux uy ux uy u2 y # • Eigenvectors: v1 ∥∇u (λ1 = |∇u|) and v2 ⊥∇u (λ2 = 0). Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Structure tensor = image feature describing the local orientation information J(∇u) = gσ ∗ (∇uσ ∇uT σ ). Isotropic struct. Linelike struct. Corners µ1 ≈ µ2 µ1 >> µ2 ≈ 0 µ1 ≥ µ2 >> 0 Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Take direction of the image gradients into account ∂u ∂t = div(D(J(∇u))∇u) • How to choose the matrix D(J(∇u))? • Keep the eigenvectors of J(∇u). • Choose the eigenvalues of D, λ1 and λ2, depending on the eigenvalues of J(∇u), µ1 and µ2. Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Edge-enhancing anisotropic diffusion ∂u ∂t = div(D(J(∇u))∇u) λ1 =    1 µ1 = 0 1 − exp −α2 µ4 1 else λ2 = 1 Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Coherence-enhancing aniso. diffusion ∂u ∂t = div(D(J(∇u))∇u) λ1 = α λ2 = ( α µ1 = µ2 α + (1 − α) exp −1 (µ1−µ2)2 else Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])

Heat PM Weickert Morphological Deconv Inpainting Weickert’s PDE • Existence and uniqueness Assume that u0 ∈ L∞(Ω), ρ ≥ 0, and σ, T > 0. Let a := ess inf u0 , b := ess sup u0 , and consider the problem      ∂u ∂t = div(D(Jρ (∇uσ |)∇u) Ω × ]0, T] , u(x, 0) = u0 (x) Ω, un = 0 ∂Ω × ]0, T] where D ∈ C∞(R2×2, R2×2),d12 (J) = d21 (J) for all symmetric matrices J, and ∀w ∈ L∞(Ω, R2), there exists a positive lower bound for the eigenvalues of D(Jρ (w)). Then, the problem has a unique solution which satisfies u ∈ C([0, T]; L2(Ω)) ∩ L2(0, T; H1(Ω)), ∂t u ∈ L2(0, T; H1(Ω)). Image processing (Weeks 2-3) -PDE’s- (11/26) M. Hachama ([email protected])