CV_2_Pyramids and Multiscale representations

Transcript

1. Computer vision Image pyramids and multiscale representations (Week 2) NHSM - 4th year - Spring 2025 - Prof. Mohammed Hachama hachamam@gmail.com http://hachama.github.io/home/
2. Outline Image pyramids and multiscale representations NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 2/8
3. Image pyramids and multiscale representations NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 3/8
4. Gaussian Scale-space Gaussian multiscale analysis • A continuous representation where an image is analyzed at different levels of blur (scale) L(x, y, σ) = I(x, y) ⋆ G(x, y, σ) NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 4/8
5. Gaussian Scale-space Gaussian multiscale analysis • A continuous representation where an image is analyzed at different levels of blur (scale) L(x, y, σ) = I(x, y) ⋆ G(x, y, σ) NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 4/8
6. Gaussian Scale-space Gaussian multiscale analysis • A continuous representation where an image is analyzed at different levels of blur (scale) L(x, y, σ) = I(x, y) ⋆ G(x, y, σ) NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 4/8
7. Gaussian Scale-space Gaussian multiscale analysis • A continuous representation where an image is analyzed at different levels of blur (scale) L(x, y, σ) = I(x, y) ⋆ G(x, y, σ) NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 4/8
8. Image Pyramids Image Pyramids • 1. Gaussian pyramid • 2. Laplacian pyramid • 3. Steerable Pyramid NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 5/8
9. Gaussian Pyramid Gaussian Pyramid NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 6/8
10. Gaussian Pyramid Gaussian Pyramid NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 6/8
11. Gaussian Pyramid Gaussian Pyramid • Gaussian filtering as anti-aliasing filters • Details get smoothed out as we move to higher levels • Only mostly large uniform regions in the original image are preserved at the higher levels • Image reconstruction from the pyramid: Not possible NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 6/8
12. Gaussian Pyramid Application 1: Template matching NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 6/8
13. Gaussian Pyramid Applications • Multi-Scale Image Representation and Analysis • Scale-Invariant Image Features • Image Compression (Progressive Encoding and transmission) • Efficient Image Processing (Reducing Computation) • Optical Flow and Motion Estimation: track motion from coarse to fine resolution. • Prevents large displacements from being missed when only analyzing the full-resolution image • Deep Learning NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 6/8
14. Laplacian Pyramid Laplacian Pyramid • Retain the residuals instead of the blurred images themselves NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
15. Laplacian Pyramid Laplacian ≈ difference-of-Gaussian (DOG) NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
16. Laplacian Pyramid Laplacian is a Bandpass Filter NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
17. Laplacian Pyramid Laplacian is a Bandpass Filter NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
18. Laplacian Pyramid Laplacian is a Bandpass Filter NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
19. Laplacian Pyramid Constructing a Laplacian Pyramid NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 7/8
20. Laplacian Pyramid Application 1: Image Blending • Blend two images smoothly by decomposing them into Gaussian and Laplacian pyramids, blending them at each level, and then reconstructing the final blended image. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
21. Laplacian Pyramid Application 1: Image Blending • Blend two images smoothly by decomposing them into Gaussian and Laplacian pyramids, blending them at each level, and then reconstructing the final blended image. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
22. Laplacian Pyramid Application 1: Image Blending • Blend two images smoothly by decomposing them into Gaussian and Laplacian pyramids, blending them at each level, and then reconstructing the final blended image. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
23. Laplacian Pyramid Application 1: Image Blending NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
24. Laplacian Pyramid Application 1: Image Blending NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
25. Laplacian Pyramid Application 1: Image Blending • When blending two images directly, sharp edges at the boundary can be noticeable. By blending low-frequency components smoothly and high-frequency details selectively, the transition between images becomes natural. • The blending mask is also pyramidal, meaning it changes smoothly across scales. This ensures that low-frequency components are blended over a larger region, while high-frequency details are only blended near the boundary. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
26. Laplacian Pyramid Application 1: Image Blending NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
27. Laplacian Pyramid Application 2: Image compression • Quantize Laplacian coefficients to reduce precision (e.g., rounding to integers). • Higher levels (coarser details) tolerate aggressive quantization, while finer levels require finer quantization to preserve details. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
28. Laplacian Pyramid Application 2: Eulerian Video Magnification • Enhances small variations in color and motion • Pulse detection: monitoring tiny skin color changes due to blood flow • Breathing analysis: amplifying chest movements. • Material deformation detection: visualizing tiny vibrations in structures. NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8
29. Laplacian Pyramid Application 2: Eulerian Video Magnification • Laplacian pyramid to separate different frequency bands. • Pixel values are filtered over time to enhance (by a factor ×α) specific frequency ranges (e.g., heartbeats at 0.5–1.5 Hz). NHSM - 4th year: Computer vision - pyramids (Week 2) - M. Hachama (hachamam@gmail.com) 8/8