Image Processing 1 : 1.Introduction

Digital Image Processing Introduction (Week 1) NHSM - 4th year
- Fall 2024 - Prof. Mohammed Hachama [email protected] http://hachama.github.io/home/

Outline Introduction Image characteristics Digitization Geometric Transformations NHSM - 4th
year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 2/23

Introduction NHSM - 4th year: Digital Image Processing - Introduction
(Week 1) - M. Hachama ([email protected]) 3/23

Information about the course • Image processing courses • Introduction
to image processing (S7 - UEF721): Image = matrix • Computer vision (S8 - UEF823): More applications • Image processing (S9 - UEF911): Image = function (more elaborated mathematics) • UEF721 Course: 15 weeks • 1 lecture: slides, no written notes (see references if need be!) • 1 tutorial/practical session • 3 assignments (Chap. 1,2,3) • 3 individual projects (Chap. 4,5,6) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Career opportunities • Work in
the field of signal/image/video processing, computer vision, or data science in general (in both industry or academia). • Be able to understand and implement recent publications in that field. • Understand latest machine learning and computer vision techniques. Many deep learning concepts are based on tools that will be introduced in this class. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Prerequisites • Linear algebra •
Differential calculus • Probability and statistics • Fourier transform • Basics of optimization • Python programming NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Evaluation • 3 Assignments =
1 points each, report submission • 3 Projects = 3 points each, 10 minutes oral presentation • CC = Assignments/3 + Projects/9 + Midterm/8 + participation (bonus)/2 • Exam = 20/20 • Individual work, sharp deadlines NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Assignments • Assignment 1: Image
manipulation, characteristics, geometric transformations • Assignment 2: spatial processing • Assignment 3: frequency domain processing • Projects • Project 1: Image compression - Fourier, Wavelets, Dictionaries • Project 2: Image restoration - Denoising, Debluring, Inpinting • Project 3: Image segmentation - Edges detection, Region classification NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Books NHSM - 4th year:
Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Similar courses • CS 131
Computer Vision: Foundations and Applications (Stanford University) • Introduction to Computer Vision, CS5670, Spring 2023, Cornell Tech, www.cs.cornell.edu/courses/cs5670/2024sp/ • 16-385 Computer Vision, Spring 2020, Carnegie Mellon, www.cs.cmu.edu/∼16385/ • YouTube lectures • Image and video Processing: From Mars to Hollywood with a Strop at the Hospital, Guillermo Sapiro • Intro to Digital Image Processing, Rich Radke • Various codes • www.numerical-tours.com NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Information about the course • Outline • Introduction (1 week)
• Spatial Processing (2 weeks) • Frequency-domain processing (2 weeks) • Image Compression (1 week) • Exercises (1 week) • Mid-Term • Image Restoration (3 weeks) • Image Segmentation (3 weeks) • Evaluations (projects) (1 week) • Links • Course: http://github.com/hachama/introimg • Professor: https//hachama.github.io/home/ NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 4/23

Why automatic image processing? NHSM - 4th year: Digital Image
Processing - Introduction (Week 1) - M. Hachama ([email protected]) 5/23

Why automatic image processing? Importance of using automatic image processing!
NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 5/23

What is an image? Image = representation of a scene
• Scene: Set of objects that are in the field of vision • Light source: Sun, ambient lighting, spotlights, ... NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

What is an image? Image = matrix of intensities NHSM
- 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

What is an image? Image = matrix of intensities •
Image = numbers! • Red, Green, Blue (RGB), K = 3 • RGB: Usual color-space for acquisition and display • Color-spaces for different purposes: HSV (Hue, Saturation, Value), YUV, YCbCr. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

Discretization NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

Quantization NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

What is an image? Intensities = information • Thermal Imaging:
temperature degrees NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

What is an image? Intensities = information • Magnetic resonance
imaging (MRI) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

What is an image? Intensities = information • X-ray scans

What is an image? Intensities = information • Radar Imaging

What is an image? Image = A mathematical object •
Matrix/Tensor • Function I : E −→ F • Distribution • Random field • Surface/sub-manifold • ... NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 6/23

Image processing (1) What is image processing? • Transformation of
a digital image into another image or extracting features, primitives, or information. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 7/23

Image processing (1) What is image processing? NHSM - 4th

Image processing (2) Fundamental problems • Restoration NHSM - 4th

Image processing (2) Fundamental problems • Deconvolution NHSM - 4th

Image processing (2) Fundamental problems • Contours Detection NHSM -
4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 8/23

Image processing (2) Fundamental problems • Segmentation NHSM - 4th

Image processing (2) Fundamental problems • Matching NHSM - 4th

Image processing (2) Fundamental problems • Inpainting NHSM - 4th

Image processing (2) Fundamental problems • Rigid Registration NHSM -

Image processing (2) Fundamental problems • Non-Rigid Registration NHSM -

Image processing (2) Fundamental problems • Image decomposition NHSM -

Image processing (2) Fundamental problems • Image fusion NHSM -

Image processing (3) Applications • Pattern recognition NHSM - 4th

Image processing (3) Applications • Optical character recognition (OCR) NHSM

Image processing (3) Applications • Scene recognition NHSM - 4th

Image processing (3) Applications • Inspection in manufacturing NHSM -

Image processing (3) Applications • Compression NHSM - 4th year:

Image processing (3) Applications • Medical imaging NHSM - 4th

Image processing (3) Applications • Geographic Information Systems NHSM -

Image processing (3) Applications • Special effects NHSM - 4th

Image processing (3) Applications • Biometric identification NHSM - 4th

Image processing (3) Applications • Self-driving cars NHSM - 4th

Image processing (3) Applications • Augmented Reality NHSM - 4th

Image processing (3) Applications • Virtual Reality NHSM - 4th

Image characteristics NHSM - 4th year: Digital Image Processing -
Introduction (Week 1) - M. Hachama ([email protected]) 10/23

Image = matrix • Image = Matrix (I : {1,
.., N} × {1, ..., M} −→ [0, 1]) • Pixel = indices + intensity NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

.., N} × {1, ..., M} −→ [0, 1]) • Pixel = indices + intensity • Frame = Origin + Axes NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

.., N} × {1, ..., M} −→ [0, 1]) • Pixel = indices + intensity • Frame = Origin + Axes • Pixel neighborhood system: 4-neighbors (2 horizontals + 2 verticals) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

Image = matrix Pixel neighbors • 4-neighbors (2 horizontals +
2 verticals) (x + 1, y), (x − 1, y), (x, y + 1), (x, y − 1) • 4 diagonal (x + 1, y + 1), (x + 1, y − 1), (x − 1, y + 1), (x − 1, y − 1) • 8-Neighbors • Some pixels can call outside the image! NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

Image = matrix Distances between pixels • Given two pixels
p(x, y) and q(s, t) • Manhattan distance: D1 (p, q) = |x − s|+|y − t| ? ? ? ? ? ? ? ? ? ? ? ? 0 ? ? ? ? ? ? ? ? ? ? ? ? NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

p(x, y) and q(s, t) • Euclidean Distance: D2 (p, q) = [(x − s)2 + (y − t)2]1/2 ? ? ? ? ? ? ? ? ? ? ? ? 0 ? ? ? ? ? ? ? ? ? ? ? ? NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

p(x, y) and q(s, t) • chessboard distance: D3 (p, q) = maximum(|x − s|, |y − t|) ? ? ? ? ? ? ? ? ? ? ? ? 0 ? ? ? ? ? ? ? ? ? ? ? ? NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

Image = matrix Image = Vector representation NHSM - 4th

Image = matrix Image = Vector representation Linear algebra tools:
linear operator, matrix decomposition, norms, optimization, ... NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 11/23

Types of regions NHSM - 4th year: Digital Image Processing
- Introduction (Week 1) - M. Hachama ([email protected]) 12/23

Characteristics For an image I: • Dynamic: max I −
min I • Luminance: Lum = Mean (I) • Contrast: C1 = N i=1 M j=1 (I(i, j) − Lum)2 N.M , C2 = max I − min I max I + min I NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 13/23

Format and storage • A bit (Binary-Digit) = ”The basic
element with which the computer works. Its value is 1 or 0. • 8 bits = 28 = 256 possible values (16 bits = 65.536). • 1 Byte = 1 octet = 8 bits. • Binary Image 512 × 512: 512 × 512 × 1 = 262144 bits = 32768 bytes. • 512 × 512 8-bit gray-level image: 512 × 512 × 1 = 262144 bytes. • 512 × 512 color image, with a byte used to store each color: 512 × 512 × 3 = 786432 bytes. • 2 hours color video: 1000x1000x24x30x60x120 = 5184 × 109 bits = 648Go! NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 14/23

Format and storage • A bit (Binary-Digit) = ”The basic
element with which the computer works. Its value is 1 or 0. • 8 bits = 28 = 256 possible values (16 bits = 65.536). • 1 Byte = 1 octet = 8 bits. • Binary Image 512 × 512: 512 × 512 × 1 = 262144 bits = 32768 bytes. • 512 × 512 8-bit gray-level image: 512 × 512 × 1 = 262144 bytes. • 512 × 512 color image, with a byte used to store each color: 512 × 512 × 3 = 786432 bytes. • 2 hours color video: 1000x1000x24x30x60x120 = 5184 × 109 bits = 648Go! We definitely need compression! NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 14/23

Format and storage • TIFF(.tif, .tiff): Tagged Image File Format,
no compression, high quality, large size. Good for printing. • JPEG (.jpg, .jpeg): Joint Photographic Experts Group, Lossy compression, reduced size. Good for digital cameras, web use. • GIF: Graphics Interchange Format, used for web graphics, can be animated, limited to 256 colors, allow for transparency, small size. • PNG: Portable Network Graphics , lossless, support 16 million colors. • EPS (.eps) Encapsulated PostScript, vector file type, can be opened in applications such as Adobe Illustrator or CorelDRAW. • RAW Image Files (.raw, .cr2, .nef, .orf, .sr2, dicom, ...): created by a camera or scanner, hold a lot of information (intensities + metadata), need to be processed in an editor. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 14/23

Digitization NHSM - 4th year: Digital Image Processing - Introduction
(Week 1) - M. Hachama ([email protected]) 15/23

Digitization Sampling + quantification NHSM - 4th year: Digital Image

Quantification (1) Tonal resolution: 1,2, ..., 8 bits NHSM -

Quantification (1) Tonal resolution NHSM - 4th year: Digital Image

Quantification (1) Application 1: Compression by reducing the tonal resolution
% Loading OpenCV library import cv2 as cv % Quantification ratio Q = 32 % Lecture of the input image im1 = cv.imread(’data/lake.jpg’,0) % Compression im2=(im1.astype(np.float32)/Q).astype(np.uint8) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 17/23

Original image/1 NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 17/23

Image size/2 NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 17/23

Quantification (1) Application 1: Compression//False contours Image size/16 NHSM -

Quantification (1) Application 1: Compression//Dithering: adding noise Image size/16 NHSM

Quantification (2) Application 2: Steganography Decimal/binary Representation 241 = (?
) × 20 + (? ) × 21 + (? ) × 22 + (? ) × 23 + (? ) × 24 + (? ) × 25 + (? ) × 26 + (? ) × 27 241 120 60 30 15 7 3 1 0 1 0 0 0 1 1 1 1 − NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 18/23

Quantification (2) Application 2: Steganography Bitplanes: Original / Bit 1

Quantification (2) Application 2: Steganography (Secret message) NHSM - 4th

Quantification (2) Application 2: Steganography (modification od bit 7) NHSM

Quantification (2) Application 2: Steganography (modification of bits 7, 6
and 5) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 18/23

Sampling Spatial resolution: number of pixels NHSM - 4th year:

Sampling Spatial resolution: number of pixels • dpi: dots per
inch (1 inch = 2.54 cm), ppp : points par pouce • Number of ”physical” pixels in an inch of a printed document. • ppc: points per centimeter ; ppm : points per millimeter • ppi (pixels per inch): # of ”squared” pixels in an inch of a screen. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

Sampling Spatial resolution: number of pixels • Computation: we need
to use three parameters • Physical dimensions of the final image (on a screen, printed) or of the source image (page to be scanned) - e.g., 30 x 20 cm • Final resolution - e.g., 118 ppc • Image size (in pixels) - e.g., 3072 x 2048 pixels NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

Sampling Spatial resolution: number of pixels • Example 1. You
want to print a 13 x 10 cm photo. Your printer asks you to provide images with a resolution of at least 300 dpi. • minimal width: 13x300/2,54=1535 pixels • minimal height: 10x300/2,54=1181 pixels. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

Sampling Spatial resolution: number of pixels • Example 2. A
screen set at 75 ppi displays a photo with dimensions of 15 x 12 cm. What is the size of the image in pixels? • 75ppi=75/2,54 pixels/cm • Largeur de l’image : 15*75/2,54 = 443 px • Hauteur de l’image : 12*75/2,54 = 354 px NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

Sampling Aliasing = Distortion in the signal due to a
low sampling frequency NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

low sampling frequency • The bands change direction and thickness NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

low sampling frequency • The bands change direction and thickness. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 19/23

Geometric Transformations NHSM - 4th year: Digital Image Processing -
Introduction (Week 1) - M. Hachama ([email protected]) 20/23

Transformations Geometric Transformations NHSM - 4th year: Digital Image Processing
- Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Direct Transformation: (i′, j′) = T(i,
j) NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Direct Transformation: (i′, j′) = T(i,
j) • Points without images, Points outside the image NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Inverse Transformation: (i, j) = T−1(i′,
j′) • Interpolation: Nearest neighbor, Bi-linear, ... NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Translation: i′ j′ = 1 0
0 1 i j + ti tj • Rotation: x′ y′ = cos θ sin θ − sin θ cos θ x y + 0 0 • Coordinates relative to the center of the image • Exercise: Implement and run simulations. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Rotation: Coordinates relative to the center
of the image NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Transformations Geometric Transformations • Rotation: Interpolation by Nearest neighbor NHSM

Transformations Geometric Transformations • Rotation: Bi-linear Interpolation NHSM - 4th

Transformations Geometric Transformations • Change of scale: i′ j′ =
αi 0 0 αj i j + 0 0 • Affine Deformation: i′ j′ = a1 a2 a3 a4 i j + b1 b2 • Exercise: Implement and run simulations. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 21/23

Image Interpolation • Image interpolation is the process of estimating
pixel values at non-integer coordinates. • It is used in various applications such as resizing, rotating, and warping images. • Types of Image Interpolation • Replication • Nearest Neighbor Interpolation • Bilinear Interpolation • Bicubic Interpolation NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation Pixel replication • Produces a “jagged” result since
each individual square pixel is made larger. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation Nearest Neighbor Interpolation • The simplest method where
the value of the nearest pixel is assigned to the interpolated pixel. • Formula: I(x′, y′) = I(round(x), round(y)) Where: • I(x′, y′) is the intensity value at the point (x′, y′) in the output image. • I(x, y) is the intensity value at the point (x, y) in the input image. • round(x) and round(y) round the coordinates to the nearest integer, representing the nearest pixel in the input image. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation Bilinear Interpolation • Uses a weighted average of
the closest 2 × 2 neighbors (four nearest pixels) to estimate the new pixel value. The interpolation is performed first in one direction (e.g., x-axis) and then in the other (y-axis), resulting in a smoother image than Nearest Neighbor. • Formula: I(x′, y′) = 1 i=0 1 j=0 I(x + i, y + j)(1 − |x′ − (x + i)|)(1 − |y′ − (y + j)|) The weights (1 − |x′ − (x + i)|) and (1 − |y′ − (y + j)|) are proportional to the distance of (x′, y′) from the corresponding neighboring pixels. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation Bicubic Interpolation • Considers the closest 4 ×
4 neighbors (16 pixels) to estimate the new pixel value using cubic polynomials. • Formula: I(x′, y′) = 2 i=−1 2 j=−1 I(x + i, y + j)W (i, x′)W (j, y′) where W (k, t) is the cubic weight function. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation Comparison • Nearest neighbor is fast but may
produce blocky images. • Bilinear provides smoother images, but may blur details. • Bicubic gives better quality with more computational cost. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 22/23

Image Interpolation What about image warping? NHSM - 4th year:

Assignment 1 Write a python code that depends on the
OpenCV library and that performs the following: 1. Read, display and save an image. 2. Manipulate pixels intensities and image channels. 3. Transform a matrix into a vector and vice-versa. 4. Compute and display some characteristics such as dynamic, luminance and contrast. 5. Perform compression by reducing the tonal resolution. 6. Perform different geometric transformations. 7. Perform a steganography. 8. Perform subsampling and upsampling. 9. Perfom image warping into a circle. NHSM - 4th year: Digital Image Processing - Introduction (Week 1) - M. Hachama ([email protected]) 23/23

Image Processing 1 : 1.Introduction

Image Processing 1 : 1.Introduction

More Decks by Mohammed Hachama

Other Decks in Education

Featured

Transcript