Introduction to Image Processing: 5.Segmentation

Digital Image Processing Segmentation (Week 10-13) NHSM - 4th year
- Fall 2024 - Prof. Mohammed Hachama [email protected] http://hachama.github.io/home/

Outline Introduction Edge Detection Simple Detectors Segmentation: Regions NHSM -
4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 2/25

Introduction NHSM - 4th year: Digital Image Processing - Segmentation
(Week 10-13) - M. Hachama ([email protected]) 3/25

What is segmentation? • Partitioning an image into a minimal
number of connected regions Ri (in a specified sense) • Each region must satisfy a property P(Ri ) Contour/Region Duality NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 4/25

What is segmentation? Contour/Region Duality NHSM - 4th year: Digital
Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 4/25

Contours NHSM - 4th year: Digital Image Processing - Segmentation

What is an edge? • It is a boundary that
separates two objects in an image. • A discontinuity in the image. • A sharp and strong change in intensity. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 6/25

What is an edge? • An edge is a sharp
and strong change in intensity. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 6/25

What is an edge? • Origins of edges NHSM -

What is an edge? • Types of edges NHSM -

Outline Introduction Edge Detection Simple Detectors Segmentation: Regions NHSM -

Simple detectors • Detection of points: Apply the following mask
and threshold the resulting image    −1 −1 −1 −1 8 −1 −1 −1 −1    NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 8/25

Simple detectors • Line detection: Filters defining 4 directions E-W,
NW-SE, N-S, NE-SW    −1 −1 −1 2 2 2 −1 −1 −1       2 −1 −1 −1 2 −1 −1 −1 2       −1 2 −1 −1 2 −1 −1 2 −1       −1 −1 2 −1 2 −1 2 −1 −1    NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 8/25

Simple detectors • Line detection: Example NHSM - 4th year:
Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 8/25

Simple detectors • Line detection: Examples NHSM - 4th year:

Derivative methods: Gradient • Derivative of an image and edges
NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 9/25

Derivative methods: Gradient • Derivative of an image and edges
• Norm: G = G2 x + G2 y ∼ Gx + Gy • Direction: θ = arctan(Gy /Gx ) NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 9/25

Derivative methods: Gradient • Example of edge detection NHSM -

Derivative methods: Gradient • Simple approximations of the discrete derivative
−1 1 −1 1 −1 0 1 −1 0 1 • Roberts filters: directional derivatives 1 0 0 −1 0 1 −1 0 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 9/25

Derivative methods: Gradient • Prewitt filters: image smoothing + image
derivative −1 −1 −1 0 0 0 1 1 1 −1 0 1 −1 0 1 −1 0 1 • Averaging + Derivation    −1 0 1 −1 0 1 −1 0 1    =    1 1 1    ∗ (−1 0 1) Edge detection less sensitive to noise. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 9/25

Derivative methods: Gradient • Sobel filters −1 −2 −1 0
0 0 1 2 1 −1 0 1 −2 0 2 −1 0 1 • Gaussian + Derivation    −1 0 1 −2 0 1 −1 0 1    =    1 2 1    ∗ (−1 0 1) Edge detection less sensitive to noise NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 9/25

Derivative methods: Gradient • Example of edge detection NHSM -

Contour detection examples • Contours Detection • Robert gradient, |I|=
I2 x + I2 y and a threshold T = 5. I = 9 9 9 9 9 9 9 2 2 9 8 9 9 9 9 2 2 2 9 9 9 9 9 9 3 2 2 9 9 9 9 9 2 2 2 2 7 9 9 9 9 2 2 2 2 9 9 9 9 2 2 2 2 2 9 9 9 9 2 2 2 4 2 9 9 9 2 2 2 2 2 2 9 9 2 2 2 2 1 2 2 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

I2 x + I2 y and a threshold T = 5. Rx = 1 0 0 −1 , Ix = 1 0 0 0 0 7 7 0 0 −1 0 0 0 6 0 0 0 0 0 0 7 7 1 0 0 0 0 0 7 0 0 0 −2 0 0 7 7 0 0 0 0 0 0 7 0 0 −2 0 0 0 7 7 0 0 0 2 0 7 7 0 0 1 0 0 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

I2 x + I2 y and a threshold T = 5. Ry = 0 1 −1 0 , Iy = 0 1 0 0 0 0 0 0 −1 0 0 0 0 −7 −1 0 0 0 0 0 0 1 0 0 2 0 0 0 −7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 −7 0 0 0 −2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 1 0 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

I2 x + I2 y and a threshold T = 5. |I|= 1 1 0 0 0 7 7 0 1 1 0 0 0 9.2 1 0 0 0 0 0 7 7.1 1 0 2 0 0 0 9.9 0 0 0 2 0 0 7 7 0 0 0 0 0 0 9.9 0 0 2 2 0 0 7 7 0 0 2 2 0 7 7 0 0 1 1 0 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

I2 x + I2 y and a threshold T = 5. Edge = 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 1 1 0 0 0 0 0 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

Contour detection examples • Edge detection with the Sobel filter
Ix NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

Iy NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

Gradient norm NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

Edges: NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 10/25

Derivative methods: Laplacian • Contours Detection NHSM - 4th year:

Derivative methods: Laplacian • Several discrete approximations of the Laplacian
exist.    0 1 0 1 −4 1 0 1 0       1 1 1 1 −8 1 1 1 1       1 4 1 4 −20 4 1 4 1    • Isotropic: Symmetric w.r.t. rotations. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 11/25

Derivative methods: Laplacian • The Laplacian, like all edge detectors,
is very sensitive to noise. • Smoothing (Gaussian) the image before detecting edges. • Edges = zero-crossings of the Laplacian of Gaussian LoG ∗ I = ∆ ∗ G ∗ I = ∆G ∗ I LoG(x, y) = − 1 πσ4 1 − x2 + y2 2σ2 e− x2+y2 2σ2 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 11/25

Derivative methods: Laplacian • Laplacian Filter NHSM - 4th year:

Derivative methods: Laplacian • Edges = zero-crossings of the Laplacian
NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 11/25

Canny method • 1. Apply a Gaussian filter to the
image • Low-pass filter to remove noise • 2. Calculate the gradient intensity in the image • Sobel filter in X and Y directions • Calculate the magnitude |G|= |Gx |+|Gy | • 3. Calculate the gradient directions in the image • Gradient direction θ = arctan(Gy /Gx ) • Round the directions to the nearest multiple of 45 degrees NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 12/25

Canny method • 4. Non-maximum suppression • If the gradient
magnitude at a pixel is less than the gradient magnitudes of its two neighbors along the gradient direction, then set the magnitude of that pixel to zero. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 12/25

Canny method • 5. Edge thresholding (hysteresis) • Uses two
thresholds: a high threshold (Sh) and a low threshold (Sb) • For each pixel • If ∥∇I(x, y)∥< Sb, then the pixel is set to zero (non-edge). • If ∥∇I(x, y)∥> Sh, then the pixel is an edge. • If Sb ≤ ∥∇I(x, y)∥≤ Sh, then the pixel is an edge if it is connected to another pixel already accepted as an edge. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 12/25

Canny method • Example NHSM - 4th year: Digital Image
Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 12/25

Edge Detection Chain • No operator is perfect for detecting
edges • In practice, we obtain incomplete edges • There are superfluous pixels • There are gaps • There are errors in edge pixel positioning and orientation • An edge detection operator is just the first step in the segmentation chain NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 13/25

Edge Detection Chain • There are global approaches for edge
detection • We do not only look for edge pixels • We seek the complete edge • We look for multiple pixels corresponding to an edge • How to define the edge? Problem? • Different techniques • Here: Hough Transform NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 13/25

Hough method • Global approach: lines representation • Plan x
− y : yi = axi + b • An infinite number of lines pass through (xi , yi ) and only one line for (a, b). • Parametric plan a − b: b = −xi a + yi • Only one line for (xi , yi ) and an infinite nuber of lines pass through (a, b) NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 14/25

Hough method • 1. Edge Detection (Gradient, Laplacian, ...) •
2. For each (xi , yi ) in the x-y plane (detected by the edge detector), ”draw” a line in the a-b parameter plane • 3. The intersections (cells) of lines in the a-b plane indicate the actual lines present in the x-y plane NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 14/25

Hough method • Problem: The slope approaches infinity for lines
that are close to vertical! • Solution: Represent using polar coordinates ρ = x cos θ + y sin θ NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 14/25

Hough method • Application NHSM - 4th year: Digital Image
Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 14/25

Application: document scanner Building a document scanner: • Step 1:
Detect edges. • Step 2: Use the edges in the image to find the contour (outline) representing the piece of paper being scanned. • Step 3: Apply a perspective transform to obtain the top-down view of the document. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 15/25

Segmentation NHSM - 4th year: Digital Image Processing - Segmentation

Thresholding • Thresholding is a simple and very popular method
for digital image processing • It is a classification method, not a region segmentation method • Pixel-based approach (not region or contour) • Thresholding can be • Global: a single threshold for the entire image • Local: a threshold for a portion of the image • Adaptive: a threshold that adjusts according to different parts of the image NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 17/25

Global Thresholding Basic Principle • Basic thresholding (2 classes): •
If value(pixel) ≥ threshold then value(pixel) = 1 • If value(pixel) < threshold then value(pixel) = 0 • The result of thresholding is a binary image • 0 or 1 (sometimes transformed to 0:255 for display) • Problem: Choosing the threshold! NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 18/25

Global Thresholding • Threshold Selection: Histogram Thresholding NHSM - 4th
year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 18/25

Global Thresholding • Threshold Selection: Simple Histogram Thresholding NHSM -

Global Thresholding • Threshold Selection: Histogram Thresholding • Advantages •
Universal, real-time, simplicity • Works well on multi-modal histograms • Disadvantages • Requires knowledge of the number of classes. • Appearance of false elements (no spatial component considered). • Number of modes often equals the expected number of classes. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 18/25

Global Thresholding • Threshold Selection: Histogram Thresholding • How to
find the right threshold (T)? • A value obtained through testing • The average value of gray levels • The median value between the maximum and minimum gray levels • A value that balances the two sections of the histogram • There are automatic algorithms to find the threshold: Averaging, Otsu, Kittler, ... NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 18/25

Global Thresholding • Averaging Method 1. Choose an initial threshold
S = S0 . 2. Divide the pixels into two classes using S. 3. Update the threshold S as the average of the two mean gray levels of the two classes S′ = m1+m2 2 . 4. If |S − S′|< δ, stop. S = S′. Otherwise, set S = S′ and return to step 2. Initial image (S0 = 76), Segm./S1 = 174.66+17.7 2 = 96.18, Final segmentation NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 18/25

Global Thresholding • Otsu’s Method NHSM - 4th year: Digital

Local or Adaptive Thresholding • Global thresholding: Problems NHSM -

Local or Adaptive Thresholding • Divide the image into sub-images
• Threshold each sub-image independently • The 4 corner sub-images are not processed because variance < 100 NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 19/25

Local or Adaptive Thresholding • Example NHSM - 4th year:

Region growing • ”Grow a region starting with a small
initial region (for example, a single pixel).” • ”Criterion for adding a pixel to a region:” • ”The difference between the intensity of the new pixel and the average intensity of the pixels in the region does not exceed a predefined threshold δ.” NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 20/25

Region growing • Example 1 NHSM - 4th year: Digital

Region growing • Exemple 1 NHSM - 4th year: Digital

Split-and-Merge • Splitting Step: Recursively divide any non-homogeneous block based
on a defined criterion, such as variance, max-min difference, etc. • Dividing a block results in 4 sub-blocks. • Attributes of each sub-block are recalculated. • Merging Step: Combine adjacent blocks representing homogeneous regions based on a defined criterion. • Example: Homogeneity criterion — A region is homogeneous if all its pixels have the same intensity. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 21/25

Split-and-Merge • Example: Split step NHSM - 4th year: Digital

Split-and-Merge • Example: merging step NHSM - 4th year: Digital

Split-and-Merge • Examples : Different thresholds NHSM - 4th year:

Split-and-Merge • Examples: Different thresholds NHSM - 4th year: Digital

Ligne de partage des eaux (Watershed) NHSM - 4th year:

Connected Component Labeling • Assign a unique number (label) to
each region. • Perform two passes over the image. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 23/25

Connected Component Labeling • First pass: from left to right
and top to bottom. • Assign to each pixel in a region the smallest label among its top and left neighbors, or a new label. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 23/25

Connected Component Labeling • Second pass: from right to left
and bottom to top. • Assign to each pixel in a region the smallest label among its own and its bottom and right neighbors’ labels. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 23/25

Connected Component Labeling • With two passes, we have completed
the labeling of regions. Sometimes more than two passes are needed. For example: spiral regions! • Continue the passes, in one direction then the other, until no more label changes occur. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 23/25

Classification • Classification = assigning a set of objects (pixels)
into different groups (classes) based on their characteristics or features (Gray level, color, gradient, local statistics, ...) • Types of classification • Supervised: The characteristics of the classes are known a priori. Examples: Minimum distance, k-nearest neighbors, statistical models (probability distributions of models), ... • Unsupervised (clustering): Classification is done based on the data and from the data directly. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 24/25

Classification • The k-Nearest Neighbors Classifier • Classes are defined
by a few reference elements. • For a new object (pixel), the k closest reference objects are determined. • Each new object is assigned to the class of the majority of its neighbors. • The Minimum-Distance-to-Mean Classifier • Classes are defined by their centers. • Each new object is assigned to the class of the nearest center. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 24/25

Classification: K-means • We want to divide the points into
k groups (clusters). • k is given in advance (a parameter of the algorithm). • The center of a group is defined as the average of the elements (pixels) in the group. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 24/25

Classification: K-means • 1. Partition the data into k non-empty
subsets. • 2. Calculate the centers of the current groups. • 3. Data points are assigned to the group whose center is closest. • 4. Return to step 2. • Stop when the groups are approximately constant. NHSM - 4th year: Digital Image Processing - Segmentation (Week 10-13) - M. Hachama ([email protected]) 24/25

Classification NHSM - 4th year: Digital Image Processing - Segmentation

Conclusion NHSM - 4th year: Digital Image Processing - Segmentation

Introduction to Image Processing: 5.Segmentation

Introduction to Image Processing: 5.Segmentation

More Decks by Mohammed Hachama

Other Decks in Education

Featured

Transcript