Image processing with scikit-image and Dash

Slide 1

Slide 1 text

Image processing with scikit-image and Dash Emmanuelle Gouillart plotly and scikit-image team @EGouillart

Slide 2

Slide 2 text

Image processing for science and business

Slide 3

Slide 3 text

What is scikit-image? An open-source (BSD) generic image processing library for the Python language (and NumPy data arrays)

Slide 4

Slide 4 text

What is scikit-image? An open-source (BSD) generic image processing library for the Python language (and NumPy data arrays) for 2D & 3D images scientiﬁc images rather than Instagram Filters simple API & gentle learning curve

Slide 5

Slide 5 text

Image processing for science Manipulations of images to Transform them for other purposes: color balance, enhancement, filtering, inpainting, ... Extract information from them: classification, measurements... a) flltering b) feature extraction c) segmentation d) measures non-local means denoising total variation denoising blob detection super-pixels skeleton & local diameter particle properties segmentation from markers ridge detection

Slide 6

Slide 6 text

What scikit-image is not No deep learning mostly because of architecture (GPU) constraints But a great tool for pre-processing: intensity and size normalization, data augmentation, ... post-processing: instance segmentation, measures...

Slide 7

Slide 7 text

Slide 8

Slide 8 text

The Scientiﬁc Python ecosystem 1 75 ... 32 ... 10 3 ... 2 8 23 ... 36 ... 1 13 ... 0 12 43 ... 6 ... 66 ... 97 1 8 5 ... 73 ... 78 98 ... 9 4 10 ... 55 ... 10 3 ... 9 1 41 ... 98 ... 38 75 ... 6 ... visualization Development environment machine learning image properties object properties image ﬁle(s) array

Slide 9

Slide 9 text

First steps with scikit-image >>> from skimage import io , filters , measure >>> im = io.imread(’image_file.png’) >>> im.shape # 2-D array (512 , 512) >>> threshold = filters. threshold_otsu (im) # float >>> im_binary = im < threshold # numpy mask >>> labels = measure.label(im_binary) >>> type(im), type(labels) # images are numpy arrays (, ) >>> labels [0, 0] # pixels are array elements 0 x Also works with 3-D images Most functions work for 2D & 3D Images are NumPy arrays Pixels are array elements Simple API relying on functions new = function(image) Optional parameters: keyword arguments

Slide 10

Slide 10 text

Manipulating images as numerical (numpy) arrays Pixels are arrays elements import numpy as np image = np.ones ((5, 5)) image [0, 0] = 0 image [2, :] = 0 x

Slide 11

Slide 11 text

Manipulating images as numerical (numpy) arrays Pixels are arrays elements import numpy as np image = np.ones ((5, 5)) image [0, 0] = 0 image [2, :] = 0 x >>> coffee.shape (400, 600, 3) >>> red channel = coffee[..., 0] >>> image 3d = np.ones((100, 100, 100))

Slide 12

Slide 12 text

API of scikit-image skimage ﬁlters restoration segmentation ... denoise_bilateral input array + optional parameters output (array) submodule module function variables

Slide 13

Slide 13 text

Consistent names

Slide 14

Slide 14 text

Consistent names filters . median ( image , selem=None , ← out=None , . . . ) morphology . binary_erosion ( image , ← selem=None , out=None ) restoration . denoise_bilateral (← image , win_size=None , . . . )

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Combining scikit-image and scikit-learn Feature extraction followed by classification Daisy features (skimage) Random Forest classifier (sklearn) Back to image processing: Gaussian filtering, thresholding and mathematical morphology (skimage).

Slide 17

Slide 17 text

Datasheet Package statistics http://scikit-image.org/ Release 0.15 (1 - 2 release per year) > 200 contributors, 5-10 maintainers 20000 unique visitors / month Among 1000 best ranked packages on PyPi

Slide 18

Slide 18 text

Documentation at a glance: galleries of examples Powered by sphinx-gallery

Slide 19

Slide 19 text

Getting started: ﬁnding documentation

Slide 20

Slide 20 text

Auto documenting your API with links to examples

Slide 21

Slide 21 text

Auto documenting your API with links to examples

Slide 22

Slide 22 text

Picture denoising

Slide 23

Slide 23 text

Picture denoising

Slide 24

Slide 24 text

Denoising tomography images In-situ imaging of phase separation in silicate melts Wanted: gallery examples using scientific datasets From basic (generic) to advanced (specific) filters

Slide 25

Slide 25 text

Denoising tomography images Histogram of pixel values From basic (generic) to advanced (speciﬁc) ﬁlters bilateral = restoration.denoise_bilateral(dat) bilateral = restoration.denoise_bilateral(dat, sigma_range← =2.5, sigma_spatial=2) tv = restoration.denoise_tv_chambolle(dat, weight=0.5)

Slide 26

Slide 26 text

Massive data processing and parallelization Competitive environment: some other tools use GPUs, Spark, etc. scikit-image uses NumPy! Roadmap: experiment with numba, Pythran Divide into blocks: use util.view as blocks to iterate conveniently over blocks Parallel processing: use joblib.Parallel or dask (da.map overlap) Better integration desirable (border eﬀects) experimental apply parallel function

Slide 27

Slide 27 text

Experimental chunking and parallelization

Slide 28

Slide 28 text

Keeping interaction easy for large data from joblib import Memory memory = Memory ( cachedir=’ . / c a c h e d i r ’ , verbose=0) @memory . cache def mem_label ( x ) : r e t u r n measure . label ( x ) @memory . cache def mem_threshold_otsu ( x ) : r e t u r n filters . threshold_otsu ( x ) [ . . . ] val = mem_threshold_otsu ( dat ) objects = dat > val median_dat = mem_median_filter ( dat , 3) val2 = mem_threshold_otsu ( median_dat [ objects ] ) liquid = median_dat > val2 segmentation_result = np . copy ( objects ) . astype ( np . uint8 ) segmentation_result [ liquid ] = 2 aggregates = mem_binary_fill_holes ( objects ) cores = mem_binary_erosion ( aggregates , np . ones ((10 , 10 , ← 10) ) )

Slide 29

Slide 29 text

More interaction for faster discovery: widgets

Slide 30

Slide 30 text

Interaction with images: annotations

Slide 31

Slide 31 text

More interaction for faster discovery: web applications made easy Package: http://github.com/plotly/dash-canvas pip install dash-canvas Gallery: https://dash-canvas.plotly.host/ Based on react-sketch, fabricJS and Dash.

Slide 32

Slide 32 text

https://dash-canvas.plotly.host/segmentation/

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Available components dash html components HTML elements html.H5 dash core components reactive components dcc.Slider, dcc.Dropdown Charts (plotly and others) Interactive DataTable Specialized libraries: daq, dash-bio Any ReactJS component wrapped w/ Dash

Slide 40

Slide 40 text

dash-canvas: drawing and transformation of annotations Modular tools for annotations and selections DashCanvas ( i d=’ canvas−c o l o r ’ , lineWidth=5, filename=” image . jpg ” , hide_buttons=[ ’ zoom ’ ,← ’ pan ’ ] , ) Functions to transform annotations (eg to Numpy arrays used by skimage) from dash_canvas import ← utils

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

Gallery of examples https://dash-canvas.plotly.host/

Slide 43

Slide 43 text

Roadmap of dash-canvas Richer interaction with images Annotations added by callbacks e.g. load annotation geometry Annotations triggering callbacks (click, release, hover... 3-D images and timeseries Richer annotations & visualizations More examples for the gallery itk-snap