Graph signal processing for the study of multivariate physiological signals Laurent Oudre [email protected] May, 26th 2023 Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 1 / 45

The Centre Borelli The Centre Borelli Fusion of two labs : Ÿ The Centre de math´ ematiques et de leurs applications (CMLA) : applied mathematics for the study of complex phenomena and data Ÿ The Cognition & Action Group (CognacG) : quantification and study of human and animal behavior Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 2 / 45

Main scientific questions How to quantify the human behavior Ÿ Adventure launched since 2012 : interdisciplinary collaboration between mathematicians, physicians, neuroscientists, engineers, biologists, etc... Ÿ Implementation of measurement chains “pipelines”, platforms and intelligent tools but also of procedures for analysis, measurement and processing of data Ÿ Creation of tools for diagnostic assistance, inter-individual comparison and longitudinal follow-up Ÿ Integration into a clinical environment and interaction between algorithms and medical/neuroscience experts Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 3 / 45

First usecase Study of EEG data Ÿ Study of EEG recorded during general anesthesia Ÿ 32 sensors at 256 Hz Ÿ How can we learn a structure and use it to process the signals ? Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 4 / 45

Second usecase Study of 3D upper-limb movements Ÿ Study of upper-limb movements with 3D markers Ÿ Around 30 sensors recording the 3D positions over time (100 Hz) Ÿ How can we take into account the skeleton structure for the study of these time series? Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 5 / 45

Graph Signal Processing Ÿ In most practical applications, the different dimensions of a multivariate signal xrtsare linked Ÿ Notion of correlation between recorded variables (ex: pressure/temperature/precipitation) Ÿ Sensor networks, body sensors, social networks...: spatial proximity, interactions... Ÿ These links can be explicitly be modeled through a graph structure: Graph Signal Processing [Ortega et al., 2018] Ÿ Each multivariate sample xrtsis assumed to be carried on the graph Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 6 / 45

Background Contents 1. Background 1.1 Concepts and definitions 1.2 The field of GSP 1.3 Graph Fourier Transform 1.4 Bandlimitedness and smoothness 1.5 Outline 2. Graph learning 3. Interpolation of missing samples 4. Conclusion Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 7 / 45

Background Concepts and definitions What is a graph ? A graph G is a triplet (V, E, W) Ÿ V is a finite set of D nodes or vertices (usually t1, 2, ..., Nu) Ÿ E € V ¢V is a set of edges Ÿ W : E Ý Ñ R is a map from the set of edges to scalar values Here : undirected graph, positive weights Wpi, jqencodes the strength of the relationship between dimensions i and j Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 8 / 45

Background Concepts and definitions Laplacian of a graph L D ¡W Ÿ W (weight or adjacency matrix) : Wi,j " Wpi, jq if pi, jq € E 0 Otherwise Ÿ D (degree matrix) : diagonal matrix with Di,i ¸ j Wi,j Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 9 / 45

Background Concepts and definitions Laplacian of a graph By construction of the Laplacian matrix Ÿ dx € RN , xT Lx 1 2 ¸ pi,j q€E Wi,j pxi ¡xj q2 ¥ 0 Ÿ The constant vector 1N is an eigenvector for matrix L associated to eigenvalue λ1 0 Obvious as the sum of the matrix along the rows/column is equal to zero Ÿ The number of connected components in the graph is equal to the number of eigenvalues equal to zero Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 10 / 45

Background The field of GSP What is a graph signal ? -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 Graph signal Given a graph G of N nodes, a graph signal is an array x € ΩN that associates an element of Ω to each node of G. Ÿ Ω R : Simple signal Ÿ Ω RT : Time signal Ÿ Ω Rd : Multivariate signal Ÿ Ω Rd ¢T : Multivariate temporal signal In most illustrations we will consider a single sample xrtsthat belongs to RN and will therefore define ONE graph signal Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 11 / 45

Background The field of GSP Example N nodes: one per signal dimension Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 12 / 45

Background The field of GSP Example 1 2 3 4 5 6 7 8 Dimension 1 lies on node 1, 2 on node 2, etc. Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 13 / 45

Background The field of GSP Example 1 2 3 4 5 6 7 8 Edges model links between dimensions Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 14 / 45

Background The field of GSP Example 1 2 3 4 5 6 7 8 1.1 0.2 2.2 1.1 2.1 1.7 2.0 1.2 1.1 2.7 0.4 1.3 1.1 2.0 Weights model the strengths of these links Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 15 / 45

Background The field of GSP Example 1.3 1.6 0.3 1.9 -0.6 2.1 1.8 -0.1 1.1 0.2 2.2 1.1 2.1 1.7 2.0 1.2 1.1 2.7 0.4 1.3 1.1 2.0 Visualization of one multivariate sample xrtson the graph Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 16 / 45

Background The field of GSP How to visualize graph signals? Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 17 / 45

Background The field of GSP How to visualize graph signals? Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 18 / 45

Background The field of GSP Examples Different kinds of signals : Ÿ Sensor networks (meteorology, population flux, energy consumption) Ÿ Interaction networks (social networks, communication networks, ...) Ÿ Economy based signals (market dependencies, stocks) Ÿ Image processing (intensity, color) Ÿ Cloud points (position, color) Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 19 / 45

Background The field of GSP Tasks Ÿ Several machine learning tasks can be extended to graph signals [Ortega et al., 2018]: Ÿ Sampling/compression: choose the most relevant nodes (i.e. dimensions) to reconstruct the whole data Ÿ Graph inference: learn the graph structure from data [Mateos et al., 2019] Ÿ Denoising/filtering: use the graph structure to remove noise, outliers... [Chen et al., 2014] Ÿ Interpolation: use the graph structure to reconstruct missing data [Narang et al., 2013] Ÿ Classification, event detection, anomaly detection, prediction... Ÿ Use the structure to improve performances on multivariate time series Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 20 / 45

Background Graph Fourier Transform Graph Fourier Transform Given a signal G with only one connex component, we compute the eigen-decomposition of its Laplacian L : L U λ1 ... λN UT , 0 λ1 λ2 ¤ ... ¤ λN Ÿ λi are interpretable as frequencies (see later for a more intuitive definition) λ1 0 : DC component Ÿ ui is the eigenvector associated to frequency λi Graph Fourier Transform The Graph Fourier Transform (GFT) ˆ x of a graph signal x € RN is defined as ˆ x UT x Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 21 / 45

Background Graph Fourier Transform Example ˆ x UT x -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 i 1 2 3 4 5 6 7 8 9 10 λi 0 0.5 1 1.5 2 2.5 3 Eigenvalues Eigenvalues λi can be interpreted as spatial frequencies Low frequencies : global phenomena, high frequencies : local phenomena Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 22 / 45

Background Graph Fourier Transform Example ˆ x UT x -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 Second eigenvector -0.4 -0.2 0 0.2 0.4 Third eigenvector -0.6 -0.4 -0.2 0 0.2 Eigenvectors u2 and u3 Can model symmetries, anti-symmetries, spatial phenomena Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 23 / 45

Background Graph Fourier Transform Example ˆ x UT x -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 λi 0 1 2 3 |ˆ si |2 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 Spectral representation Graph spectrum Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 24 / 45

Background Bandlimitedness and smoothness Bandlimitedness λi 0 1 2 3 |ˆ si |2 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 Spectral representation Bandlimitedness Ÿ Common assumption in signal processing : sparsity of the spectrum Ÿ In SP : bandlimitedness of signals (baseband, wideband...) is used for sampling, denoising Ÿ In GSP : same notion but on the graph spectrum x is K-bandlimited iff. }ˆ x} 0 K Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 25 / 45

Background Bandlimitedness and smoothness Example λi 0 1 2 3 |ˆ si |2 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 Spectral representation λi 0 1 2 3 |ˆ si |2 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 Spectral representation Filtering by removing all frequencies except for the 4 most dominant frequencies Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 26 / 45

Background Bandlimitedness and smoothness Example -0.2 -0.15 -0.1 -0.05 0 0.05 0.1 -0.25 -0.2 -0.15 -0.1 -0.05 0 0.05 4-bandlimited approximation Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 27 / 45

Background Bandlimitedness and smoothness Smoothness Ÿ Intuitively, signal values taken on adjacent nodes should be quite similar Ÿ Notion of smoothness for a graph signal x : Spxq xT Lx 1 2 ¸ pi,j q€E Wi,j pxi ¡xj q2 Ÿ Spxqis small if pxi ¡xj q2 is small for large Wi,j Ÿ Careful! This quantity in counterintuitive: large smoothness is achieved for non-smooth signals and vice-versa! Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 28 / 45

Background Bandlimitedness and smoothness Example Smoothness : 1.1623 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 Smoothness : 0.020407 -0.14 -0.12 -0.1 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 0.06 Smoothness decreases as the graph signal becomes more smooth Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 29 / 45

Background Bandlimitedness and smoothness Interpretation of the eigenvectors/eigenvalues Ÿ For the eigenvectors of the Laplacian ui , we have Spui q uT i Lui λi Ÿ New interpretation of the eigenvalues λi : smoothness of the associated eigenvector Ÿ For one connex component, λ1 0 and u1 1D Constant eigenvector: perfect smoothness! Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 30 / 45

Background Bandlimitedness and smoothness How to interpret the graph spectrum Parallels can be drawn between standard signal processing and graph signal processing: Ÿ Notion of smoothness and low-frequency approximation Useful for denoising, interpolation... Ÿ Notion of sparsity and bandlimitedness Useful for subsampling and reconstruction Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 31 / 45

Background Outline Outline 1. Graph learning from graph signals, based on the bandlimitedness and smoothness assumptions (with application to EEG/brain data) Ÿ P. Humbert, B. Le Bars, L. Oudre, A. Kalogeratos, and N. Vayatis. Learning Laplacian Matrix from Graph Signals with Sparse Spectral Representation. Journal of Machine Learning Research, 22(195):1-47, 2021. Ÿ P. Humbert, L. Oudre, and C. Dubost. Learning spatial filters from EEG signals with Graph Signal Processing methods. In Proceedings of the International Conference of the IEEE Engineering in Medecine and Biology Society (EMBC), Guadalajara, Mexico, 2021. Ÿ B. Le Bars, P. Humbert, L. Oudre, and A. Kalogeratos. Learning laplacian matrix from bandlimited graph signals. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 2937-2941, Brighton, UK, 2019. 2. Graph signal interpolation, based on “non-smooth” assumption (with application to 3D movement analysis) Ÿ A. Mazarguil, L. Oudre, and N. Vayatis. Non-smooth interpolation of graph signals. Signal Processing, 196:108480, 2022. Ÿ A. Mazarguil, L. Oudre, and N. Vayatis. Localized interpolation for graph signals. In Proceedings of the European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands, 2020. Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 32 / 45

Graph learning Contents 1. Background 2. Graph learning 2.1 Problem formulation 2.2 Results 3. Interpolation of missing samples 4. Conclusion Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 33 / 45

Graph learning Problem formulation Graph inference -0.5 0 0.5 1 1.5 -0.5 0 0.5 1 1.5 Ÿ Aim : Given a collection of n observed graph signals ty pk qun k 1 of size N, learn the graph G that best explains the structure observed in the signals Ÿ Assumption : The graph signals Y should be bandlimited and smooth for G Ÿ Inputs : Y ry p1 q , ¤¤¤ , y pn qs € RN ¢n : input graph signals Ÿ Outputs : L UΛUT : Laplacian matrix of G ˆ Y : GFT of signals Y on G Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 34 / 45

Graph learning Problem formulation Problem formulation min ˆ Y,U,Λ ∥Y ¡Uˆ Y∥2 F α∥Λ1 {2 ˆ Y∥2 F β∥ˆ Y∥S s.t. $ ' ' & ' ' % UT U IN , u1 1 c N 1N , pa q pUΛUT q k,l ¤ 0 k $ l, pb q Λ diag p0, λ2, . . . , λN q © 0, pc q tr pΛq N € R ¦. pd q Ÿ ∥Y ¡Uˆ Y∥2 F : Y should be close to the inverse GFT of its spectral representation in G Ÿ ∥Λ1 {2 ˆ Y∥2 F : Y should be smooth on G Ÿ ∥ˆ Y∥S : sparsity constraint on the frequency representation of Y Constraints : L UΛUT should be a Laplacian matrix (symmetric, semi positive) Resolution with alternate minimization Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 35 / 45

Graph learning Results Synthetic data Ÿ Two synthetic graphs : Random Geometric Graph (RGG) and Erdos-R´ enyi Graph (ER) Ÿ Noisy graph signals with n 1000, N 20 and 10-bandlimited Comparison of the adjacency matrices Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 36 / 45

Graph learning Results Real data: temperature data Ÿ Temperature in Brittany (32 weather stations, 747 graph signals) Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 37 / 45

Graph learning Results Real data: fMRI data Ÿ 40 subjects (20 healthy, 20 ADHD), N 39 regions of interest Ÿ We also used the graph to classify the subjects : 65% (52.5% with standard correlation graphs) Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 38 / 45

Interpolation of missing samples Contents 1. Background 2. Graph learning 3. Interpolation of missing samples 3.1 Problem formulation 3.2 Results 4. Conclusion Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 39 / 45

Interpolation of missing samples Problem formulation Interpolation of missing samples Ÿ Aim : Given a multivariate time series Y of n samples recorded on N sensors with missing values and a graph G, reconstruct the missing values Ÿ Assumption : The missing graph signal values can be reconstructed by using the neighborhood nodes Ÿ Inputs : Y € RN ¢n : input graph signals U and K : sets of unknown/known samples G : graph Ÿ Outputs : YU : missing samples imputation Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 40 / 45

Interpolation of missing samples Problem formulation Problem formulation min Y,A,b ∥Y ¡pAY b1T N q∥2 F µ LocpAq where Ÿ ∥Y ¡pAY b1T N q∥2 F : Y should follow a linear structural equation model Ÿ LocpAq ° i,j d2 G pi, jqa2 i,j is a localization term. This penalty ensures that the contributions for signal reconstruction on node i is mostly carried on a set of nodes that are close (according to the geodesic distance on the graph) to i Biconvex problem according to Y, pA, bq Resolution with alternate minimization Two resolution methods: one based on closed form solution (heavy) and one relaxed iterative method Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 41 / 45

Interpolation of missing samples Results Real data Ÿ Normalized Root Mean Square Error (in dB) for several datasets Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 42 / 45

Interpolation of missing samples Results Is the graph important ? Ÿ Comparison of several distances on synthetic data Ÿ Using the graph information is especially relevant when the percentage of missing data is large Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 43 / 45

Conclusion Contents 1. Background 2. Graph learning 3. Interpolation of missing samples 4. Conclusion Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 44 / 45

Conclusion Conclusion Ÿ Interpretable assumptions such as smoothness or bandlimitesness can be useful for several tasks (sampling, learning, interpolation...) Ÿ Graph Signal Processing allows to take into account the relationships and correlation observed in multivariate data Ÿ Versatile framework : graphs can model several types of interactions Ÿ Various applications in healthcare : sensor networks, EEG, multisensors... Laurent Oudre Graph signal processing for the study of multivariate physiological signals May, 26th 2023 45 / 45