Time-Varying rPPG Signal Separation via Block-Sparse Signal Model

Transcript

1. Time-Varying rPPG Signal Separation via Block-Sparse Signal Model Kosuke Kurihara1, Yoshihiro Maeda2, Daisuke Sugimura3, Takayuki Hamamoto1 1Tokyo University of Science 2Shibaura Institute of Technology 3Tokyo Metropolitan University Background Proposed Method ⚫ PPG signal: blood volume changes in cardiac cycles • Useful for healthcare and emotion estimation Jointly optimize rPPG signal and separation vector unit-norm constraint ⚫ Contact PPG sensors impose a burden in daily use → Demand for non-contact measurement Point 2 Point 1 Smoothness across adjacent windows Point 1: Block-sparse model in time-frequency domain Applications ⚫ rPPG is quasi-periodic due to stable cardiac cycles → me freq en m nents min te r ss time in s ⚫ This characteristic can be modeled as block-sparse ⚫ Non-contact Remote PPG measurement (rPPG) • Blood volume changes cause subtle skin color changes • rPPG signal can be extracted from a facial patch Introduce weighted norm: block-wise sparsity Signal extraction Pulse Skin RGB video R G B RGB signal Cardiac cycle R P rPPG signal Q • Problem: rPPG is very weak (< 2 bits in 8-bit [1]) Easily degraded by illumination fluctuations R 1.8/255 G 1.4/255 B 1.9/255 0 RGB video (8-bit) U S Short-time Fourier transform (STFT) STFT Time → PPG signal -th frequency block of Spectrogram of PPG Point 2: Time-varying signal separation Ground truth PPG Pixel average T Weight of -th frequency block Block-sparse → Camera Vessel ⚫ Promote block-sparseness of rPPG by regularization term Freq. Light by grouping all time windows of each frequency bin into a block 5 10 ⚫ Divide RGB signal into short time windows by sliding operator ⚫ Build a signal separation model for each time window → En ble → time [s] time series signal tive se r ti n t fl t ting ill min ti n Input RGB signal Related Work Separation model for -th time window ⚫ Independent component analysis [2] • Assumes rPPG and noises are independent rPPG ⚫ Chrominance analysis via dichromatic model [3, 4] • Assumes rPPG lies in diffuse components RGB Separation Noise Concatenate all time windows ⚫ Autocorrelation analysis [5] • Exploits temporal periodicity of rPPG signal Separation model for all time windows Our method: Precise rPPG model + Time-varying separation ➔ Accurate rPPG signal extraction ➔ Extracted rPPG signal Experiments ⚫ Dataset: UBFC-RPPG [6] • Camera: Logitech C920, Ground truth: pulse oximeter ⚫ Evaluation setting • Single : evaluate on each patch individually • Multi : evaluate aggregated rPPG from all patches using principal component analysis ⚫ Evaluation metrics • SNR [dB] : extracted rPPG vs. Ground truth PPG • Heart rate estimation accuracy (using extracted rPPG) : mean absolute error (MAE) [bpm], success rate (SR) [%] Evaluation results Method Green [1] ICA [2] CHROM [3] POS [4] PVM [5] MTTS-CAN [7] PhysFormer [8] Ours References Single SNR(↑) MAE(↓) SR(↑) -4.09 27.1 15.0 -2.45 42.4 8.8 -6.56 38.2 8.7 1.66 59.6 3.3 -6.82 72.5 6.3 n/a n/a n/a n/a n/a n/a 2.67 16.2 27.3 Multiple SNR(↑) MAE(↓) SR(↑) -5.81 28.0 23.5 8.86 15.8 67.1 -6.29 35.4 13.2 7.77 15.8 54.4 -0.96 78.4 8.6 0.71 25.4 21.1 4.13 21.3 41.3 13.98 5.1 74.8 [1] W. Verkruysse et al., Opt. Express, 2008 [5] R. Macwan et al., IEEE/CVF CVPRW, 2018 n t n t n t Dataset detail Subjects 47 Videos 49 Resolution 640×480 Bit depth 8-bit Frame rate 30 fps Duration ≈ 1 min reen reen TT sF rmer rs rs r r n tr t 1 n tr t time [s] Example rPPG (single, subject #47) [2] M. Poh et al., IEEE TBE, 2011 [6] S. Bobbia et al., Pattern Recognit. Lett., 2019 [3] G. de Haan et al., IEEE TBE, 2013 [7] X. Liu et al., NeurIPS, 2020 1 time [s] Example rPPG (multi, subject #15) [4] W. Wang et al., IEEE TBE, 2017 [8] Z. Yu et al., IEEE/CVF CVPR, 2022