Shigemi Ishida1, Masato Uchino1, Chengyu Li1, Shigeaki Tagashira2, and Akira Fukuda1 Kyushu University1, Kansai University2 ITSC 2019 Design of Acoustic Vehicle Detector with Steady-Noise Suppression 1

Background 2 [1] N. Buch et al., “Vehicle localisation and classification in urban CCTV streams”, ITS World Congress (2009) Motorbike detection problem nVehicle detection is one of the core technologies in ITS nExisting vehicle detectors • Intrusive: Loop coils, photoelectric tubes →Require roadwork • Non-intrusive : Laser, ultrasound →Installed above roads, high cost • CCTV-based [1] →Low accuracy in bad weather →Low-cost high accuracy vehicle detectors

L R Acoustic Vehicle Detector(1/2) nDetect vehicle sound[2] • Estimate direction of sound source using 2 mics and calculate sound delay on 2 mics n Sound map • A map of time difference of vehicle sound 3 L R Sound map [2],S. Ishida, J. Kajimura, M. Uchino et al., “SAVeD: Acoustic vehicle detector with speed estimation capable of sequential vehicle detection,” in Proc. IEEE Conf. Intelligent Transportation Systems (ITSC), Nov. 2018, pp. 906–912.

Acoustic Vehicle Detector(2/2) nCross-correlation function • Maximum at t=∆t for signals with delay of ∆t • Find peak of cross-correlation function • We used GCC (generalized cross- correlation function) nRANSAC • Robust estimation method for model fitting • Fit S-curves to the sound map to detect passing vehicles 4 Peak 0.8ms = $ ! " + s1(t), s2(t): sound on the mics

Issue nHow to detect vehicles in the steady noise condition? • Rain causes steady noise on the two microphones • The steady noise generates a noise peak and weaken a vehicle sound peak 5 −1.0 −0.5 0.0 0.5 1.0 Sound delay [ms] −0.04 −0.03 −0.02 −0.01 0.00 0.01 0.02 0.03 0.04 0.05 GCC -1 -0.5 0 0.5 1 320 325 330 335 340 345 350 355 360 Sound Delay ∆t [ms] Time t [s] Incorrect sound-delay estimation Noise points make difficult to detect

nMathematically exclude the influence of steady noise in a sound delay estimation • Formulate the noise influence • Store and update the noise data →We perform signal processing on the stored noise to move GCC noise peaks out of the sound map 6 Key Idea

Design Overview 1. Retrieve sound on 2 microphones 2. Estimate a vehicle passing in the probabilistic vehicle detector. 3. Update the noise data in Noise Storage when no vehicle is passing 4. Remove noise peaks by performing signal processing 5. Draw sound map and detect vehicles using RANSAC Fitting 7 Signal processing FFT Probabilistic Vehicle Detector LPF LPF FFT Noise Storage Sound Mapper Vehicle Detector Sound Retriever Noise Updater Thresholding

Steady-Noise Suppression nMathematical approach • Control GCC by performing signal processing and changing stored noise phase nLet V and N and " be vehicle sounds, noise and stored noise • ! ! = ! + ! + ! • ! " = " + " +#$"%&' " lWhen no vehicle is passing ( ≈ ! ) • GCC peak is dominated by stored noise phase lWhen a vehicle is passing (V ≫ ! ) • GCC peak corresponds to vehicle sound delay ∆ 8 -1 -0.5 0 0.5 1 320 325 330 335 340 345 350 355 360 Sound Delay ∆t [ms] Time t [s] -1 -0.5 0 0.5 1 320 325 330 335 340 345 350 355 360 Sound Delay ∆t [ms] Time t [s] ( > 1) signal processing

nUpdate the noise data in the noise storage • Rain-noise frequency components change over time →We control the noise by switching the noise update timing • We employ our ultra low-power vehicle detection method[3] Noise Updater 9 Sound Delay On Noise Update Switch Off Probability of Vehicle Passing Time Threshold Sound Delay robability of Vehicle Threshold Sound Delay Probability of Vehicle Threshold [3] K. Kubo, C. Li, S. Ishida et al., “Design of ultra low power vehicle detector utilizing discrete wavelet transform,” in Proc. ITS AP Forum, May 2018, pp. 1052–1063.

Experiment Environment nVehicle sound collected on camp ⦁ 2-lane road (1-lane in each dir) ⦁ 48kHz, 16bit, Mic separation D=30cm ⦁ AZDEN SGM-990 mic ⦁ Sony HDR=MV1 recorder ⦁ Record video as ground truth nNormal rain condition: • 93 vehicles passed • Recorded for approximately 23 minutes nHeavy rain condition: • 165 vehicles passed • Recorded for approximately 45 minutes 10

Normal Rain Condition Heavy Rain Condition w/ Noise Suppressor w/o Noise Suppressor w/ Noise Suppressor w/o Noise Suppressor L to R R to L Total L to R R to L Total L to R R to L Total L to R R to L Total Precision 1.00 0.98 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Recall 0.77 0.98 0.86 0.67 0.93 0.79 0.79 0.87 0.82 0.57 0.69 0.62 F-measure 0.87 0.98 0.92 0.80 0.96 0.88 0.88 0.93 0.90 0.73 0.82 0.76 Experiment Result 11

FP Detection Problem nFP was caused by sequentially passing vehicles • Detection algorithm based on a RANSAC fitting →Sequentially passing vehicles cause miss fitting nSteady-noise suppressor revealed sequential vehicle problem 12 Time t [second] Sound delay [second]

Summary nSteady Noise Suppressor • Control GCC by performing signal processing and changing stored noise phase • Store and update noise data • Detector with steady noise suppressor detected vehicles with F-measures of 0.92 and 0.90 in normal and heavy rain conditions, respectively. nSequential Vehicle Problem. • Need to improve detection algorithm for the sequential vehicle problem 13

14 Thank you!!

Detect vehicle sound Detect vehicle sound ◦ Estimate direction of sound source using stereo mics at sidewalk Advantages ◦Monitors multiple lanes at one side ◦Low deployment cost with low height configuration ◦Detect all types of vehicles 15

Acoustic Vehicle Detector(3/3) 16 t t O t t O t t O t t O nRANSAC • Robust estimation method for model fitting nProcess 1. Randomly sample a point 2. Estimate an S-curve based on the sampled point 3. Total distances 4. Repeat 1)~3) and completefitting with the minimum sum

Steady-Noise Suppression nRemove noise mathematically • Let V and N and ! be vehicle sounds, noise and stored noise • is noise supplement factor, is noise delay • The sound signal GCC - .! - ." / is: , , nWhen no vehicle is passing ( ≈ ! ) nWhen a vehicle is passing (V ≫ ! ) 17 ¯ S1(f) = V1(f) + N1(f) + ¯ N1(f) ¯ S2(f) = V2(f) + N2(f) + e j2⇡f⌧ ¯ N2(f) h RP |¯ n1 ||¯ n2 | (t) i MAX > ⇥ RP n1n2 (t) ⇤ MAX (1) ⇥ RP v1v2 (t) ⇤ MAX > h RP |¯ n1 ||¯ n2 | (t) i MAX (1) RP ¯ s1 ¯ s2 (t) = RP v1v2 (t) + RP n1n2 (t) + RP |¯ n1 ||¯ n2 | (t) RP v1v2 (t) = Z Gv1v2 (f) |G¯ s1 ¯ s2 (f)| ej2⇡ft dfRP n1n2 (t) = Z RP v1v2 (t) = Z Gv1v2 (f) |G¯ s1 ¯ s2 (f)| ej2⇡ft dfRP n1n2 (t) = Z Gn1n2 (f) |G¯ s1 ¯ s2 (f)| ej2⇡ft dfRP |¯ n1 ||¯ n2 | (t) RP v1v2 (t) = Z Gv1v2 (f) |G¯ s1 ¯ s2 (f)| ej2⇡ft dfRP n1n2 (t) = Z Gn1n2 (f) |G¯ s1 ¯ s2 (f)| ej2⇡ft dfRP |¯ n1 ||¯ n2 | (t) = Z G|¯ n1 ||¯ n2 | (f) |G¯ s1 ¯ s2 (f)| ej2⇡f(t ⌧) df ¯ S1(f) = V1(f) + N1(f) + ¯ N1(f) ¯ S2(f) = V2(f) + N2(f) + e j2⇡f⌧ ¯ N2(f) -1 -0.5 0 0.5 1 320 325 330 335 340 345 350 355 360 Sound Delay ∆t [ms] Time t [s] -1 -0.5 0 0.5 1 320 325 330 335 340 345 350 355 360 Sound Delay ∆t [ms] Time t [s] ( > 1)

Normal Rain Condition Heavy Rain Condition w/ Noise Suppressor w/o Noise Suppressor w/ Noise Suppressor w/o Noise Suppressor L to R R to L Total L to R R to L Total L to R R to L Total L to R R to L Total TP 39 41 80 34 39 73 74 62 136 54 49 103 FN 12 1 13 17 3 20 20 9 29 40 22 62 FP 0 1 1 0 0 0 0 0 0 0 0 0 Precision 1.00 0.98 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Recall 0.77 0.98 0.86 0.67 0.93 0.79 0.79 0.87 0.82 0.57 0.69 0.62 F-measure 0.87 0.98 0.92 0.80 0.96 0.88 0.88 0.93 0.90 0.73 0.82 0.76 Experiment Result 18

© 2019 Shigemi ISHIDA, distributed under CC BY-NC 4.0