#45 Computer Vision & Deep Learning applied to GPS signals

Sujet: Application of Computer Vision Deep Learning techniques in domain of GPS signals

Speaker: Evgenii Munin, PhD Candidate at ENAC

Les signaux GPS sont souvent soumis aux différents types d'événements provoquants la degradation de positionnement. Dans beaucoup des cas, ce type d'erreur est dû à la présence des obstacles (dites de canion urban: bâtiment, voitures etc.). Ces obstacles par réflection génèrent des signaux multiples qui peuvent biaiser le positionnement du récepteur GPS.

Dans cette présentation, nous expliquerons une façon de détecter les erreurs liées aux trajets multiples en utilisant l'approche de Computer Vision. Nous allons décrire le detecteur qui intervient au niveau de récepteur GPS et qui travaille avec le signal brute. Puis nous parlerons de la question de la possibilité de faire tourner ce type de détecteur sur un module de basse consommation d'énergie en utilisant I'Intel Neural Compute Stick VPU.

Transcript

1. Application of Computer Vision DL techniques in domain of GPS signals Toulouse Data Science: Data Talk #45 Evgenii Munin PhD Candidate @ ENAC
2. • PhD candidate at ENAC ◦ Optim Lab. Work on data analysis in GNSS • Experience ◦ Anomaly detection in GNSS ◦ Development of ML tools to analyze driver behaviour ◦ Geo data analysis (car traffic use cases) • Links ◦ Linkedin: https://www.linkedin.com/in/evgenii-munin-01932a143/ ◦ Github: https://github.com/EvgeniiMunin About me: Evgenii Munin 2018 M2 Recherche opérationnelle 2019 DS Intern Data Scientist 2020 PhD Candidate
3. • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo Outline
4. Outline • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo
5. Intro to GPS. What is it about?
6. Intro to GPS Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
7. Intro to GPS Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
8. Intro to GPS Is it the only navigation service provider ? Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
9. What the most frequent kinds of errors can happen Atmospheric/ Ionospheric errors Multipath effects Satellite clock errors Common errors in GPS Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
10. What the most frequent kinds of errors can happen Atmospheric/ Ionospheric errors Multipath effects Satellite clock errors Common errors in GPS Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
11. GPS Antenna GPS receiver pipeline
12. Antenna Front-End GPS Antenna Filter Amplifier A/D Converter GPS receiver pipeline
13. Antenna Front-End Signal Processing GPS Antenna Filter Amplifier A/D Converter Acquisition Tracking Frame Synchronization GPS receiver pipeline
14. Antenna Front-End Signal Processing Navigation solution GPS Antenna Filter Amplifier A/D Converter Acquisition Tracking Measurement Nav Solution Frame Synchronization GPS receiver pipeline
15. Antenna Front-End Signal Processing Navigation solution GPS Antenna Filter Amplifier A/D Converter Acquisition Tracking Measurement Nav Solution Frame Synchronization Antenna Hardware Close to client solution We want to operate with raw signal before the client side GPS receiver pipeline
16. Outline • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo
17. Signal Processing Acquisition Tracking Frame Synchronization Signal representation
18. Raw 1D signal - Pseudo Random Noise - Low Signal-to-Noise Ratio - Encoded Sat information - No estimation of Incoming Sat info Signal Processing Acquisition Tracking Frame Synchronization Signal representation
19. Raw 1D signal Processed 2D spectre. ACF - Pseudo Random Noise - Low Signal-to-Noise Ratio - Encoded Sat information - No estimation of Incoming Sat info - High Signal-to-Noise Ratio - Computed estimation of freq, delay for Sat - Still encoded Sat information Signal Processing Acquisition Tracking Frame Synchronization Signal representation
20. Normal signal: - Frequency: Doppler - Time: Propagation delay - Phase - Carrier-to-Noise ratio (some analogue of SNR) Multipath bias measurement: - Frequency: Multipath Doppler - Time: Multipath Propagation delay - Multipath Phase - Amplitude coefficient Important signal parameters We need these parameters to obtain the frequency-time representations of signal on receiver output
21. What processed signal looks like ? - Transform I/Q correlator outputs in 2-channel images; - Transform physical scale (Hz, ms) in pixels; - Define image resolution as number of pixels on each axis. Important signal parameters
22. Maybe it is available open source? - The data is mostly “client oriented”: - Already estimated longitude, latitude, time - No direct Multipath reference Need specific raw receiver data - Acquire and label ourselves - Difficult to generate enough samples - Generate synthetically - Maybe less real life - Can generate and automatically label as much as I want Let’s generate !!! What about data ?
23. Outline • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo
24. - Training is carried out offline with synthetic data; - Trained model is copied and exported to NCS VPU dongle; - Model is run in real-time over real signals for assessment. - Available open source: https://github.com/EvgeniiMunin/gnss-multipath-detector Pipeline
25. - Assess the performance of the detector and runtime using Intel Neural Compute Stick (NCS) dongle - Has zoo of pretrained models - Tensorflow, Caffe - Support custom models - Link to intel docs: https://software.intel.com/content/www/us/en/develop/articles/intel-movidius-neural-co mpute-stick.html Pipeline
26. - Models are lightweight are chosen with < 10M parameters; - Fine-tuned models gathered into ensemble; - Weighted average used. Various weights are assigned to each model. Architecture BaseCNN DenseNet121 MobileNet W1 W2 W3 Weights optimized with Nelder-Mead method Blending Prediction proba
27. - Uniformly distributed C/N0 ratio in the interval [42, 48] dBHz; - Shift, scale, shear, rotate; - Test-time augmentation (TTA) on test data. Augmentations
28. - Attention map represents the relative importance of image areas; - Introduction of attention maps into pipeline helps to reinforce points of interest in image. Trick to increase model performance
29. Outline • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo
30. - Show capability to detect multipath before next correlation cycle → Detection time constraint Ti = 20 ms; - Show low computational and power consumption w.r.t. GNSS receiver constraints → Simulation on Intel NCS VPU dongle; - Check performance of the detection pipeline and compare with a benchmark. Recall objectives
31. - Objective: Demonstrate that multipath prediction model integrated into NCS VPU can operate in real-time: - Inference time is less than Ti = 20 ms (duration of navigation bit). - Frozen signal parameters: C/N0 = 40 dBHz, ∆θ MP = 0, α MP = 0.7. Mann-Whitney nonparametric test results on comparison of runtime distributions Results on runtime limits test
32. Single models + ensembles Results on performance tests Single models + ensembles with attention mapping and TTA
33. C/N0 = [40...45] dBHz Benchmark results: Alexis Louis, Mathieu Raimondi. Neural Network based Evil WaveForms Detection. Airbus Defence and Space. 2020. https://www.ion.org/publications/abstract.cfm?articleID=17651 C/N0 = 40 dBHz Conclusion: - Results are comparable - Benchmark capable to detect EWF - Our model gives higher performance on Multipath detection Single models + ensembles with attention mapping and TTA Results on performance tests
34. Outline • GNSS Intro • Signal representation • Pipeline description • Some results • Options to deploy in prod • Demo
35. Detector Tensorflow frozen graph .h5 .pb .ir: NCS + Raspberry Pi GNSS receiver Receiver embedded solution How to deploy this detector ?
36. Detector Tensorflow frozen graph .h5 .pb .ir: NCS + Raspberry Pi GNSS receiver Receiver embedded solution Cloud solution (Used for Demo) Detector .h5 Flask App Deploy on GCP Kubernetes cluster How to deploy this detector ?
37. Thank you for attention !!! Let’s Demo Links ◦ Arxiv paper: https://arxiv.org/abs/1911.02347 ◦ Repo: https://github.com/EvgeniiMunin/gnss-multipath-detector ◦ Linkedin:https://www.linkedin.com/in/evgenii-munin-01932a143/ ◦ Github: https://github.com/EvgeniiMunin ◦ Mail: munin.evgenii@gmail.com