#45 Computer Vision & Deep Learning applied to GPS signals

Sujet: Application of Computer Vision Deep Learning techniques in domain of GPS signals

Speaker: Evgenii Munin, PhD Candidate at ENAC

Les signaux GPS sont souvent soumis aux différents types d'événements provoquants la degradation de positionnement. Dans beaucoup des cas, ce type d'erreur est dû à la présence des obstacles (dites de canion urban: bâtiment, voitures etc.). Ces obstacles par réflection génèrent des signaux multiples qui peuvent biaiser le positionnement du récepteur GPS.

Dans cette présentation, nous expliquerons une façon de détecter les erreurs liées aux trajets multiples en utilisant l'approche de Computer Vision. Nous allons décrire le detecteur qui intervient au niveau de récepteur GPS et qui travaille avec le signal brute. Puis nous parlerons de la question de la possibilité de faire tourner ce type de détecteur sur un module de basse consommation d'énergie en utilisant I'Intel Neural Compute Stick VPU.

Transcript

1. Application of Computer
   Vision DL techniques in
   domain of GPS signals
   Toulouse Data Science:
   Data Talk #45
   Evgenii Munin
   PhD Candidate @ ENAC
   

   View Slide

2. ● PhD candidate at ENAC
   ○ Optim Lab. Work on data analysis in GNSS
   ● Experience
   ○ Anomaly detection in GNSS
   ○ Development of ML tools to analyze driver behaviour
   ○ Geo data analysis (car traffic use cases)
   ● Links
   ○ Linkedin: https://www.linkedin.com/in/evgenii-munin-01932a143/
   ○ Github: https://github.com/EvgeniiMunin
   About me: Evgenii Munin
   2018
   M2 Recherche
   opérationnelle
   2019
   DS Intern Data
   Scientist
   2020
   PhD
   Candidate
   

   View Slide

3. ● GNSS Intro
   ● Signal representation
   ● Pipeline description
   ● Some results
   ● Options to deploy in prod
   ● Demo
   Outline
   

   View Slide

4. Outline
   ● GNSS Intro
   ● Signal representation
   ● Pipeline description
   ● Some results
   ● Options to deploy in prod
   ● Demo
   

   View Slide

5. Intro to GPS. What is it about?
   

   View Slide

6. Intro to GPS
   Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
   

   View Slide

7. Intro to GPS
   Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
   

   View Slide

8. Intro to GPS
   Is it the only navigation service provider ?
   Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
   

   View Slide

9. What the most frequent kinds of errors can happen
   Atmospheric/
   Ionospheric
   errors
   Multipath
   effects
   Satellite clock
   errors
   Common errors in GPS
   Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
   

   View Slide

10. What the most frequent kinds of errors can happen
    Atmospheric/
    Ionospheric
    errors
    Multipath
    effects
    Satellite clock
    errors
    Common errors in GPS
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589
    

    View Slide

11. GPS Antenna
    GPS receiver pipeline
    

    View Slide

12. Antenna Front-End
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    GPS receiver pipeline
    

    View Slide

13. Antenna Front-End Signal Processing
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Frame
    Synchronization
    GPS receiver pipeline
    

    View Slide

14. Antenna Front-End Signal Processing Navigation solution
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Measurement
    Nav Solution
    Frame
    Synchronization
    GPS receiver pipeline
    

    View Slide

15. Antenna Front-End Signal Processing Navigation solution
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Measurement
    Nav Solution
    Frame
    Synchronization
    Antenna
    Hardware
    Close to
    client
    solution
    We want to operate with raw
    signal before the client side
    GPS receiver pipeline
    

    View Slide

16. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo
    

    View Slide

17. Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation
    

    View Slide

18. Raw 1D signal
    - Pseudo Random Noise
    - Low Signal-to-Noise Ratio
    - Encoded Sat information
    - No estimation of Incoming
    Sat info
    Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation
    

    View Slide

19. Raw 1D signal
    Processed 2D spectre. ACF
    - Pseudo Random Noise
    - Low Signal-to-Noise Ratio
    - Encoded Sat information
    - No estimation of Incoming
    Sat info
    - High Signal-to-Noise Ratio
    - Computed estimation of freq,
    delay for Sat
    - Still encoded Sat information
    Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation
    

    View Slide

20. Normal signal:
    - Frequency: Doppler
    - Time: Propagation delay
    - Phase
    - Carrier-to-Noise ratio
    (some analogue of SNR)
    Multipath bias measurement:
    - Frequency: Multipath Doppler
    - Time: Multipath Propagation delay
    - Multipath Phase
    - Amplitude coefficient
    Important signal parameters
    We need these parameters to obtain the frequency-time
    representations of signal on receiver output
    

    View Slide

21. What processed signal looks like ?
    - Transform I/Q correlator outputs
    in 2-channel images;
    - Transform physical scale (Hz,
    ms) in pixels;
    - Define image resolution as
    number of pixels on each axis.
    Important signal parameters
    

    View Slide

22. Maybe it is available open source?
    - The data is mostly “client oriented”:
    - Already estimated longitude, latitude,
    time
    - No direct Multipath reference
    Need specific raw receiver data
    - Acquire and label ourselves
    - Difficult to generate enough
    samples
    - Generate synthetically
    - Maybe less real life
    - Can generate and automatically
    label as much as I want
    Let’s generate !!!
    What about data ?
    

    View Slide

23. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo
    

    View Slide

24. - Training is carried out offline with synthetic data;
    - Trained model is copied and exported to NCS VPU dongle;
    - Model is run in real-time over real signals for assessment.
    - Available open source: https://github.com/EvgeniiMunin/gnss-multipath-detector
    Pipeline
    

    View Slide

25. - Assess the performance of the detector and runtime using
    Intel Neural Compute Stick (NCS) dongle
    - Has zoo of pretrained models
    - Tensorflow, Caffe
    - Support custom models
    - Link to intel docs:
    https://software.intel.com/content/www/us/en/develop/articles/intel-movidius-neural-co
    mpute-stick.html
    Pipeline
    

    View Slide

26. - Models are lightweight are chosen with < 10M parameters;
    - Fine-tuned models gathered into ensemble;
    - Weighted average used. Various weights are assigned to each model.
    Architecture
    BaseCNN DenseNet121 MobileNet
    W1 W2 W3
    Weights optimized with
    Nelder-Mead method
    Blending
    Prediction
    proba
    

    View Slide

27. - Uniformly distributed C/N0 ratio in the
    interval [42, 48] dBHz;
    - Shift, scale, shear, rotate;
    - Test-time augmentation (TTA) on test
    data.
    Augmentations
    

    View Slide

28. - Attention map represents the relative
    importance of image areas;
    - Introduction of attention maps into pipeline
    helps to reinforce points of interest in
    image.
    Trick to increase model performance
    

    View Slide

29. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo
    

    View Slide

30. - Show capability to detect multipath before next correlation cycle
    → Detection time constraint Ti = 20 ms;
    - Show low computational and power consumption w.r.t. GNSS
    receiver constraints
    → Simulation on Intel NCS VPU dongle;
    - Check performance of the detection pipeline and compare with
    a benchmark.
    Recall objectives
    

    View Slide

31. - Objective: Demonstrate that multipath prediction model integrated into NCS VPU
    can operate in real-time:
    - Inference time is less than Ti = 20 ms (duration of navigation bit).
    - Frozen signal parameters: C/N0 = 40 dBHz, ∆θ MP = 0, α MP = 0.7.
    Mann-Whitney nonparametric test results on
    comparison of runtime distributions
    Results on runtime limits test
    

    View Slide

32. Single models + ensembles
    Results on performance tests
    Single models + ensembles with attention mapping and TTA
    

    View Slide

33. C/N0 = [40...45] dBHz
    Benchmark results: Alexis Louis, Mathieu Raimondi. Neural Network based Evil
    WaveForms Detection. Airbus Defence and Space. 2020.
    https://www.ion.org/publications/abstract.cfm?articleID=17651
    C/N0 = 40 dBHz Conclusion:
    - Results are comparable
    - Benchmark capable to detect EWF
    - Our model gives higher performance on
    Multipath detection
    Single models + ensembles with attention mapping and TTA
    Results on performance tests
    

    View Slide

34. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo
    

    View Slide

35. Detector
    Tensorflow
    frozen graph
    .h5 .pb
    .ir: NCS + Raspberry Pi
    GNSS receiver
    Receiver embedded solution
    How to deploy this detector ?
    

    View Slide

36. Detector
    Tensorflow
    frozen graph
    .h5 .pb
    .ir: NCS + Raspberry Pi
    GNSS receiver
    Receiver embedded solution
    Cloud solution (Used for Demo)
    Detector
    .h5
    Flask App
    Deploy on GCP
    Kubernetes
    cluster
    How to deploy this detector ?
    

    View Slide

37. Thank you for attention !!!
    Let’s Demo
    Links
    ○ Arxiv paper: https://arxiv.org/abs/1911.02347
    ○ Repo: https://github.com/EvgeniiMunin/gnss-multipath-detector
    ○ Linkedin:https://www.linkedin.com/in/evgenii-munin-01932a143/
    ○ Github: https://github.com/EvgeniiMunin
    ○ Mail: munin.evgenii@gmail.com
    

    View Slide