Upgrade to Pro — share decks privately, control downloads, hide ads and more …

#45 Computer Vision & Deep Learning applied to GPS signals

#45 Computer Vision & Deep Learning applied to GPS signals

Sujet: Application of Computer Vision Deep Learning techniques in domain of GPS signals

Speaker: Evgenii Munin, PhD Candidate at ENAC

Les signaux GPS sont souvent soumis aux différents types d'événements provoquants la degradation de positionnement. Dans beaucoup des cas, ce type d'erreur est dû à la présence des obstacles (dites de canion urban: bâtiment, voitures etc.). Ces obstacles par réflection génèrent des signaux multiples qui peuvent biaiser le positionnement du récepteur GPS.

Dans cette présentation, nous expliquerons une façon de détecter les erreurs liées aux trajets multiples en utilisant l'approche de Computer Vision. Nous allons décrire le detecteur qui intervient au niveau de récepteur GPS et qui travaille avec le signal brute. Puis nous parlerons de la question de la possibilité de faire tourner ce type de détecteur sur un module de basse consommation d'énergie en utilisant I'Intel Neural Compute Stick VPU.

Toulouse Data Science

November 25, 2020
Tweet

More Decks by Toulouse Data Science

Other Decks in Education

Transcript

  1. Application of Computer
    Vision DL techniques in
    domain of GPS signals
    Toulouse Data Science:
    Data Talk #45
    Evgenii Munin
    PhD Candidate @ ENAC

    View full-size slide

  2. ● PhD candidate at ENAC
    ○ Optim Lab. Work on data analysis in GNSS
    ● Experience
    ○ Anomaly detection in GNSS
    ○ Development of ML tools to analyze driver behaviour
    ○ Geo data analysis (car traffic use cases)
    ● Links
    ○ Linkedin: https://www.linkedin.com/in/evgenii-munin-01932a143/
    ○ Github: https://github.com/EvgeniiMunin
    About me: Evgenii Munin
    2018
    M2 Recherche
    opérationnelle
    2019
    DS Intern Data
    Scientist
    2020
    PhD
    Candidate

    View full-size slide

  3. ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo
    Outline

    View full-size slide

  4. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo

    View full-size slide

  5. Intro to GPS. What is it about?

    View full-size slide

  6. Intro to GPS
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589

    View full-size slide

  7. Intro to GPS
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589

    View full-size slide

  8. Intro to GPS
    Is it the only navigation service provider ?
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589

    View full-size slide

  9. What the most frequent kinds of errors can happen
    Atmospheric/
    Ionospheric
    errors
    Multipath
    effects
    Satellite clock
    errors
    Common errors in GPS
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589

    View full-size slide

  10. What the most frequent kinds of errors can happen
    Atmospheric/
    Ionospheric
    errors
    Multipath
    effects
    Satellite clock
    errors
    Common errors in GPS
    Link: https://medium.com/@penrosewang/introduction-to-gnss-some-basics-2dc8cb716589

    View full-size slide

  11. GPS Antenna
    GPS receiver pipeline

    View full-size slide

  12. Antenna Front-End
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    GPS receiver pipeline

    View full-size slide

  13. Antenna Front-End Signal Processing
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Frame
    Synchronization
    GPS receiver pipeline

    View full-size slide

  14. Antenna Front-End Signal Processing Navigation solution
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Measurement
    Nav Solution
    Frame
    Synchronization
    GPS receiver pipeline

    View full-size slide

  15. Antenna Front-End Signal Processing Navigation solution
    GPS Antenna
    Filter Amplifier
    A/D
    Converter
    Acquisition
    Tracking
    Measurement
    Nav Solution
    Frame
    Synchronization
    Antenna
    Hardware
    Close to
    client
    solution
    We want to operate with raw
    signal before the client side
    GPS receiver pipeline

    View full-size slide

  16. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo

    View full-size slide

  17. Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation

    View full-size slide

  18. Raw 1D signal
    - Pseudo Random Noise
    - Low Signal-to-Noise Ratio
    - Encoded Sat information
    - No estimation of Incoming
    Sat info
    Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation

    View full-size slide

  19. Raw 1D signal
    Processed 2D spectre. ACF
    - Pseudo Random Noise
    - Low Signal-to-Noise Ratio
    - Encoded Sat information
    - No estimation of Incoming
    Sat info
    - High Signal-to-Noise Ratio
    - Computed estimation of freq,
    delay for Sat
    - Still encoded Sat information
    Signal Processing
    Acquisition
    Tracking
    Frame
    Synchronization
    Signal representation

    View full-size slide

  20. Normal signal:
    - Frequency: Doppler
    - Time: Propagation delay
    - Phase
    - Carrier-to-Noise ratio
    (some analogue of SNR)
    Multipath bias measurement:
    - Frequency: Multipath Doppler
    - Time: Multipath Propagation delay
    - Multipath Phase
    - Amplitude coefficient
    Important signal parameters
    We need these parameters to obtain the frequency-time
    representations of signal on receiver output

    View full-size slide

  21. What processed signal looks like ?
    - Transform I/Q correlator outputs
    in 2-channel images;
    - Transform physical scale (Hz,
    ms) in pixels;
    - Define image resolution as
    number of pixels on each axis.
    Important signal parameters

    View full-size slide

  22. Maybe it is available open source?
    - The data is mostly “client oriented”:
    - Already estimated longitude, latitude,
    time
    - No direct Multipath reference
    Need specific raw receiver data
    - Acquire and label ourselves
    - Difficult to generate enough
    samples
    - Generate synthetically
    - Maybe less real life
    - Can generate and automatically
    label as much as I want
    Let’s generate !!!
    What about data ?

    View full-size slide

  23. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo

    View full-size slide

  24. - Training is carried out offline with synthetic data;
    - Trained model is copied and exported to NCS VPU dongle;
    - Model is run in real-time over real signals for assessment.
    - Available open source: https://github.com/EvgeniiMunin/gnss-multipath-detector
    Pipeline

    View full-size slide

  25. - Assess the performance of the detector and runtime using
    Intel Neural Compute Stick (NCS) dongle
    - Has zoo of pretrained models
    - Tensorflow, Caffe
    - Support custom models
    - Link to intel docs:
    https://software.intel.com/content/www/us/en/develop/articles/intel-movidius-neural-co
    mpute-stick.html
    Pipeline

    View full-size slide

  26. - Models are lightweight are chosen with < 10M parameters;
    - Fine-tuned models gathered into ensemble;
    - Weighted average used. Various weights are assigned to each model.
    Architecture
    BaseCNN DenseNet121 MobileNet
    W1 W2 W3
    Weights optimized with
    Nelder-Mead method
    Blending
    Prediction
    proba

    View full-size slide

  27. - Uniformly distributed C/N0 ratio in the
    interval [42, 48] dBHz;
    - Shift, scale, shear, rotate;
    - Test-time augmentation (TTA) on test
    data.
    Augmentations

    View full-size slide

  28. - Attention map represents the relative
    importance of image areas;
    - Introduction of attention maps into pipeline
    helps to reinforce points of interest in
    image.
    Trick to increase model performance

    View full-size slide

  29. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo

    View full-size slide

  30. - Show capability to detect multipath before next correlation cycle
    → Detection time constraint Ti = 20 ms;
    - Show low computational and power consumption w.r.t. GNSS
    receiver constraints
    → Simulation on Intel NCS VPU dongle;
    - Check performance of the detection pipeline and compare with
    a benchmark.
    Recall objectives

    View full-size slide

  31. - Objective: Demonstrate that multipath prediction model integrated into NCS VPU
    can operate in real-time:
    - Inference time is less than Ti = 20 ms (duration of navigation bit).
    - Frozen signal parameters: C/N0 = 40 dBHz, ∆θ MP = 0, α MP = 0.7.
    Mann-Whitney nonparametric test results on
    comparison of runtime distributions
    Results on runtime limits test

    View full-size slide

  32. Single models + ensembles
    Results on performance tests
    Single models + ensembles with attention mapping and TTA

    View full-size slide

  33. C/N0 = [40...45] dBHz
    Benchmark results: Alexis Louis, Mathieu Raimondi. Neural Network based Evil
    WaveForms Detection. Airbus Defence and Space. 2020.
    https://www.ion.org/publications/abstract.cfm?articleID=17651
    C/N0 = 40 dBHz Conclusion:
    - Results are comparable
    - Benchmark capable to detect EWF
    - Our model gives higher performance on
    Multipath detection
    Single models + ensembles with attention mapping and TTA
    Results on performance tests

    View full-size slide

  34. Outline
    ● GNSS Intro
    ● Signal representation
    ● Pipeline description
    ● Some results
    ● Options to deploy in prod
    ● Demo

    View full-size slide

  35. Detector
    Tensorflow
    frozen graph
    .h5 .pb
    .ir: NCS + Raspberry Pi
    GNSS receiver
    Receiver embedded solution
    How to deploy this detector ?

    View full-size slide

  36. Detector
    Tensorflow
    frozen graph
    .h5 .pb
    .ir: NCS + Raspberry Pi
    GNSS receiver
    Receiver embedded solution
    Cloud solution (Used for Demo)
    Detector
    .h5
    Flask App
    Deploy on GCP
    Kubernetes
    cluster
    How to deploy this detector ?

    View full-size slide

  37. Thank you for attention !!!
    Let’s Demo
    Links
    ○ Arxiv paper: https://arxiv.org/abs/1911.02347
    ○ Repo: https://github.com/EvgeniiMunin/gnss-multipath-detector
    ○ Linkedin:https://www.linkedin.com/in/evgenii-munin-01932a143/
    ○ Github: https://github.com/EvgeniiMunin
    ○ Mail: [email protected]

    View full-size slide