Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing AR Systems For Speed

Diana Hu
September 27, 2018

Designing AR Systems For Speed

Strangeloop 2018
https://www.thestrangeloop.com/2018/designing-augmented-reality-systems-for-speed.html

Building Augmented Reality (AR) systems is a software engineering art. It requires making choices among a set of trade-offs. On one hand, you have complex computer vision and machine learning algorithms that burn many CPU cycles; and on the other hand, you have the conflicting goal of getting the system to be as lean as possible since AR runs on resource-limited hardware such as wearables or mobile devices. This talk will explore how designing AR software is very much like the problem of designing distributed systems.

Like distributed systems, in AR: - We also need highly concurrent systems to take advantage of the many cores in cellphones. Concepts like message queues and "producers-consumers" become valuable tools to reason about computation abstractions on images and sensor data. - We also need highly available systems with an instant response due to the requirement of real-time interactions from the users with AR experiences. - We also need caching mechanisms to enable data locality for AR data retrieval.

Diana Hu

September 27, 2018
Tweet

More Decks by Diana Hu

Other Decks in Technology

Transcript

  1. Reasoning About the Future
    Source: https://en.wikipedia.org/wiki/Koomey%27s_law
    Koomey’s Law

    View Slide

  2. Reasoning About the Future
    Source: https://en.wikipedia.org/wiki/Koomey%27s_law
    Koomey’s Law
    In 1985, Feynman calculated an efficiency upper limit of Factor
    100B since then we've only managed to achieve a Factor 40k

    View Slide

  3. Reasoning About the Future
    Edholm’s Law
    Source: IEEE Spectrum: Edholm's Law of Bandwidth

    View Slide

  4. Reasoning About the Future
    Edholm’s Law
    Source: IEEE Spectrum: Edholm's Law of Bandwidth
    “we may someday see the end of wireline”

    View Slide

  5. How should we design
    AR Systems?
    Better Abstractions

    View Slide

  6. System Design Principles
    Distributed Systems
    ● Decrease processing time
    ○ Co-location
    ○ Caching
    ○ Remoting
    ● Sharding
    ● Scalability via concurrency
    ○ Pooling
    ○ Parallelization

    View Slide

  7. System Design Principles
    Distributed Systems
    ● Decrease processing time
    ○ Co-location
    ○ Caching
    ○ Remoting
    ● Sharding
    ● Scalability via concurrency
    ○ Pooling
    ○ Parallelization
    AR Systems
    ● Real time by default
    ○ Efficient networking
    ○ Response time
    ● Many devices
    ● Concurrency to take advantage of
    Many cores
    ○ Lock free programming
    ○ Actor models

    View Slide

  8. Speeding up Networking
    Life is in real-time

    View Slide

  9. View Slide

  10. Solution Space

    View Slide

  11. Solution Space

    View Slide

  12. Solution Space

    View Slide

  13. Our Ideal Solution

    View Slide

  14. Real-time P2P AR Technology
    100’s ms

    View Slide

  15. Real-time P2P AR Technology
    100’s ms 10’s ms

    View Slide

  16. P2P Networking

    View Slide

  17. Looks Familiar?

    View Slide

  18. Speeding up Computation
    CV is hard

    View Slide

  19. Traditional AR SLAM Pipeline (10,000ft view)

    View Slide

  20. It’ll be Hard to Run in Real-Time
    Source: https://doi.org/10.1016/j.robot.2017.10.019

    View Slide

  21. Can We do Better?

    View Slide

  22. Actor Models to the Rescue
    This will probably be its own talk in the future

    View Slide

  23. Actor Models to the Rescue
    This will probably be its own talk in the future

    View Slide