Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Designing AR Systems For Speed

05ee7b9a450069f210aac00cd5edd630?s=47 Diana Hu
September 27, 2018

Designing AR Systems For Speed

Strangeloop 2018
https://www.thestrangeloop.com/2018/designing-augmented-reality-systems-for-speed.html

Building Augmented Reality (AR) systems is a software engineering art. It requires making choices among a set of trade-offs. On one hand, you have complex computer vision and machine learning algorithms that burn many CPU cycles; and on the other hand, you have the conflicting goal of getting the system to be as lean as possible since AR runs on resource-limited hardware such as wearables or mobile devices. This talk will explore how designing AR software is very much like the problem of designing distributed systems.

Like distributed systems, in AR: - We also need highly concurrent systems to take advantage of the many cores in cellphones. Concepts like message queues and "producers-consumers" become valuable tools to reason about computation abstractions on images and sensor data. - We also need highly available systems with an instant response due to the requirement of real-time interactions from the users with AR experiences. - We also need caching mechanisms to enable data locality for AR data retrieval.

05ee7b9a450069f210aac00cd5edd630?s=128

Diana Hu

September 27, 2018
Tweet

Transcript

  1. Reasoning About the Future Source: https://en.wikipedia.org/wiki/Koomey%27s_law Koomey’s Law

  2. Reasoning About the Future Source: https://en.wikipedia.org/wiki/Koomey%27s_law Koomey’s Law In 1985,

    Feynman calculated an efficiency upper limit of Factor 100B since then we've only managed to achieve a Factor 40k
  3. Reasoning About the Future Edholm’s Law Source: IEEE Spectrum: Edholm's

    Law of Bandwidth
  4. Reasoning About the Future Edholm’s Law Source: IEEE Spectrum: Edholm's

    Law of Bandwidth “we may someday see the end of wireline”
  5. How should we design AR Systems? Better Abstractions

  6. System Design Principles Distributed Systems • Decrease processing time ◦

    Co-location ◦ Caching ◦ Remoting • Sharding • Scalability via concurrency ◦ Pooling ◦ Parallelization
  7. System Design Principles Distributed Systems • Decrease processing time ◦

    Co-location ◦ Caching ◦ Remoting • Sharding • Scalability via concurrency ◦ Pooling ◦ Parallelization AR Systems • Real time by default ◦ Efficient networking ◦ Response time • Many devices • Concurrency to take advantage of Many cores ◦ Lock free programming ◦ Actor models
  8. Speeding up Networking Life is in real-time

  9. None
  10. Solution Space

  11. Solution Space

  12. Solution Space

  13. Our Ideal Solution

  14. Real-time P2P AR Technology 100’s ms

  15. Real-time P2P AR Technology 100’s ms 10’s ms

  16. P2P Networking

  17. Looks Familiar?

  18. Speeding up Computation CV is hard

  19. Traditional AR SLAM Pipeline (10,000ft view)

  20. It’ll be Hard to Run in Real-Time Source: https://doi.org/10.1016/j.robot.2017.10.019

  21. Can We do Better?

  22. Actor Models to the Rescue This will probably be its

    own talk in the future
  23. Actor Models to the Rescue This will probably be its

    own talk in the future