Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evoking Magic Realism with Augmented Reality Technology

Evoking Magic Realism with Augmented Reality Technology

QCon.ai 2019 talk https://www.infoq.com/presentations/realism-augmented-reality

Exploring how building a real-world system is more of a software engineering art. It requires making choices among a set of tradeoffs.

05ee7b9a450069f210aac00cd5edd630?s=128

Diana Hu

April 17, 2019
Tweet

Transcript

  1. EVOKING MAGIC REALISM WITH AUGMENTED REALITY Diana Hu QCon.ai 2019

  2. Hello! • Director of Engineering & Head of AR Platform

    at Niantic • Co-Founder & CTO @Escher Reality, a YC ‘17 company acquired by Niantic • Building large scale ML & CV production systems since 2012 • Worked as data scientist @Intel Labs & OnCue TV @sdianahu
  3. None
  4. https://mastersreview.com/literary-terms-magical-realism-science-fiction-and-fantasy/

  5. A Magic Insight into Reality Matter-of-fact inclusion of fantastical elements

    into seemingly realistic fiction Magic Reality
  6. Augmenting Reality Matter-of-fact inclusion of fantastical elements into the physical

    world seemingly realistic fiction Digital Magic Physical Reality
  7. Principles For Magical Realism in AR Exploration Exercise Social Discover

    the world around us Most of the time, a simple walk will do Real-world, meaningful Interactions
  8. Exploration

  9. Exercise

  10. Shared & Social

  11. None
  12. MAKING THE DIGITAL BELIEVABLE (AKA How do we build this?)

  13. Niantic Real World Platform

  14. A Brief on AR Technology NEED TO UNDERSTAND THE WORLD

    NEED FOR VISUALS NEED TO CREATE
  15. Understanding The World

  16. AR Systems for Human Perception Rayner K, Smith TJ, Malcolm

    GL, Henderson JM. Eye Movements and Visual Encoding During Scene Perception. Psychological science. 2009;20(1):6-10. doi:10.1111/j.1467- 9280.2008.02243.x. The Miller Response- Time Test ~100ms => instantaneous. <~1 second => fast enough >10 seconds => lose user.
  17. NEED FOR SPEED AS A “MATTER- OF-FACT” Building Believable AR

  18. AR System Design Principles EFFICIENT NETWORKING CONCURRENT PROGRAMMING

  19. Speeding up Networking Life is in real-time

  20. None
  21. Solution Space

  22. Solution Space

  23. Solution Space

  24. Our Ideal Solution

  25. Real-time P2P AR Technology 100’s ms

  26. Real-time P2P AR Technology 100’s ms 10’s ms

  27. Speeding up Computation Computer Vision is hard

  28. Traditional AR SLAM Pipeline (10,000ft view)

  29. Inputs Images 1080x720 @ 30-60 Hz IMU 3x3 100s-1000s Hz

  30. Feature Extraction

  31. Building AR Maps

  32. Source:https://spectrum.ieee.org/automaton/robotics/home-robots/piaggio-cargo-robot

  33. It’ll be Hard to Run in Real-Time Source: https://doi.org/10.1016/j.robot.2017.10.019

  34. Can We do Better?

  35. Actor Models to the Rescue This will probably be its

    own talk in the future
  36. Actor Models to the Rescue This will probably be its

    own talk in the future
  37. Summary • Magical realism in AR • Making the digital

    believable • Building AR systems for Human perception – Need for Speed
  38. Credits Thanks @Pete Turner, and the rest of the AR

    Eng team
  39. THANKS! Any Questions? Find me @sdianahu We are hiring! https://www.nianticlabs.com/jobs/