The Reality Side Of AR

The excitement so far with ARKit has centered on making AR objects react to the real world. How about making AR objects part of that world? In this talk, Alex will explore the use of Core Location to create an AR experience anchored on real-world locations to enhance everything from retail and museum fixed navigation through dynamic crowd control applications.

Transcript

1. THE REALITY SIDE OF AR ARKIT + CORE ML + CORE LOCATION
2. ALEX CURYLO 2  DESIGN AWARDS 200+ SKUS @TROLLWERKS AGODA.COM, BKK
3. IN A MATTER OF HOURS, THE WORLD WILL BE BUZZING WITH TALK OF AUGMENTED REALITY … IT’S GOT ALL THE MAKINGS OF A PLATFORM SHIFT … APPLE IS ESSENTIALLY TURNING A LIGHT SWITCH ON. Alex Kantrowitz, BuzzFeed 9/12: “WHY AUGMENTED REALITY IS ABOUT TO TAKE OVER YOUR WORLD”
4. … EAGERLY DOWNLOADED IT, ANTICIPATING AR MAGIC, ONLY TO COME AWAY DEEP IN A PERSONAL "TROUGH OF DISILLUSIONMENT". Charlie Fink, Forbes 9/25: “APPLE’S IOS 11 AR NOT READY FOR PRIME TIME”
5. WHAT YOU GET WITH ARKIT UNDER THE HOOD TECHNOLOGY ▸ Visual Inertial Odometry: camera + CoreMotion = calibration-free movement detection ▸ Scene Understanding: detect horizontal planes (tables, floors) and track objects ▸ Lighting Estimation: Apply lighting to virtual objects as camera sensor detects on real objects ▸ iOS 11.0 + A9 processor (6S/SE)
6. WHAT YOU CAN DO WITH THAT THE ARTIFICIAL SIDE OF AR ▸ Interior Design IKEA, Lowe’s ▸ Gimmicks tossed in a few hours Ruler/selfie/drawing apps, most games ▸ Useless novelty Catalogs, the rest of the games “VR and AR will be the death of pop-up ads and pre-roll videos” — mmmm, no.
7. ARKIT LOOKS AMAZING. BUT IT'S GOING TO BE A PARTY TRICK UNTIL THE REALITY SIDE OF AR IS MORE THAN JUST A BACKDROP. @stevenbjohnson
8. THE SECRET TO SUCCESS THE REALITY SIDE OF AR ▸ Core ML Understand scene content ▸ Core Location Understand real world location ▸ ARCloud “A machine-readable 1:1 scale model of the real world. Our AR devices are the real-time interface to this parallel virtual world which is perfectly overlaid onto the physical world.” — Matt Miesnieks
9. LET’S GET STARTED ARKIT + CORE ML DEMO ▸
10. HTTP://WWW.MADEWITHARKIT.COM/IDEAS WHY ARKIT? ▸ Runs on all 6S and later devices ▸ 500+ million @ 50% iOS 11 ▸ SceneKit 3D views ▸ SpriteKit 2D views ▸ Metal for custom engines ▸ Unity support ▸ Unreal Engine support
11. WORLD TRACKING VISUAL-INERTIAL ODOMETRY (VIO) 1. Find matching points across multiple images 2. Compute essential matrices (homogenous normalized image coordinates y = y’ in 3D) 3. Decompose the essential matrices to obtain relative poses between image pairs 4. Performing "bundle adjustment" (BA) to tweak estimates http://www.nec-labs.com/~manu/pdf/icra13_monocular.pdf
12. SCENE UNDERSTANDING WHERE THE MAGIC HAPPENS ▸ Plane Detection ▸ Hit testing ▸ Light estimation Only horizontal plane detection is supported currently!
13. RENDERING TIME TO DO YOUR PART
14. WALKTHROUGH XCODE 9 PROJECT TEMPLATE
15. WALKTHROUGH SELECT YOUR RENDERING TECHNOLOGY
16. WALKTHROUGH CHECK IT OUT
17. WALKTHROUGH ARSCNVIEW
18. WALKTHROUGH SHIP.SCN
19. WALKTHROUGH TEXTURE.PNG
20. WALKTHROUGH VIEW CONTROLLER
21. WALKTHROUGH SESSION MANAGEMENT
22. API PART 1
23. API PART 2
24. API PART 3
25. API PART 4
26. HTTPS://GITHUB.COM/PROJECTDENT/ARKIT-CORELOCATION ARKIT AND CORE LOCATION
27. HTTPS://GITHUB.COM/EH3RRERA/ARKITANIMATION ARKIT AND MIXAMO MODEL ANIMATION
28. HTTPS://GITHUB.COM/NOVATECCONSULTING/FACERECOGNITION-IN-ARKIT ARKIT AND FACE RECOGNITION
29. RESOURCES ▸ https://developer.apple.com/arkit/ ▸ http://www.madewitharkit.com ▸ https://videos.raywenderlich.com/screencasts/948- introduction-to-arkit-getting-started ▸ https://github.com/markdaws/arkit-by-example ▸ http://metalkit.org/2017/07/29/using-arkit-with-metal.html ▸ https://medium.com/@azamsharp/arkit-for-ios-developers- book-now-available-78fc20b48d06 ▸ http://www.alexcurylo.com/2017/06/23/arkit-and-kaboodle/
30. СПАСИБО! HTTPS://CAREERSATAGODA.COM У НАС ЕСТЬ КРУТЫЕ ВАКАНСИИ В БАНГКОКЕ, ПРИЕЗЖАЙТЕ К НАМ!