技術と意味から捉えるARの活用方法(dotFes2018VIETNAM Da Nang)

本スライドはdotFes2018ベトナム ダナンでのセッションの日本語訳バージョンです。
https://www.dotfes.jp/2018vietnam/jp/schedule_detail.html#detail_s3

iOSのARKit、AndroidのARCoreなど、ARを簡単に実現できるフレームワークが増え、AR技術を用いた事例が増えてきました。
このセッションではARについて技術と意味の両側面から捉えなおすことで、ARの活用方法を考えていきます。

Links
・る https://r-u.co.jp/
・@024t910 https://twitter.com/024t910

・ARKit Examples https://arkit.r-u.co.jp/
・[Example 1]Suikawari https://arkit.r-u.co.jp/entertainment.html
・[Example 2]Runner https://arkit.r-u.co.jp/sport.html
・[Example 3]AR Cleaner https://arkit.r-u.co.jp/utility.html

・[Example 1 "Body-Measuring"] https://arkit.r-u.co.jp/measuring.html
・[Example 2 "Control Robot"] https://arkit.r-u.co.jp/robotics.html
・[Example 3 "Avatar-System"] https://r-u.co.jp/?page=anicon
・[Example 4 "Hologram"] https://www.youtube.com/watch?v=F6dbYRRzjio

・AR Quick Look Gallery https://developer.apple.com/jp/arkit/gallery/

Transcript

1. ٕज़ͱҙຯ͔Βଊ͑Δ
   "3ͷ׆༻ํ๏ Δגࣜձࣾ
   େ੢୓ਓ 
   !
   EPU'FT
   
   7*&5/".
   %B/BOH
2. ࣗݾ঺հ
3. Δגࣜձࣾ େ੢୓ਓʢΔ
   ୅දऔక໾ʣ
   $POUFOUT
   1MBOOFS
   
   5FDIOJDBM
   %JSFDUPS
   "QQMJDBUJPO
   8FC
   %JHJUBM
   *OTUBMMBUJPO 
 1SPEVDU
   1SPUPUZQJOH
   FUD
   5XJUUFS
   !U
   
4. WORKS Լه63-ͷฐࣾXFCαΠτΛɺ֬͝ೝ͍ͩ͘͞ɻ
 IUUQTSVDPKQ
   
5. "3ʹ͍ͭͯߟ͑Δ
6. ͦ΋ͦ΋"3ͬͯԿ͚ͩͬʁ Augmented Reality [֦ு͞Εͨݱ࣮]
7. "3ͷྺ࢙ 1965 1993 1994 1999 2007 2008 2009 2011 Ultimate Display
 (Ivan Sutherland) KARMA
 (Columbia Univ.) NaviCam
 ( J. Rekimoto and K. Nagao) FLARToolKit
 (@saqoosh) iPhone
 (Apple) ηΧΠΧϝϥ
 (Tonchidot) Vuforia
 (PTC) 2017 ARKit
 (Apple) ARCore
 (Google) 2014 Tango
 (Google) 2016 Hololens
 (Microsoft) 2018 Magic Leap One
 (Magic Leap) AR Quick Look WebXR
 (Mozilla) Kinect
 (Microsoft) iPhone X ARToolKit
 (Kato, H., Billinghurst, M.) NyARToolKit NyARToolKit
 (@nyatla) Information Overlay Marker Based AR AR for WEB AR Library SmartPhone Geolocation (GPS + commpass) Mobile AR
 Library IR Tracking AR Infrastructure Native AR Library Pokemon GO
 (Niantic)
8. ࣅͨ΋ͷͱͯ͠ VR MR XR Virtual Reality Mixed Reality Extended Reality
9. VR MR XR Virtual Reality Mixed Reality Extended Reality 
 σδλϧٕज़Λ༻͍
 ݱ࣮ʹ৘ใΛ෇Ճͯ͠
 Ξ΢τϓοτ͞Εͨ΋ͷ ͱͯ͠࿩ΛਐΊ·͢ ຊηογϣϯͰ͸ AR
10. "3ͷମݧઃܭ
11. "3ͷاըΛߟ͑Δ্ͰେࣄͳϙΠϯτ Ϣʔβʔͱݱ࣮ͱͷ
 Ґஔؔ܎Λಋ͘
 ϙδγϣϯτϥοΩϯάٕज़ ը໘্ʹCGΛग़ٕ͢ज़ ARͱ͸
12. ٕज़ͱॲཧ଎౓ͷਐาʹΑΓ
 ϢʔβʔͷҐஔతͳ੍ݶ͕औΓ෷ΘΕ͖ͯͨ 1965 1993 1994 1999 2007 2008 2009 2011 Ultimate Display
 (Ivan Sutherland) KARMA
 (Columbia Univ.) NaviCam
 ( J. Rekimoto and K. Nagao) FLARToolKit
 (@saqoosh) iPhone
 (Apple) ηΧΠΧϝϥ
 (Tonchidot) Vuforia
 (PTC) 2017 ARKit
 (Apple) ARCore
 (Google) 2014 Tango
 (Google) 2016 Hololens
 (Microsoft) 2018 Magic Leap One
 (Magic Leap) AR Quick Look WebXR
 (Mozilla) Kinect
 (Microsoft) iPhone X ARToolKit
 (Kato, H., Billinghurst, M.) NyARToolKit NyARToolKit
 (@nyatla) Information Overlay Marker Based AR AR for WEB AR Library Geolocation (GPS + commpass) Mobile AR
 Library IR Tracking AR Infrastructure Native AR Library Pokemon GO
 (Niantic) SmartPhone
13. Ґஔؔ܎ͷมԽ σΟεϓϨΠݻఆ
 ϢʔβʔͱϚʔΧʔ͕Ұॹʹಈ͘ ϚʔΧʔ͕ݻఆ
 ϢʔβʔͱσΟεϓϨΠ͕Ұॹʹಈ͘ ϚʔΧʔແ͠
 ϢʔβʔͱσΟεϓϨΠ͕Ұॹʹࣗ༝ʹಈ͘ ϚʔΧʔແ͠
 ϢʔβʔͱσΟεϓϨΠ͕ผʑʹࣗ༝ʹಈ͘
14. ਎ମੑΛ׆͔ͨ͠ମݧઃܭ σΟεϓϨΠͷ޲͜͏ଆͷग़དྷࣄ
    ͔Β
 ݱ࣮ʹى͍ͬͯ͜Δ͜ͱͷ֦ு
    ʹ͢ΔͨΊʹ ? σΟεϓϨΠͷ޲͜͏ଆͷग़དྷࣄ ݱ࣮ʹى͍ͬͯ͜Δ͜ͱͷ֦ு ϙδγϣϯτϥοΩϯάͷಛੑΛ
    ߟྀ͢Δ͜ͱ͕େࣄ AR Effect
15. Our Examples https://arkit.r-u.co.jp/
16. Example 1 https://arkit.r-u.co.jp/entertainment.html
17. Example 2 https://arkit.r-u.co.jp/sport.html
18. Example 3 https://arkit.r-u.co.jp/utility.html
19. ମݧઃܭͷϙΠϯτ ݱ࣮ͰͷϢʔβʔͷߦಈʹ
 ج͍ͮͨग़དྷࣄ
 
 ࣗ෼ͷ͍Δݱ࣮ͷԆ௕ ? ݱ࣮ʹى͍ͬͯ͜Δ͜ͱͷ֦ு 
 User Action 
 AR Effect Ϣʔβʔͷ਎ମੑ͋Γ͖Ͱ
 ϦΞΫγϣϯΛฦ͢
20. ମݧઃܭͷϙΠϯτ ͲΜͳମݧΛ͍͔ͤͨ͞ɹ͕ॏཁ ? 
 User Action 
 AR Effect ԿΛग़͔͢ɹͰ͸ͳͯ͘
21. ׂΔؾ࣋ͪྑ͞ΛੜΉ
    ͨΊʹ
    ΧϝϥʹखΛөͤ͞
 ࣮ࡍʹࢦΛ஄͘ Example 1
22. ૸ΓͷεϐʔυΛ࣮ײͤ͞Δ
    ͨΊʹ
    Ϣʔβʔ͕૸Γͳ͕Β
 ࢖͏ίϯςϯπʹ͢Δ Example 2
23. Example 3 ૟আػΛ͔͚ͨ৔ॴΛ஌Δ
    ͨΊʹ
    ࣮ࡍʹ૟আػΛ͔͚ͨҐஔΛ
 ϙδγϣϯτϥοΩϯά͢Δ
24. "3ͷ͜Ε͔Β
25. ݱঢ়ͷ"3ٕज़͕ಘҙͳ͜ͱ Λೝࣝ͢Δ Target Position Floor Wall Camera Position
26. ͦΕҎ֎ͷೝࣝ͸ࣗ෼Ͱ͠ͳ͍ͱ͍͚ͳ͍ ͦ͜Ͱ׆༻Ͱ͖Δͷ͕ ը૾ೝࣝ΍෺ཧηϯαʔ
27. "3ٕज़ΛΑΓൃలͤ͞ΔςΫϊϩδʔ ը૾ೝࣝ ʜ ػցֶश ʜ OpenCV dlib YOLO CoreML MLKIt OpenPose ෺ཧηϯα ʜ LiDAR VR Tracker ΦϯσόΠε Sensor Module ʜ FaceTracking
 GPS
 Compass Acceleration
 Gyro
 Microphone ʜ
28. Example 1 “Body Measuring” ARKit OpenCV + Floor https://arkit.r-u.co.jp/measuring.html
29. Example 2 “Control Robot” ARKit konashi + + Roomba Send Tracking Data Control Motor Through a Serial Port https://arkit.r-u.co.jp/robotics.html
30. Example 3 “Avatar System” ARKit FaceTracking Motion Sensor Controller + + "OJ$PO ˜
    
    8POEFSMFBHVF
    *OD 
 6OJUZ$IBO ˜
    6OJUZ
    5FDIOPMPHJFT
    +BQBO6$- https://r-u.co.jp/?page=anicon
31. Example 4 “Hologram” VIVE Tracker Micro Projector + "TTPDJBUF
    $PNQPOZ
    
    ,PLZP
    *OD https://www.youtube.com/watch?v=F6dbYRRzjio
32. ͞Βʹɺ͜Ε͔Βઌͷ"3 Display Handheld-Display
 (Smart Phone) Wearable
 (Glasses / Goggles) Retinal Projection
33. ΠϯλϥΫγϣϯίετ͕Լ͕Δ ͔͟͢ɹͷͧ͘
 ͱ͍ͬͨߦҝΛ͠ͳͯ͘΋
 ৘ใΛड͚Δ͜ͱ͕Ͱ͖ΔΑ͏ʹ
34. "3৘ใͷΦʔόʔϨΠͷมԽ ҙ͕ࣝͲ͜ʹ޲͍͍ͯΔ͔͕
 ॏཁͳϙδγϣϯτϥοΩϯάͷཁૉʹͳΔ ݻఆΧϝϥͷө૾ʹ ΧϝϥΛ޲͚ͨ΋ͷʹ ໨ઢΛ޲͚ͨ΋ͷʹ
35. ݟΔ͚ͩͰ৘ใ΍ମݧΛ૿෯ͤ͞Δੈք ݱ࣮ੈքͰϢʔβʔ͕ͲΜͳߦಈɾৼΔ෣͍Λ͍ͯ͠Δ͔
 τϥοΩϯά͢Δ͜ͱ͕ମݧσβΠϯͷ伴ʹ
36. "QQFOEJY
37. "3$PSF͸࣮͸J04ʹରԠɻ
    ڞ௨ػೳ͚ͩͰ͍͍ͳΒɺ
 "3$PSF 6OJUZΛ࢖ͬͯ։ൃ͢Δͷ͕͓͢͢Ίɻ ARKit ARCore or ?
38. ARKit ARCore Object Detection 
 FaceTracking Android Support Plane Detection(Horizontal & Vertical) Position Tracking Share World Tracking DataɾMulti User
 (ARWorldMap / Cloud Anchors) Image Tracking (Image Marker AR) Lighting Estimation
39. "3
    2VJDL
    -PPL
    (BMMFSZ https://developer.apple.com/jp/arkit/gallery/
40. Reference source • Page6.
 ɾFile:Augmented-reality.jpg - Wikimedia Commons https://commons.wikimedia.org/wiki/File:Augmented-reality.jpg
 Description:Монгол: Augmented Reality Технологийг IKEA брэнд үйл ажиллагаандаа ашиглаж байна
 Date:20 March 2016 Source:Own work Author:OyundariZorigtbaatar • Page7.
 ɾ֦ுݱ࣮ײٕज़ͷকདྷల๬(2012 ྐྵຊ७Ұ)
 ɾThe Sword of Damocles - Head Mount Display(1964, Ivan Sutherland)
 ɾKARMA(1993, Columbia University Computer Graphics and User Interfaces Lab Steven Feiner, Blair MacIntyre, Dorée Seligmann)
 https://graphics.cs.columbia.edu/projects/karma/karma.html
 ɾ"The World through the Computer:Computer Augmented Interaction with Real World Environments”,Proceedings of UIST’95(1995 J. Rekimoto and K. Nagaoʣ
 ɾʮ߈֪ػಈୂʯʮి೴ίΠϧʯͷੈքΛ࣮ݱʂ - ARToolKitΛ࢖֦ͬͨுݱ࣮ײϓϩάϥϛϯάʢ2007 ޻ֶφϏ ڮຊ ௚) http://kougaku-navi.net/ARToolKit/
 ɾ͍·ɺΈ͓͖͍ͯͨ΢ΣϒαΠτ ୈ2ճ(2009 gihyo.jp - Lançamento ) http://gihyo.jp/design/serial/01/website-pickup/0002
 ɾiPhone (Original)(2015 iMore) https://www.imore.com/iphone-2g
 ɾηΧΠΧϝϥऴྃͷ͓஌Βͤ……։ൃݩɾ಴ஐυοτʮ໨ࢦͨ͠ࢥ૝͸ఘΊ͍ͯͳ͍(2013 TechCrunch Japan) https://jp.techcrunch.com/2013/12/17/the_end_of_sekai/
 ɾXbox & Kinect - Stories https://news.microsoft.com/presskits/xbox/
 ɾGoogle expands Project Tango tablet sales to more countries(2015 ZDNet) https://www.zdnet.com/article/google-expands-project-tango-tablet-sales-to-more-countries/
 ɾMicrosoft Hololens https://www.microsoft.com/en-us/hololens
 ɾMagic Leap One: Creator Edition https://www.magicleap.com/magic-leap-one
 ɾWebXR Viewer on iOS App Store(Mozilla) https://itunes.apple.com/us/app/webxr-viewer/id1295998056?mt=8
 ɾAR Quick Look brings augmented reality to Safari in iOS 12 (2018 apple insider ) https://appleinsider.com/articles/18/07/13/ar-quick-look-brings-augmented-reality-to-safari-in-ios-12
 ɾʰPokémon GOʱͷ༡ͼํ(2017 Niantic) https://www.pokemongo.jp/howto/play/
 ɾiPhone X - Appleʢ೔ຊʣhttps://www.apple.com/jp/iphone-x/
41. Reference source • Page8.
 ɾͦ͜ʹ੢ӝ͸͋Δͷ͔͍ʢ2017 ෌Ӭܟͱ૿ా༤ଠͱେ੢୓ਓʣhttps://www.youtube.com/watch?v=d5w8pYwOBoU
 ɾMicrosoft Hololens https://www.microsoft.com/en-us/hololens
 ɾMinority Report made today's technology possible, says production designer Alex McDowell (2015 dezeen) https://www.dezeen.com/2015/11/06/minority-report-sci-fi-movie- steven-spielberg-future-of-technology-predictions-possible-production-designer-alex-mcdowell/ • Page11
 ɾFile:Augmented-reality.jpg - Wikimedia Commons https://commons.wikimedia.org/wiki/File:Augmented-reality.jpg
 Description:Монгол: Augmented Reality Технологийг IKEA брэнд үйл ажиллагаандаа ашиглаж байна
 Date:20 March 2016 Source:Own work Author:OyundariZorigtbaatar
 ɾAppleɺARKit 2Λൃද - Apple (೔ຊ) https://www.apple.com/jp/newsroom/2018/06/apple-unveils-arkit-2/ • Page25.
 ɾScanning and Detecting 3D Objects | Apple Developer Documentation https://developer.apple.com/documentation/arkit/scanning_and_detecting_3d_objects?language=objc
 ɾARKit By Example — Part 2: Plane Detection + Visualization https://blog.markdaws.net/arkit-by-example-part-2-plane-detection-visualization-10f05876d53
 ɾPart 1: ARKit Wall and Plane Detection for iOS 11.3 | Collective Idea https://collectiveidea.com/blog/archives/2018/04/30/part-1-arkit-wall-and-plane-detection-for-ios-11.3
42. Reference source • Page26.
 ɾVIVE Tracker https://www.vive.com/jp/vive-tracker/
 ɾCMU-Perceptual-Computing-Lab/openpose: OpenPose: Real-time multi-person keypoint detection library for body, face, and hands estimation https://github.com/CMU-Perceptual-Computing-Lab/ openpose
 ɾYOLO: Real-Time Object Detection https://pjreddie.com/darknet/yolo/
 ɾML Kit | Google Developers https://developers.google.com/ml-kit/
 ɾdlib C++ Library http://dlib.net/
 ɾOpenCV library https://opencv.org/
 ɾ௒Ի೾ڑ཭ηϯαʔɹ̝̘ʵ̨̧̌̐: ηϯαҰൠ ळ݄ిࢠ௨঎-ిࢠ෦඼ɾωοτ௨ൢɹhttp://akizukidenshi.com/catalog/g/gM-11009/
 ɾXbox & Kinect - Stories https://news.microsoft.com/presskits/xbox/
 ɾiPhone X - Appleʢ೔ຊʣhttps://www.apple.com/jp/iphone-x/
 ɾଌҬηϯα σʔλग़ྗλΠϓ/UST-10/20LX ঎඼ৄࡉ | ๺ཅిػגࣜձࣾhttps://www.hokuyo-aut.co.jp/search/single.php?serial=16 • Page29
 ɾ800γϦʔζʛϩϘοτ૟আػ ϧϯό | ΞΠϩϘοτެࣜαΠτ https://www.irobot-jp.com/product/800series/index.html
 ɾkonashi - A physical computing toolkit for iPhone, iPod touch and iPad http://konashi.ux-xu.com/ • Page30.
 ɾWonderleague corp. http://www.wonderleague.co.jp/ • Pge31.
 ɾVIVE Tracker https://www.vive.com/jp/vive-tracker/
 ɾPicoPro – Celluon Inc. https://www.celluon.com/picopro-3/