Hack Reality - An Introduction To Making Apps for Apple Vision Pro by Matt Heaney

Talk by Matt Heaney

At WWDC 2023, Apple made a game changing announcement: they introduced the Vision Pro and visionOS, their first steps into spacial computing. Their new headset allows apps to break out of iPhone and iPad screens and blend seamlessly into spaces around us.

For users, this is incredibly exciting.

But as developers, this goes another level: we get to make apps to be used on the headset! Whether you’re an experienced developer for Apple products or brand new to the game, this is the perfect time to get involved.

The best part is: it’s easier than you may think to get and running with developing for visionOS. There’s a lot of new terms and tools we can use, but this talk has you covered.

In this talk, we’ll start at the very top, and we’ll look at:

What impact could the Vision Pro have on developing for Apple products?
Navigating the Vision Pro simulator in Xcode
The difference between apps that use Windows, Volumes and Spaces
How to get up and going building your first Vision Pro apps
My goal with this talk is not only to provide you with the information you need to get up and running, but also to inspire you to make your first projects for the Vision Pro

Transcript

1. Matt Heaney :) An Introduction To Making Apps for Apple Vision Pro Hack Reality
2. Who Am I? • I’m Matt Heaney • Based in Nottingham • Contract Senior iOS engineer • Passionate about giving back to the iOS community • This is my fi rst ever in-person talk at an event! • I love UK Ice Hockey 🏒 (#letsGoPanthers)
3. I firmly believe the Vision Pro is the future of tech
4. “I firmly believe the Vision Pro is the future of tech” - Me, 19th March 2024, Leeds Mobile
12. $3,500
14. These apps hack the reality around us.
15. Sounds great! I kinda want to make a Vision Pro app! What do I need to make apps for Vision Pro?
16. “Sounds great! I kinda want to make a Vision Pro app! What do I need to make apps for Vision Pro?” - You (maybe), 19th March 2024, Leeds Mobile
17. What do you need? • Mac - this needs to be a Mac with an Apple Silicon • Xcode 15.2 or above • You do not need a Vision Pro
18. Developing for Vision Pro is a great excuse if you want to buy the headset and need to justify spending £3,500
19. “Developing for Vision Pro is a great excuse if you want to buy the headset and need to justify spending £3,500” - Not me, not now, not here
20. The VisionOS simulator
21. Resizing Windows
22. Moving Windows
23. The VisionOS simulator
24. Moving Around The Environment Interact Tool Look Around Move Around Orbit Zoom/ Walk Reset
25. “Awesome, what do I need to know to start making Vision Pro apps?” - You (hopefully), 19th March 2024, Leeds Mobile
26. VisionOS App Core Concepts
27. Windows - Flat 2D views - The user can move and resize the windows - The user can have multiple windows open - Imagine notes, emails, search engines all around the room now - Don’t undersell how powerful these are - SwiftUI fi rst
28. Volumes - A cube space - Can be used to show 3D models - The user can have multiple volumes open - Imagine 3D models of the Vision Pro around the room - Makes use of RealityKit
29. Spaces - We can use full immersive spaces - Can change the entire environment around us - Imagine we’re changing this environment to be at the WWDC 2023 announcement - Makes use of RealityKit and ARKit
30. Apps Can Use A Mixture
31. “Ok, this is making sense! How can I start working with these windows then?” - You (hopefully), 19th March 2024, Leeds Mobile
32. Simply knowing SwiftUI will get you so far with making windows for Vision Pro.
34. The Vision Pro App Style - Glass effect - Blend into the environment - No bold, intense colours when you can avoid it - We get this glassy background by default
35. Glass Colours - Not using bold colours certainly doesn’t mean don’t use colours
36. Ornaments - As windows aren’t limited to the size of a device, we can have additional components outside of windows - These additional control components are called ornaments - This keeps focus on essential information - We have 3 main different variants of ornaments
37. Ornaments - Tab Bar
38. Ornaments - Tab Bar
39. Ornaments - Toolbar
40. Ornaments - Toolbar
41. Ornaments - Custom Ornaments
42. Ornaments - Custom Ornaments
43. Multiple Windows - The Vision Pro is perfect for apps with multiple windows - We can quickly and easily break information out to new views to improve that user experience
44. Multiple Windows
45. Multiple Windows
46. Multiple Windows
47. Starting Window Sizes
48. Starting Window Sizes
49. These tips put you in a great position to get up and running with 2D content for Vision Pro
50. “Great! But what about working with 3D objects?” - You (hopefully), 19th March 2024, Leeds Mobile
51. Volumes
52. Displaying 3D Models
53. Reality Composer Pro
54. USDZ Files - Universal Scene Description Zip file - 3D model file format - Made of different aspects of 3D models - such as the model, textures, and animations
55. USDZ Files
56. USDZ Files
57. Reality Composer Pro
58. Adding These Models to our View
59. Adding These Models - To A Window?
60. Adding These Models
61. Adding These Models
62. Recap Simulators, Concepts, Windows, Volumes, and Too Many Quotes - You know how to get setup, and how to use the simulator - You know what the 3 main concepts are (Windows, Volumes, Spaces) - You know how to use SwiftUI to make window views for your app - You know how to use 3D models in your app
63. You should now be in a great place to start making your own Vision Pro apps!
64. “Matt, that was a great introduction! How can I follow you on social media?” - You (hopefully), 19th March 2024, Leeds
65. Matt Heaney :) An Introduction To Making Apps for Apple Vision Pro Hack Reality