A look at visionOS & Vision Pro / why it's January 9, 2007 again.

I was there, in the room, in San Francisco, at Moscone, January 2007, when Steve said “Are you getting it?”.

That was the beginning of my career as an iOS developer, with the SDK coming just 1 year later. But the love for the iPhone didn’t wait for the SDK.

Then I went through it all: the iPad, longer and bigger iPhones, the same with the iPad, Apple TV, Apple Watch etc … and every year came a new iPhone, a new iOS (and at the launch of iOS 7 I almost had to cry, where had my skeumorphism gone?!).

And then there was always the other side of me: Green politics and an attempted switch to full-time politics in 2024, which unfortunately failed in 2023.

The day after my political defeat there was WWDC and the presentation of Vision Pro and I was back. Motivated again because visionOS and Vision Pro are at least as revolutionary (actually more so) to me as the iPhone and iOS, or iPhoneOS. In this talk I summarise what some of you may already know, but everyone will definitely learn something.

I have been intensively involved with the new platform since the keynote. Even more intensively since the SDK came out. Even more intensively since I was at Apple in Munich. The revolution has already begun.

At best, not everyone may have noticed it yet. In this lecture we will take a closer look at the Vision Pro and its hardware as well as the different possibilities of the SDK, i.e. visionOS. We will look at how you can show your existing iPad and iPhone apps in Shared Space, as well as immersive Spaces and 3D objects.

Transcript

2. > Who is StuFF mc? [ !" , #$ , %& , '( , ) ا َل ْ ع َ ر َ ب ِ ي * ة ُ ] Yes!, Ja!, Oui !, ¡Si!, ن ع م ! As a kid/teen I had 2 passions: My C64 (later PC) & Hip Hop It's just Me, Myself and eyeOS ! I once started the Pomcast the first french-speaking podcast for the Mac community. ! Then came the iPhone, then Swift, then SwiftUI, then visionOS Always on the Edge! ! Ethically & politicaly involved for the # since 13 years #Vegan ! Tried and failed to run for & parliament — so I'm back. Fired up, ready to (re-)go! 2 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
3. I WROTE A BOOK ABOUT PRIVACY and I also give talks about Ethics 3 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
4. What are we gonna use the Vision Pro for? 4 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
5. 5 — Casey Neistat: The thing no one will say about Apple Vision Pro
6. VISION PRO ! Array of cameras and sensors " Spatial audio close to your ears # Two 4.5K micro-OLED displays $ (28!) Light Seal conforming to your face % Solo Knit Band with Fit Dial & ... and the horrible Dual Loop Band 6 — There are 28 Light Seal Sizes. Check out "Light Seal Sizes Decoded"
7. VISION PRO ! Battery: 2h of use / all-day (plugged) " Prescription glasses mounted inside # Digital Crown: $ + Immersion level % Top button: Spatial Videos/Photos #% Force Quit pressing both buttons 7 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
8. YouTube: MyGadgetsWorld: How to Shutdown and Force Close and optimize AVP
9. CAMERAS/SENSORS Outside ! 3D-mapping: 4 Sensors (TrueDepth) Lidar sensor (sender & receiver) ⬅⬇⬆➡ Head & Hand Tracking: 6 Cams & 2 HR Cams > 1 billion pixels/sec IR Illumination Sensors 9 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
10. CAMERAS/DISPLAYS Inside ! Eye tracking: LEDs & 4 IR cameras ! Invisible light patterns to eyes ! 23 Millions Pixels ! An iMac on each Eye! ! Total 12 Cameras on Vision Pro! 10 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
11. DISPLAY Outside ! EyeSight reveals your eyes 1 ! Tell if you’re using apps ! ... or fully immersed in an experience ! This image is the marketing " 1 So yeah, this is no passthrough and not really your eyes. Super cringe. 11 — @carrascomolina.com: Having Visions with eyeOS. ! Image: Apple Vision Pro
12. YouTube: Apple Vision Pro Hands-On Review
13. PROCESSORS M2 Chip R1 Chip Runs visionOS Process input 12 cams, 4 sensors, 6 mics Advanced Computer vision Stream images to displays < 12ms Delivers graphics ? + R2 13 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
14. Screw, Screw, Screw 14 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
15. iFixit • Vision Pro Teardown: Behind the Complex and Creepy Tech
16. SOFTWARE COMPATIBILITY ! Most iOS Apps will run fine... * ! Rotation with tapping a button ! No API deprecated prior to iOS 14 ! No UIDeviceOrientation " / UIScreen ! UITabBar have no leading or trailing ** ! No Pencil! Use #if !os(visionOS) ** Tab bars have a different design and do not lay out leading-to-trailing. No Tagbanwa for you! * Prefers iPad at Landscape, if only iPhone then Portrait 16 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
17. TOOLS ! Xcode 15.4/16 Beta — No Intel, worst case use 15.0 Beta 8 ! visionOS Simulator (with AR Anchors 3 and FaceTime Simulator) ! Reality Composer (iOS)/(Pro on Mac, with Debugger) ! Reality Converter beta (converts .obj, .gltf, .usd to USDZ) or Blender! ! Parallax Previewer 3 GitHub/tracyhenry: Sample Code (not from ) on how to stick an entity to a Wall or Table • For more control, use Data Providers 17 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
18. DEBUGGING ! Simulator has some possibilities! ! ⚠ Don't put things fix in front of you! ! SideCar projects Xcode to Vision Pro ! Occlusion Mesh, Surfaces, ... ! NOW RealityKit Debugger since visionOS 2! 18 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
19. 19 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
20. The Icon ! 3 sRGB 1024x1024 Images ! Background (Color? 6) ! Middle + Front ! Use SF Symbols's Copy Image as... 6 FB13078913: Allow providing a Color and/or Gradient... 20 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
21. Navigation ! By using your (2?) eyes, hands, and voice. ! Gestures: Tap (finger tips), pinch, rotate, ... ! Keyboard: Virtual or Bluetooth ! Mouse or Trackpad 21 — Video: Apple: Introducing Apple Vision Pro
22. Navigation ! Game Controller ! Pencil? Use your finger! 9 9 See RealityKitDrawingApp.xcodeproj 22 — Video: Apple: Introducing Apple Vision Pro
23. Look at me, I'm hovering! "When you look it hovers" 4 Button("Hover") { } // Auto HStack { ... } .padding() .contentShape(.hoverEffect, .rect(cornerRadius: 100) ) .hoverEffect() 4 WWDC23: Elevate your windowed app for spatial Computing 23 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
24. BETTER HOVERS ! visionOS 2 brings changes struct ProfileButtonStyle: ButtonStyle { func makeBody(configuration: Configuration) -> some View { configuration.label .hoverEffect(.highlight) .clipShape(.capsule) .hoverEffect { effect, isActive, _ in effect.scaleEffect(isActive ? 1.5 : 1) } .padding() } } 24 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
25. Raise ur glass! ! visionOS's colorScheme is Dark ! ! Compatible Apps use Light Mode! ! Adopt Vibrancy / Vibrant Materials. ! Use SwiftUI Semantic Styles. ! PlantIcon(plant) .background (.fill.tertiary, in: .circle) 25 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
26. Sheets / Alerts ! Sheets will always be presented modally ! Like macOS ! e.g. PhotoPicker as well ! Will push the content back (Z-axis) ! Alert/Sheet takes the original Z-Axis 26 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
27. Swap Windows (2.0) struct MyApp: App { var body: some Scene { WindowGroup { ContentView() } WindowGroup(id: "window") { PushView() } } } struct ContentView: View { @Environment(\.pushWindow) var pushWindow var body: some View { Button("Push") { pushWindow(id: "window") } } } struct PushView: View { @Environment(\.dismissWindow) var dismissWindow var body: some View { Button("Dismiss") { dismissWindow() } } } 27 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
28. Windows? 3D! Normal Windows (UIKit5 or SwiftUI) work Some system stuff will get 3D effects For example ornaments / TabView Volumes (3D Cubes, basically) Now resizable in visionOS 2! They've always been movable... 5 Even with Objective-C, but Storyboards and alike aren't highly supported. 28 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
29. World.xcodeproj ! Amazing Hello World Sample App from  ! placementGestures8 ! EarthEntity.swift > func update 8 PlacementGesturesModifier.swift > PlacementGesturesModifier 29 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
30. LOAD A 3D OBJECT IN SWIFTUI Model3D(named: "Earth") { model in model.resizable() } placeholder: { ProgressView() } Model3D(named: "Earth") { phase in switch phase { case .empty: Text("Waiting" ) case .failure(let error): Text("Error \(error)") case .success(let model): model.resizable() } } 30 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
31. No Windows, No Volume, Immersive Space! 31 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
32. struct VisionApp: App { @Environment(\.openImmersiveSpace) var openImmersiveSpace var body: some Scene { WindowGroup { Button("Open Immersive Space") { Task { await openImmersiveSpace(id: "immersiveSpace") } } } ImmersiveSpace(id: "immersiveSpace") { ContentView() .placementGestures(initialPosition: Point3D([475, -1200.0, -1200.0])) } } } 32 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
33. DISMISS @Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace dismissImmersiveSpace() // Only one can be shown at a time ⛔ Unable to present another Immersive Space when one already requested 33 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
34. IMMERSION STYLE Not in the Shared Space? 3 Modes Mixed Progressive Full7 7 Destination Video Sample from , Updated for visionOS 2 34 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
35. AN IMMERSIVE APP import RealityKitContent // Open Package in Reality Composer Pro @main struct WorldApp: App { var body: some Scene { ImmersiveSpace { RealityView { content in content.add( try await Entity(named: "Earth", in: realityKitContent) ) } } } } 35 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
36. CHANGE IMMERSION @main struct WorldApp: App { @State private var style: ImmersionStyle = .mixed var body: some Scene { ImmersiveSpace(id: "solar") { SolarSystem(immersionStyle: $style) } .immersionStyle(selection: $style, in: .mixed, .progressive, .full) } } 36 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
37. CHANGE IMMERSION SolarSystem(immersionStyle: $currentStyle) .simultaneousGesture(EntityGesture(MagnifyGesture()) .onChanged { value in var scale = value.gestureValue.magnification style = if scale > 5 { .progressive } else if scale > 10 { .full } else { .mixed } } 37 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
38. ATTACHMENTS RealityView { content, attachments in // ... make entity and add it to content if let text = attachments.entity(for: "text") { entity.addChild(text) } } attachments: { Attachment(id: "text") { Text("SWIFTUI!") } } ! Please provide a more convenient way to attach SwiftUI Views 38 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
39. HappyBeam.xcodeproj ! Game interface in SwiftUI ! " -gesture detection with ARKit ! Supports several kinds of input ! Hands: In front # or on your laps (tap) ! Game Controller 39 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
40. Use macOS inside Vision Pro ! Look at your Mac screen, it will turn private! ! 2.0: ! Larger screens ! You can be in an immersive environment. ! Future? ! Each windows as a part of visionOS? 40 — @wattmaller1: You can use your Mac Virtual Display in an immersive environment!
41. Other Sample Apps Swift Splash Use RealityKit to create an interactive ride in visionOS. Diorama Design scenes for your visionOS app using Realtiy Composer Pro. 41 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
42. 2024 Sample Apps RealityKit Drawing App Draw with your finger thanks to low-level mesh Spaceship Immersive game w/ RK audio, simulation, & rendering features 42 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.
43. 2024 Sample Apps Object Tracking Track real-world objects w/ ref. objects trained with Create ML BOTAnist Leverages RK and RC Pro to let you build a custom robot botanist 43 — manu@carrascomolina.com • A look at visionOS & Vision Pro / why it's January 9, 2007 again.