Rapid Cross-Platform AR Development with React Native

Augmented Reality's popularity is growing at an ever increasing rate, and it might seem as if native iOS and Android developers are the only ones who have the skills to keep up.

Fortunately, a platform called Viro was created to enable web and mobile developers to develop cross-platform AR/VR experiences through the React Native framework.

We'll take a brief look at what AR is and how it works, and then take a deep dive into Viro and everything it provides to get you moving rapidly with AR development.

We'll focus on a few very interesting pieces that Viro provides, such as 3D models, portals, image recognition, and real life object recognition.

By the end of this session, you will have the skills and inspiration to start working with AR immediately!

Transcript

2. PID PID RA PID RA PID Cross Platform Cross Platform ATIVE ATIVE REACT NATIVE REACT NATIVE WITH WITH WITH WITH Alex Hinson n @amhinson
5. PID PID RA PID RA PID Cross Platform Cross Platform ATIVE ATIVE REACT NATIVE REACT NATIVE WITH WITH WITH WITH Alex Hinson n @amhinson
6. Y WHY How What
7. What is AR ?
8. creates a virtual environment that is closed off to the real world places virtual objects in the real world
10. Why AR Right Now ?
11. expensive / inaccessible quality hardware cutting edge gaming
12. ARKit & ARCore Pokemon GO, Instagram, Snapchat
13. How Does AR Work ?
14. Tracking dering Rendering
15. 3d rendering lighting & shadows physics animations Rendering
16. built by Apple not cross platform SceneKit
17. built for games cross platform not JS or React based Unity / Unreal
18. cross platform JS/React based
19. What is Viro ?
21. ARKit & ARCore Cardboard, Daydream, & Gear VR
22. Why would you use Viro ?
23. ar Familiar l Powerful Accessible
24. How does viro work ?
25. high performance native 3D rendering engine custom React Native components
26. Viro Viro d Testbed App App
27. warning: lots of code ahead Examples
28. <ViroARSceneNavigator apiKey={API_KEY} initialScene={{ scene: this.renderARScene }} />
29. <ViroARScene onTrackingUpdated={this.onInitialized} > // … </ViroARScene>
30. onInitialized = (state) => { if (state === ViroConstants.TRACKING_NORMAL) { this.setState({ tracking: true }); } };
31. <ViroARScene onTrackingUpdated={this.onInitialized}> {this.state.tracking && ( <ViroText text="Airship" position={[0, 0, -1]} style={{ color: '#FF2A13' }} /> )} </ViroARScene>
33. <ViroARScene onTrackingUpdated={this.onInitialized}> {this.state.tracking && ( <ViroText text="Airship" position={[0, 0, -1]} style={{ color: '#FF2A13' }} /> )} </ViroARScene>
35. SHapes
36. <ViroBox height={0.2} length={0.2} width={0.2} materials={['white']} />
38. ViroMaterials.createMaterials({ white: { lightingModel: 'Blinn', diffuseColor: 'rgb(231,231,231)' } });
39. <ViroAmbientLight color=“#FFFFFF" intensity={150} /> <ViroDirectionalLight color="#FFFFFF" direction={[0.5, -1, 0.5]} castsShadow={true} /> // ...
41. <ViroSphere radius={0.2} materials={[‘white']} />
43. <ViroSphere radius={0.2} materials={['moon']} /> // ... ViroMaterials.createMaterials({ moon: { shininess: 2.0, lightingModel: 'Blinn', diffuseTexture: require('./res/moon.jpg') } });
46. <ViroSphere radius={0.2} onDrag={position => this.setState({ x: position[0], y: position[1], z: position[2] }) } position={[this.state.x, this.state.y, this.state.z]} materials={['moon']} />
48. ViroAnimations.registerAnimations({ spin: { properties: { rotateY: '+=45' }, duration: 2000 } });
49. <ViroSphere radius={0.2} onDrag={position => this.setState({ x: position[0], y: position[1], z: position[2] }) } position={[this.state.x, this.state.y, this.state.z]} materials={['moon']} animation={{ name: 'spin', run: true, loop: true }} />
51. Custom 3d Obj ects
54. <Viro3DObject source={require('./res/dino.vrx')} position={[0, 0, -1]} scale={[0.1, 0.1, 0.1]} type="VRX" />
56. final ARScene scene = new ARScene(); scene.setListener(new ARScene.Listener() { public void onAnchorFound(ARAnchor anchor, ARNode arNode) { // Make sure anchor is a plane if (anchor.getType() == ARAnchor.Typea.PLANE) { ARPlaneAnchor planeAnchor = (ARPlaneAnchor) anchor; // Ensure this is a horizontal plane of the right size if (planeAnchor.getAlignment() == ARPlaneAnchor.Alignment.HORIZONTAL && planeAnchor.getWidth() > 1.0 && planeAnchor.getHeight() > 1.0) { Object3D dinosaur = new Object3D(); dinosaur.loadModel("file://android-asset/dinosaur.fbx", Object3D.Type.FBS); arNode.addChildNode(dinosaur); } } } });
57. <ViroARPlaneSelector> <Viro3DObject source={require("./res/dinosaur.vrx")} type="VRX" /> </ViroARPlaneSelector>
60. <ViroSpotLight innerAngle={5} outerAngle={45} direction={[0, -1, -0.2]} position={[0, 3, 1]} color="#ffffff" castsShadow={true} shadowOpacity={0.5} /> // ... Viro3DObject <ViroQuad rotation={[-90, 0, 0]} width={4} height={4} arShadowReceiver={true} />
62. <Viro3DObject source={require('./res/dino.vrx')} position={[0, 0, 0]} scale={[0.1, 0.1, 0.1]} type="VRX" onClick={() => this.setState({ runAnimation: true }) } animation={{ name: 'move', run: this.state.runAnimation, loop: true }} />
63. ViroAnimations.registerAnimat ions({ move: { properties: { positionZ: '+=0.2' }, duration: 1000 } });
65. Portals
67. <ViroPortalScene passable={true}> <ViroPortal position={[0, 0, -1]} scale={[0.1, 0.1, 0.1]} > <Viro3DObject source={require(‘./res/ship_window.vrx')} type="VRX" /> </ViroPortal> <Viro360Image source={require(‘./res/360_island.jpg')} /> </ViroPortalScene>
69. <Viro360Video source={require('./res/360_surf.mp4')} loop={true} />
71. Image Recognition
73. <ViroARImageMarker target="airshipLogo" onAnchorFound={() => this.setState({ runAnimation: true }) } > // ... Objects to render when target is found </ViroARImageMarker>
74. ViroARTrackingTargets.createTargets({ airshipLogo: { source: require('./res/logo.png'), orientation: 'Up', physicalWidth: 0.03 } });
75. ViroAnimations.registerAnimations({ slideUp: { properties: { positionZ: 0, opacity: 1.0 }, easing: 'Bounce', duration: 700 }, rotate: { properties: { rotateZ: '+=45' }, duration: 200 } });
76. <ViroNode opacity={0} position={[0, -0.01, 0.05]} animation={{ name: ‘slideUp', run: this.state.runAnimation }} > <ViroText text="Airship" extrusionDepth={2} scale={[0.1, 0.1, 0.1]} rotation={[-90, 0, 0]} animation={{ name: 'rotate', run: this.state.runAnimation, loop: true }} /> </ViroNode>
77. <ViroARImageMarker target="airshipLogo" onAnchorFound={() => this.setState({ runAnimation: true })}> <ViroNode opacity={0} position={[0, -0.01, 0.05]} animation={{ name: ‘slideUp', run: this.state.runAnimation }}> <ViroText text="Airship" extrusionDepth={2} scale={[0.1, 0.1, 0.1]} rotation={[-90, 0, 0]} animation={{ name: ‘rotate', run: this.state.runAnimation, loop: true }} /> </ViroNode> </ViroARImageMarker> // … ViroARTrackingTargets.createTargets({ airshipLogo: { source: require('./res/mark_with_bg.png'), orientation: 'Up', physicalWidth: 0.03 } }); ViroAnimations.registerAnimations({ slideUp: { properties: { positionZ: 0, opacity: 1.0 }, easing: 'Bounce', duration: 700 }, rotate: { properties: { rotateZ: ‘+=45' }, duration: 200 } });
81. <ViroARImageMarker target="videoImage"> <ViroVideo source={require('./video.mp4')} loop={true} rotation={[-90, 0, 0]} height={0.08} width={0.15} /> </ViroARImageMarker>
84. Object Recognition
85. <ViroARObjectMarker target={target.targetName}> // ... Something to render </ViroARObjectMarker> // ... ViroARTrackingTargets.createTargets({ tvRemote: { source: require('./res/tvRemote.arobject'), type: 'Object' }, dvdRemote: { source: require('./res/dvdRemote.arobject'), type: 'Object' } });
87. const TARGETS = [ { displayText: 'TV', targetName: 'tvRemote' }, { displayText: 'DVD', targetName: 'dvdRemote' } ]; // ... {TARGETS.map(target => ( <ViroARObjectMarker target={target.targetName}> <ViroText text={target.displayText} extrusionDepth={2} scale={[0.15, 0.15, 0.15]} position={[0, 0.1, 0]} textAlign="center" /> </ViroARObjectMarker> ))}
89. Viro & react NAtive
91. <View style={{ flex: 1 }}> <ViroARSceneNavigator apiKey={API_KEY} initialScene={{ scene: this.renderARScene }} /> <Text>Score: {this.state.score}</Text> <TouchableOpacity onPress={this.restartGame}> <Text>Restart</Text> </TouchableOpacity> </View>
93. state = { dollarBills: [] }; // ... // Inside setInternal this.setState(({ dollarBills }) => ({ dollarBills: [ ...dollarBills, { position: this.randomXYZNumber(-2, 2), rotation: this.randomXYZNumber(-45, 45) } ] }));
94. dollarBills.map((item, index) => ( <ViroBox key={index} position={item.position} rotation={item.rotation} onClick={() => this.onDollarSelected(index)} height={0.2} length={0.01} width={0.6} materials={['billFace']} /> ))
97. github.com/amhinson/ar-demo Demo A pp
98. repo: github.com/amhinson/ar-demo Thanks Alex Hinson @amhinson