React VR workshop

Slides from presentation done during SwingDev's WebVR meetup workshop about React VR

Transcript

1. www.swingdev.io React VR workshop Building Cannon Shooter game in React VR with Mirek Ciastek, SwingDev’s Front-end Developer
2. www.swingdev.io Agenda 1. Introduction and recap 2. Project setup and building basic scene 3. Loading models and adding interaction 4. Adding particles and sounds 5. Adding score and weapon types 6. Summary
3. www.swingdev.io Cannon Shooter
4. www.swingdev.io Introduction and recap
5. www.swingdev.io React VR React VR lets you build VR apps using only JavaScript. It uses the same design as React, letting you compose a rich VR world and UI from declarative components.
6. www.swingdev.io It’s more complicated than that Main Thread React VR Runtime React Native WebGL/WebVR OVRUI Three.js WebWorker React VR Application Code React Native React Web Browser Asynchronous Bridge
7. www.swingdev.io • JSX and React API • Props and State • Input handlers - onEnter, onExit, onMove, onInput, onClick • Fonts and Text Core features
8. www.swingdev.io 3D space • OpenGL coordinates • units in meters • rotation in degrees • Y axis points up, Z axis points to user
9. www.swingdev.io React VR is based on React React is a declarative, efficient, and flexible JavaScript library for building user interfaces. • JSX syntax - like HTML in JavaScript • component oriented - everything is a component • props are for passing data to component • state allows to update component • React is only for views - the rest you need to add by your own
10. www.swingdev.io class ShoppingList extends React.Component { render() { return ( <div className="shopping-list"> <h1>Shopping List for {this.props.name}</h1> <ul> <li>Instagram</li> <li>WhatsApp</li> <li>Oculus</li> </ul> </div> ); } } React example
11. www.swingdev.io <View> <Pano source={asset('chess-world.jpg')}/> <Text style={{ backgroundColor: '#777879', fontSize: 0.8, layoutOrigin: [0.5, 0.5], paddingLeft: 0.2, paddingRight: 0.2, textAlign: 'center', textAlignVertical: 'center', transform: [{translate: [0, 0, -3]}], }}> hello </Text> </View> React VR example
12. www.swingdev.io Project setup and building basic scene
13. www.swingdev.io Requirements • Node.js • NPM or Yarn • React VR CLI • Git (version control) or SourceTree
14. www.swingdev.io https://github.com/SwingDev/reactvr-workshop or short version https://bit.ly/2qzdp5k
15. www.swingdev.io Setup project git clone git@github.com:SwingDev/reactvr-workshop.git Clone repository cd reactvr-workshop && git checkout stage_1 Go to project directory and checkout to start branch yarn install && yarn start Run the app Go to http://localhost:8081/vr?hotreload
16. www.swingdev.io Project structure !"" /actions !"" /components !"" /libs !"" /modules !"" /reducers !"" /static_assets !"" /store !"" /utils !"" /views !"" /vr #"" index.vr.js #"" /components !"" App.js !"" Cannon.js !"" Ground.js !"" HUD.js !"" Sky.js !"" Summary.js #"" Wall.js #"" /views !"" /CustomModel #"" /ParticlePool
17. www.swingdev.io Let’s make Sky const vertexShader = ` varying vec3 vWorldPosition; void main() { vec4 worldPosition = modelMatrix * vec4( position, 1.0 ); vWorldPosition = worldPosition.xyz; gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 ); } `; const fragmentShader = ` uniform vec3 topColor; uniform vec3 bottomColor; uniform float offset; uniform float exponent; varying vec3 vWorldPosition; void main() { float h = normalize( vWorldPosition + offset ).y; gl_FragColor = vec4( mix( bottomColor, topColor, max( pow( max( h , 0.0), exponent ), 0.0 ) ), 1.0 ); } `; Sky.js
18. www.swingdev.io Let’s make Sky const Sky = () => ( <Sphere radius={400} widthSegments={32} heightSegments={15} materialParameters={{ vertexShader, fragmentShader, uniforms, side: THREE.BackSide, }} /> ); const uniforms = { topColor: { value: new THREE.Color(0x0077ff), }, bottomColor: { value: new THREE.Color(0xffffff), }, offset: { value: Y_OFFSET, }, exponent: { value: 0.6, }, }; Sky.js
19. www.swingdev.io Ground const Ground = () => ( <Plane dimWidth={500} dimHeight={500} style={{ transform: [{ translateY: -Y_OFFSET, }, { rotateX: -90, }], }} materialParameters={{ color: 0xffc880, specular: 0x050505, }} lit /> ); Ground.js
20. www.swingdev.io Add Sky and Ground to App.js import Sky from './Sky'; import Ground from './Ground'; class App extends React.Component { render() { return ( <View> <Sky /> <Ground /> </View> ); } } App.js
21. www.swingdev.io Add lights to scene import { View, AmbientLight, DirectionalLight, } from 'react-vr'; class App extends React.Component { render() { return ( <View> <AmbientLight intensity={0.4} /> <DirectionalLight /> <Sky /> <Ground /> </View> ); } } App.js
22. www.swingdev.io Loading models and adding interaction
23. www.swingdev.io glTF glTF (GL Transmission Format) is a file format for 3D scenes and models using the JSON standard. It is described by its creators as the "JPEG of 3D."
24. www.swingdev.io How glTF look like
25. www.swingdev.io Create Cannon component render() { return ( <View style={this.props.style}> <View> <CustomModel source={asset('cannon/cannon_head_separate.gltf')} style={CANNON_STYLE} material={MATERIAL} /> </View> <CustomModel source={asset('cannon/cannon_legs_separate.gltf')} style={CANNON_STYLE} material={MATERIAL} /> </View> ); } import { CustomModel } from '../views/CustomModel/component'; Cannon.js
26. www.swingdev.io Make Wall of boxes renderBox = (box) => { const { id, x, y, file, } = box; return ( <VrButton key={id} > <CustomModel source={asset(file || 'box/box.gltf')} material={BOX_MATERIAL} style={{ transform: [{ translate: [x, y, 0], }, { scale: [BOX_SCALE, BOX_SCALE, BOX_SCALE], }], }} /> </VrButton> ); } render() { const { boxes } = this.state; return ( <View style={this.props.style}> {boxes.map(this.renderBox)} </View> ); } Wall.js
27. www.swingdev.io Add Cannon and Wall to App.js class App extends React.Component { render() { return ( <View> <AmbientLight intensity={0.4} /> <DirectionalLight /> <Sky /> <Ground /> <Wall style={WALL_STYLE} /> <Cannon style={CANNON_STYLE} /> </View> ); } }
28. www.swingdev.io Adding click listener to boxes <VrButton key={id} onClick={() => this.handleHit(box)} > {/* Custom model component */} </VrButton> handleHit = (box) => { const { id, x, y } = box; const { boxes } = this.state; const { updatedBoxes, boxesToRemove, } = getUpdatedBoxes(boxes, id, weapon); this.setState({ boxes: updatedBoxes, }, () => this.afterHitUpdate(boxesToRemove)); }; afterHitUpdate = (boxesToRemove) => { const { boxes } = this.state; this.props.addHit( getPoints(boxesToRemove), boxesToRemove.length, ); }; Wall.js
29. www.swingdev.io Link Cannon’s head to camera componentDidMount() { this.rotateListener = RCTDeviceEventEmitter.addListener( 'onReceivedHeadMatrix', this.handleReceiveMatrix, ); } componentWillUnmount() { this.rotateListener(); } handleReceiveMatrix = () => { const [x, y] = VrHeadModel.rotation(); this.setState({ headRotateX: getClamped(x), headRotateY: getClamped(y), }); }; Cannon.js
30. www.swingdev.io Update Cannon’s render function render() { const { headRotateX, headRotateY } = this.state; return ( <View style={this.props.style}> <View style={{ transform: [{ translate: [0, 0.5, 0], }, { rotateX: headRotateX, }, { rotateY: headRotateY, }], }} > <CustomModel source={asset('cannon/cannon_head_separate.gltf')} style={CANNON_STYLE} material={MATERIAL} /> </View> <CustomModel source={asset('cannon/cannon_legs_separate.gltf')} style={CANNON_STYLE} material={MATERIAL} /> </View> ); }
31. www.swingdev.io Adding particles and sounds
32. www.swingdev.io Show particles on box’s hit Wall.js state = { boxes: getBoxesProps(), explosionPosition: new THREE.Vector3(0, 0, -5), showExplosion: false, }; handleHit = (box) => { const { id, x, y } = box; const { boxes, explosionPosition } = this.state; NativeModules.ShotBridge.emitShot(); const { updatedBoxes, boxesToRemove, } = getUpdatedBoxes(boxes, id); this.setState({ boxes: updatedBoxes, explosionPosition: explosionPosition.clone().set(x, y, -5), showExplosion: true, }, () => this.afterHitUpdate(boxesToRemove)); }; afterHitUpdate = (boxesToRemove) => { this.props.addHit( getPoints(boxesToRemove), boxesToRemove.length, ); this.setState({ showExplosion: false, }); };
33. www.swingdev.io Add particles to Wall’s boxes Wall.js import { ParticlePool } from '../views/ParticlePool/component'; render() { const { boxes, explosionPosition, showExplosion } = this.state; return ( <View style={this.props.style}> {boxes.map(this.renderBox)} <ParticlePool type='explosion' show={showExplosion} style={{ transform: [{ translate: [ explosionPosition.x - 0.25, explosionPosition.y + 0.5, explosionPosition.z - 1, ], }], }} /> </View> ); }
34. www.swingdev.io Update Cannon’s state on shot Cannon.js state = { headRotateX: 0, headRotateY: 0, showSmoke: false, }; componentDidMount() { this.rotateListener = RCTDeviceEventEmitter.addListener( 'onReceivedHeadMatrix', this.handleReceiveMatrix, ); this.shotListener = RCTDeviceEventEmitter.addListener( 'shot', this.handleShot, ); } componentWillUnmount() { this.rotateListener(); this.shotListener(); } handleShot = () => { this.setState({ showSmoke: true, }, () => { this.setState({ showSmoke: false, }); }); };
35. www.swingdev.io Add particles to Cannon <View style={{ transform: [{ translate: [0, 0.5, 0], }, { rotateX: headRotateX, }, { rotateY: headRotateY, }], }} > <CustomModel source={asset('cannon/cannon_head_separate.gltf')} style={CANNON_STYLE} material={MATERIAL} /> <ParticlePool type='smoke' show={showSmoke} style={{ transform: [{ translate: [0, 0.5, -1], }], }} /> </View>
36. www.swingdev.io Add sound to Wall’s component Wall.js state = { boxes: getBoxesProps(), soundPlayState: 'pause', explosionPosition: new THREE.Vector3(0, 0, -5), showExplosion: false, }; render() { // start of render() return ( <View style={this.props.style}> boxes.map(this.renderBox) <Sound autoPlay={false} source={{ mp3: asset('box-explosion.mp3'), }} playControl={soundPlayState} onEnded={this.handleSoundEnd} /> {/* rest of the component */} </View> ); }
37. www.swingdev.io Play sound on hit Wall.js handleSoundEnd = () => { this.setState({ soundPlayState: 'stop', }); }; handleHit = (box, weapon = weapons.CANNONBALL) => { const { id, x, y } = box; const { boxes, explosionPosition } = this.state; NativeModules.ShotBridge.emitShot(); const { updatedBoxes, boxesToRemove, } = getUpdatedBoxes(boxes, id, weapon); this.setState({ boxes: updatedBoxes, soundPlayState: 'play', explosionPosition: explosionPosition.clone().set(x, y, -5), showExplosion: true, }, () => this.afterHitUpdate(boxesToRemove)); };
38. www.swingdev.io Add sound to Cannon’s component Cannon.js state = { headRotateX: 0, headRotateY: 0, soundPlayState: 'pause', showSmoke: false, }; handleSoundEnd = () => { this.setState({ soundPlayState: 'stop', }); }; handleShot = () => { this.setState({ soundPlayState: 'play', showSmoke: true, }, () => { this.setState({ showSmoke: false, }); }); }; render() { const { headRotateX, headRotateY, showSmoke, soundPlayState, } = this.state; return ( <View style={this.props.style}> {/* start of the component */} <Sound autoPlay={false} source={{ mp3: asset('cannon-shot.mp3'), }} playControl={soundPlayState} onEnded={this.handleSoundEnd} /> </View> ); }
39. www.swingdev.io Adding score and weapon types
40. www.swingdev.io Make component for game summary Summary.js class Summary extends React.Component { render() { const { score, hits, time } = this.props.player; return ( <View style={{ ...this.props.style, ...ROOT_STYLE }}> <Text style={TEXT_STYLE}> Your points: {score} </Text> <Text style={TEXT_STYLE}> Your hits: {hits} </Text> <Text style={TEXT_STYLE}> Your time: {getFormattedTime(time)} </Text> <VrButton onClick={() => this.props.updateFinishedStatus(false)}> <Text style={BUTTON_STYLE}> Restart game </Text> </VrButton> </View> ); } }
41. www.swingdev.io Create game’s HUD HUD.js class HUD extends React.Component { render() { const { score } = this.props.player; return ( <Text style={{ ...this.props.style, backgroundColor: 'rgba(0, 0, 0, 0.6)', fontSize: 0.25, fontWeight: '400', layoutOrigin: [0.5, 0.5], paddingLeft: 0.05, paddingRight: 0.05, textAlign: 'center', textAlignVertical: 'center', position: 'absolute', }} > {score} </Text> ); } }
42. www.swingdev.io Show Summary when game ends Wall.js render() { const { hasFinished } = this.props.player; const { boxes, explosionPosition, showExplosion, soundPlayState, } = this.state; return ( <View style={this.props.style}> {(!hasFinished) ? ( boxes.map(this.renderBox) ) : ( <Summary style={SUMMARY_STYLE} /> )} {/* rest of the component */} </View> ); } finishGame() { this.props.updateFinishedStatus(true); this.setState({ boxes: getBoxesProps(), soundPlayState: 'pause', }); } afterHitUpdate = (boxesToRemove) => { const { boxes } = this.state; this.props.addHit( getPoints(boxesToRemove), boxesToRemove.length, ); this.setState({ showExplosion: false, }); if (boxes.length === 0) { this.finishGame(); } }; import Summary from './Summary';
43. www.swingdev.io Attach HUD to Cannon’s head Cannon.js render() { // start of render() return ( <View style={this.props.style}> <View style={{ transform: [{ translate: [0, 0.5, 0], }, { rotateX: headRotateX, }, { rotateY: headRotateY, }], }} > <HUD style={HUD_STYLE} /> {/* rest of the Cannon’s head */} </View> {/* rest of the component */} </View> ); } import HUD from './HUD';
44. www.swingdev.io Handle weapon types Wall.js renderBox = (box) => { const { id, x, y, file, } = box; return ( <VrButton key={id} onClick={() => this.handleHit(box)} onLongClick={() => this.handleHit(box, weapons.ROCKET)} longClickDelayMS={SPECIAL_WEAPON_DELAY} > <CustomModel source={asset(file || 'box/box.gltf')} material={BOX_MATERIAL} style={{ transform: [{ translate: [x, y, 0], }, { scale: [0.03, 0.03, 0.03], }], }} /> </VrButton> ); } handleHit = (box, weapon = weapons.CANNONBALL) => { const { id, x, y } = box; const { boxes, explosionPosition } = this.state; NativeModules.ShotBridge.emitShot(); const { updatedBoxes, boxesToRemove, } = getUpdatedBoxes(boxes, id, weapon); this.setState({ boxes: updatedBoxes, soundPlayState: 'play', explosionPosition: explosionPosition.clone().set(x, y, -5), showExplosion: true, }, () => this.afterHitUpdate(boxesToRemove)); };
45. www.swingdev.io
46. www.swingdev.io Questions?
47. www.swingdev.io Further reading • Getting started with React VR • Oculus Developer Center • Prototyping with React VR • Getting Started with WebVR • Bring Virtual Reality to Your Browser with React VR • Learn React VR (Chapter 8 | Building a VR Video App) • glTF Overview
48. www.swingdev.io https://twitter.com/mciastek https://medium.com/@mciastek https://github.com/mciastek #webvrmeetup Where to find me:
49. www.swingdev.io Thank you