Modeling the Physical World with Three.js

Modeling the Physical World with Three.js

Three.js is one of the easiest (and most fun) ways to build and deploy 3D applications. At Lyft, it’s at the core of several tools that are critical to our self-driving car project.

In this talk, I focus on using three.js to model the physical world — something that’s pretty important if you’re trying to build a self-driving car. I cover the basics of 3D programming and introduce the abstractions provided by three.js. Finally, I demo a few of our internal tools and discuss how they were built.

Dad50df1d9acad462018f77360434de6?s=128

Patrick Dubroy

June 21, 2018
Tweet

Transcript

  1. Modeling the Physical
 World with Three.js Berlin.js · June 21,

    2018
  2. Patrick Dubroy @dubroy lyft.com/level5

  3. • Opened Dec. 1 • 18 engineers now - Goal

    of 35 in 2018 - Space for ~100 in 2019 • HD Mapping & localization • Frontend: viz + internal tools • We are hiring!
  4. Three.js • Raw WebGL is very low-level • Three.js provides:

    - Scene graph API - Math library (Vector3, Matrix4, Quaternion, etc.) - Other helpers for audio, animation, etc.
  5. <!DOCTYPE html> <head> <title>My first three.js app</title> <style> body {

    margin: 0; } canvas { width: 100%; height: 100% } </style> </head> <script src="js/three.js"></script> <script> // Our Javascript will go here. </script> Hello Three.js
  6. const {innerWidth, innerHeight} = window; const scene = new THREE.Scene();

    const camera = new THREE.PerspectiveCamera( 75, innerWidth / innerHeight, 0.1, 1000); const renderer = new THREE.WebGLRenderer(); renderer.setSize(innerWidth, innerHeight); document.body.appendChild(renderer.domElement); Creating a scene
  7. const geometry = new THREE.BoxGeometry(1, 1, 1); const material =

    new THREE.MeshBasicMaterial({ color: 0x648fff }); const cube = new THREE.Mesh(geometry, material); scene.add(cube); camera.position.z = 5; Creating an object
  8. function animate() { requestAnimationFrame(animate); cube.rotation.x += 0.01; cube.rotation.y += 0.01;

    renderer.render(scene, camera); } animate(); Rendering the scene ▶
  9. const {innerWidth, innerHeight} = window; const scene = new THREE.Scene();

    const camera = new THREE.PerspectiveCamera( 75, innerWidth / innerHeight, 0.1, 1000); const renderer = new THREE.WebGLRenderer(); renderer.setSize(innerWidth, innerHeight); document.body.appendChild(renderer.domElement); Creating a scene
  10. Coordinates

  11. (0, 0) X Y Viewport Coordinates 2D

  12. (0, 0) X Y Viewport (0, 0, 0) X Y

    Viewport Z Coordinates 2D 3D
  13. None
  14. Near plane Far plane

  15. Near plane Far plane

  16. const {innerWidth, innerHeight} = window; const scene = new THREE.Scene();

    const camera = new THREE.PerspectiveCamera( 75, innerWidth / innerHeight, 0.1, 1000); const renderer = new THREE.WebGLRenderer(); renderer.setSize(innerWidth, innerHeight); document.body.appendChild(renderer.domElement); Creating a scene Field of view (degrees) Aspect ratio Distance to
 near plane Distance to
 far plane
  17. const geometry = new THREE.BoxGeometry(1, 1, 1); const material =

    new THREE.MeshBasicMaterial({ color: 0x648fff }); const cube = new THREE.Mesh(geometry, material); scene.add(cube); camera.position.z = 5; Creating an object
  18. Demo: Map Visualization ▶

  19. function createLineWithPointsAndColor( points: THREE.Vector3[], color: number) { const geometry =

    new THREE.Geometry(); for (const p of points) { geometry.vertices.push(p); } const material = new THREE.LineBasicMaterial({ color }); return new THREE.Line(geometry, material); } Rendering Lanes
  20. Understanding
 the scene graph

  21. … …

  22. Transformations … …

  23. (.39, 0, -.6) Transformations • Each object defines its own

    coordinate system or frame • An object’s pose is its position and rotation relative to its parent - obj.rotation / obj.position … … (1.2, 0, 1.9)
  24. • Each object defines its own coordinate system or frame

    • An object’s pose is its position and rotation relative to its parent - obj.rotation / obj.position Transformations … … (.39, 0, -.6) (1.2, 0, 1.9) (-.39, 0, .6) (-1.2, 0, -1.9)
  25. • Each object defines its own coordinate system or frame

    • An object’s pose is its position and rotation relative to its parent - obj.rotation / obj.position Transformations … … (.39, 0, -.6) (1.2, 0, 1.9) (-.39, 0, .6) (-1.2, 0, -1.9)
  26. • Each object defines its own coordinate system or frame

    • An object’s pose is its position and rotation relative to its parent - obj.rotation / obj.position - obj.matrix Transformations … … (.39, 0, -.6) (1.2, 0, 1.9) (-.39, 0, .6) (-1.2, 0, -1.9)
  27. … … vec.applyMatrix4(lidar.matrix); const m = new THREE.Matrix4() .getInverse(camera.matrix); vec.applyMatrix4(m);

  28. Demo: Colored Point Cloud ▶

  29. None
  30. None
  31. None
  32. More:
 threejs.org Cartographer point cloud viewer We are hiring frontend

    engineers in Munich: lyft.com/level5 Thanks!