Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Modeling the Physical World with Three.js

Modeling the Physical World with Three.js

Three.js is one of the easiest (and most fun) ways to build and deploy 3D applications. At Lyft, it’s at the core of several tools that are critical to our self-driving car project.

In this talk, I focus on using three.js to model the physical world — something that’s pretty important if you’re trying to build a self-driving car. I cover the basics of 3D programming and introduce the abstractions provided by three.js. Finally, I demo a few of our internal tools and discuss how they were built.

Patrick Dubroy

June 21, 2018
Tweet

More Decks by Patrick Dubroy

Other Decks in Programming

Transcript

  1. Modeling the Physical

    World with Three.js
    Berlin.js · June 21, 2018

    View full-size slide

  2. Patrick Dubroy
    @dubroy
    lyft.com/level5

    View full-size slide

  3. • Opened Dec. 1
    • 18 engineers now
    - Goal of 35 in 2018
    - Space for ~100 in 2019
    • HD Mapping & localization
    • Frontend: viz + internal tools
    • We are hiring!

    View full-size slide

  4. Three.js
    • Raw WebGL is very low-level
    • Three.js provides:
    - Scene graph API
    - Math library (Vector3, Matrix4, Quaternion, etc.)
    - Other helpers for audio, animation, etc.

    View full-size slide



  5. My first three.js app
    <br/>body { margin: 0; }<br/>canvas { width: 100%; height: 100% }<br/>


    <br/>// Our Javascript will go here.<br/>
    Hello Three.js

    View full-size slide

  6. const {innerWidth, innerHeight} = window;
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
    75, innerWidth / innerHeight, 0.1, 1000);
    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(innerWidth, innerHeight);
    document.body.appendChild(renderer.domElement);
    Creating a scene

    View full-size slide

  7. const geometry = new THREE.BoxGeometry(1, 1, 1);
    const material = new THREE.MeshBasicMaterial({
    color: 0x648fff
    });
    const cube = new THREE.Mesh(geometry, material);
    scene.add(cube);
    camera.position.z = 5;
    Creating an object

    View full-size slide

  8. function animate() {
    requestAnimationFrame(animate);
    cube.rotation.x += 0.01;
    cube.rotation.y += 0.01;
    renderer.render(scene, camera);
    }
    animate();
    Rendering the scene

    View full-size slide

  9. const {innerWidth, innerHeight} = window;
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
    75, innerWidth / innerHeight, 0.1, 1000);
    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(innerWidth, innerHeight);
    document.body.appendChild(renderer.domElement);
    Creating a scene

    View full-size slide

  10. (0, 0) X
    Y
    Viewport
    Coordinates
    2D

    View full-size slide

  11. (0, 0) X
    Y
    Viewport
    (0, 0, 0)
    X
    Y
    Viewport
    Z
    Coordinates
    2D 3D

    View full-size slide

  12. Near plane
    Far plane

    View full-size slide

  13. Near plane
    Far plane

    View full-size slide

  14. const {innerWidth, innerHeight} = window;
    const scene = new THREE.Scene();
    const camera = new THREE.PerspectiveCamera(
    75, innerWidth / innerHeight, 0.1, 1000);
    const renderer = new THREE.WebGLRenderer();
    renderer.setSize(innerWidth, innerHeight);
    document.body.appendChild(renderer.domElement);
    Creating a scene
    Field of view
    (degrees)
    Aspect ratio
    Distance to

    near plane
    Distance to

    far plane

    View full-size slide

  15. const geometry = new THREE.BoxGeometry(1, 1, 1);
    const material = new THREE.MeshBasicMaterial({
    color: 0x648fff
    });
    const cube = new THREE.Mesh(geometry, material);
    scene.add(cube);
    camera.position.z = 5;
    Creating an object

    View full-size slide

  16. Demo: Map Visualization

    View full-size slide

  17. function createLineWithPointsAndColor(
    points: THREE.Vector3[], color: number) {
    const geometry = new THREE.Geometry();
    for (const p of points) {
    geometry.vertices.push(p);
    }
    const material =
    new THREE.LineBasicMaterial({ color });
    return new THREE.Line(geometry, material);
    }
    Rendering Lanes

    View full-size slide

  18. Understanding

    the scene graph

    View full-size slide

  19. Transformations
    … …

    View full-size slide

  20. (.39, 0, -.6)
    Transformations
    • Each object defines its own coordinate
    system or frame
    • An object’s pose is its position and
    rotation relative to its parent
    - obj.rotation / obj.position
    … …
    (1.2, 0, 1.9)

    View full-size slide

  21. • Each object defines its own coordinate
    system or frame
    • An object’s pose is its position and
    rotation relative to its parent
    - obj.rotation / obj.position
    Transformations
    … …
    (.39, 0, -.6)
    (1.2, 0, 1.9)
    (-.39, 0, .6)
    (-1.2, 0, -1.9)

    View full-size slide

  22. • Each object defines its own coordinate
    system or frame
    • An object’s pose is its position and
    rotation relative to its parent
    - obj.rotation / obj.position
    Transformations
    … …
    (.39, 0, -.6)
    (1.2, 0, 1.9)
    (-.39, 0, .6)
    (-1.2, 0, -1.9)

    View full-size slide

  23. • Each object defines its own coordinate
    system or frame
    • An object’s pose is its position and
    rotation relative to its parent
    - obj.rotation / obj.position
    - obj.matrix
    Transformations
    … …
    (.39, 0, -.6)
    (1.2, 0, 1.9)
    (-.39, 0, .6)
    (-1.2, 0, -1.9)

    View full-size slide

  24. … …
    vec.applyMatrix4(lidar.matrix);
    const m = new THREE.Matrix4()
    .getInverse(camera.matrix);
    vec.applyMatrix4(m);

    View full-size slide

  25. Demo: Colored Point Cloud

    View full-size slide

  26. More:

    threejs.org
    Cartographer point cloud viewer
    We are hiring
    frontend engineers in Munich:
    lyft.com/level5
    Thanks!

    View full-size slide