Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Implementing a lightning fast route search engine in Typescript

Implementing a lightning fast route search engine in Typescript

In the context of building a mobile app for public transport, I had to implement a route search engine for the region of Oise, France. It was a more complex journey than I expected and I'd like to give a glimpse on how I implemented the Connection Scan Algorithm in Typescript, heavily leveraging the async features of node.

Titouan Galopin

April 05, 2018
Tweet

More Decks by Titouan Galopin

Other Decks in Technology

Transcript

  1. Implementing a lightning fast route
    search engine in Typescript
    SFNode April 2018

    View Slide

  2. Symfony since 2015 (PHP framework)
    Titouan Galopin
    @tgalopin
    Created bus.io in 2015
    (public transport app : https://getbus.io)

    View Slide

  3. Agenda
    1. Goal
    2. The routing problem
    3. Historic solutions
    4. Connection Scan Algorithm
    5. Implementation

    View Slide

  4. Goal

    View Slide

  5. Let users find the quickest way to
    go from a point A (coordinates) to
    point B by bus

    View Slide

  6. “I want to go from the Golden
    Gate Bridge to LinkedIn
    Headquarters leaving at 6:58”

    View Slide

  7. The routing problem

    View Slide

  8. This problem is not new,
    mathematicians already thought about it

    View Slide

  9. This problem is not new,
    mathematicians already thought about it
    Fortunately, in Computer Science
    we love Mathematics!

    View Slide

  10. Equivalent problem in Maths:
    Shortest Path Problem (SPP)

    View Slide

  11. Graphs

    View Slide

  12. Graphs
    Invented in 1735 by Euler

    View Slide

  13. Graphs
    Invented in 1735 by Euler
    Powerful model for Computer Science
    (from AI to blockchain, compilers, ...)

    View Slide

  14. Non-oriented Oriented
    Source : https://fr.wikipedia.org/wiki/Théorie_des_graphes

    View Slide

  15. Source : https://en.wikipedia.org/wiki/Shortest_path_problem
    SPP : weighted oriented graphs

    View Slide

  16. Historic solutions

    View Slide

  17. Djikstra algorithm: 1956
    O(n * log n)
    n: number of nodes m : number of edges

    View Slide

  18. Djikstra algorithm: 1956
    O(n * log n)
    Bellman–Ford algorithm: 1958
    O(n * m)
    n: number of nodes m : number of edges

    View Slide

  19. Djikstra algorithm: 1956
    O(n * log n)
    Bellman–Ford algorithm: 1958
    O(n * m)
    A* algorithm: 1968
    Extension of Djikstra, O(n * log n) but much
    better in real life, most used today
    n: number of nodes m : number of edges

    View Slide

  20. Still today, the best generic algorithm
    to solve SPP is in
    O(n * log n)
    n: number of nodes m : number of edges

    View Slide

  21. What about more specific
    problems like public transports?

    View Slide

  22. Main difference :
    the bus/train/… has specific timetables
    => additional constraint
    => better complexity?

    View Slide

  23. Connection Scan Algorithm

    View Slide

  24. Connection Scan Algorithm
    Developed in Germany in 2012

    View Slide

  25. Connection Scan Algorithm
    Developed in Germany in 2012
    Targets SPP with timetables

    View Slide

  26. Connection Scan Algorithm
    Developed in Germany in 2012
    Targets SPP with timetables
    Overall complexity: O(c * log c)
    Runtime complexity: O(c) Linear!

    View Slide

  27. Connection
    A to B
    7:42 to 7:45

    View Slide

  28. A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17
    E to F
    7:20 to 7:29
    List of connections instead of Graph
    ...
    ...

    View Slide

  29. Implementation

    View Slide

  30. Preparation (before the query)
    Create a list of all the connections
    ordered by ascending departing time
    O(c * log c)

    View Slide

  31. Runtime (during the query)

    View Slide

  32. // Map of the the fastest ways to go to each stop
    // node name => fastest connection to go to the node
    let inConnection: StringMap = {};
    // Map of earliest arrival time per node
    // node name => arrival time
    // (418 = 6 * 60 + 58)
    let arrivalTimes: StringMap = { A: 418 };

    View Slide

  33. for (let i in connections) {
    const connection = connections[i];
    // If we never arrived to this connection’s departure stop,
    // we never will as connections are ordered by departure time
    if (typeof arrivalTimes[connection.departure_stop] === 'undefined') {
    continue;
    }
    // If the connection leaves before we arrive at its departure, it can't be used
    if (arrivalTimes[connection.departure_stop] > connection.departure_time) {
    continue;
    }
    // If there already was a connection but the previous connection arrived
    // earlier, keep the previous one
    if (arrivalTimes[connection.arrival_stop] < connection.arrival_time) {
    continue;
    }
    // Otherwise, we know for sure this connection is better than what we had
    arrivalTimes[connection.arrival_stop] = connection.arrival_time;
    inConnection[connection.arrival_stop] = connection;
    }
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17

    View Slide

  34. for (let i in connections) {
    const connection = connections[i];
    // If we never arrived to this connection’s departure stop,
    // we never will as connections are ordered by departure time
    if (typeof arrivalTimes[connection.departure_stop] === 'undefined') {
    continue;
    }
    // If the connection leaves before we arrive at its departure, it can't be used
    if (arrivalTimes[connection.departure_stop] > connection.departure_time) {
    continue;
    }
    // If there already was a connection but the previous connection arrived
    // earlier, keep the previous one
    if (arrivalTimes[connection.arrival_stop] < connection.arrival_time) {
    continue;
    }
    // Otherwise, we know for sure this connection is better than what we had
    arrivalTimes[connection.arrival_stop] = connection.arrival_time;
    inConnection[connection.arrival_stop] = connection;
    }
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17

    View Slide

  35. for (let i in connections) {
    const connection = connections[i];
    // If we never arrived to this connection’s departure stop,
    // we never will as connections are ordered by departure time
    if (typeof arrivalTimes[connection.departure_stop] === 'undefined') {
    continue;
    }
    // If the connection leaves before we arrive at its departure, it can't be used
    if (arrivalTimes[connection.departure_stop] > connection.departure_time) {
    continue;
    }
    // If there already was a connection but the previous connection arrived
    // earlier, keep the previous one
    if (arrivalTimes[connection.arrival_stop] < connection.arrival_time) {
    continue;
    }
    // Otherwise, we know for sure this connection is better than what we had
    arrivalTimes[connection.arrival_stop] = connection.arrival_time;
    inConnection[connection.arrival_stop] = connection;
    }
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17

    View Slide

  36. for (let i in connections) {
    const connection = connections[i];
    // If we never arrived to this connection’s departure stop,
    // we never will as connections are ordered by departure time
    if (typeof arrivalTimes[connection.departure_stop] === 'undefined') {
    continue;
    }
    // If the connection leaves before we arrive at its departure, it can't be used
    if (arrivalTimes[connection.departure_stop] > connection.departure_time) {
    continue;
    }
    // If there already was a connection but the previous connection arrived
    // earlier, keep the previous one
    if (arrivalTimes[connection.arrival_stop] < connection.arrival_time) {
    continue;
    }
    // Otherwise, we know for sure this connection is better than what we had
    arrivalTimes[connection.arrival_stop] = connection.arrival_time;
    inConnection[connection.arrival_stop] = connection;
    }
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17

    View Slide

  37. for (let i in connections) {
    const connection = connections[i];
    // If we never arrived to this connection’s departure stop,
    // we never will as connections are ordered by departure time
    if (typeof arrivalTimes[connection.departure_stop] === 'undefined') {
    continue;
    }
    // If the connection leaves before we arrive at its departure, it can't be used
    if (arrivalTimes[connection.departure_stop] > connection.departure_time) {
    continue;
    }
    // If there already was a connection but the previous connection arrived
    // earlier, keep the previous one
    if (arrivalTimes[connection.arrival_stop] < connection.arrival_time) {
    continue;
    }
    // Otherwise, we know for sure this connection is better than what we had
    arrivalTimes[connection.arrival_stop] = connection.arrival_time;
    inConnection[connection.arrival_stop] = connection;
    }
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17

    View Slide

  38. A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    B to E
    7:11 to 7:18
    G to D
    7:08 to 7:11
    D to E
    7:12 to 7:17
    // We know these times are
    // the earliest possible arrival
    // times for each stop
    arrivalTimes:
    A: 6:58
    B: 7:05
    C: 7:09
    D: 7:11
    E: 7:15
    // Moreover, we get a list of the
    // fastest way to get to each node
    inConnection:
    B:
    C:
    D:
    E:
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15

    View Slide

  39. inConnection:
    B:
    C:
    D:
    E:
    A to B
    7:00 to 7:05
    B to C
    7:06 to 7:09
    B to D
    7:07 to 7:11
    C to E
    7:10 to 7:15
    let steps = [];
    let current = 'E';
    while (current !== 'A') {
    let step = inConnection[current];
    current = step['departure_stop'];
    steps.push(step);
    }
    // Result: A -> B -> C -> E

    View Slide

  40. More features!
    Handle walks between stops and at the
    start/end of the journey
    Select the 5 best results in the next minutes

    View Slide

  41. But how to keep performance?
    Async!

    View Slide

  42. OSRM for walks
    Modern C++ routing engine for shortest
    paths in road networks
    http://project-osrm.org

    View Slide

  43. Search leaving at time T
    Prepare in advance
    the connections
    Load the
    connections
    CSA Result 1
    Query
    Search leaving at time T + 1
    Search leaving at time T + 2
    ...
    Select best
    results
    User
    Prepare in advance
    the walks between
    stops
    Find walks from
    start point
    Find walks to
    end point
    CSA Result 2
    Search leaving at time T + 3

    View Slide

  44. In production
    https://getbus.io
    370 nodes
    10998 connections
    20 results computed, 10 bests selected
    150ms per query

    View Slide

  45. Thanks!
    Questions?

    View Slide