Upgrade to Pro — share decks privately, control downloads, hide ads and more …


Sunny Ng
March 20, 2019


Inspired by frustrated tweets and the fact that MTA New York City Transit now provides data that drives countdown clocks to every station to the public, goodservice.io was created to automatically detect irregular train headways and delays.

Sunny Ng

March 20, 2019


  1. goodservice.io Sunny Ng Transit Techies NYC #7 - March 20,

  2. About Me Full-stack Software Engineer @ Harry’s by day Public

    transit nerd all times, occasionally bikes
  3. December 31, 2017 NYCT finished implementing countdown clocks on all

  4. Inspiration

  5. Inspiration

  6. What if we automatically detect irregular headways?

  7. None
  8. MTA NYCT Real-Time Feeds • GTFS-RT standard + extensions •

    9 endpoints • IRT, A/C/E/S (Rockaway/Franklin), N/Q/R/W, B/D/F/M, L, SIR, G, J/Z, 7 • Updates every 30 seconds
  9. MTA NYCT Real-Time Feeds Arrival Time Stop ID Stop Name

    (not in the feed) 9:28:25 am A24N 59 St–Columbus Cir 9:38:37 am A15N 125 St 9:41:37 am A12N 145 St 9:45:37 am A09N 168 St 9:48:37 am A07N 175 St 9:49:37 am A06N 181 St 9:51:37 am A05N 190 St 9:55:07 am A03N Dyckman St 9:58:07 am A02N Inwood–207 St Trip ID: 072200_A..N Direction: 0 Route ID: A Train ID: 1A 1155 FAR/207 Timestamp: 9:37:49 am
  10. MTA NYCT Static Feeds • GTFS Standard • Zip file

    of CSV files for routes, trips, stops, stop times, etc. • Provides scheduled information for a time period • 19,891 trips, 548,745 stop times
  11. Initial Implementation • Identify “key stations” • Calculate actual headway

    using real-time feed • Calculate scheduled headway using static-feed • Compare scheduled and actual headways
  12. • Minimum or average/median headway is not important • Routes

    can be very long • A train’s run time is 98 minutes each way • Parts of each route can be in different schedule periods • Incidents affecting one part/direction of the route may not affect others
  13. Uneven Headways * From MTA’s Review of the G Line

    (July 10, 2013) Scheduled departures at Bergen St (2013)
  14. Washington Heights Central Park W 8 Avenue Fulton St Lefferts

    Blvd Branch Rockaway (Far Rockaway Branch) Rockaway (Rockaway Park Branch) via Chambers St
  15. Splitting by Lines • Routing is transient • Cannot completely

    rely on scheduled data • When was the last time you’ve seen the M train run up to 71 Av until midnight? • Provide a view of grouped routes for each line • Need to differentiate between local/express per line • GTFS-RT doesn’t explicitly tell us if a train is running local/express
  16. Revised Implementation • Identify physical lines and seed key stations

    • Make minimal assumptions about where a route will go • Service pattern for a route may change within the hour • Allow multiple concurrent segments • Ability to detect service changes for free
  17. Express or Local? e.g. Broadway Line (Uptown & Queens) 34

    St–Herald Sq 14 St–Union Sq 23 St 28 St • Trip includes 28 St: local • Trip includes 14 St–Union Sq, but not 28 St: express • Trip only includes stop at 34 St—Herald Sq: !
  18. Detecting Delays

  19. Statuses • Delay: Any train’s estimated arrival time at the

    terminal has increased by >= 5 minutes while in between the same two stations • Service Change: Any train is stopping at different stations than what are scheduled • Not Good: Difference in maximum scheduled and actual headway >= 3 minutes • No Service: Route scheduled to run but no trains detected • Not Scheduled: Route not scheduled to run • Good Service: None of the above, hooray!
  20. Demo

  21. View service levels by route

  22. View service levels by line

  23. Adapting to late night/weekend service changes

  24. Performance stats for each route

  25. Slack Integration

  26. Twitter Bot - @goodservice_io

  27. What’s next?

  28. What’s next? • Real-time service map • Detect trains slowing

    down • Analyze frequent locations of slow downs • Analyze accuracy of countdown clocks • Mobile app???
  29. @_blahblahblah & @goodservice_io blahblahblah-/goodservice sunnyng medium.com/good-service Thank you!