Decision Algorithms In Production.

Advice for researchers and analysts from a coder on transitioning a mathematical model from proof-of-concept to production. Presented at the AI Startups Conference http://aistartupsconference.org

Transcript

1. Decision Algorithms In Production Philip Thomas Co-Founder and CEO philip@staffjoy.com AI Startups Conference 2016
2. Advice for researchers from a coder on transitioning a model to production
4. Staffjoy makes workers happier and businesses more profitable by replacing their manual scheduling with automation
5. Staffjoy automates scheduling by solving NP-hard math problems
6. Monday 9am-5pm Monday 10am-6pm Tuesday 10am-6pm Tuesday 11am-7pm
8. Proofs of concepts are not production code
10. We applied software engineering techniques to stabilize our algorithms
14. Problem: Slow Calculations
15. Solution: Simplify the problem
16. Hard Problem Solution
17. Hard Problem Solution Easy Problem Easy Problem
20. Most errors are due to code changes
21. Problem: Managing changes and rollbacks
22. Solution: Docker
23. Docker packages complex applications like a binary
24. Stage Server
25. What happens when something breaks on a server?
26. Problem: Monitoring in Production
27. Solution: Logging
29. Production services interact with other services
30. Problem: Running code
31. Solution: Switch from Julia to Python
33. Chomp Calculates shifts from forecasts Mobius Schedules workers in their preferred shifts
35. Problem: Had to roll back changes too often
36. Solution: Backtesting
37. Build a library of historical data that caused problems
38. Tests we run: • Formatting • Linting • Unit tests • Backtesting • Speed benchmarks
39. Problem: Calculating the same problem multiple times is inefficient
40. Solution: Caching
41. Subproblem Match old problems? Calculate NO YES Cache
42. Users don’t care about “feasibility”
43. Problem: Rules get broken with real users
44. Solution: Elegantly downgrade
45. Nice To Haves Soft Constraints Hard Constraints
46. Summary • Simplify the problem • Make builds repeatable • Log everything • Backtest • Cache • Downgrade elegantly
48. We’re Hiring: Dev-ops engineer Front-end engineer Product designer
49. Decision Algorithms In Production Philip Thomas Co-Founder and CEO philip@staffjoy.com AI Startups Conference 2016