Evolving Sustainable Data Pipelines

Full post here:

Transcript

2. ESP Evolving Sustainable {data} Pipelines Anna Smith - @OMGannaks 29 January 2015
3. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
4. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
6. 1. 2. 3.
7. WHERE THE MAGIC HAPPENS
8. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
9. DEMAND website user interaction transactions SUPPLY warehouse inventory allocation
10. DEMAND website user interaction transactions SUPPLY warehouse inventory allocation INTERACTION reservation calendar
14. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
15. PHASE 0
17. YUP
18. KEEP IT TOGETHER
19. PHASE 1 primordial soup
20. STABILITY
21. PHASE 2 process
22. orderwarehouse.job
23. $ ./runjob.py orderwarehouse.job $ ./runjob.py orderwarehouse.job --show $ ./runjob.py orderwarehouse.job --only 2
24. runjob.py
25. ADAPTING ensuring data quality
26. PHASE 3 exposing weaknesses
27. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
28. dependency manager
31. A bit about RTR A bit about our analytics Some evolution stuff A bit about Luigi The future bits
32. PHASE 4 ownership
33. RELIABILITY
34. COMMUNICATION
35. THE FUTURE @OMGannaks asmith@renttherunway.com