Taming the Data Monster

Data warehouse projects are complex because they want all the data, from all the source systems, over all time, with quality, to answer any conceivable question. Agile approaches tackle complexity by breaking complex projects into smaller pieces. Using slicing, pairing, a test first approach and rapid deployment we explore how to tame this inherent complexity.

Transcript

1. How to tackle the Data Monster Martin Chesbrough, intern @ Everest
3. It’s easy … All the data from all the source systems recording all the business processes over all time with every process change captured whether documented or not and we need to trust the data
4. Some of the small changes I find useful - Slicing - Pairing - Test First - Deploying
5. Slice-a-palooza! An Everest Academy workshop
8. Deploy as soon as you can
10. Jeff Sutherland, Neil Harrison and Joel Riddle, Teams that Finish Early Accelerate Faster (2014) “ Happiness is one of the best metrics [for productivity] because it is a predictive indicator.“
11. Summary Data projects can be hard ❖ They’re BIG ❖ They’re complex ❖ They take a long time ❖ Solving data quality is hard ❖ Deploying Make sure you are not the 6th failed data project ❖ Slice the work finely ❖ Learn faster through pairing ❖ Test the smallest thing first ❖ Then deploy the smallest and simplest thing None of this is easy, but it doesn’t get easier if you delay So start now
12. Questions? Find me at martin.Chesbrough@Everest.engineering Or linkedin.com/in/martinchesbrough