Operationalising Data Science on Cloud Foundry

Delivered at CF Summit Europe 2016 in Frankfurt, 28th September 2016

The first part of this talk argues why Cloud Foundry is ideal for operationalising and scaling machine learning models.

The core of the talk describes a scalable cloud native architecture for operationalising data science models based on microservices. There is a video demo of this architecture in action.

The last part of the talk explains the limits of Cloud Foundry in the context of data science and what still has to be achieved for Cloud Foundry to become an end-to-end data science platform.

Transcript

1. Operationalising Data Science on CF Ian Huston
2. Who am I? • Data Scientist working with clients at Pivotal Labs • Cloud Foundry user • Community Buildpack writer @ianhuston ihuston
3. Everybody wants systems that are smarter, everybody wants systems that are more predictive, everybody wants everything scored, everybody wants to understand what’s the next best offer, next best opportunity, how to make things a little bit more efficient. Marc Benioff Forbes CIO Summit, March 7, 2016 “ ”
4. What’s the problem? If your Machine Learning model is not in production, it does not provide business value. A slide deck does not count as production!
5. Who has this problem? Data Scientists Want their model to make an impact Developers Want to add ML to their app CIOs/CDOs Want return on ‘Big Data’ investment Our Clients Want to implement their first ML models
6. Day 1 Problems Load and Transform Data Train the Predictive Model Connect to Incoming Data Apply Model Take Action Run the Model
7. ‘Scoring As A Service’ Ingest Data Build Model elsewhere with offline data Serve Result or Take Action Apply Model Store Model
8. ‘CF powered Learning’ Ingest Data Build Model in CF or elsewhere Serve Result or Take Action Apply Model Store Model Batch Update
9. In-Stream (Online) Learning Ingest Data Serve Result or Take Action Update & Apply Model
10. How can I do this? Build it: Spring Cloud Data Flow Marketplace data services Spring Boot Python ML microservices Use it: Initial offerings from GE, IBM, Alpine, Bosch
11. http://moves.cfapps.pez.pivotal.io/about
12. Day 2 Problems Which predictions were made with the old model or new? Do I need to continue serving the old predictions? Which library versions were used with old model and new? Has the data schema changed in the underlying system? Can I provide the right inputs for the different model versions? Can I replay the stream? Update the Model!
13. Using a Model Service • Version control for model • Parse data with varying schemas • Serve appropriate model version based on consuming app • Store underlying data for model re-training and reproducibility Ingest Data Model Service Serve Result or Take Action Apply Model Store Model
14. What’s Next for Data Science on CF? More data services More demos of data science/ML models Examples of successful projects Building blocks for building your own ML services dsoncf.com @ianhuston