Making machine learning model
deployment boring
Thomas Kluiters, ING
Ruurtjan Pul, BigData Republic
Slide 2
Slide 2 text
ING wil softwaregeorienteerde bank worden (steeds meer producten zijn digitaal)
en data driven zijn
en dus ML gebruiken
Slide 3
Slide 3 text
ING wil softwaregeorienteerde bank worden (steeds meer producten zijn digitaal)
en data driven zijn
en dus ML gebruiken
Slide 4
Slide 4 text
Machine learning flips the equation
Classical
programming
Rules
Data
Answers
Slide 5
Slide 5 text
Machine learning flips the equation
Classical
programming
Rules
Data
Answers
Machine
learning
Answers
Data
Rules
Slide 6
Slide 6 text
Machine learning helps many companies
●
Match candidates and vacancies
●
Recommend customer treatments to call center agents
●
Pick the right time for airplane maintenance
Slide 7
Slide 7 text
Machine learning helps many companies
●
Match candidates and vacancies
●
Recommend customer treatments to call center agents
●
Pick the right time for airplane maintenance
Slide 8
Slide 8 text
Machine learning helps many companies
●
Match candidates and vacancies
●
Recommend customer treatments to call center agents
●
Pick the right time for airplane maintenance
Slide 9
Slide 9 text
Machine learning requires many capabilities
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Feature engineering
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Business value
Compliance and Risk
Integration Process
Domain knowledge
Slide 10
Slide 10 text
Machine learning requires many capabilities
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Feature engineering
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Business value
Compliance and Risk
Integration Process
Domain knowledge
Slide 11
Slide 11 text
Machine learning requires many capabilities
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Programming
CI/CD
Feature engineering
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Business value
Compliance and Risk
Integration Process
Domain knowledge
Slide 12
Slide 12 text
Machine learning requires many capabilities
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Feature engineering
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Statistics
Algorithms
Math
Scripting
Monitoring
Stability
Reliability
Infrastructure
Business value
Compliance and Risk
Integration Process
Domain knowledge
Slide 13
Slide 13 text
Machine learning requires many capabilities
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Feature engineering
Data plumbing
Monitoring
Stability
Reliability
Infrastructure
Statistics
Algorithms
Math
Scripting Programming
CI/CD
Data plumbing
Business value
Compliance and Risk
Integration Process
Domain knowledge
Slide 14
Slide 14 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 15
Slide 15 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 16
Slide 16 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 17
Slide 17 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 18
Slide 18 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 19
Slide 19 text
Expertise oriented teams do not work
Data lab
Product team
Data engineering
Operations
Slide 20
Slide 20 text
Full stack teams don’t scale
Feature team
Slide 21
Slide 21 text
Full stack teams don’t scale
Slide 22
Slide 22 text
Full stack teams don’t scale
Slide 23
Slide 23 text
Centralized teams should handle cross-cutting concerns
Feature teams
Centralized team
Slide 24
Slide 24 text
The machine learning platform
Slide 25
Slide 25 text
The machine learning platform
● The platform is a service built by ING engineers, for ING engineers and data scientists
● The platform is responsible for hosting machine learning models, and orchestrating them
Slide 26
Slide 26 text
The machine learning platform
Slide 27
Slide 27 text
The machine learning platform
● Orchestrator as an
interface between
models and outside
world
● Kafka for streaming
data
● REST API for real-time
and batch
Slide 28
Slide 28 text
● Allow teams to fully embrace the power and capabilities
of machine learning
● Only software engineers, data scientists and product
owners are responsible
The machine learning platform
Slide 29
Slide 29 text
A journey: the feedback use case
● ING gathers user feedback
● Feedback covers many categories
Slide 30
Slide 30 text
● Automatically categorize user feedback using
machine learning
● Establish business goals
Inception of the idea
Product owner
Slide 31
Slide 31 text
● Data scientists iteratively build a model
● Incorporate feedback from product owner
● Configure the model according to platform standards
Building the model
Data scientist
Slide 32
Slide 32 text
● Data scientist and software engineer agree on configuration
of the model
● Software engineer builds interface for consuming the platform
● Data scientist deploys model on pipeline
Packaging the model
Software engineer
& data scientist
Slide 33
Slide 33 text
● Automated pipelines deploy the model on the platform
● Technologies such as Ansible, Docker and Gitlab CI
● No actions are required by the platform team
The packaged model
Platform
Slide 34
Slide 34 text
The productionized model
● Once in production, data scientists,
software engineers and product
owners can monitor their model
● The model can be changed iteratively
and quickly deployed again
Slide 35
Slide 35 text
The productionized model, monitoring
Slide 36
Slide 36 text
The productionized model, versioning
● Models are tracked by their
version number
● Using historic data we can track
the performance of the models
Slide 37
Slide 37 text
Centralized teams should handle cross-cutting concerns
ING’s platform team provides machine learning
deployment as a service
Allowing data scientists to easily and quickly deploy
models makes machine learning boring
Key takeaways