Model as a Service up and running in AWS

Presentation made at Papis'17 (https://www.papis.io/2017/program/talks/)

https://www.papis.io/2017/program/talks-list/model-as-a-service-up-and-running-in-aws-lia-bifano

Transcript

1. Model as a Service up and running in AWS https://github.com/liabifano/ml-aws
2. whoami - I’m Lia Bifano and I’m a statistician - - -
3. whoami - I’m Lia Bifano and I’m a statistician - I’m from Brazil! - -
4. whoami - I’m Lia Bifano and I’m a statistician - I’m from Brazil! - Former Data scientist at -
5. whoami - I’m Lia Bifano and I’m a statistician - I’m from Brazil! - Former Data scientist at - Faculté Informatique et Communications Sciences at
6. Objectives - Share experience regarding deploy online models -
7. Objectives - Share experience regarding deploy online models - Show the main steps to deploy your own model in AWS
8. Objectives - Share experience regarding deploy online models - Show the main steps to deploy your own model in AWS Step-by-step
9. - Now, we have to deploy it.. - Sure! Here it is my Jupyter Notebook..
10. Inputs: do it before something happens DB of Addresses User You Model App time
11. Inputs: do it before something happens DB of Addresses User You Model App time insert into address (id, address) values (1, `Av1`) (1, Av1) select address from address where id=1 output=150 ok
12. Inputs: do it before something happens DB of Addresses User You Model App time ok insert into address (id, address) values (1, `Av1`) update address set address=`Av2` where id=1 (1, Av1) select address from address where id=1 select address from address where id=1 (1, Av2) output=150 ok
13. Inputs: do it before something happens DB of Addresses User You Model App time ok insert into address (id, address) values (1, `Av1`) update address set address=`Av2` where id=1 (1, Av1) select address from address where id=1 select address from address where id=1 (1, Av2) select output from model where id=1 (1, 150) output=150 and not 200.. ok Datomic was helpful, but ...
14. Schema Evolution (for retrain) - More relevant to inputs than outputs but can be applied as well
15. Schema Evolution (for retrain) - More relevant to inputs than outputs but can be applied as well Choices and Consequences - Store in a single table “schema-on-write” database and do migrations (slow down the retrain process) - -
16. Schema Evolution (for retrain) - More relevant to inputs than outputs but can be applied as well Choices and Consequences - Store in a single table “schema-on-write” database and do migrations (slow down the retrain process) - Store in a “schema-on-read” database and build an ETL to get information later -
17. Schema Evolution (for retrain) - More relevant to inputs than outputs but can be applied as well Choices and Consequences - Store in a single table “schema-on-write” database and do migrations (slow down the retrain process) - Store in a “schema-on-read” database and build an ETL to get information later - Store in different tables and deal with multiple tables
18. General Good Practices - Check schema and enforce types just before running the model
19. Int and Float to datetime At least it throws a error...
20. General Good Practices - Check schema and enforce types just before run the model - Predictions should be pure The function result value cannot depend on any hidden information or state and also don’t cause any side effect
21. age = today() - date_of_birth age = request_timestamp - date_of_birth
22. General Good Practices - Check schema and enforce types just before run the model - Predictions should be pure The function result value cannot depend on any hidden information or state and also don’t cause any side effect - Tests
23. Machine Learning Model
24. Machine Learning Model Go to model code
25. But it works on my machine! - Docker containers puts your machine in production - “It is a way to package software in a format that can run isolated on a shared operational system” - Docker’s documentation - Might “agnostic” to the programming language as long it respects the contracts - - - -
26. But it works on my machine! - Docker containers puts your machine in production - “It is a way to package software in a format that can run isolated on a shared operational system” - Docker’s documentation - Might “agnostic” to the programming language as long it respects the contracts - How? - Write a recipe with all dependencies, called Dockerfile - -
27. But it works on my machine! - Docker containers puts your machine in production - “It is a way to package software in a format that can run isolated on a shared operational system” - Docker’s documentation - Might “agnostic” to the programming language as long it respects the contracts - How? - Write a recipe with all dependencies, called Dockerfile - Build a image based on the Dockerfile (inert and immutable) -
28. But it works on my machine! - Docker containers puts your machine in production - “It is a way to package software in a format that can run isolated on a shared operational system” - Docker’s documentation - Might “agnostic” to the programming language as long it respects the contracts - How? - Write a recipe with all dependencies, called Dockerfile - Build a image based on the Dockerfile (inert and immutable) - Create an instance of the image, called docker container
29. But it works on my machine! - Docker containers puts your machine in production - “It is a way to package software in a format that can run isolated on a shared operational system” - Docker’s documentation - Might “agnostic” to the programming language as long it respects the contracts - How? - Write a recipe with all dependencies, called Dockerfile - Build a image based on the Dockerfile (inert and immutable) - Create an instance of the image, called docker container If the image is a class, then the container would be a running time object… Go to Dockerfile
30. What can I do in AWS? - Offers a set of cloud computing services to run web application - Buy machine to store, compute and run tasks or applications - Interaction through console or clients (boto, JuliaCloud, aws.signature, awscli)
31. Brief overview AWS - EC2 (Elastic Compute Cloud) - “Machines” with preconfigured template that are able to run tasks / services with docker containers - Allows choose optimized machines - - - -
32. Brief overview AWS - EC2 (Elastic Compute Cloud) - “Machines” with preconfigured template that are able to run tasks / services with docker containers - Allows choose optimized machines - ECS (EC2 Container Services) - Manage cluster docker containers (recovery and scheduling, network distribution, autoscaling) - -
33. Brief overview AWS - EC2 (Elastic Compute Cloud) - “Machines” with preconfigured template that are able to run tasks / services with docker containers - Allows choose optimized machines - ECS (EC2 Container Services) - Manage cluster docker containers (recovery and scheduling, network distribution, autoscaling) - ECR (EC2 Container Registry) - Store docker images to run containers inside EC2 instances (DockerHub and Quay.io)
34. Brief overview AWS - RDS (Amazon Relational Database Service) - Managed Relational Databases (MySQL, PostgreSQL, AWS Aurora, among others) - -
35. Brief overview AWS - RDS (Amazon Relational Database Service) - Managed Relational Databases (MySQL, PostgreSQL, AWS Aurora, among others) - Cloudformation - Recipe about how your stack (set of resources) is configured
36. Stack Overview - Scale of each model’s version independently - Let the consumer choose which version is required Go to cfs/scripts
37. To summarize... - Save features / inputs is important and how to deal with it - How to pack your dependencies, docker overview - Overview of AWS - How to deploy each model’s version in a different stack
38. Thanks for listening! I’m @liavbifano Check it out: https://github.com/liabifano/ml-aws Huge thanks to: PAPIs organizers Nubank team and friends SafariBooks