Coding For Cloud

How to design and develop a cloud-native machine learning application in Python

Transcript

1. Coding For Cloud Harjindersingh Mistry Bargava Subramanian Red Hat
2. Grace • Smart individual • Just finished MBA Mark • A geek • Just completed engineering
3. Grace & Mark Online Store
4. Hey Mark, We don’t have enough money to buy our own servers ! So, can we rent some ? That sounds like a good idea, Grace ! That’s called Cloud Computing. I remember what Prof told about it ...
5. Cloud Computing • Computers on rent • Some guy has lot of computer infrastructure ◦ He wants to give it on rent to make money • Very similar to car-rental service like Europcar • Cloud-native Application is a program that is designed and developed specifically for cloud computing architecture
6. Alright, I have finished the implementation ! Cloud, Take it and go !
7. Hey dude, customers are reporting that our service is very slow :( Please check what is wrong there. Oh! is it ? But I had used best algo! Ahha, there seems to be a scalability issue. Let me call my Prof and get his suggestions ...
8. Horizontal Scalability • Think of data in terms of standard ‘collections’ • Write functional as much as possible • Let the data flow through functions: ◦ If data is static then we can do Parallel Processing ◦ If data size is not fully known then we can be Reactive ◦ There is also a possibility of mixing them together by using Stream Processing
9. Alright, I have done cool clustering and my architecture is reactive now! Cloud, …. take it and go !
10. Wow Mark, the customers seem to be happy with performance now :) But one customer has reported a bug ! Can you please fix it ? Oh! is it ? Let me check that. It might be something related to “data-access” module.
11. Ok ! So, I have fixed the “data-access” module. Cloud, …. take it and go one more time !
12. Hey dude, what are your programming ?!? There is another bug reported :( Oh! is it ? Let me check that.
13. Oh man, this debugging is taking forever :( I need to configure the whole system to repro the scenario ! ... Finally, I got it !!! Bug is inside “business-logic” module. I fixed it. Let me test the whole app. Come on Mark ! How much more time do you need ?
14. Ok ! So, the tests passed. Cloud, …. take it and go one more time ! Wait, now I remember what Prof said about Micro Services ...
15. Micro Services • Divide the whole application into smaller services, called Micro Services • It gives flexibility in testing and deploying in small independent chunks • This improves maintainability a lot
16. Ok ! So, I have done Micro Services :) Cloud, …. take it and go !
17. Awesome Mark ! We are doing good ! How about adding one new feature into our store ! Oh yes ! That will be fun :) It has been while since I coded something meaningful !
18. Alright ! The new feature is ready ! But I somehow don’t like all micro services sharing same computing resources :( Let me talk to Prof ...
19. Containers • Deploy services by using Containers • Containers increase speed of deployment • Developer can simulate production environment locally • Moreover, each service runs in its own environment ◦ So if one service goes down, others are still safe
20. So, now I have each micro service containerized separately ! That looks good. Cloud, take it and go !
21. Crux of the Story • Current business model demands speed ◦ Need to experiment: It’s ok to fail, but fail fast ◦ Need the ability to react: Everything cannot be planned • But, the software has to be good in terms of: ◦ Correctness ◦ Performance ◦ Maintainability ◦ Operability
22. Cloud Native Application • It helps us experiment with an idea • We can achieve those attributes of good software as follows: ◦ Correctness: Test cases. This cannot change :) ◦ Performance: Horizontal Scalability ◦ Maintainability: Micro Services ◦ Operability: Containers
23. Cloud Native Application: Example • Movie Recommendation Service • Cloud Native Machine Learning Application in Python • There are two parts: ◦ Offline Training Job: ◦ Online Scoring Service: Movie Data Training Job ML Model Cloud Storage User Input + Model Scoring Service Recommendations
24. Layers of Cloud Native Application Deployment Output Channel Functionality Data Access S3 Local Files ...
25. Functionality Layer • Contains core business logic • As much ‘functional’ as possible i.e. immutability and statelessness • Data flows through pipeline of functions: • For example: Input Data Function-1 ... Function-N Output Result Input Data Read/Split Transform Train ML Model
26. Data Access Layer • Provides abstraction on various data sources • Helps in testing the core logic locally Abstract Class / Interface S3 Local Files ...
27. Output Channel Layer • Different channels for different consumers • For micro service, it will have REST component Functionality CLI REST ...
28. Deployment Layer • Contains the scripts for deployments ◦ Docker files and/or YAML files for containers ◦ Python scripts with calls to vendor-specific APIs e.g. AWS API run_job_flow()
29. Demo • Functionality Layer: Training • Functionality Layer: Scoring • Deployment Layer: OpenShift Input Data Read/Split Transform Train ML Model User Input + Model Scoring Service Recommendations
30. Not Covered • Continuous Integration / Continuous Delivery • Security • Monitoring • Vendor APIs for deployment
31. Code, Slides and Contact • Code: ◦ https://github.com/harjinder-hari/cloud_native_analytics • Slides: ◦ https://speakerdeck.com/harjinderhari/coding-for-cloud • Contact: ◦ Harjinder email: harjinder.v2@gmail.com ◦ Bargava twitter: @bargava
32. Conclusion • Current business model demands speed ◦ Need to experiment: It’s ok to fail, but fail fast ◦ Need the ability to react: Everything cannot be planned • Cloud Native Applications is a way to achieve it ◦ We saw ▪ Layers involved in a cloud native application ▪ Techniques to achieve the required speed ▪ Everything is written in Python ! :)
33. Images Courtesy [1] Grace and Mark https://image.freepik.com/free-vector/happy-university-students_23-2147531065.jpg [2] Professor https://image.freepik.com/free-vector/science-teacher_23-2147509483.jpg [3] Rocket https://image.freepik.com/free-vector/launching-business-concept_23-2147504957.jpg