Implementing ML Research Papers: Practical Strategies for Bringing Research into Product Application

Integrating ML research into real-world product applications can be challenging. However, it is crucial for staying at the forefront of innovation. This session focuses on equipping ML practitioners and product developers with practical strategies to effectively implement ML research papers in product development cycles.

I will practically cover 6 key concepts with a case study:
1. Understanding the relevance and impact of ML research papers in product development
2. Techniques for effectively reading, interpreting, and extracting actionable insights from ML research papers
3. Strategies for integrating research-based algorithms into product applications
4. Overcoming challenges in adapting ML research findings to real-world scenarios (data, scalability, performance, etc.)
5. Best practices for rigorous experimentation, evaluation, and validation of research-driven product solutions
6. How ML and software product meets.
By the end of the session, participants will be equipped with a practical framework that guides them through the process of implementing ML research papers..

Transcript

1. Building and Enlightening Data Professionals in Africa. An annual conference for all data practitioners in Africa. #DataFestAfrica23 #DFA23
2. Implementing ML Research Papers: Practical Strategies for Bringing Research into Product Application Sam Ayo @officialsamayo
3. Meet Sam Ayo - AI/ML Engineer, Data Scientist and Head of Engineering - Extensive experience in software, data, and AI consulting - Providing insights into AI's transformative power Sam Ayo @officialsamayo
4. Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clark
5. Agenda • Selecting Research Papers for Implementation • Effective ML Research Paper Interpretation • Strategies for bringing research-based ML into product apps
6. Selecting Research Papers for Implementation
7. ML research papers; Getting started Machine Learning (ML) research papers are scholarly documents that present novel findings, developments, and insights related to the field of machine learning.
8. The relevance and impact of ML research papers in product apps • Convey important breakthroughs • Share research work • Introduce novel practices • Help academia and industries implement in real-time • Build your technical portfolio • Advance Technological Growth in industries
9. Criteria for Choosing Research Papers • Is it built on any of the AI milestones; e.g transformers, CNN? • Was it written by reputable authors or reputable orgs/teams? • How many citations? • What year of publication? • Is it relevant to your Problem? • Is it built using any of the community frameworks? e.g. pytorch, tensorflow, jax?
10. Identify Papers with Practical Applicability • What is it benchmarked against? • What metric was used to evaluate the performance? • Ensure it aligns with business goals and user needs
11. Most Research papers introduce complex methodologies that do not fit products in reality.
12. Effective ML Research Paper Interpretation
13. What do I need to know? • Paper Structure Understanding • Abstract and Introduction • Related Work • Discussion and Conclusion • Experiments and Results • Citations and References
14. Interpreting ML Papers Like most papers, this section elaborates more on the topic, including a review of other related works, the datasets used, the research problem, and a description of the proposed solution invented/innovated. Abstract and Introduction
15. This section is very crucial because that’s exactly where you make a decision of how relevant/significant the research is and if it highlights any novelty. Interpreting ML Papers Discussion and Conclusion
16. Interpreting ML Papers This is where it gets exciting. It highlights the model architecture specifications e.g number of neuron layers, dropouts, regularizers etc. Experiments and Results
17. Take detailed notes as you read and interpret the paper’s model architecture.
18. Strategies for bringing research-based ML into product apps
19. Bringing ML into production environment no be beans.
20. The Critical Questions • How will the Predictions be served? • How will the model be served? • What Stacks do I need? • How should model experiments design be done? • How will ML meet the software system?
21. Integrating ML? • Batch inference • Real-time inference • Streaming inference • Edge inference 1. Serving model Predictions
22. Integrating ML? • One-to-one • One-to-all 2. Serving model weights
23. Integrating ML? Experimentation is at the heart of the Machine Learning profession. We progress because we experiment and it begins in a notebook. 3. Design model experiments
24. Design model experiments A ML notebook should be: • headlined, • descriptive and • documented sectionally.
25. Design model experiments A ML notebook should be: • headlined, • descriptive and • documented sectionally.
26. Design model experiments A ML notebook should be: • headlined, • descriptive and • documented sectionally.
27. Let’s look at it together https://github.com/Stosan/heart-failure- prediction-dfa2023
28. Integrating ML? You need a set of tools to check for everything throughout the ML system architecture and lifecycle. 4. Determine the Stacks
29. Determine the stack Feature Store CICD Containers Model Registry Model Server Performance monitoring Experiment Tracking ML Tools for the entire lifecycle
30. Model Server with FastAPI A lightweight API for house the model and model metadata attributes, handle inference requests and return predictions. It would typically include self-contained functions for data processing, dynamic artifact loading, training and prediction logic.
31. Model Server with FastAPI Dockerize the model server and deploy to a cluster with appropriate network overlays, authentication protocols, and application routes. Containerizing with Docker
32. Let’s look at it together https://github.com/Stosan/fastapi-model-serving
33. System software
34. Integrating ML? • Monolithic integration • single service integration • Microservice integration 5. Model meets the software system
35. Monolithic integration Single service integration Microservice integration The ML service code base is integrated within the rest of the backend code base. The ML service code base is deployed on a single server, with elastic load balancing for scaling. The ML service code base is deployed such that components get their own services. The entire system process is slowed down by the ML service, the model size and computation requirements usually add additional load on the backend servers. Usually considered if the inference process is very light to run. The model size can be complex without putting load pressure on the rest of the infrastructure. This is typically the easiest way to deploy a model while ensuring scalability, maintainability and reliability. This is a relief system for the entire codebase. It ensures the different components of the ML system can be reused for different purposes. For example, the ML inference manager at RadioAdSpread. www.radioadspread.com
36. Python is not enough!
37. Python is not enough!
38. Challenges in adapting ML research to product apps • The cost of wrong predictions is cheap • This is a lot to keep track of! • Model build language often differs from legacy codebase • ML inference must be asynchronous • low latency(p99)
39. Challenges with ML in production There are three typical issues that degrade the performance of deployed ML models. • Concept drift • Locality • Data quality
40. Machine learning engineering is very experimental, even in production.
41. THANK YOU! Sam Ayo @officialsamayo