Machine Learning on Production

Machine Learning on Production

Transcript

1. MACHINE LEARNING ON PRODUCTION EKO KURNIAWAN KHANNEDY
2. MACHINE LEARNING ON PRODUCTION EKO KURNIAWAN KHANNEDY ▸ Principal Software Development Engineer at blibli.com ▸ Part of Research and Development Team ▸ eko.k.khannedy@gdn-commerce.com
3. HAL YANG PALING SULIT ITU ADALAH MEMBAWA MACHINE LEARNING KE PRODUCTION …. MACHINE LEARNING ON PRODUCTION
4. MACHINE LEARNING ON PRODUCTION AGENDA ▸ The Hard Part ▸ Best Practice ▸ Machine Learning in blibli.com
5. THE HARD PART MACHINE LEARNING ON PRODUCTION
6. MACHINE LEARNING ON PRODUCTION DATA ▸ Data Too Big ▸ Unstructured Data ▸ Document Oriented and Master Detail Data ▸ Continuous Data ▸ Imbalance Data ▸ Wild Data
7. MACHINE LEARNING ON PRODUCTION PREPROCESSING ▸ Feature Extraction ▸ Too Many Features Extraction Makes Process Too Long
8. MACHINE LEARNING ON PRODUCTION TRAINING ▸ Batch Training ▸ Sequential Algorithm ▸ Validation
9. BEST PRACTICE MACHINE LEARNING ON PRODUCTION
10. DATA
11. MACHINE LEARNING ON PRODUCTION DATA TOO BIG ▸ Load data to memory. ▸ Streaming the datasource. ▸ Split data into multiple nodes. ▸ Use memory-file database.
12. MACHINE LEARNING ON PRODUCTION UNSTRUCTURED DATA ▸ Analyse Your Data ▸ Find Characteristic of Your Data ▸ Find Best Approachment for that case.
13. MACHINE LEARNING ON PRODUCTION DOCUMENT ORIENTED AND MASTER DETAIL DATA ▸ Analyse Your Data ▸ Find the Best Way to Treat The Data
14. MACHINE LEARNING ON PRODUCTION CONTINUOUS DATA ▸ Wide the range that use in normalization process. ▸ Consider it as a missing value.
15. MACHINE LEARNING ON PRODUCTION IMBALANCE DATA ▸ Down Sampling. ▸ Up Sampling.
16. MACHINE LEARNING ON PRODUCTION WILD DATA ▸ Use Default Value. ▸ Use Average Value. ▸ Use Machine Learning to Predict Missing Value.
17. PREPROCESSING
18. MACHINE LEARNING ON PRODUCTION FEATURE EXTRACTION ▸ Add as Many Facts as Possible ▸ Remove Irrelevant Feature
19. MACHINE LEARNING ON PRODUCTION TOO MANY FEATURES EXTRACTION MAKES PROCESS TOO LONG ▸ Use Non-Blocking Process ▸ Use Event Driven Process ▸ Use Parallel Process
20. TRAINING
21. MACHINE LEARNING ON PRODUCTION BATCH TRAINING ▸ Use Real Time Training ▸ Scheduled Training
22. MACHINE LEARNING ON PRODUCTION SEQUENTIAL ALGORITHM ▸ Distributed The Data ▸ Parallel The Algorithm
23. MACHINE LEARNING ON PRODUCTION VALIDATION ▸ Split Validation ▸ Cross Validation ▸ Parallel The Validation
24. MACHINE LEARNING IN BLIBLI.COM MACHINE LEARNING ON PRODUCTION
25. MACHINE LEARNING ON PRODUCTION FRAUD PREVENTION PLATFORM RESTFULL MASTER DATA CLIENT MACHINE LEARNING ENGINE PREPROCESSING ENGINE THIRD PARTY
 SERVICE
26. MACHINE LEARNING ON PRODUCTION MACHINE LEARNING ENGINE RESTFULL METADATA
 DATA CLIENT TRAINING
 ENGINE TRAINING
 DATA CLASSIFICATION
 ENGINE
27. THANKS