Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Machine Learning on Production
Search
Eko Kurniawan Khannedy
March 18, 2016
Technology
140
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Machine Learning on Production
Machine Learning on Production
Eko Kurniawan Khannedy
March 18, 2016
Transcript
27 slides
MACHINE LEARNING ON PRODUCTION EKO KURNIAWAN KHANNEDY
MACHINE LEARNING ON PRODUCTION EKO KURNIAWAN KHANNEDY ▸ Principal Software
Development Engineer at blibli.com ▸ Part of Research and Development Team ▸
[email protected]
HAL YANG PALING SULIT ITU ADALAH MEMBAWA MACHINE LEARNING KE
PRODUCTION …. MACHINE LEARNING ON PRODUCTION
MACHINE LEARNING ON PRODUCTION AGENDA ▸ The Hard Part ▸
Best Practice ▸ Machine Learning in blibli.com
THE HARD PART MACHINE LEARNING ON PRODUCTION
MACHINE LEARNING ON PRODUCTION DATA ▸ Data Too Big ▸
Unstructured Data ▸ Document Oriented and Master Detail Data ▸ Continuous Data ▸ Imbalance Data ▸ Wild Data
MACHINE LEARNING ON PRODUCTION PREPROCESSING ▸ Feature Extraction ▸ Too
Many Features Extraction Makes Process Too Long
MACHINE LEARNING ON PRODUCTION TRAINING ▸ Batch Training ▸ Sequential
Algorithm ▸ Validation
BEST PRACTICE MACHINE LEARNING ON PRODUCTION
DATA
MACHINE LEARNING ON PRODUCTION DATA TOO BIG ▸ Load data
to memory. ▸ Streaming the datasource. ▸ Split data into multiple nodes. ▸ Use memory-file database.
MACHINE LEARNING ON PRODUCTION UNSTRUCTURED DATA ▸ Analyse Your Data
▸ Find Characteristic of Your Data ▸ Find Best Approachment for that case.
MACHINE LEARNING ON PRODUCTION DOCUMENT ORIENTED AND MASTER DETAIL DATA
▸ Analyse Your Data ▸ Find the Best Way to Treat The Data
MACHINE LEARNING ON PRODUCTION CONTINUOUS DATA ▸ Wide the range
that use in normalization process. ▸ Consider it as a missing value.
MACHINE LEARNING ON PRODUCTION IMBALANCE DATA ▸ Down Sampling. ▸
Up Sampling.
MACHINE LEARNING ON PRODUCTION WILD DATA ▸ Use Default Value.
▸ Use Average Value. ▸ Use Machine Learning to Predict Missing Value.
PREPROCESSING
MACHINE LEARNING ON PRODUCTION FEATURE EXTRACTION ▸ Add as Many
Facts as Possible ▸ Remove Irrelevant Feature
MACHINE LEARNING ON PRODUCTION TOO MANY FEATURES EXTRACTION MAKES PROCESS
TOO LONG ▸ Use Non-Blocking Process ▸ Use Event Driven Process ▸ Use Parallel Process
TRAINING
MACHINE LEARNING ON PRODUCTION BATCH TRAINING ▸ Use Real Time
Training ▸ Scheduled Training
MACHINE LEARNING ON PRODUCTION SEQUENTIAL ALGORITHM ▸ Distributed The Data
▸ Parallel The Algorithm
MACHINE LEARNING ON PRODUCTION VALIDATION ▸ Split Validation ▸ Cross
Validation ▸ Parallel The Validation
MACHINE LEARNING IN BLIBLI.COM MACHINE LEARNING ON PRODUCTION
MACHINE LEARNING ON PRODUCTION FRAUD PREVENTION PLATFORM RESTFULL MASTER DATA
CLIENT MACHINE LEARNING ENGINE PREPROCESSING ENGINE THIRD PARTY SERVICE
MACHINE LEARNING ON PRODUCTION MACHINE LEARNING ENGINE RESTFULL METADATA DATA
CLIENT TRAINING ENGINE TRAINING DATA CLASSIFICATION ENGINE
THANKS
More Decks by Eko Kurniawan Khannedy
See All by Eko Kurniawan Khannedy
Monolith to Event-Driven Microservices
khannedy
1
290
Refactoring
khannedy
0
390
Multi-Datacenter Kafka at Blibli.com
khannedy
2
1.6k
QA Tools - Research and Development
khannedy
0
330
Reactive Puzzle
khannedy
0
230
Event-Driven Architecture
khannedy
1
2.1k
Resilience Engineering with Hystrix and Spring
khannedy
1
610
Mocking for Unit Test using Mockito
khannedy
1
370
Centralized Configuration using Consul and Spring Cloud
khannedy
2
760
Other Decks in Technology
See All in Technology
MCPでネットワーク障害調査を自動化する前に直すべき3つのこと
yama3133
1
660
Go でトランザクション、どう書いてますか?
knwoop
1
580
Amazon Bedrock AgentCore の基礎とセキュリティ・ガバナンス
ren8k
0
270
AIエージェントのアーキテクチャ設計
nsatoh
0
540
ノードが消える前提で設計する!Karpenterで金融システムを止めないためのノウハウ大全
daitak
1
250
[DBRE Summit 26] DBRE はデータベース運用をどう変えたのか
_awache
0
130
Kiro のリモートワークスペース: IDE は手元のままで、ワークスペースをクラウド、コンテナ、自前のマシンに置く
inariku
PRO
1
170
そのイメージ、どのCIが作った? Artifact Attestationsで来歴を固定する / Which CI system created that image? Pinning provenance with Artifact Attestations
chmikata
0
270
AWS Well-Arhitected Frameworkセキュリティの柱 について
miu_crescent
PRO
2
250
現場に渡し、構造で守る AI エージェントを活用したセルフサーブのデータ基盤の運用
legalontechnologies
PRO
0
110
Terraform 0 to 100:100個以上のリポジトリを常に最新にし続けるための自動化
sue445
3
1.4k
Is the Website Dying?
miriamschwab
0
160
Featured
See All Featured
WCS-LA-2024
lcolladotor
0
1k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
21
1.7k
技術選定の審美眼(2025年版) / Understanding the Spiral of Technologies 2025 edition
twada
PRO
121
130k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
davidcarrasco
1
320
Thoughts on Productivity
jonyablonski
76
5.5k
Mobile First: as difficult as doing things right
swwweet
225
10k
The Pragmatic Product Professional
lauravandoore
37
7.5k
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
techseoconnect
PRO
0
500
Making Projects Easy
brettharned
120
6.9k
New Earth Scene 8
popppiees
5
2.9k
Game over? The fight for quality and originality in the time of robots
wayneb77
1
340
Raft: Consensus for Rubyists
vanstee
143
7.9k