experience_presentation.pdf

Transcript

1. Pavel Yudin Experience
2. 2. Experience • ML (since 2014) • Recommender systems, ranking, search, ML infra, NLP, CV • Metric pushing / Infra = 60/40 • VK, Meta, Snap, Microsoft, Cambridge • System engineer (since 2011) • Compilers, programming languages, networking, backend • Education: • 2 Masters (CS, Cambridge + top 1 Russian tech uni.) • compilers / programming languages • Bachelor • CS/math/physics – equal split
3. 3. VK, Director of Machine Learning • AI Org • Leading a department to build the ML platform layer for the entire company • Dual role – both managerial and technical • Customers • Main: Ads, Search, Recommendations, etc • Efforts: • 16+ main platforms + specialized ones later • Status: • 4 -> 1.0+ version • Inference, Notebook platform, ML training platform, Model Store • 1 alpha version: • Unified ranking platform • 2 design phase: • Feature Engineering platform, Feature Store • 2 helped to design: • FeatureFlow, StreamFlow
4. 4. Meta, E6 Software engineer • Meta business suite (Business Identity Recommendations team) • Leading team efforts to build new and maintain existing ML systems to drive user engagement • Co-steered (with manager) the hiring, improved processes. • Product (Meta Business Suite) • Directions: • Business Feed • Feed, Categories, Pages you might follow panel • Platform/Infra efforts • HPO, FS, Slice & data analysis, Data lineage, Logging, Monitoring, alerting • Notifications • ML Privacy • Ad hoc projects: • Marketing efforts, Ads/Revenue
5. 5. Meta
6. 6. Meta
7. 7. Snap • Monetization (Ad ranking) team • Ad ranking quality improvements to drive revenue / decrease cost of serving / improve ad score • Product (Snap Ads) • Launched projects: • Multiple ranking model improvements • Novel offline validation technique • AutoML service (leading 3 senior eng) • Auto HPO, AutoFS, AutoFIS, NAS, Snapshot averaging
8. 8. Microsoft • Bing Maps (local) team • Coming up with novel ideas -> moving them into production • Local search quality improvements (multiple markets) • Products (Bing Maps, Bing Images) • Launched projects: • LiveMaps (best short paper award, SIGIR, 17: link, leading 2 eng + labeling team/10) • ML proj for data cleaning • Several ML ranking models for Bing Maps (retrieval + heavy layer + selection stage) • New architecture for BingGC reducer
9. 9. Microsoft LiveMaps (with Bing Images) Bing Maps
10. 10. LLMs experience • VK • Serving 20+ different LLMs on inference platform • Copilot supported on Notebook platform • Helped R&D department with infra / cluster management / training • Broad understanding about all aspects related to LLMs: • Applied part: • LLMs architecture, Building an instruction dataset, Pre-training models, supervised fine tuning, RLHF, Evaluation, Quantization, New Trends, • Engineering/Infra part: • Running LLMs, Building a vector storage, Retrieval Augmented Generation, Advanced RAG, Inference optimization, Deploying LLMs, Securing LLMs
11. VK, Director of Machine Learning • Dual role • Managerial and technical • Hiring/Firing/Perf eval + arch design • ~27 people in my team + ~10 v-team + 8 open HC • Senior+ engineers mostly • Goal • Build ML platform layer for the company • Multiyear goal • ~16 main platforms
12. Broad picture
13. Team allocation & data flow
14. High level results • Inference platform • ~100 K QPS • ~400 models, • ~500 services, • ~2000 GPUs • ML training platform • ~50 prod pipelines • Notebook platform • WAU – ~200 users • Unified ranking platform • ~50 models • Model store • ~400 models
15. My Role • Architectural • logical + physical architecture • Roadmap creation / tracking • Team management • Hiring / firing / coaching / perf evaluation