AI Company

AI Company -- Upstage

Transcript

1. AI Company Upstage Upstudy Hyperconnect Director of AI Sungjoo Ha June 17th, 2022 Sungjoo Ha 1
2. AI Company • য়ט੄ ઱ઁ: AI company • ੋఠ֔ ӝࣿਸ ੜ ഝਊೠ ӝস਷ internet companyۄ ࠛ۷Ҋ ੉ח ݽ߄ੌ द؀ө૑ ੉য઎׮ • Amazon, Alphabet, Facebook, Alibaba, Tencent, etc. • AI द؀੄ AI companyח ޖ঺੉ ׮ܳө? • ੿׹਷ ݽܰ૑݅ ѐੋ੸ਵ۽ ࢤп೮؍ ݻ о૑ ನੋ౟ী ؀ೠ ࣗѐ Sungjoo Ha 2
3. ࣳೝށ + ਢࢎ੉౟ ≠ Internet Company Sungjoo Ha 3
4. Jeff Bezos in 1997 In the book space, there are more than three million different books worldwide active and in print at any given time across all languages, so when you have that many items, you can literally build a store online that couldn't exist any other way. 1 1 https://youtu.be/rWRbTnE1PEM Sungjoo Ha 4
5. Aggregators • Zero marginal cost • ٣૑ణ ই੉మਸ ౸ݒೞחؘী ୶о ࠺ਊ੉ ٜ૑ ঋ਺ • Distributionী ࠺ਊ੉ ٜ૑ ঋ਺ • Transactionী ࠺ਊ੉ ٜ૑ ঋ਺ • ੉ܳ ӓೠਵ۽ ഝਊೠ Ѫ੉ ҳӖ੉ա ಕ੉झ࠘җ э਷ super-aggregator2 • ੋఠ֔ী ઓ੤ೞח Ѫ੉ value proposition਷ ইש • ੋఠ֔੉ ࢜۽੉ ઱ח о஖ܳ ߉ইٜ੉Ҋ ੋఠ֔੉ ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझܳ ݅ٝ 2 https://stratechery.com/concept/aggregation-theory/ Sungjoo Ha 5
6. Internet-Enabled Technology • Internet द؀੄ ӝࣿ • ੋఠ֔ द؀ী ਢಕ੉૑ח ׮ ੓૑݅ ੋఠ֔ਵ۽ ੋ೧ оמ೧૓ ӝࣿਸ ഝਊೞח ӝস਷ ઁೠ੸ • ਬ੷ ೯زী ؀ೠ ࣻ૘ਸ ߄ఔਵ۽ ࢎਊ੗ܳ ੉೧ೞח Ѫ • ೠ ߊ աইоࢲ A/B test ೞח Ѫ3 • 1֙ী ೠ ف ߣ ߓನೞ؍ Ѫ੉ • Continuous integration4, continuous deployment ١ਵ۽ ݒੌݒੌ ߓನ • ӓױ੸ਵ۽ ૣ਷ ੉ఠۨ੉࣌ ࢎ੉௿ਸ ٜ݅যղҊ product-market-fitਸ ଺ӝ ਤ೧ ࡅܰѱ ਑૒੉ח Ѫ੉ оמ೧૗ • ੉ܳ оמாೞח ઑ૒ ҳઑ 4 Martin Fowlerо ੘ࢿೠ 2006֙ Ӗীࢲ CI ਊযо ࢎਊ 3 Google਷ ੉޷ 2000֙ী A/B పझ౟ܳ दب Sungjoo Ha 6
7. Learnings From The Past • Internet company੄ ৘द۽ࠗఠ ࢤп೧ࠁח AI company • AIо ইפݶ ࢿ݀غ૑ ঋח ࠺ૉפझ • ӝઓী ੹ഃ ࠛоמೞ؍ Ѫ੉ оמ೧૑ח Ѫ • ઑӘ ؊ ևѱ ೧ࢳ೧ࠁݶ AI द؀ী оמ೧૓ ࢜۽਍ ӝࣿਸ ੜ ഝਊೞ ח ഥࢎ Sungjoo Ha 7
8. Algorithm + Compute + Data • ML ޙઁীࢲ ݏׯڰܻח ف о૑ ޙઁ • ݽ؛ ࢿמ੉ ೟णࣇীࢲ ծ਷ ҃਋ • underfitting: ݽ؛ ࣻਊ۱ ૐо • ݽ؛ ࢿמ੉ పझ౟ࣇীࢲ ծ਷ ҃਋ • overfitting: ؘ੉ఠ ୶о • അ؀੄ ML ӝࣿ਷ ݽ؛੄ ࣻਊ۱(capacity)ਸ ޖೠ੿ טܾ ࣻ ੓਺ • ੉ܳ جܻӝ ਤೠ ҅࢑ ੗ਗҗ • ୽࠙ೠ ন੄ ؘ੉ఠ݅ ੓ਵݶ ҅ࣘ ࢿמ੉ જই૗ • 2010֙؀ ୡ߈ ੉੹ীח ੉۠ ѐ֛੉ হ঻ਵݴ ୭Ӕ ٜয ੉۠ ൒ ܴ੉ ؊ ъചغҊ ੓਺ Sungjoo Ha 8
9. AI Strategy • ׮ܲ ೒ۨ੉যٜ਷ যڃ ࢤпਸ ೞҊ ੓ਸө? AI ੹ۚ੉ ަө? • ׮ٜ ৈ۞ оמࢿਸ ࠁݶࢲ ߬౴ೞҊ ੓਺ • Google: AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস • OpenAI: Ѣ؀ ݽ؛ী ؀ೠ ъೠ ޺਺ਸ ాೠ ௾ ߬౴ • Tesla: അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ ా೤ Sungjoo Ha 9
10. AI Strategy -- Google • AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস • Google Brain੉ա Deepmind৬ э਷ ୹઺ೠ োҳࣗ5 • TPUܳ ੌଧх஖ ઁ੘6 • ੹ࣁ҅੄ ݽٚ ؘ੉ఠܳ ൚ࣻ • ׮নೠ AI ੽Ӕ ߑߨਸ ನ౟ಫܻয়۽ ਍৔ • AI investment fund • AI M&A • Scaling, neuroscience, ... 6 2015֙ द੘ 5 GBח 2011֙ founding, DMח 2014֙ acquisition Sungjoo Ha 10
11. AI Strategy -- OpenAI Conviction that if you take the right stuff, which already exists, and apply and mix it with a lot of data and a lot of compute, that it will in fact work. And so that was the missing piece.7 • Scaling hypothesis੄ ௾ ߬౴ 7 https://www.youtube.com/watch?v=13CZPWmke6A&t=950s Sungjoo Ha 11
12. AI Strategy -- Tesla • അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ ా೤ • Data engine • Auto labeling8 • Shadow mode 8 https://youtu.be/j0z4FweCy4M?t=5157 Sungjoo Ha 12
13. ӝઓഥࢎ + AI/ML/DL ≠ AI Company Sungjoo Ha 13
14. Zero Marginal Content • AIо ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझח ޖ঺ੌ ө? • ݻ о૑ ൦౟ח ੓਺ • Zero marginal cost content creation • GPT-3 (copilot), DALL·E 2 • Sampling റ ࢎۈ੉ ࢶఖೞѱ ೞח Ѫ • Super-human decision making • AlphaGo, AlphaFold Sungjoo Ha 14
15. AI-Enabled Technology • ؀ӝসٜ੉ঠ ׮নೠ ߬౴ਸ ೡ ࣻ ੓૑݅ ੘਷ ഥࢎח যڌѱ? • AI द؀ীח ੋఠ֔ द؀੄ ൒ܴҗ ࠺तೞѱ AI ݽ؛਷ ׮ٜ ࢎਊೡ Ѫ • ৈӝীࢲ ب୹ػ 2ର੸ੋ ѐ֛/ӝࣿ/ޙചܳ ঴݃ա ੜ ഝਊೞջب ઺ਃೠ ನੋ౟о ؼ Ѫ • ݃஖ A/B పझ౟ܳ ೞח ഥࢎ৬ ইצ ഥࢎо ա׊Ҋ • CI/CDܳ ഝਊೞח ഥࢎ৬ ইפפ ഥࢎо ա׊ח Ѫ୊ۢ Sungjoo Ha 15
16. Learned Business Logic • ࠺ૉפझ ۽૒ -> ݽ؛ Үജ • ࠺ૉפझ ۽૒: A ੉ݶ B ੉׮ • ೐۽Ӓېݠٜ੉ ٜ݅যղח ௏٘੄ ݆਷ ࠗ࠙੉ ࠺ૉפझ ۽૒ • ੉ѱ ޤо ׮ؘܲ? ೐۽Ӓېݠо ೞܖݶ ٜ݅؍Ѧ ҡ൤ ML ॄࢲ ೠ׳ Ѧ۰ࢲ ݅٘ח Ѣ ইפঠ? • ࢎۈࠁ׮ ੜ ೡ ࣻ ੓ח ҃਋о ੓਺ • A੉ݶ B੉׮ীࢲ A੄ ઑѤ੉ ցޖցޖ ࠂ੟೮׮ݶ ࢎۈ਷ ޅೣ • Software 2.0 Sungjoo Ha 16
17. Software Rot • ࠺ૉפझ ۽૒ب ࣗ೐౟ਝযب ॅ਺ • ઱߸ ജ҃੉ ߄Շӝ ٸޙ • ࢜۽਍ ӝמ੉ ࢤӝҊ, ઁಿਸ ੉Ցয оҊ੗ ೞח ߑೱ੉ ߄ՇҊ, ࢎਊ੗о ߄ՇҊ... Ӓېࢲ ॅ਺ • ࠁా ੉Ѧ যڌѱ ؀਽ೞջ? ࣗ೐౟ਝয ূ૑פযо ௏٘ܳ ࣻ੿ • A੉ݶ B ೮חؘ ੉ઁח A੉ݶ C ೧ঠ ೠ׮ • Ӕؘ ੉ѱ ݽ؛੉঻׮ݶ • ૑ࣘ੸ਵ۽ ؘ੉ఠ ࣻ૘ೞҊ ੉ܳ ߄ఔਵ۽ ݽ؛੉ ঌইࢲ ജ҃ী ؀਽ • ؘ੉ఠ ऺ੉ݶ ࢿמب ੼੼ ؊ જই૕ ࣻ ੓਺ Sungjoo Ha 17
18. Example • ോܻझ౮ vs ݽ؛ • द੘਷ ࢎۈ੉ ݅ٚ ോܻझ౮੉ જওਵա ૑ ࣘ੸ਵ۽ ࢿמ ೞۅ • ݽ؛਷ द੘਷ աࡍਵա ૑ࣘ੸ਵ۽ ࢿמ ࢚ थ Sungjoo Ha 18
19. Ideal • ݽٚ ੄ࢎ Ѿ੿੉ ೙ਃೠ Ҕী ݽ؛ਸ ഝਊ • ݽٚ Ѫ੄ ੗زച • ҳӖ੄ ৘द • ௥݂ झாેਸ ز੸ਵ۽ ؀਽ೞৈ ؘ੉ఠࣃఠ੄ պߑ࠺ܳ 40 ಌࣃ౟ ੺х • ౠ൤ ࠺ૉפझ/೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ ஖ജೡ ࣻ ੓ਵݶ ߬झ౟ • Ӓ۞ݶ ҅ࣘ೧ࢲ ૑ࣘ੸ਵ۽ ઁಿ੉ ѐࢶغח ҃೷ਸ ೡ ࣻ ੓ਸ Ѫ Sungjoo Ha 19
20. Easy? • औա? No • ML/ঌҊ્ܻ, ҅࢑੗ਗ, ؘ੉ఠ ׮ ы୾ঠ ೣ • ׮݅ ॶݽ੓ӝ ਤ೧ ೦࢚ ఠޖפ হ੉ ݆਷ ҅࢑੗ਗҗ ؘ੉ఠо ೙ਃೞ૑ח ঋ਺ • ೡ ࣻח ੓ח ੿ب • Ӓېب ݆ਸࣻ۾ ੉ٙ • ӒܻҊ അ੤੄ ӝࣿ۽ب оמ • ҙ۲ػ ੉ঠӝ ઑӘ ؊ ೧ࠁݶ... Sungjoo Ha 20
21. ML Model • ೠ੿ػ ؘ੉ఠ۽ જ਷ Ѿҗܳ ঳যղ۰ݶ ݽ؛ ஏݶ੄ Ҋ޹੉ ݆੉ ೙ਃ • AI ӝࣿ ߊ੹ ࣘبо ցޖ ࡅܴ • ੉੹ী ղܽ Ѿۿ੉ ࢜۽਍ ӝࣿ੉ աয়ݶࢲ ੤Ҋغযঠ ೞח ҃਋ب ݆਺ • ୭न ӝࣿ زೱਸ ੜ ࢓ಝঠ ೣ • ݽ؛۞ٜ੉ ࣗ೐౟ਝয ূ૑פয݂ب ੜೞח Ѫ਷ ൨ٝ • ݽ؍ೠ ஶప੉ց ӝ߈੄ ূ૑פয݂ী ؀ೠ Ҋ޹ ೙ਃ • ӝࣿ੉ ੗Բ݅ ߄Շযࢲ ࡅܰѱ ٮۄо׮ࠁݶ ߸҃੉ ݆਺ • ഈস ҳઑী ؀ೠ Ҋ޹ ೙ਃ Sungjoo Ha 21
22. Compute • ௾ ݽ؛ਸ ٜ݅ӝ ਤ೧ ҅࢑ ੗ਗ੉ ೙ਃ • ҅࢑ ੗ਗਸ ؘ੉ఠ۽ ߄Բח ҃਋ب ݆਺ • AlphaGo • Data augmentation • ௾ ݽ؛਷ ࢲࡂب য۰਑ Sungjoo Ha 22
23. MLOps • ML ݽ؛җ ࣗ೐౟ਝয੄ ਬӝ੸ੋ Ѿ೤ਸ ਤೠ Ҋ޹ ೙ਃ • അ؀੸ੋ بҳ੄ ѐߊ • ؘ੉ఠ ౵੉೐ۄੋ • ૑ࣘ੸ਵ۽ ݽ؛ ೟णೞח ౵੉೐ۄੋ • ݽ؛ ࢿמ ੗ز ಣо • ੗ز ݽ؛ ߓನ • Shadow mode Sungjoo Ha 23
24. Data • ઁಿਸ ా೧ ؘ੉ఠܳ ദٙೡ ࣻ ੓ח ؘ੉ఠ ੹ۚ੉ ೙ਃ • যڃ ؘ੉ఠܳ যڌѱ ࣻ૘ೡө? • যڌѱ ੿׹ਸ ঳ਸө? • Negative sampling ١ • ഑਷ human-in-the-loop दझమਸ যڌѱ ੜ ਍৔ೡө? • ݽ؛੉ ੜ ೞҊ ੓ח૑ যڌѱ ૑ࣘ੸ਵ۽ ݽפఠ݂ೞ૑? • ా೤ ؘ੉ఠ ਝযೞ਋झ • Feature store • ؘ੉ఠ ੿೤ࢿ • ML ࢿמী о੢ ௾ ৔ೱਸ ઱ח ࠗ࠙ • ߡӒ ೴౴੄ োࣘ Sungjoo Ha 24
25. AI PM • ೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ যڌѱ ஖ജೡ ࣻ ੓ਸө? • ੉ܳ ా೧ ࢎਊ੗ٜ੄ ݅઒بо য়ܰҊ ؊ ݆਷ ؘ੉ఠܳ ٜ݅যղҊ ؊ ա਷ ઁಿਵ۽ ੉য૑ח • ੿׹ਸ ঳ӝ ए਍ ҃਋ب ੓Ҋ য۰਍ ҃਋ب ੓Ҋ • AI ӝࣿਸ ੉೧ೞҊ ੉ী ݏ୾ ઁಿਸ э੉ ҳࢿ೧ঠ ೣ • ӝദী ؀ೠ ੄ࢎ ࣗాਸ ਤೠ ࢜۽਍ ঱য/بҳ/೐۽ࣁझ ೙ਃ Sungjoo Ha 25
26. TikTok • ߑ؀ೠ ন੄ ௑బஎ • ন਷ ૕ਸ ׸ࠁೠ׮ח Ѻ঱ೞী ؀ӏݽ ݃ா౴ਸ ాೠ ݽٚ פ஖੄ ൚ࣻ • ࣚऔѱ ௑బஎܳ ٜ݅ ࣻ ੓ب۾ ب৬઱ח ׮নೠ ઁ੘ بҳ ઁҕ • ݺߔೠ ੿׹ਸ ૒੽੸ਵ۽ ঳ਸ ࣻ ੓ח ߑधਵ۽ UIܳ ҳࢿ • ೠ ߣী ೞա੄ ࠺٣য়݅ ࠁݴ ࢎਊ੗੄ swipe ৈࠗ۽ ஂೱਸ ݺߔೞѱ ഛੋ • implicit feedback -> explicit feedback • ৘ஏ ӝ߈੄ ࢎਊ੗ ݽ؛݂ • ױࣽ൤ ੋध(recognition)ীࢲ Ӓ஖ח Ѫ੉ ইפۄ ࢎਊ੗о যڌѱ ೯زೞ৓ח૑ ݺߔೠ ೖ٘ߔਸ ߉ب۾ ҳࢿ Sungjoo Ha 26
27. Culture • ੌೞח ߑध, ޙചب ઺ਃ • AI ӝദ੗ۄחѱ ই૒ ઓ੤ೞ૑ח ঋ਺ • Ӓ۞׮ࠁפ э੉ যڌѱ ੌೡ૑ী ؀೧ࢲب Ҋ޹೧ࠊঠ ೣ • ݽٚ ઑ૒/ҳࢿਗ੉ ੉ী ؀ೠ Ӓܿਸ ݠ݁ࣘী ыҊ ӝৈ೧ঠ ೣ • ੄ࢎ Ѿ੿ӂ੗ٜب ੄޷੓ח ࣻળਵ۽ AIী ؀೧ ੉೧೧ঠ • ઑ૒ҳઑب ੉۞ೠ ղਊਸ ߈৔೧ঠ • Conway੄ ߨ஗ Sungjoo Ha 27
28. Conclusion • AI ӝࣿਸ ా೧ ੹ী ೡ ࣻ হ؍ Ѫਸ ೡ ࣻ ੓ח ഥࢎٜ੉ ࢤӡ Ѫ • AI ӝࣿ ੗୓݅ ੜ ഝਊೞח Ѫਵ۽ ੉޷ ର߹ചח оמ • ӝࣿী ؀ೠ ੉೧৬ ઑ૒ ҳઑ ߂ সޖ ߑध ߂ ޙച ١ ׮নೠ ஏݶীࢲ Ҋ۰೧ঠ AI ӝࣿਸ ੜ ഝਊೡ ࣻ ੓਺ • ੄ࢎ Ѿ੿੉ ੉ܖয૑ח ݽٚ Ҕীࢲ ݽ؛੉ ഝਊؼ ৈ૑о ੓਺ • ೐۽؋౟੄ ઺ਃೠ ޙઁܳ AI ޙઁ۽ ஖ജೞݶ ੉࢚੸ • ੉ܳ ߄ఔਵ۽ ML ݽ؛੄ ѐࢶ੉ ೐۽؋౟੄ ѐࢶਵ۽ ੉য૑Ҋ ੉ח ؊ ݆਷ ؘ੉ఠ۽ ੉য ૑ח ࢶࣽജ ҳઑܳ ٜ݅যյ ࣻ ੓਺ Sungjoo Ha 28