AI Company

AI Company -- Upstage

Transcript

1. AI Company
   Upstage Upstudy
   Hyperconnect
   Director of AI
   Sungjoo Ha
   June 17th, 2022
   Sungjoo Ha
   1
   

   View Slide

2. AI Company
   • য়ט੄ ઱ઁ: AI company
   • ੋఠ֔ ӝࣿਸ ੜ ഝਊೠ ӝস਷ internet companyۄ ࠛ۷Ҋ ੉ח
   ݽ߄ੌ द؀ө૑ ੉য઎׮
   • Amazon, Alphabet, Facebook, Alibaba, Tencent, etc.
   • AI द؀੄ AI companyח ޖ঺੉ ׮ܳө?
   • ੿׹਷ ݽܰ૑݅ ѐੋ੸ਵ۽ ࢤп೮؍ ݻ о૑ ನੋ౟ী ؀ೠ ࣗѐ
   Sungjoo Ha
   2
   

   View Slide

3. ࣳೝށ + ਢࢎ੉౟
   ≠ Internet Company
   Sungjoo Ha
   3
   

   View Slide

4. Jeff Bezos in 1997
   In the book space, there are more than three
   million different books worldwide active and
   in print at any given time across all
   languages, so when you have that many
   items, you can literally build a store online
   that couldn't exist any other way. 1
   1 https://youtu.be/rWRbTnE1PEM
   Sungjoo Ha
   4
   

   View Slide

5. Aggregators
   • Zero marginal cost
   • ٣૑ణ ই੉మਸ ౸ݒೞחؘী ୶о ࠺ਊ੉ ٜ૑ ঋ਺
   • Distributionী ࠺ਊ੉ ٜ૑ ঋ਺
   • Transactionী ࠺ਊ੉ ٜ૑ ঋ਺
   • ੉ܳ ӓೠਵ۽ ഝਊೠ Ѫ੉ ҳӖ੉ա ಕ੉झ࠘җ э਷ super-aggregator2
   • ੋఠ֔ী ઓ੤ೞח Ѫ੉ value proposition਷ ইש
   • ੋఠ֔੉ ࢜۽੉ ઱ח о஖ܳ ߉ইٜ੉Ҋ ੋఠ֔੉ ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझܳ ݅ٝ
   2 https://stratechery.com/concept/aggregation-theory/
   Sungjoo Ha
   5
   

   View Slide

6. Internet-Enabled Technology
   • Internet द؀੄ ӝࣿ
   • ੋఠ֔ द؀ী ਢಕ੉૑ח ׮ ੓૑݅ ੋఠ֔ਵ۽ ੋ೧ оמ೧૓ ӝࣿਸ ഝਊೞח ӝস਷ ઁೠ੸
   • ਬ੷ ೯زী ؀ೠ ࣻ૘ਸ ߄ఔਵ۽ ࢎਊ੗ܳ ੉೧ೞח Ѫ
   • ೠ ߊ աইоࢲ A/B test ೞח Ѫ3
   • 1֙ী ೠ ف ߣ ߓನೞ؍ Ѫ੉
   • Continuous integration4, continuous deployment ١ਵ۽ ݒੌݒੌ ߓನ
   • ӓױ੸ਵ۽ ૣ਷ ੉ఠۨ੉࣌ ࢎ੉௿ਸ ٜ݅যղҊ product-market-fitਸ ଺ӝ ਤ೧ ࡅܰѱ ਑૒੉ח Ѫ੉ оמ೧૗
   • ੉ܳ оמாೞח ઑ૒ ҳઑ
   4 Martin Fowlerо ੘ࢿೠ 2006֙ Ӗীࢲ CI ਊযо ࢎਊ
   3 Google਷ ੉޷ 2000֙ী A/B పझ౟ܳ दب
   Sungjoo Ha
   6
   

   View Slide

7. Learnings From The Past
   • Internet company੄ ৘द۽ࠗఠ ࢤп೧ࠁח AI company
   • AIо ইפݶ ࢿ݀غ૑ ঋח ࠺ૉפझ
   • ӝઓী ੹ഃ ࠛоמೞ؍ Ѫ੉ оמ೧૑ח Ѫ
   • ઑӘ ؊ ևѱ ೧ࢳ೧ࠁݶ AI द؀ী оמ೧૓ ࢜۽਍ ӝࣿਸ ੜ ഝਊೞ
   ח ഥࢎ
   Sungjoo Ha
   7
   

   View Slide

8. Algorithm + Compute
   + Data
   • ML ޙઁীࢲ ݏׯڰܻח ف о૑ ޙઁ
   • ݽ؛ ࢿמ੉ ೟णࣇীࢲ ծ਷ ҃਋
   • underfitting: ݽ؛ ࣻਊ۱ ૐо
   • ݽ؛ ࢿמ੉ పझ౟ࣇীࢲ ծ਷ ҃਋
   • overfitting: ؘ੉ఠ ୶о
   • അ؀੄ ML ӝࣿ਷ ݽ؛੄ ࣻਊ۱(capacity)ਸ ޖೠ੿ טܾ ࣻ ੓਺
   • ੉ܳ جܻӝ ਤೠ ҅࢑ ੗ਗҗ
   • ୽࠙ೠ ন੄ ؘ੉ఠ݅ ੓ਵݶ ҅ࣘ ࢿמ੉ જই૗
   • 2010֙؀ ୡ߈ ੉੹ীח ੉۠ ѐ֛੉ হ঻ਵݴ ୭Ӕ ٜয ੉۠ ൒
   ܴ੉ ؊ ъചغҊ ੓਺
   Sungjoo Ha
   8
   

   View Slide

9. AI Strategy
   • ׮ܲ ೒ۨ੉যٜ਷ যڃ ࢤпਸ ೞҊ ੓ਸө? AI ੹ۚ੉ ަө?
   • ׮ٜ ৈ۞ оמࢿਸ ࠁݶࢲ ߬౴ೞҊ ੓਺
   • Google: AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস
   • OpenAI: Ѣ؀ ݽ؛ী ؀ೠ ъೠ ޺਺ਸ ాೠ ௾ ߬౴
   • Tesla: അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ ా೤
   Sungjoo Ha
   9
   

   View Slide

10. AI Strategy -- Google
    • AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস
    • Google Brain੉ա Deepmind৬ э਷ ୹઺ೠ োҳࣗ5
    • TPUܳ ੌଧх஖ ઁ੘6
    • ੹ࣁ҅੄ ݽٚ ؘ੉ఠܳ ൚ࣻ
    • ׮নೠ AI ੽Ӕ ߑߨਸ ನ౟ಫܻয়۽ ਍৔
    • AI investment fund
    • AI M&A
    • Scaling, neuroscience, ...
    6 2015֙ द੘
    5 GBח 2011֙ founding, DMח 2014֙ acquisition
    Sungjoo Ha
    10
    

    View Slide

11. AI Strategy -- OpenAI
    Conviction that if you take the right stuff,
    which already exists, and apply and mix it
    with a lot of data and a lot of compute, that it
    will in fact work. And so that was the missing
    piece.7
    • Scaling hypothesis੄ ௾ ߬౴
    7 https://www.youtube.com/watch?v=13CZPWmke6A&t=950s
    Sungjoo Ha
    11
    

    View Slide

12. AI Strategy -- Tesla
    • അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ
    ా೤
    • Data engine
    • Auto labeling8
    • Shadow mode
    8 https://youtu.be/j0z4FweCy4M?t=5157
    Sungjoo Ha
    12
    

    View Slide

13. ӝઓഥࢎ + AI/ML/DL
    ≠ AI Company
    Sungjoo Ha
    13
    

    View Slide

14. Zero Marginal Content
    • AIо ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझח ޖ঺ੌ
    ө?
    • ݻ о૑ ൦౟ח ੓਺
    • Zero marginal cost content
    creation
    • GPT-3 (copilot), DALL·E 2
    • Sampling റ ࢎۈ੉ ࢶఖೞѱ ೞח Ѫ
    • Super-human decision making
    • AlphaGo, AlphaFold
    Sungjoo Ha
    14
    

    View Slide

15. AI-Enabled Technology
    • ؀ӝসٜ੉ঠ ׮নೠ ߬౴ਸ ೡ ࣻ ੓૑݅ ੘਷ ഥࢎח যڌѱ?
    • AI द؀ীח ੋఠ֔ द؀੄ ൒ܴҗ ࠺तೞѱ AI ݽ؛਷ ׮ٜ ࢎਊೡ Ѫ
    • ৈӝীࢲ ب୹ػ 2ର੸ੋ ѐ֛/ӝࣿ/ޙചܳ ঴݃ա ੜ ഝਊೞջب ઺ਃೠ
    ನੋ౟о ؼ Ѫ
    • ݃஖ A/B పझ౟ܳ ೞח ഥࢎ৬ ইצ ഥࢎо ա׊Ҋ
    • CI/CDܳ ഝਊೞח ഥࢎ৬ ইפפ ഥࢎо ա׊ח Ѫ୊ۢ
    Sungjoo Ha
    15
    

    View Slide

16. Learned Business Logic
    • ࠺ૉפझ ۽૒ -> ݽ؛ Үജ
    • ࠺ૉפझ ۽૒: A ੉ݶ B ੉׮
    • ೐۽Ӓېݠٜ੉ ٜ݅যղח ௏٘੄ ݆਷ ࠗ࠙੉ ࠺ૉפझ ۽૒
    • ੉ѱ ޤо ׮ؘܲ? ೐۽Ӓېݠо ೞܖݶ ٜ݅؍Ѧ ҡ൤ ML ॄࢲ ೠ׳ Ѧ۰ࢲ ݅٘ח Ѣ ইפঠ?
    • ࢎۈࠁ׮ ੜ ೡ ࣻ ੓ח ҃਋о ੓਺
    • A੉ݶ B੉׮ীࢲ A੄ ઑѤ੉ ցޖցޖ ࠂ੟೮׮ݶ ࢎۈ਷ ޅೣ
    • Software 2.0
    Sungjoo Ha
    16
    

    View Slide

17. Software Rot
    • ࠺ૉפझ ۽૒ب ࣗ೐౟ਝযب ॅ਺
    • ઱߸ ജ҃੉ ߄Շӝ ٸޙ
    • ࢜۽਍ ӝמ੉ ࢤӝҊ, ઁಿਸ ੉Ցয оҊ੗ ೞח ߑೱ੉ ߄ՇҊ, ࢎਊ੗о ߄ՇҊ... Ӓېࢲ ॅ਺
    • ࠁా ੉Ѧ যڌѱ ؀਽ೞջ? ࣗ೐౟ਝয ূ૑פযо ௏٘ܳ ࣻ੿
    • A੉ݶ B ೮חؘ ੉ઁח A੉ݶ C ೧ঠ ೠ׮
    • Ӕؘ ੉ѱ ݽ؛੉঻׮ݶ
    • ૑ࣘ੸ਵ۽ ؘ੉ఠ ࣻ૘ೞҊ ੉ܳ ߄ఔਵ۽ ݽ؛੉ ঌইࢲ ജ҃ী ؀਽
    • ؘ੉ఠ ऺ੉ݶ ࢿמب ੼੼ ؊ જই૕ ࣻ ੓਺
    Sungjoo Ha
    17
    

    View Slide

18. Example
    • ോܻझ౮ vs ݽ؛
    • द੘਷ ࢎۈ੉ ݅ٚ ോܻझ౮੉ જওਵա ૑
    ࣘ੸ਵ۽ ࢿמ ೞۅ
    • ݽ؛਷ द੘਷ աࡍਵա ૑ࣘ੸ਵ۽ ࢿמ ࢚
    थ
    Sungjoo Ha
    18
    

    View Slide

19. Ideal
    • ݽٚ ੄ࢎ Ѿ੿੉ ೙ਃೠ Ҕী ݽ؛ਸ ഝਊ
    • ݽٚ Ѫ੄ ੗زച
    • ҳӖ੄ ৘द
    • ௥݂ झாેਸ ز੸ਵ۽ ؀਽ೞৈ ؘ੉ఠࣃఠ੄ պߑ࠺ܳ 40 ಌࣃ౟ ੺х
    • ౠ൤ ࠺ૉפझ/೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ ஖ജೡ ࣻ ੓ਵݶ ߬झ౟
    • Ӓ۞ݶ ҅ࣘ೧ࢲ ૑ࣘ੸ਵ۽ ઁಿ੉ ѐࢶغח ҃೷ਸ ೡ ࣻ ੓ਸ Ѫ
    Sungjoo Ha
    19
    

    View Slide

20. Easy?
    • औա? No
    • ML/ঌҊ્ܻ, ҅࢑੗ਗ, ؘ੉ఠ ׮ ы୾ঠ ೣ
    • ׮݅ ॶݽ੓ӝ ਤ೧ ೦࢚ ఠޖפ হ੉ ݆਷ ҅࢑੗ਗҗ ؘ੉ఠо ೙ਃೞ૑ח ঋ਺
    • ೡ ࣻח ੓ח ੿ب
    • Ӓېب ݆ਸࣻ۾ ੉ٙ
    • ӒܻҊ അ੤੄ ӝࣿ۽ب оמ
    • ҙ۲ػ ੉ঠӝ ઑӘ ؊ ೧ࠁݶ...
    Sungjoo Ha
    20
    

    View Slide

21. ML Model
    • ೠ੿ػ ؘ੉ఠ۽ જ਷ Ѿҗܳ ঳যղ۰ݶ ݽ؛ ஏݶ੄ Ҋ޹੉ ݆੉ ೙ਃ
    • AI ӝࣿ ߊ੹ ࣘبо ցޖ ࡅܴ
    • ੉੹ী ղܽ Ѿۿ੉ ࢜۽਍ ӝࣿ੉ աয়ݶࢲ ੤Ҋغযঠ ೞח ҃਋ب ݆਺
    • ୭न ӝࣿ زೱਸ ੜ ࢓ಝঠ ೣ
    • ݽ؛۞ٜ੉ ࣗ೐౟ਝয ূ૑פয݂ب ੜೞח Ѫ਷ ൨ٝ
    • ݽ؍ೠ ஶప੉ց ӝ߈੄ ূ૑פয݂ী ؀ೠ Ҋ޹ ೙ਃ
    • ӝࣿ੉ ੗Բ݅ ߄Շযࢲ ࡅܰѱ ٮۄо׮ࠁݶ ߸҃੉ ݆਺
    • ഈস ҳઑী ؀ೠ Ҋ޹ ೙ਃ
    Sungjoo Ha
    21
    

    View Slide

22. Compute
    • ௾ ݽ؛ਸ ٜ݅ӝ ਤ೧ ҅࢑ ੗ਗ੉ ೙ਃ
    • ҅࢑ ੗ਗਸ ؘ੉ఠ۽ ߄Բח ҃਋ب ݆਺
    • AlphaGo
    • Data augmentation
    • ௾ ݽ؛਷ ࢲࡂب য۰਑
    Sungjoo Ha
    22
    

    View Slide

23. MLOps
    • ML ݽ؛җ ࣗ೐౟ਝয੄ ਬӝ੸ੋ Ѿ೤ਸ ਤೠ Ҋ޹ ೙ਃ
    • അ؀੸ੋ بҳ੄ ѐߊ
    • ؘ੉ఠ ౵੉೐ۄੋ
    • ૑ࣘ੸ਵ۽ ݽ؛ ೟णೞח ౵੉೐ۄੋ
    • ݽ؛ ࢿמ ੗ز ಣо
    • ੗ز ݽ؛ ߓನ
    • Shadow mode
    Sungjoo Ha
    23
    

    View Slide

24. Data
    • ઁಿਸ ా೧ ؘ੉ఠܳ ദٙೡ ࣻ ੓ח ؘ੉ఠ ੹ۚ੉ ೙ਃ
    • যڃ ؘ੉ఠܳ যڌѱ ࣻ૘ೡө?
    • যڌѱ ੿׹ਸ ঳ਸө?
    • Negative sampling ١
    • ഑਷ human-in-the-loop दझమਸ যڌѱ ੜ ਍৔ೡө?
    • ݽ؛੉ ੜ ೞҊ ੓ח૑ যڌѱ ૑ࣘ੸ਵ۽ ݽפఠ݂ೞ૑?
    • ా೤ ؘ੉ఠ ਝযೞ਋झ
    • Feature store
    • ؘ੉ఠ ੿೤ࢿ
    • ML ࢿמী о੢ ௾ ৔ೱਸ ઱ח ࠗ࠙
    • ߡӒ ೴౴੄ োࣘ
    Sungjoo Ha
    24
    

    View Slide

25. AI PM
    • ೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ যڌѱ ஖ജೡ ࣻ ੓ਸө?
    • ੉ܳ ా೧ ࢎਊ੗ٜ੄ ݅઒بо য়ܰҊ ؊ ݆਷ ؘ੉ఠܳ ٜ݅যղҊ ؊
    ա਷ ઁಿਵ۽ ੉য૑ח
    • ੿׹ਸ ঳ӝ ए਍ ҃਋ب ੓Ҋ য۰਍ ҃਋ب ੓Ҋ
    • AI ӝࣿਸ ੉೧ೞҊ ੉ী ݏ୾ ઁಿਸ э੉ ҳࢿ೧ঠ ೣ
    • ӝദী ؀ೠ ੄ࢎ ࣗాਸ ਤೠ ࢜۽਍ ঱য/بҳ/೐۽ࣁझ ೙ਃ
    Sungjoo Ha
    25
    

    View Slide

26. TikTok
    • ߑ؀ೠ ন੄ ௑బஎ
    • ন਷ ૕ਸ ׸ࠁೠ׮ח Ѻ঱ೞী ؀ӏݽ ݃ா౴ਸ ాೠ ݽٚ פ஖੄ ൚ࣻ
    • ࣚऔѱ ௑బஎܳ ٜ݅ ࣻ ੓ب۾ ب৬઱ח ׮নೠ ઁ੘ بҳ ઁҕ
    • ݺߔೠ ੿׹ਸ ૒੽੸ਵ۽ ঳ਸ ࣻ ੓ח ߑधਵ۽ UIܳ ҳࢿ
    • ೠ ߣী ೞա੄ ࠺٣য়݅ ࠁݴ ࢎਊ੗੄ swipe ৈࠗ۽ ஂೱਸ ݺߔೞѱ ഛੋ
    • implicit feedback -> explicit feedback
    • ৘ஏ ӝ߈੄ ࢎਊ੗ ݽ؛݂
    • ױࣽ൤ ੋध(recognition)ীࢲ Ӓ஖ח Ѫ੉ ইפۄ ࢎਊ੗о যڌѱ ೯زೞ৓ח૑ ݺߔೠ ೖ٘ߔਸ ߉ب۾
    ҳࢿ
    Sungjoo Ha
    26
    

    View Slide

27. Culture
    • ੌೞח ߑध, ޙചب ઺ਃ
    • AI ӝദ੗ۄחѱ ই૒ ઓ੤ೞ૑ח ঋ਺
    • Ӓ۞׮ࠁפ э੉ যڌѱ ੌೡ૑ী ؀೧ࢲب Ҋ޹೧ࠊঠ ೣ
    • ݽٚ ઑ૒/ҳࢿਗ੉ ੉ী ؀ೠ Ӓܿਸ ݠ݁ࣘী ыҊ ӝৈ೧ঠ ೣ
    • ੄ࢎ Ѿ੿ӂ੗ٜب ੄޷੓ח ࣻળਵ۽ AIী ؀೧ ੉೧೧ঠ
    • ઑ૒ҳઑب ੉۞ೠ ղਊਸ ߈৔೧ঠ
    • Conway੄ ߨ஗
    Sungjoo Ha
    27
    

    View Slide

28. Conclusion
    • AI ӝࣿਸ ా೧ ੹ী ೡ ࣻ হ؍ Ѫਸ ೡ ࣻ ੓ח ഥࢎٜ੉ ࢤӡ Ѫ
    • AI ӝࣿ ੗୓݅ ੜ ഝਊೞח Ѫਵ۽ ੉޷ ର߹ചח оמ
    • ӝࣿী ؀ೠ ੉೧৬ ઑ૒ ҳઑ ߂ সޖ ߑध ߂ ޙച ١ ׮নೠ ஏݶীࢲ Ҋ۰೧ঠ AI ӝࣿਸ ੜ
    ഝਊೡ ࣻ ੓਺
    • ੄ࢎ Ѿ੿੉ ੉ܖয૑ח ݽٚ Ҕীࢲ ݽ؛੉ ഝਊؼ ৈ૑о ੓਺
    • ೐۽؋౟੄ ઺ਃೠ ޙઁܳ AI ޙઁ۽ ஖ജೞݶ ੉࢚੸
    • ੉ܳ ߄ఔਵ۽ ML ݽ؛੄ ѐࢶ੉ ೐۽؋౟੄ ѐࢶਵ۽ ੉য૑Ҋ ੉ח ؊ ݆਷ ؘ੉ఠ۽ ੉য
    ૑ח ࢶࣽജ ҳઑܳ ٜ݅যյ ࣻ ੓਺
    Sungjoo Ha
    28
    

    View Slide