Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AI Company

AI Company

AI Company -- Upstage

Sungjoo Ha

June 17, 2022
Tweet

More Decks by Sungjoo Ha

Other Decks in Technology

Transcript

  1. AI Company
    Upstage Upstudy
    Hyperconnect
    Director of AI
    Sungjoo Ha
    June 17th, 2022
    Sungjoo Ha
    1

    View Slide

  2. AI Company
    • য়ט੄ ઱ઁ: AI company
    • ੋఠ֔ ӝࣿਸ ੜ ഝਊೠ ӝস਷ internet companyۄ ࠛ۷Ҋ ੉ח
    ݽ߄ੌ द؀ө૑ ੉য઎׮
    • Amazon, Alphabet, Facebook, Alibaba, Tencent, etc.
    • AI द؀੄ AI companyח ޖ঺੉ ׮ܳө?
    • ੿׹਷ ݽܰ૑݅ ѐੋ੸ਵ۽ ࢤп೮؍ ݻ о૑ ನੋ౟ী ؀ೠ ࣗѐ
    Sungjoo Ha
    2

    View Slide

  3. ࣳೝށ + ਢࢎ੉౟
    ≠ Internet Company
    Sungjoo Ha
    3

    View Slide

  4. Jeff Bezos in 1997
    In the book space, there are more than three
    million different books worldwide active and
    in print at any given time across all
    languages, so when you have that many
    items, you can literally build a store online
    that couldn't exist any other way. 1
    1 https://youtu.be/rWRbTnE1PEM
    Sungjoo Ha
    4

    View Slide

  5. Aggregators
    • Zero marginal cost
    • ٣૑ణ ই੉మਸ ౸ݒೞחؘী ୶о ࠺ਊ੉ ٜ૑ ঋ਺
    • Distributionী ࠺ਊ੉ ٜ૑ ঋ਺
    • Transactionী ࠺ਊ੉ ٜ૑ ঋ਺
    • ੉ܳ ӓೠਵ۽ ഝਊೠ Ѫ੉ ҳӖ੉ա ಕ੉झ࠘җ э਷ super-aggregator2
    • ੋఠ֔ী ઓ੤ೞח Ѫ੉ value proposition਷ ইש
    • ੋఠ֔੉ ࢜۽੉ ઱ח о஖ܳ ߉ইٜ੉Ҋ ੋఠ֔੉ ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझܳ ݅ٝ
    2 https://stratechery.com/concept/aggregation-theory/
    Sungjoo Ha
    5

    View Slide

  6. Internet-Enabled Technology
    • Internet द؀੄ ӝࣿ
    • ੋఠ֔ द؀ী ਢಕ੉૑ח ׮ ੓૑݅ ੋఠ֔ਵ۽ ੋ೧ оמ೧૓ ӝࣿਸ ഝਊೞח ӝস਷ ઁೠ੸
    • ਬ੷ ೯زী ؀ೠ ࣻ૘ਸ ߄ఔਵ۽ ࢎਊ੗ܳ ੉೧ೞח Ѫ
    • ೠ ߊ աইоࢲ A/B test ೞח Ѫ3
    • 1֙ী ೠ ف ߣ ߓನೞ؍ Ѫ੉
    • Continuous integration4, continuous deployment ١ਵ۽ ݒੌݒੌ ߓನ
    • ӓױ੸ਵ۽ ૣ਷ ੉ఠۨ੉࣌ ࢎ੉௿ਸ ٜ݅যղҊ product-market-fitਸ ଺ӝ ਤ೧ ࡅܰѱ ਑૒੉ח Ѫ੉ оמ೧૗
    • ੉ܳ оמாೞח ઑ૒ ҳઑ
    4 Martin Fowlerо ੘ࢿೠ 2006֙ Ӗীࢲ CI ਊযо ࢎਊ
    3 Google਷ ੉޷ 2000֙ী A/B పझ౟ܳ दب
    Sungjoo Ha
    6

    View Slide

  7. Learnings From The Past
    • Internet company੄ ৘द۽ࠗఠ ࢤп೧ࠁח AI company
    • AIо ইפݶ ࢿ݀غ૑ ঋח ࠺ૉפझ
    • ӝઓী ੹ഃ ࠛоמೞ؍ Ѫ੉ оמ೧૑ח Ѫ
    • ઑӘ ؊ ևѱ ೧ࢳ೧ࠁݶ AI द؀ী оמ೧૓ ࢜۽਍ ӝࣿਸ ੜ ഝਊೞ
    ח ഥࢎ
    Sungjoo Ha
    7

    View Slide

  8. Algorithm + Compute
    + Data
    • ML ޙઁীࢲ ݏׯڰܻח ف о૑ ޙઁ
    • ݽ؛ ࢿמ੉ ೟णࣇীࢲ ծ਷ ҃਋
    • underfitting: ݽ؛ ࣻਊ۱ ૐо
    • ݽ؛ ࢿמ੉ పझ౟ࣇীࢲ ծ਷ ҃਋
    • overfitting: ؘ੉ఠ ୶о
    • അ؀੄ ML ӝࣿ਷ ݽ؛੄ ࣻਊ۱(capacity)ਸ ޖೠ੿ טܾ ࣻ ੓਺
    • ੉ܳ جܻӝ ਤೠ ҅࢑ ੗ਗҗ
    • ୽࠙ೠ ন੄ ؘ੉ఠ݅ ੓ਵݶ ҅ࣘ ࢿמ੉ જই૗
    • 2010֙؀ ୡ߈ ੉੹ীח ੉۠ ѐ֛੉ হ঻ਵݴ ୭Ӕ ٜয ੉۠ ൒
    ܴ੉ ؊ ъചغҊ ੓਺
    Sungjoo Ha
    8

    View Slide

  9. AI Strategy
    • ׮ܲ ೒ۨ੉যٜ਷ যڃ ࢤпਸ ೞҊ ੓ਸө? AI ੹ۚ੉ ަө?
    • ׮ٜ ৈ۞ оמࢿਸ ࠁݶࢲ ߬౴ೞҊ ੓਺
    • Google: AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস
    • OpenAI: Ѣ؀ ݽ؛ী ؀ೠ ъೠ ޺਺ਸ ాೠ ௾ ߬౴
    • Tesla: അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ ా೤
    Sungjoo Ha
    9

    View Slide

  10. AI Strategy -- Google
    • AI 3ਃࣗܳ ૓੿ೠ ੄޷۽ ݽف ы୶Ҋ ੓ח ݻ উغח ӝস
    • Google Brain੉ա Deepmind৬ э਷ ୹઺ೠ োҳࣗ5
    • TPUܳ ੌଧх஖ ઁ੘6
    • ੹ࣁ҅੄ ݽٚ ؘ੉ఠܳ ൚ࣻ
    • ׮নೠ AI ੽Ӕ ߑߨਸ ನ౟ಫܻয়۽ ਍৔
    • AI investment fund
    • AI M&A
    • Scaling, neuroscience, ...
    6 2015֙ द੘
    5 GBח 2011֙ founding, DMח 2014֙ acquisition
    Sungjoo Ha
    10

    View Slide

  11. AI Strategy -- OpenAI
    Conviction that if you take the right stuff,
    which already exists, and apply and mix it
    with a lot of data and a lot of compute, that it
    will in fact work. And so that was the missing
    piece.7
    • Scaling hypothesis੄ ௾ ߬౴
    7 https://www.youtube.com/watch?v=13CZPWmke6A&t=950s
    Sungjoo Ha
    11

    View Slide

  12. AI Strategy -- Tesla
    • അઓೞח ӝࣿਸ ӓೠਵ۽ ഝਊೞח ߡ౭ஸ
    ా೤
    • Data engine
    • Auto labeling8
    • Shadow mode
    8 https://youtu.be/j0z4FweCy4M?t=5157
    Sungjoo Ha
    12

    View Slide

  13. ӝઓഥࢎ + AI/ML/DL
    ≠ AI Company
    Sungjoo Ha
    13

    View Slide

  14. Zero Marginal Content
    • AIо ইפݶ ࢿ݀ೡ ࣻ হח ࠺ૉפझח ޖ঺ੌ
    ө?
    • ݻ о૑ ൦౟ח ੓਺
    • Zero marginal cost content
    creation
    • GPT-3 (copilot), DALL·E 2
    • Sampling റ ࢎۈ੉ ࢶఖೞѱ ೞח Ѫ
    • Super-human decision making
    • AlphaGo, AlphaFold
    Sungjoo Ha
    14

    View Slide

  15. AI-Enabled Technology
    • ؀ӝসٜ੉ঠ ׮নೠ ߬౴ਸ ೡ ࣻ ੓૑݅ ੘਷ ഥࢎח যڌѱ?
    • AI द؀ীח ੋఠ֔ द؀੄ ൒ܴҗ ࠺तೞѱ AI ݽ؛਷ ׮ٜ ࢎਊೡ Ѫ
    • ৈӝীࢲ ب୹ػ 2ର੸ੋ ѐ֛/ӝࣿ/ޙചܳ ঴݃ա ੜ ഝਊೞջب ઺ਃೠ
    ನੋ౟о ؼ Ѫ
    • ݃஖ A/B పझ౟ܳ ೞח ഥࢎ৬ ইצ ഥࢎо ա׊Ҋ
    • CI/CDܳ ഝਊೞח ഥࢎ৬ ইפפ ഥࢎо ա׊ח Ѫ୊ۢ
    Sungjoo Ha
    15

    View Slide

  16. Learned Business Logic
    • ࠺ૉפझ ۽૒ -> ݽ؛ Үജ
    • ࠺ૉפझ ۽૒: A ੉ݶ B ੉׮
    • ೐۽Ӓېݠٜ੉ ٜ݅যղח ௏٘੄ ݆਷ ࠗ࠙੉ ࠺ૉפझ ۽૒
    • ੉ѱ ޤо ׮ؘܲ? ೐۽Ӓېݠо ೞܖݶ ٜ݅؍Ѧ ҡ൤ ML ॄࢲ ೠ׳ Ѧ۰ࢲ ݅٘ח Ѣ ইפঠ?
    • ࢎۈࠁ׮ ੜ ೡ ࣻ ੓ח ҃਋о ੓਺
    • A੉ݶ B੉׮ীࢲ A੄ ઑѤ੉ ցޖցޖ ࠂ੟೮׮ݶ ࢎۈ਷ ޅೣ
    • Software 2.0
    Sungjoo Ha
    16

    View Slide

  17. Software Rot
    • ࠺ૉפझ ۽૒ب ࣗ೐౟ਝযب ॅ਺
    • ઱߸ ജ҃੉ ߄Շӝ ٸޙ
    • ࢜۽਍ ӝמ੉ ࢤӝҊ, ઁಿਸ ੉Ցয оҊ੗ ೞח ߑೱ੉ ߄ՇҊ, ࢎਊ੗о ߄ՇҊ... Ӓېࢲ ॅ਺
    • ࠁా ੉Ѧ যڌѱ ؀਽ೞջ? ࣗ೐౟ਝয ূ૑פযо ௏٘ܳ ࣻ੿
    • A੉ݶ B ೮חؘ ੉ઁח A੉ݶ C ೧ঠ ೠ׮
    • Ӕؘ ੉ѱ ݽ؛੉঻׮ݶ
    • ૑ࣘ੸ਵ۽ ؘ੉ఠ ࣻ૘ೞҊ ੉ܳ ߄ఔਵ۽ ݽ؛੉ ঌইࢲ ജ҃ী ؀਽
    • ؘ੉ఠ ऺ੉ݶ ࢿמب ੼੼ ؊ જই૕ ࣻ ੓਺
    Sungjoo Ha
    17

    View Slide

  18. Example
    • ോܻझ౮ vs ݽ؛
    • द੘਷ ࢎۈ੉ ݅ٚ ോܻझ౮੉ જওਵա ૑
    ࣘ੸ਵ۽ ࢿמ ೞۅ
    • ݽ؛਷ द੘਷ աࡍਵա ૑ࣘ੸ਵ۽ ࢿמ ࢚

    Sungjoo Ha
    18

    View Slide

  19. Ideal
    • ݽٚ ੄ࢎ Ѿ੿੉ ೙ਃೠ Ҕী ݽ؛ਸ ഝਊ
    • ݽٚ Ѫ੄ ੗زച
    • ҳӖ੄ ৘द
    • ௥݂ झாેਸ ز੸ਵ۽ ؀਽ೞৈ ؘ੉ఠࣃఠ੄ պߑ࠺ܳ 40 ಌࣃ౟ ੺х
    • ౠ൤ ࠺ૉפझ/೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ ஖ജೡ ࣻ ੓ਵݶ ߬झ౟
    • Ӓ۞ݶ ҅ࣘ೧ࢲ ૑ࣘ੸ਵ۽ ઁಿ੉ ѐࢶغח ҃೷ਸ ೡ ࣻ ੓ਸ Ѫ
    Sungjoo Ha
    19

    View Slide

  20. Easy?
    • औա? No
    • ML/ঌҊ્ܻ, ҅࢑੗ਗ, ؘ੉ఠ ׮ ы୾ঠ ೣ
    • ׮݅ ॶݽ੓ӝ ਤ೧ ೦࢚ ఠޖפ হ੉ ݆਷ ҅࢑੗ਗҗ ؘ੉ఠо ೙ਃೞ૑ח ঋ਺
    • ೡ ࣻח ੓ח ੿ب
    • Ӓېب ݆ਸࣻ۾ ੉ٙ
    • ӒܻҊ അ੤੄ ӝࣿ۽ب оמ
    • ҙ۲ػ ੉ঠӝ ઑӘ ؊ ೧ࠁݶ...
    Sungjoo Ha
    20

    View Slide

  21. ML Model
    • ೠ੿ػ ؘ੉ఠ۽ જ਷ Ѿҗܳ ঳যղ۰ݶ ݽ؛ ஏݶ੄ Ҋ޹੉ ݆੉ ೙ਃ
    • AI ӝࣿ ߊ੹ ࣘبо ցޖ ࡅܴ
    • ੉੹ী ղܽ Ѿۿ੉ ࢜۽਍ ӝࣿ੉ աয়ݶࢲ ੤Ҋغযঠ ೞח ҃਋ب ݆਺
    • ୭न ӝࣿ زೱਸ ੜ ࢓ಝঠ ೣ
    • ݽ؛۞ٜ੉ ࣗ೐౟ਝয ূ૑פয݂ب ੜೞח Ѫ਷ ൨ٝ
    • ݽ؍ೠ ஶప੉ց ӝ߈੄ ূ૑פয݂ী ؀ೠ Ҋ޹ ೙ਃ
    • ӝࣿ੉ ੗Բ݅ ߄Շযࢲ ࡅܰѱ ٮۄо׮ࠁݶ ߸҃੉ ݆਺
    • ഈস ҳઑী ؀ೠ Ҋ޹ ೙ਃ
    Sungjoo Ha
    21

    View Slide

  22. Compute
    • ௾ ݽ؛ਸ ٜ݅ӝ ਤ೧ ҅࢑ ੗ਗ੉ ೙ਃ
    • ҅࢑ ੗ਗਸ ؘ੉ఠ۽ ߄Բח ҃਋ب ݆਺
    • AlphaGo
    • Data augmentation
    • ௾ ݽ؛਷ ࢲࡂب য۰਑
    Sungjoo Ha
    22

    View Slide

  23. MLOps
    • ML ݽ؛җ ࣗ೐౟ਝয੄ ਬӝ੸ੋ Ѿ೤ਸ ਤೠ Ҋ޹ ೙ਃ
    • അ؀੸ੋ بҳ੄ ѐߊ
    • ؘ੉ఠ ౵੉೐ۄੋ
    • ૑ࣘ੸ਵ۽ ݽ؛ ೟णೞח ౵੉೐ۄੋ
    • ݽ؛ ࢿמ ੗ز ಣо
    • ੗ز ݽ؛ ߓನ
    • Shadow mode
    Sungjoo Ha
    23

    View Slide

  24. Data
    • ઁಿਸ ా೧ ؘ੉ఠܳ ദٙೡ ࣻ ੓ח ؘ੉ఠ ੹ۚ੉ ೙ਃ
    • যڃ ؘ੉ఠܳ যڌѱ ࣻ૘ೡө?
    • যڌѱ ੿׹ਸ ঳ਸө?
    • Negative sampling ١
    • ഑਷ human-in-the-loop दझమਸ যڌѱ ੜ ਍৔ೡө?
    • ݽ؛੉ ੜ ೞҊ ੓ח૑ যڌѱ ૑ࣘ੸ਵ۽ ݽפఠ݂ೞ૑?
    • ా೤ ؘ੉ఠ ਝযೞ਋झ
    • Feature store
    • ؘ੉ఠ ੿೤ࢿ
    • ML ࢿמী о੢ ௾ ৔ೱਸ ઱ח ࠗ࠙
    • ߡӒ ೴౴੄ োࣘ
    Sungjoo Ha
    24

    View Slide

  25. AI PM
    • ೐۽؋౟੄ ઱ਃ ޙઁܳ AI ޙઁ۽ যڌѱ ஖ജೡ ࣻ ੓ਸө?
    • ੉ܳ ా೧ ࢎਊ੗ٜ੄ ݅઒بо য়ܰҊ ؊ ݆਷ ؘ੉ఠܳ ٜ݅যղҊ ؊
    ա਷ ઁಿਵ۽ ੉য૑ח
    • ੿׹ਸ ঳ӝ ए਍ ҃਋ب ੓Ҋ য۰਍ ҃਋ب ੓Ҋ
    • AI ӝࣿਸ ੉೧ೞҊ ੉ী ݏ୾ ઁಿਸ э੉ ҳࢿ೧ঠ ೣ
    • ӝദী ؀ೠ ੄ࢎ ࣗాਸ ਤೠ ࢜۽਍ ঱য/بҳ/೐۽ࣁझ ೙ਃ
    Sungjoo Ha
    25

    View Slide

  26. TikTok
    • ߑ؀ೠ ন੄ ௑బஎ
    • ন਷ ૕ਸ ׸ࠁೠ׮ח Ѻ঱ೞী ؀ӏݽ ݃ா౴ਸ ాೠ ݽٚ פ஖੄ ൚ࣻ
    • ࣚऔѱ ௑బஎܳ ٜ݅ ࣻ ੓ب۾ ب৬઱ח ׮নೠ ઁ੘ بҳ ઁҕ
    • ݺߔೠ ੿׹ਸ ૒੽੸ਵ۽ ঳ਸ ࣻ ੓ח ߑधਵ۽ UIܳ ҳࢿ
    • ೠ ߣী ೞա੄ ࠺٣য়݅ ࠁݴ ࢎਊ੗੄ swipe ৈࠗ۽ ஂೱਸ ݺߔೞѱ ഛੋ
    • implicit feedback -> explicit feedback
    • ৘ஏ ӝ߈੄ ࢎਊ੗ ݽ؛݂
    • ױࣽ൤ ੋध(recognition)ীࢲ Ӓ஖ח Ѫ੉ ইפۄ ࢎਊ੗о যڌѱ ೯زೞ৓ח૑ ݺߔೠ ೖ٘ߔਸ ߉ب۾
    ҳࢿ
    Sungjoo Ha
    26

    View Slide

  27. Culture
    • ੌೞח ߑध, ޙചب ઺ਃ
    • AI ӝദ੗ۄחѱ ই૒ ઓ੤ೞ૑ח ঋ਺
    • Ӓ۞׮ࠁפ э੉ যڌѱ ੌೡ૑ী ؀೧ࢲب Ҋ޹೧ࠊঠ ೣ
    • ݽٚ ઑ૒/ҳࢿਗ੉ ੉ী ؀ೠ Ӓܿਸ ݠ݁ࣘী ыҊ ӝৈ೧ঠ ೣ
    • ੄ࢎ Ѿ੿ӂ੗ٜب ੄޷੓ח ࣻળਵ۽ AIী ؀೧ ੉೧೧ঠ
    • ઑ૒ҳઑب ੉۞ೠ ղਊਸ ߈৔೧ঠ
    • Conway੄ ߨ஗
    Sungjoo Ha
    27

    View Slide

  28. Conclusion
    • AI ӝࣿਸ ా೧ ੹ী ೡ ࣻ হ؍ Ѫਸ ೡ ࣻ ੓ח ഥࢎٜ੉ ࢤӡ Ѫ
    • AI ӝࣿ ੗୓݅ ੜ ഝਊೞח Ѫਵ۽ ੉޷ ର߹ചח оמ
    • ӝࣿী ؀ೠ ੉೧৬ ઑ૒ ҳઑ ߂ সޖ ߑध ߂ ޙച ١ ׮নೠ ஏݶীࢲ Ҋ۰೧ঠ AI ӝࣿਸ ੜ
    ഝਊೡ ࣻ ੓਺
    • ੄ࢎ Ѿ੿੉ ੉ܖয૑ח ݽٚ Ҕীࢲ ݽ؛੉ ഝਊؼ ৈ૑о ੓਺
    • ೐۽؋౟੄ ઺ਃೠ ޙઁܳ AI ޙઁ۽ ஖ജೞݶ ੉࢚੸
    • ੉ܳ ߄ఔਵ۽ ML ݽ؛੄ ѐࢶ੉ ೐۽؋౟੄ ѐࢶਵ۽ ੉য૑Ҋ ੉ח ؊ ݆਷ ؘ੉ఠ۽ ੉য
    ૑ח ࢶࣽജ ҳઑܳ ٜ݅যյ ࣻ ੓਺
    Sungjoo Ha
    28

    View Slide