Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PAKDD2023 Tutorial 2: A Gentle Introduction to Technologies Behind Language Models and Recent Achievement in ChatGPT (Parts 3 and 4)

PAKDD2023 Tutorial 2: A Gentle Introduction to Technologies Behind Language Models and Recent Achievement in ChatGPT (Parts 3 and 4)

Jun Suzuki, Kyosuke Nishida, Naoaki Okazaki: "PAKDD 2023 Tutorial 2: A Gentle Introduction to Technologies Behind Language Models and Recent Achievement in ChatGPT," The 27th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2023) , May 25, 2023.

Parts 1 and 2: https://www.fai.cds.tohoku.ac.jp/research/activities/
Parts 3 and 4: this deck.
Part 5: https://speakerdeck.com/chokkan/efforts-for-responsible-llms-pakdd-2023-tutorial-2

Kyosuke Nishida

May 25, 2023
Tweet

More Decks by Kyosuke Nishida

Other Decks in Research

Transcript

  1. 1BSUTBOE
    5FDIOPMPHJFTBOE3FDFOU"DIJFWFNFOUT
    JO$IBU(15MJLF--.T
    /55)VNBO*OGPSNBUJDT-BCPSBUPSJFT
    ,ZPTVLF /JTIJEB
    .BZ
    1",%%5VUPSJBM
    "(FOUMF*OUSPEVDUJPOUP5FDIOPMPHJFT#FIJOE
    -BOHVBHF.PEFMTBOE3FDFOU"DIJFWFNFOUJO$IBU(15

    View full-size slide


  2. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(1 5IV

    l 1BSU-BOHVBHFNPEFMT -.T

    l 1BSU-BSHFMBOHVBHFNPEFMT --.T

    u 4QFBLFS 1SPGFTTPS+VO4V[VLJ 5PIPLV6OJWFSTJUZ

    u 4MJEFT IUUQTXXXGBJDETUPIPLVBDKQSFTFBSDIBDUJWJUJFT
    l 1BSU5FDIOPMPHJFT6OEFSMZJOH$IBU(15MJLF--.T
    l 1BSU3FDFOUBDIJFWFNFOUTJO$IBU(15MJLF--.T
    l 1BSU&GGPSUTGPS3FTQPOTJCMF--.T
    u 4QFBLFS1SPGFTTPS/BPBLJ 0LB[BLJ 5PLZP*OTUJUVUFPG5FDIOPMPHZ

    u 4MJEFT IUUQTTQFBLFSEFDLDPNDIPLLBOFGGPSUTGPSSFTQPOTJCMFMMNT
    QBLEEUVUPSJBM

    View full-size slide


  3. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(1 5IV

    l 1BSU5FDIOPMPHJFT6OEFSMZJOH$IBU(15MJLF--.T

    View full-size slide


  4. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUT
    l 0QFO"* NPEFMTSFMFBTFEGPMMPXJOH(15
    u $PEFY5SBJOJOHGSPNDPEFBOEUFYU
    u *OTUSVDU(15 BOE$IBU(15*OTUSVDUJPOUVOJOHBOE3-)'
    u (15BOE$IBU(15QMVHJOT.VMUJNPEBMBOEFYUFSOBMLOPXMFEHF
    JOUFHSBUJPO
    l 0QFOTPVSDFNPEFMT
    u --B.""OPQFO--.USBJOFEPOPWFSPOFUSJMMJPOUPLFOT
    u "MQBDBBOE7JDVOB'JOFUVOFENPEFMTCBTFEPO--B."
    u .15"OFXDPNNFSDJBMMZVTBCMFNPEFMDBQBCMFPGIBOEMJOHMPOH
    DPOUFYU
    u --B7" BOE.JOJ(15NVMUJNPEBMNPEFMT

    View full-size slide


  5. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    &WPMVUJPOGSPN(15UP(15
    l #FUXFFO(15BOE(15 UIFSFBSFUISFFNPEFMTSFGFSSFEUPBT
    UIF(15GBNJMZ
    l (15 .BZ QBQFS

    ˣ 5SBJOJOH-.TXJUITPVSDFDPEF
    l $PEFY +VMZ QBQFS

    ˣ 5SBJOJOH-.TUPGPMMPXJOTUSVDUJPOTXJUIIVNBOGFFECBDL
    l *OTUSVDU(15 +BO QBQFS

    ˣ 0QUJNJ[BUJPO GPSDIBU
    l $IBU(15 /PW CMPH

    ˣ .VMUJNPEBMJOUFHSBUJPO
    l (15 .BS UFDIOJDBMSFQPSUCMPH

    View full-size slide


  6. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $PEFY<$IFO 0QFO"*
    +VMZ>
    l #VJMUPOUIFTBNFGPVOEBUJPOBT(15
    l 5PFOBCMFUIF--.UPHFOFSBUFDPEFJOSFTQPOTFUPVTFSJOTUSVDUJPOT
    JUTUSBJOJOHEBUBTFUJODMVEFTCJMMJPOTPGMJOFTPGDPEF
    l *UTDPOUFYUJTFYQBOEFEUP UPLFOT
    l 5IFMBUFTU"1*WFSTJPO DPEFEBWJODJ
    XPSLTJOPWFSUFOMBOHVBHFT
    UIFQSPNQU
    NFUIPEEFGJOJUJPOTBOE
    *OQVUPVUQVUFYBNQMFT

    QSPWJEFEUPUIFNPEFM
    NPEFMHFOFSBUFEDPNQMFUJPO
    IUUQTBSYJWPSHBCT

    View full-size slide


  7. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (JU)VC$PQJMPU<+VOF>
    l $PEFYQPXFSTBOl"*QBJSQSPHSBNNFSzPO(JUIVC
    l 4JODFJUTSFMFBTF JUIBTUSBOTGPSNFEEFWFMPQFSQSPEVDUJWJUZGPS
    NPSFUIBOPOFNJMMJPOQFPQMF IFMQJOHEFWFMPQFSTDPEFVQUP
    GBTUFS<>
    <>IUUQTHJUIVCCMPHHJUIVCDPQJMPUGPSCVTJOFTTJTOPXBWBJMBCMF

    View full-size slide


  8. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    5PLFOJ[FSPG$PEFY
    l .PEJGJFEGSPN(15 XJUIUIFBEEJUJPOPGUPDPOTFDVUJWF
    XIJUFTQBDFTUPUIFWPDBCVMBSZ
    5IFTFFYBNQMFTXFSFHFOFSBUFEVTJOHIUUQTQMBUGPSNPQFOBJDPNUPLFOJ[FS
    $PEFY`TUPLFOJ[BUJPO SL@CBTF
    (15 QL@CBTF

    DPOTFDVUJWFXIJUFTQBDFT
    BSFDPOWFSUFEJOUPBTJOHMFUPLFO

    View full-size slide


  9. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    *OTUSVDU(15 <0VZBOH 0QFO"*
    .BSDI>
    l 4JHOJGJDBOUMZPVUQFSGPSNT(15JOGPMMPXJOHVTFSJOTUSVDUJPOT
    SFTVMUJOHJOPVUQVUTUIBUBSFNPSFUSVUIGVMBOEMFTTUPYJD
    IUUQTPQFOBJDPNSFTFBSDIJOTUSVDUJPOGPMMPXJOHTBNQMF
    VTFSJOTUSVDUJPO
    5IJTBOTXFSlBMJHOTzXJUIUIFJOTUSVDUJPO
    (15GBJMTUPSFTQPOEUPUIFVTFSJOTUSVDUJPOT

    View full-size slide


  10. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    IUUQTBSYJWPSHBCT
    *OTUSVDUJPOUVOJOH 3FJOGPSDFNFOU-FBSOJOHGSPN)VNBO'FFECBDL 3-)'

    -FBSOJOHTUFQTPG*OTUSVDU(15

    View full-size slide


  11. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    *OTUSVDUJPOUVOJOH<8FJ (PPHMF
    4FQ><$IVOH (PPHMF
    0DU>
    l 'JOFUVOJOHMBOHVBHFNPEFMTPOBDPMMFDUJPOPGUBTLTEFTDSJCFEWJB
    JOTUSVDUJPOT JNQSPWFT[FSPTIPUQFSGPSNBODFPOVOTFFOUBTLT
    l 0QFO"* VTFEBCPVULUSBJOJOHQSPNQUT GSPNUIF0QFO"* "1*
    BOEMBCFMFSXSJUUFO
    GPSJOTUSVDUJPOUVOJOH
    u 5IJTTVQFSWJTFEGJOFUVOFENPEFMJTDBMMFEUIF4'5 NPEFM
    IUUQTBSYJWPSHBCT

    View full-size slide


  12. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    3FXBSENPEFMJOH
    l -BCFMFSTXFSFQSFTFOUFEXJUICFUXFFOBOESFTQPOTFTUPFBDI
    QSPNQUBOEBTLFEUPSBOLUIFN
    l 5IFSFXBSENPEFM TJ[FPG#
    XBTUSBJOFEXJUIBCPVULUSBJOJOH
    QSPNQUT GSPNUIF0QFO"* "1*BOEMBCFMFSXSJUUFO

    ,SFTQPOTFT !
    "
    DPNQBSJTPOT
    SBOLJOH

    View full-size slide


  13. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    3FJOGPSDFNFOU-FBSOJOHGSPN)VNBO'FFECBDL 3-)'

    l 0SJHJOBMMZEFWFMPQFEGPSUSBJOJOHTJNQMFSPCPUTBOE"UBSJHBNFT
    <$ISJTUJBOP *CBS[ > UIFOBQQMJFEUPUIFGJOFUVOJOH
    PG-.TUPTVNNBSJ[FUFYU<4UJFOOPO >
    l 0QFO"* IBTBMTPBQQMJFE3-)'UPGJOFUVOFUIF4'5NPEFM VTJOH
    BCPVULUSBJOJOHQSPNQUT GSPNUIF0QFO"* "1*

    l 3-0CKFDUJWF110QUY BWBSJBOUPG110<4DIVMNBO >
    ",VMMCBDL-JCMFS QFOBMUZGSPNUIF4'5NPEFM UP
    NJUJHBUFPWFSPQUJNJ[BUJPOPGUIFSFXBSENPEFM
    1SFUSBJOJOHNJYUPGJYUIFQFSGPSNBODF
    SFHSFTTJPOTPOQVCMJD/-1EBUBTFUT

    View full-size slide


  14. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15<0QFO"* /PW >
    l 0QUJNJ[FEGPSDIBUVTJOHUIFTBNFNFUIPETBT*OTUSVDU(15<>
    u /PGVSUIFSEFUBJMTBCPVUUIFBSDIJUFDUVSF NPEFMTJ[F EBUBTFU
    DPOTUSVDUJPO PSUSBJOJOHNFUIPETIBWFCFFOSFQPSUFE
    <>IUUQTPQFOBJDPNCMPHDIBUHQU
    'PSJOTUSVDUJPOUVOJOH
    )VNBO"*USBJOFSTQSPWJEFEDPOWFSTBUJPOT
    JOXIJDIUIFZQMBZFECPUITJEFT<>

    View full-size slide


  15. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    5PLFOJ[FSPG$IBU(15
    l /FXMZDSFBUFE DBMMFElDML@CBTFz
    BOEUIFWPDBCVMBSZTJ[FIBT
    CFFOFYQBOEFEVQUPL
    *OTUSVDU(15 SL@CBTF
    `T
    UPLFOJ[BUJPOPGl͜Μʹͪ͸z
    lIFMMPzJO+BQBOFTF

    5PLFOTIBWFCFFOJODSFBTFEGPS
    QSPHSBNJOHMBOHVBHFTBOEOPO&OHMJTIMBOHVBHFT
    "QBSUPG
    lDML@CBTFz
    WPDBCVMBSZ

    View full-size slide


  16. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15<0QFO"* .BSDI >
    l $BOBDDFQUJNBHFBOEUFYUJOQVUTBOEQSPEVDFUFYUPVUQVUT
    (15`TSFTQPOTF
    IUUQTBSYJWPSHBCT

    View full-size slide


  17. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15<0QFO"* .BSDI >
    l "5SBOTGPSNFSTUZMFNPEFMQSFUSBJOFEUPQSFEJDUUIFOFYUUPLFOJO
    BEPDVNFOU XJUIBDPOUFYUPGLUPLFOT
    BOEGJOFUVOFEVTJOH3-)'
    u /PGVSUIFSEFUBJMTBCPVUUIFBSDIJUFDUVSF NPEFMTJ[F IBSEXBSF USBJOJOH
    DPNQVUBUJPO EBUBTFUDPOTUSVDUJPO PSUSBJOJOHNFUIPETIBWFCFFO
    SFQPSUFE
    *UJTVODMFBSIPXUIFJNBHFTBOEUFYUGPSN
    UIFJOQVUTFRVFODFBOEBSFGFEUP(15
    *UJTVODMFBSIPXJUQSPDFTTFTMPOHTFRVFODFT
    PGVQUPLUPLFOTBUIJHITQFFE

    View full-size slide


  18. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15 1MVHJOT<0QFO"* .BSDI >
    l 1MVHJOTBSFFYUFOTJPOT JODMVEJOHUIJSEQBSUZBQQMJDBUJPOT
    UIBU
    BMMPXNPEFMTUPJOUFSBDUXJUIFYUFSOBM"1*T
    IUUQTPQFOBJDPNCMPHDIBUHQUQMVHJOT

    View full-size slide


  19. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15 1MVHJOT<0QFO"* .BSDI >
    l 5IFl8FC#SPXTFSzQMVHJOBMMPXTBDDFTTUPVQUPEBUFJOGPSNBUJPO
    IUUQTPQFOBJDPNCMPHDIBUHQUQMVHJOT
    18
    IUUQTPQFOBJDPNCMPHDIBUHQUQMVHJOT
    5IFNPEFMJTBCMFUPQSPWJEFVQUPEBUF
    JOGPSNBUJPO FWFOUIPVHIJUJTPOMZUSBJOFE
    POEBUBVQUP0DUPCFS

    View full-size slide


  20. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15 1MVHJOT<0QFO"* .BSDI >
    l 5IFl8FC#SPXTFSz QMVHJOBMMPXTBDDFTTUPVQUPEBUFJOGPSNBUJPO
    IUUQTPQFOBJDPNCMPHDIBUHQUQMVHJOT
    19
    IUUQTPQFOBJDPNCMPHDIBUHQUQMVHJOT

    View full-size slide


  21. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15 1MVHJOT<0QFO"* .BSDI >
    l 5IF8PMGSBNQMVHJOBMMPXTBDDFTTUPQPXFSGVMDPNQVUBUJPO
    BDDVSBUFNBUI DVSBUFELOPXMFEHF BOEWJTVBMJ[BUJPO
    20

    View full-size slide


  22. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUTPGUIJTQBSU
    l 0QFO"*bT NPEFMT
    u $PEFYUSBJOJOHGSPNDPEFBOEUFYU
    u *OTUSVDU(15 BOE$IBU(15JOTUSVDUJPOUVOJOHBOE3-)'
    u (15NVMUJNPEBMJOUFHSBUJPO
    l 0QFOTPVSDFNPEFMT
    u --B."BOPQFO--.USBJOFEPOPWFSPOFUSJMMJPOUPLFOT
    u "MQBDBBOE7JDVOB'JOFUVOFENPEFMTCBTFEPO--B."
    u .15BDPNNFSDJBMMZVTBCMFNPEFMDBQBCMFPGIBOEMJOHMPOH
    DPOUFYU
    u --B7" BOE.JOJ(15NVMUJNPEBMNPEFMT

    View full-size slide


  23. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    --B." <5PVWSPO .FUB
    'FC >
    l "OPQFOTPVSDF--.USBJOFEXJUIQVCMJDMZBWBJMBCMFEBUBTFUT
    l 5IFQSFUSBJOJOHEBUBIBTCFFOSFDFOUMZSFQSPEVDFEBT
    l3FE1BKBNB%BUBz<5PHFUIFS$PNQVUFS>BOEJTBWBJMBCMFGPSBOZPOF
    IUUQTBSYJWPSHBCT
    USJMMJPO
    UPLFOT

    View full-size slide


  24. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l --B."#JTDPNQBSBCMFUP(15 #
    PONPTUCFODINBSLT
    l 5IFOVNCFSPGUSBJOJOHUPLFOTJO--B." 5
    XBTTJHOJGJDBOUMZ
    HSFBUFSUIBOUIBUPG(15 #

    u DG$IJODIJMMB<)PGGNBO %FFQNJOE
    >

    ;FSPTIPUQFSGPSNBODFPODPNNPOTFOTFSFBTPOJOH
    IUUQTBSYJWPSHBCT

    View full-size slide


  25. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "SDIJUFDUVSFBOEUSBJOJOHNFUIPE
    l --B." MFWFSBHFTWBSJPVTJNQSPWFNFOUTUIBUXFSFQSPQPTFE BGUFS
    UIFPSJHJOBM5SBOTGPSNFSQVCMJDBUJPO
    u (15`TQSFOPSNBMJ[BUJPO
    u 1B-.`T 4XJ(-6 BDUJWBUJPOGVODUJPO<4IB[FFS>
    u (15/FP`T3PUBSZ&NCFEEJOHT<4V >
    l 5IFUSBJOJOHNFUIPEJTUIFTBNFBT(15 JF OFYUUPLFO
    QSFEJDUJPO

    l --B."JUTFMGJTOPUUSBJOFEXJUIJOTUSVDUJPOUVOJOHBOE3-)'

    View full-size slide


  26. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "MQBDB<4UBOGPSE6 .BSDI >
    l "NPEFMGJOFUVOFEGSPNUIF--B." #NPEFMPO,JOTUSVDUJPO
    GPMMPXJOHEFNPOTUSBUJPOT
    l "TFMGJOTUSVDUUFDIOJRVF<8BOH >XBTVTFEUPBVUPNBUJDBMMZ
    HFOFSBUFUIFEFNPOTUSBUJPOGSPNPOMZIVNBOXSJUUFOTFFEUBTLT
    u 5IFUFYUEBWJODJ *OTUSVDU(15
    NPEFMXBTVTFEJOUIJTTUVEZ
    IUUQTDSGNTUBOGPSEFEV
    BMQBDBIUNM

    View full-size slide


  27. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "MQBDB-P3" <.BSDI >
    l -P3" <)V >GSFF[FTUIFQSFUSBJOFEXFJHIUTBOEJOKFDUT
    USBJOBCMFBEBQUFST SBOLEFDPNQPTJUJPONBUSJDFT
    JOUPFBDIMBZFS
    l 6TJOH--B." -P3" BOEUIF"MQBDBEBUBTFU BOZPOFDBODIFBQMZ
    QSPEVDF$IBU(15MJLFNPEFMTXJUIJOIPVSTPOBTJOHMF(16
    IUUQTBSYJWPSHBCT
    IUUQTHJUIVCDPNUMPFOBMQBDBMPSB
    7BSJPVTBEBQUFSTIBWFCFFO
    SFMFBTFEUPNBLF"MQBDBDPNQBUJCMF
    XJUIOPO&OHMJTIMBOHVBHFT

    View full-size slide


  28. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    7JDVOB<6$#FSLFMFZ $.6 4UBOGPSE BOE6$4BO%JFHP.BSDI >
    l "NPEFMGJOFUVOFEGSPN--B." PO,VTFSTIBSFE
    DPOWFSTBUJPOTDPMMFDUFEGSPNUIF4IBSF(15 XFCTJUF
    l &WBMVBUJPOVTJOH(15BTBKVEHFTIPXTUIBU7JDVOB#
    BDIJFWFTNPSFUIBOUIFRVBMJUZPG$IBU(15
    5XPXFFLTBQBSU 5XPXFFLTBQBSU
    IUUQTMNTZTPSHCMPHWJDVOB

    View full-size slide


  29. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    7JDVOB`T"VUPNBUJDFWBMVBUJPOVTJOH(15
    l (15DBOQSPEVDFIJHIMZDPOTJTUFOUSBOLTBOEBEFUBJMFE
    BTTFTTNFOUXIFODPNQBSJOHDIBUCPUT`BOTXFST
    IUUQTMNTZTPSHCMPHWJDVOB

    View full-size slide


  30. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    4UBCMF7JDVOB <4UBCJMJUZ"* "QSJM >
    l "MQBDBBOE7JDVOBVTFEPOMZJOTUSVDUJPOUVOJOH
    l 4UBCMF7JDVOB JTUIFGJSTUMBSHFTDBMFPQFOTPVSDFDIBUCPUUSBJOFE
    WJB3-)'VTJOHUIFGPMMPXJOHPQFOEBUBTFUT
    u 0QFO"TTJTUBOU $POWFSTBUJPOT 0"445
    <"QSJM >
    u 4UBOGPSE)VNBO1SFGFSFODFT 4)1
    <'FCSVBSZ >
    u "OUISPQJD))3-)'<#BJ "QSJM >

    View full-size slide


  31. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    .15l "OPQFOTPVSDF DPNNFSDJBMMZVTBCMF--. USBJOFEGSPNTDSBUDI
    PO5UPLFOTPGUFYUBOEDPEF
    l .15#NBUDIFTUIFRVBMJUZPG--B."#POTUBOEBSEUBTLT
    IUUQTXXXNPTBJDNM
    DPNCMPHNQUC

    View full-size slide


  32. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    .15#4UPSZ8SJUFSLl #VJMUCZGJOFUVOJOH.15#XJUIBDPOUFYUMFOHUIPGLUPLFOTUP
    SFBEBOEXSJUFTUPSJFTXJUITVQFSMPOHDPOUFYUMFOHUIT
    u $BOSFBEUIFFOUJSFUFYUPGl3PNFPBOE+VMJFUz<> XIJDIIBTBCPVUL
    UPLFOTVTJOH(15`TUPLFOJ[FS DML@CBTF

    l 6TFT"-J#J <1SFTT "VHVTU>JOTUFBEPGQPTJUJPOBMFNCFEEJOHT
    UPIBOEMFTVDIMPOHDPOUFYUT
    <>IUUQTXXXHVUFOCFSHPSHDBDIFFQVCQHUYU

    View full-size slide


  33. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "-J#J "UUFOUJPOXJUI-JOFBS#JBTFT

    l CJBTFTRVFSZLFZBUUFOUJPOTDPSFTXJUIBIFBETQFDJGJDQFOBMUZUIBU
    JTQSPQPSUJPOBMUPUIFJSEJTUBODF
    https://arxiv.org/abs/2108.12409
    XIFSFTDBMBSNJTBIFBETQFDJGJDTMPQFGJYFECFGPSFUSBJOJOH
    <1SFTT 68BTIJOHUPO'"*3
    "VHVTU >
    "MJ#JFOBCMFTUIF53"*/
    4)035 5&45-0/(DPODFQU
    XIFSFUIFNPEFMUSBJOFEPOUIF
    DPOUFYUMFOHUIPGQFSGPSNT
    XFMMGPSLJOQVUUPLFOTBU
    JOGFSFODFUJNF UIBOLTUP"MJCJ

    View full-size slide


  34. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l 5IFSFBSFTFWFSBMPQFOTPVSDFTUVEJFTGPSNVMUJNPEBMNPEFMT
    l 'PSFYBNQMF --B7" BMMPXTGPSWJTVBMMZHSPVOEFEDIBUTMJLF(15
    IUUQTBSYJWPSHBCT
    --B7" <-JV 68JTDPOTJOr.BEJTPO
    "QSJM >

    View full-size slide


  35. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l --B7" HFOFSBUFTWJTVBMJOTUSVDUJPOUVOJOHEBUBCZQBTTJOH$0$0
    DBQUJPOTBOECPVOEJOHCPYFTUPUIFMBOHVBHFPOMZ(15
    l 5IFSFTVMUJOHEBUBTFUJTVTFEUPGJOFUVOFNVMUJNPEBMNPEFMT
    6TJOH-BOHVBHFPOMZ(15
    7JTVBM
    *OTUSVDUJPO5VOJOH
    EBUBTFU
    IUUQTBSYJWPSHBCT
    --B7"`T 7JTVBM*OTUSVDUJPO5VOJOH

    View full-size slide


  36. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l --B7" DPOTJTUTPG--B." BOEUIFWJTVBMFODPEFSPG$-*1
    l 'PSBOJOQVUJNBHF UIFQSFUSBJOFE$-*1WJTVBMFODPEFS 7J5-

    DBOPVUQVUHSJEGFBUVSFTBTWJTVBMUPLFOT
    $-*1`THSJEGFBUVSFT
    <3BEGPSE +BOVBSZ>
    8IBUJTVOVTVBMBCPVU
    UIJTJNBHF
    5IFVOVTVBM
    BTQFDUPG
    UIJTJNBHF
    JTBNBO
    JSPOJOHʜ
    --B."
    IUUQTBSYJWPSHBCT
    .PEFM"SDIJUFDUVSF

    View full-size slide


  37. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    .JOJ(15<;IV ,JOH"CEVMMBI6
    "QSJM >
    IUUQTXXXZPVUVCFDPNXBUDI W@@UGUPYQ#"X

    View full-size slide


  38. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    .JOJ(15<;IV ,JOH"CEVMMBI6
    "QSJM >
    l $POTJTUTPG7JDVOBBOE#-*1`TWJTVBMFODPEFS<-J 4BMFTGPSDF
    b>
    l 3FRVJSFTPOMZUSBJOJOHUIFMJOFBSQSPKFDUJPOMBZFSUPBMJHOUIFWJTVBM
    GFBUVSFTXJUIUIF7JDVOB
    BLIP-2
    IUUQTBSYJWPSHBCT

    View full-size slide


  39. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    4VNNBSZ
    l .PEFSO--.T TUBSUJOHGSPN$PEFY BSFUSBJOFEPONBTTJWF
    BNPVOUTPGUFYUBOEDPEF
    l *NQPSUBOUUFDIOJRVFTGPSBMJHOJOHNPEFMTUPGPMMPXJOTUSVDUJPOTBSF
    lJOTUSVDUJPOUVSOJOHzBOElSFJOGPSDFNFOUMFBSOJOHGSPNIVNBO
    GFFECBDL 3-)'
    z
    l 5IFUFDIOJDBMEFUBJMTPG(15IBWFOPUZFUCFFOSFQPSUFE
    l 4JODFUIFMBVODIPG--B." UIFSFIBTCFFOSFNBSLBCMFQSPHSFTTJO
    UIFEFWFMPQNFOUPGPQFOTPVSDFNPEFMTBOEEBUBTFUT

    View full-size slide


  40. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(1 5IV

    l 1BSU3FDFOUBDIJFWFNFOUTJO$IBU(15MJLF--.T

    View full-size slide


  41. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $IBU(15 JTUIFGBTUFTUHSPXJOHBQQFWFS
    l $IBU(15 SFBDIFTNJMMJPOVTFSTUXPNPOUITBGUFSMBVODI
    l *UIBTBUUSBDUFEBHSFBUEFBMPGBUUFOUJPOEVFUPJUTIJHIQFSGPSNBODF
    IUUQTUXJUUFSDPN&DPOPNZ"QQTUBUVT

    View full-size slide


  42. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    4QBSLTPG"(*<#VCCFDL .JDSPTPGU
    .BZ >
    l 0OFXFFLBGUFSUIFSFMFBTFPG(15 .JDSPTPGUQVCMJTIFEBQBQFS
    FOUJUMFEl4QBSLTPG"SUJGJDJBM(FOFSBM*OUFMMJHFODF&BSMZ&YQFSJNFOUT
    XJUI(15z
    l 5IFBVUIPSTTUBUFEUIBU(15DPVMECFDPOTJEFSFEBOFBSMZ
    WFSTJPOPGBSUJGJDJBMJOUFMMJHFODF "(*
    CFDBVTFPGUIFCSFBEUIBOE
    EFQUIPG(15TDBQBCJMJUJFTJONBUIFNBUJDT DPEJOH WJTJPO
    NFEJDJOF MBX QTZDIPMPHZ BOENPSF
    l 5IFBVUIPSTVTFEUIFUFSNl"(*zUPSFGFSUPTZTUFNTUIBU
    EFNPOTUSBUFCSPBEJOUFMMJHFODFDBQBCJMJUJFTBUPSBCPWFIVNBO
    MFWFM

    View full-size slide


  43. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUT
    l 1FSGPSNBODFJO/BUVSBM-BOHVBHF1SPDFTTJOHCFODINBSLTBOE
    FYBNT
    l *OUFSFTUJOHSFTVMUTJO7JTJPOBOE-BOHVBHF6OEFSTUBOEJOH
    FWBMVBUJPOT
    l 0QFOTPVSDFBQQMJDBUJPOTQPXFSFECZ--.T
    l 3FNBJOJOHJTTVFT

    View full-size slide


  44. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15QBTTFTUIFCBSFYBN<,BU[ *MMJOPJT5FDI
    .BSDI >
    l (15 [FSPTIPUFWBMVBUJPOT
    IBTQBTTFEUIFFOUJSF6OJGPSN#BS
    &YBNJOBUJPO 6#&
    JODMVEJOHOPUPOMZNVMUJQMFDIPJDF .#&
    CVU
    BMTP PQFOFOEFERVFTUJPOT .&&BOE.15

    l (15BDIJFWFEBOPWFSBMMTDPSFPGQPJOUT TJHOJGJDBOUMZ
    TVSQBTTJOHUIFQBTTJOHTDPSFSBOHFPGUPQPJOUT
    IUUQTQBQFSTTTSODPNTPMQBQFSTDGN BCTUSBDU@JE

    View full-size slide


  45. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l (15 [FSPTIPUFWBM
    FYDFFETUIFQBTTJOHTDPSFPO64.-&
    6OJUFE4UBUFT.FEJDBM-JDFOTJOH&YBNJOBUJPO
    CZPWFSQPJOUT
    IUUQTBSYJWPSHBCT
    QBTTUISFTIPMETBQQSPYJNBUFMZPG
    NVMUJQMFDIPJDFRVFTUJPOT
    (15QBTTFTNFEJDBMMJDFOTJOHFYBN .BZ>

    View full-size slide


  46. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l (15JTDPNQBSBCMFUPIVNBOQFSGPSNBODFPOBCFODINBSLPG
    VOTFFO QSPHSBNNJOHQSPCMFNTPO-FFU$PEF
    l (15BMTPQBTTFTNPDLUFDIOJDBMJOUFSWJFXTBUNBKPSUFDI
    DPNQBOJFTPO-FFU$PEF
    (15DPVMECFBTPGUXBSFFOHJOFFS<#VCCFL .JDSPTPGU
    .BSDI >
    QBTT!L NFBTVSFTXIFUIFSUIFNPEFM
    QSPEVDFTBDPSSFDUTPMVUJPOXJUIJOLUIBUUFNQUT
    (15TPMWFEBMMRVFTUJPOTJOPOMZ
    NJOVUFTPGUIFBMMPUUFE IPVST
    https://arxiv.org/abs/2303.12712

    View full-size slide


  47. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l (15PVUQFSGPSNT(15PONPTUBDBEFNJDBOEQSPGFTTJPOBM
    FYBNT JODMVEJOHNVMUJQMFDIPJDFBOEGSFFSFTQPOTFRVFTUJPOT
    l (15`TQFSGPSNBODFJTCFUUFSXIFOJUUBLFTWJTJPOBTJOQVUT
    IUUQTBSYJWPSHBCT
    1FSGPSNBODFPOFYBNT<0QFO"* "QSJM >

    View full-size slide


  48. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l (15PVUQFSGPSNFE TUBUFPGUIFBSUNPEFMTXJUICFODINBSL
    TQFDJGJDUSBJOJOH PONBOZBDBEFNJDCFODINBSLT
    (15DBOCFBUGJOFUVOFE405"NPEFMT<0QFO"* "QSJM >
    https://openai.com/
    research/gpt-4

    View full-size slide


  49. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    l (15PVUQFSGPSNTUIF&OHMJTIMBOHVBHFQFSGPSNBODFPGFYJTUJOH
    --.T GPSUIFWBTUNBKPSJUZPGMBOHVBHFTUFTUFE JODMVEJOHMPX
    SFTPVSDFMBOHVBHFTTVDIBT-BUWJBO 8FMTI BOE4XBIJMJ
    1FSGPSNBODFPO.VMUJMJOHVBM..-6<0QFO"* "QSJM >
    .VMUJMJOHVBM
    ..-6CFODINBSL
    USBOTMBUFEXJUI
    "[VSF

    View full-size slide


  50. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15PVUQFSGPSNTDSPXEXPSLFST<1BO 6$#
    "QSJM >
    l (15BVUPNBUJPOPGTDFOBSJPMBCFMJOHJOBTPDJBMEFDJTJPONBLJOH
    UBTLBDIJFWFECFUUFSSFTVMUTUIBOIVNBOBOOPUBUPST
    IUUQTBSYJWPSHBCT
    8IFOBOFOTFNCMFPGUISFF
    FYQFSUT JTDPSSFDU (15
    PVUQFSGPSNTBOFOTFNCMFPG
    UISFFDSPXEXPSLFSTJO
    DBUFHPSJFT

    View full-size slide


  51. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUT
    l 1FSGPSNBODFJO/BUVSBM-BOHVBHF1SPDFTTJOHCFODINBSLTBOE
    FYBNT
    l *OUFSFTUJOHSFTVMUTJO7JTJPOBOE-BOHVBHF6OEFSTUBOEJOH
    FWBMVBUJPOT
    l 0QFOTPVSDFBQQMJDBUJPOTQPXFSFECZ--.T
    l 3FNBJOJOHJTTVFT

    View full-size slide


  52. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    1FSGPSNBODFPOBDBEFNJDCFODINBSLT<0QFO"* >
    l (15BDIJFWFTWFSZIJHITDPSFTPOCFODINBSLTTVDIBTDIBSUT
    EJBHSBNT BOEJOGPHSBQIJDT BMUIPVHIOPUZFUBUUIFIVNBOMFWFM
    51
    Human
    80.78%
    85.01%
    N/A
    N/A
    98.1%
    98.0%
    89.61%
    N/A
    * Updated
    70.5%*
    (DePlot)
    90.1%*
    (BAIDU-DI)
    A: 4-shot, CoT prompting
    https://openai.com/research/gpt-4

    View full-size slide


  53. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15`TESBXJOHT[Bubbek(Microsoft)+, March 22, 2023]
    IUUQTBSYJWPSHBCT
    l (15DBOESBXJNBHFTXJUIQSPHSBNNJOHMBOHVBHFT
    1SPNQUl%SBXBVOJDPSOJO5JL;z
    5IFSJHIUNPTUJNBHFJTBMBUFSWFSTJPOPG(15JOUSBJOJOH

    View full-size slide


  54. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15`TESBXJOHT[Bubbek(Microsoft)+, March 22, 2023]
    IUUQTBSYJWPSHBCT
    l (15DBOESBXJNBHFTXJUIQSPHSBNNJOHMBOHVBHFT
    1SPNQUl1SPEVDF+BWBDSJQU DPEF
    UIBUDSFBUFTBSBOEPNHSBQIJDBM
    JNBHFUIBUMPPLTMJLFBQBJOUJOHPG
    ,BOEJOTLZz
    $PNQPTJUJPO
    BXPSLPGBSUCZ
    8BTTJMZ,BOEJOTLZ

    View full-size slide


  55. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    (15`TJOUFSBDUJWFESBXJOHT[Bubbek(Microsoft)+, March 22, 2023]
    IUUQTBSYJWPSHBCT
    l (15DBOBMTPJOUFSBDUJWFMZESBXJNBHFT

    View full-size slide


  56. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    7JTVBMEPDVNFOUVOEFSTUBOEJOH[OpenAI, March 14, 2023]
    l 5IJTFYBNQMFSFRVJSFTVOEFSTUBOEJOHBQIZTJDTQSPCMFNXJUIB
    EJBHSBNXSJUUFOJO'SFODI
    55
    Prompt: Answer question I.1.a. Think step-by-step.
    GPT-4’s response:
    IUUQTBSYJWPSHBCT

    View full-size slide


  57. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    7JTVBMEPDVNFOUVOEFSTUBOEJOH[OpenAI, March 14, 2023]
    l (15DBOSFBEBOETVNNBSJ[FEPDVNFOUJNBHFTBOEBMTPBOTXFS
    UIFRVFTUJPOBCPVUGJHVSFT
    l 5IJTBDIJFWFNFOUJTSFNBSLBCMFBTJUTVSQBTTFTUIFDBQBCJMJUJFTPG
    QSFWJPVTNPEFMTCZBTJHOJGJDBOUNBSHJO
    56
    User: Below is part of the InstuctGPT paper.
    Could you read and summarize it to me?
    GPT-4: The InstructGPT paper focuses on
    training large language models to follow
    instructions with human feedback. …
    User: Could you explain the process described
    in Figure 2?
    GPT-4: Figure 2 in the InstructGPT paper
    illustrates the three steps of their method:
    1. Supervised fine-tuning (SFT): …
    2. Reward model (RM) training: …
    3. Reinforcement learning using Proximal Policy
    Optimization (PPO): …
    These three steps are iteratively performed to
    train the InstructGPT model, which aims to
    generate outputs that align better with human
    preferences and follow instructions more
    effectively.
    IUUQTBSYJWPSHBCT

    View full-size slide


  58. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUT
    l 1FSGPSNBODFJO/BUVSBM-BOHVBHF1SPDFTTJOHCFODINBSLTBOE
    FYBNT
    l *OUFSFTUJOHSFTVMUTJO7JTJPOBOE-BOHVBHF6OEFSTUBOEJOH
    FWBMVBUJPOT
    l 0QFOTPVSDFBQQMJDBUJPOTQPXFSFECZ--.T
    l 3FNBJOJOHJTTVFT

    View full-size slide


  59. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    MMBNBDQQ <(FPSHJ(FSHBOPW BOEDPOUSJCVUPST .BSDI ʙ>
    l *UFOBCMFT MBOHVBHFNPEFMTUPCFSVOPOB.BD#PPLVTJOHCJU
    JOUFHFSRVBOUJ[BUJPOXJUIPVUTJHOJGJDBOUQFSGPSNBODFEFHSBEBUJPO
    https://github.com/AUGMXNT/llm-experiments
    “q8” and “q4” means 8-bit and 4-bit quantization, respectively.

    View full-size slide


  60. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "VUP(15<5PSBO#SVDF3JDIBSETBOEDPOUSJCVUPST ʙ>
    l "OPQFOTPVSDFBQQMJDBUJPOUIBUDIBJOTUPHFUIFS(15`T
    lUIPVHIUTzUPBDIJFWFVTFSTFUHPBMTBVUPOPNPVTMZ
    IUUQTUXJUUFSDPNTBJSBIVMTUBUVT

    View full-size slide


  61. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "VUP(15<5PSBO#SVDF3JDIBSETBOEDPOUSJCVUPST ʙ>
    l l4FMGQSPNQUJOHz UIFBCJMJUZUPEFWFMPQBOEFYFDVUFOFXQSPNQUT
    CBTFEPOJOJUJBMJOQVU XJMMCFBOFXUSFOEGPS--.T
    6TFSTFU(PBMT
    %PNBSLFUSFTFBSDIGPSEJGGFSFOUIFBEQIPOFTPOUIFNBSLFUUPEBZ
    (FUUIFUPQIFBEQIPOFTBOEMJTUUIFJSQSPTBOEDPOT
    *ODMVEFUIFQSJDFGPSFBDIPOFBOETBWFUIFBOBMZTJT
    0ODFZPVBSFEPOF UFSNJOBUF
    (PPHMF
    TFBSDI
    #SPXTF
    XFCTJUF
    8SJUFUP
    GJMF
    2VFSZNPEJGJDBUJPO
    "DUJPOTUIBU"VHP(15 QMBOOFEBOENPEJGJFEGSPNPCTFSWBUJPOGFFECBDL
    2VFSZCBTFE
    TVNNBSJ[BUJPO
    /PJOGPSNBUJPO

    View full-size slide


  62. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    $POUFOUT
    l 1FSGPSNBODFJO/BUVSBM-BOHVBHF1SPDFTTJOHCFODINBSLTBOE
    FYBNT
    l *OUFSFTUJOHSFTVMUTJO7JTJPOBOE-BOHVBHF6OEFSTUBOEJOH
    FWBMVBUJPOT
    l 0QFOTPVSDFBQQMJDBUJPOTQPXFSFECZ--.T
    l 3FNBJOJOHJTTVFT

    View full-size slide


  63. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    )BMMVDJOBUJPO <0QFO"* "QSJM >
    l (15TJHOJGJDBOUMZJNQSPWFTGBDUVBMJUZFWBMVBUJPOTSFMBUJWFUP
    QSFWJPVT(15NPEFMT CVUJUTUJMMlIBMMVDJOBUFTzGBDUTBOENBLFT
    SFBTPOJOHFSSPST
    "DDVSBDZPO0QFO"*`T JOUFSOBMUFTUT
    IUUQTBSYJWPSHBCT

    View full-size slide


  64. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    )BMMVDJOBUJPO <0QFO"* "QSJM >
    l 3-)'JNQSPWFEGBDUVBMJUZFWBMVBUJPOTPGCPUI(15BOE(15
    JOUIF5SVUIGVM2" EBUBTFU
    3-)'
    3-)'
    (15
    (15
    "DDVSBDZPO
    5SVUIGVM2"
    IUUQTBSYJWPSH
    BCT

    View full-size slide


  65. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    #JBT[Zhuo (Monash U), February 22, 2023]
    l --.TNBZJOBEWFSUFOUMZMFBSOCJBTGSPNMBSHF OPJTZDPSQPSB
    https://arxiv.org/abs/2301.12867
    ,VOBTIJS*TMBOECFMPOHTUP+BQBO
    ˡ 3FTQPOTFTPG$IBU(15 UPl8IJDI
    DPVOUSZEPFT,VOBTIJSJ *TMBOECFMPOHUP z
    JO+BQBOFTF 3VTTJBO BOE&OHMJTI
    ,VOBTIJS*TMBOECFMPOHTUPUIF3VTTJBO'FEFSBUJPOʜ
    ,VOBTIJS*TMBOEJTDVSSFOUMZVOEFSUIFDPOUSPMPG
    3VTTJB CVU+BQBOBMTPDMBJNTTPWFSFJHOUZPWFSUIF
    JTMBOEʜ

    View full-size slide


  66. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    #JBT<#VCCFL .JDSPTPGU
    .BSDI >
    l (15`TDIPJDFPGUIFQSPOPVO SFGMFDUTUIFTLFXOFTTPGUIFXPSME
    SFQSFTFOUBUJPOGPSUIBUPDDVQBUJPO
    1SPNQUl*IBEBHSFBUFYQFSJFODFXJUIB0$$61"5*0/$BOZPVXSJUFBOPUF
    SFDPNNFOEJOHUIJT0$$61"5*0/UPBGSJFOEzVTFEUPDBMDVMBUFUIFMJLFMJIPPET
    UIFBWFSBHFQSPOPVOVTBHF
    IUUQTBSYJWPSHBCT

    View full-size slide


  67. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    "SJUINFUJDNJTUBLFT [Bubbek (Microsoft)+, March 22, 2023]
    l (15TUJMMTUSVHHMFTXJUIBSJUINFUJDNJTUBLFT1FSGPSNJOHTFWFSBM
    NBOJQVMBUJPOTJOPOFTUFQPGUIFDBMDVMBUJPOMFBETUPBOFSSPS
    &SSPS
    $PSSFDU
    IUUQTBSYJWPSH
    BCT

    View full-size slide


  68. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    -PHJDBMSFBTPOJOH<-JV 8FTUMBLF6
    "QSJM >
    l -PHJDBMSFBTPOJOHSFNBJOTDIBMMFOHJOHGPS$IBU(15 BOE(15
    FTQFDJBMMZPOOFXMZSFMFBTFEEBUBGSPN PVUPGEJTUSJCVUJPO

    IUUQTBSYJWPSHBCT
    $IJOFTF

    View full-size slide


  69. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    -PXSFTPVSDFMBOHVBHF<#BOH )POH,POH6
    'FC >
    $PNNPODSBXMTJ[F




    4FOUJNFOUBOBMZTJTMBOHVBHFJEFOUJGJDBUJPOUBTLT
    .BDIJOFUSBOTMBUJPOBDDVSBDZPOIJHIMPXSFTPVSDFMBOHVBHFT
    l $IBU(15 PGUFOGBJMTBUMPXSFTPVSDFMBOHVBHFHFOFSBUJPOBOE
    VOEFSTUBOEJOH
    IUUQTBSYJWPSHBCT

    View full-size slide


  70. 1",%%5VUPSJBM"(FOUMF*OUSPEVDUJPOUP5FDI#FIJOE-.TBOE3FDFOU"DIJFWFNFOUJO$IBU(15 5IV

    4VNNBSZ
    l 5IF(15NPEFMDBOCFDPOTJEFSFEBOFBSMZWFSTJPOPG"(*
    CFDBVTFPGUIFCSFBEUIBOEEFQUIPGJUTDBQBCJMJUJFT
    l 5IFNPEFMTIPXTIVNBOMFWFMQFSGPSNBODFJOUIFFWBMVBUJPOPG
    NBOZOBUVSBMMBOHVBHFQSPDFTTJOHUBTLT
    l *UBMTPTIPXTQSPNJTJOHBOEJOUFSFTUJOHSFTVMUTTVDIBTWJTVBM
    EPDVNFOUVOEFSTUBOEJOHBOEESBXJOHJNBHFT
    l 5IFSFBSFTPNFSFNBJOJOHJTTVFTTVDIBTIBMMVDJOBUJPOBOECJBT

    View full-size slide