Speaker Deck
Sign in
Sign up
for free
8-bit Quantization of Transformer Model
Scatter Lab Inc.
April 29, 2020
Research
0
1k
8-bit Quantization of Transformer Model
Scatter Lab Inc.
April 29, 2020
Tweet
Share
More Decks by Scatter Lab Inc.
See All by Scatter Lab Inc.
scatterlab
0
540
scatterlab
0
540
scatterlab
0
540
scatterlab
0
550
scatterlab
0
550
scatterlab
0
560
scatterlab
0
560
scatterlab
0
980
scatterlab
0
980
Other Decks in Research
See All in Research
resnant
1
1.2k
sobamchan
1
380
n1n9
0
910
shunk031
4
720
pshiko
1
600
wangzwhu
0
270
issdu
0
1.2k
shinyorke
0
1.2k
omshivaprakash
0
190
bakanaouji
1
1.3k
task4233
1
100
ysekky
1
470
Featured
See All Featured
mza
80
3.9k
lynnandtonic
266
16k
hatefulcrawdad
257
16k
eileencodes
112
24k
skipperchong
4
490
sferik
607
54k
phodgson
84
3.8k
holman
287
130k
chrislema
231
16k
scottboms
250
11k
addyosmani
494
110k
matthewcrist
70
7k
Transcript
#JU2VBOUJ[BUJPOPG5SBOTGPSNFS.PEFM .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH
ݾର ݾର &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM "CTUSBDU *OUSPEVDUJPO .FUIPE
3FTVMU
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS /FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM
"CTUSBDU &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • *$.- *OUFM • ߓࣘب UISPVHIQVUೱ࢚ਸਵݶࢲب#-&6TDPSFBDDVSBDZ݅ڄয • *OUFMDQVী୭ച
• 5FOTPS'MPX • '1 *OU 4
*OUSPEVDUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ୭न*OUFM$16ٜWFDUPSJ[FEOFVSBMOFUXPSLJOTUSVDUJPO 7//* ٜਸನೣ • ѐCJUܳ'." 'VTFE.VMUJQMZBOE"EE 0QFSBUJPOਸجܻחѪਸ$ZDMF۽ࣻ೯
• .BJO$POUSJCVUJPO • '1 */5RVBOUJ[BUJPOਸ݅405"#-&64DPSFೞۅ݅ਵ۽ܖযն • 1FSGPSNBODF0QUJNJ[BUJPO • .BU.VM • 2VBOUJ[FE.BU.VM(SBQI0QUJNJ[BUJPO • *OQVU1JQFMJOF0QUJNJ[BUJPO • 1BSBMMFM&YFDVUJPO 5
.PEFM%FTDSJQUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ೨ब'1 */5۽QSFDJTJPOਸ൞ࢤೞ؊ۄب࠙ࢿמੜաৢஹನքܳ2VBOUJ[F • 4PGUNBY -BZFS/PSNBMJ[BUJPOҗэ҃ח*/5۽աఋղӝী൨ٚч݆ই ֫BDD൞ࢤ࢚ؽ • .FBO
7BSJBODF &YQ١҅*/5۽աఋղӝী൨ٝ 6
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ӝઓ݆ॳ؍-JOFBS2VBOUJ[BUJPOߑधࢎਊ • .BY .JOਸ҅೧ঠೞ۽-JOFBSTDBOਸਃ۽ೣ • 2VBOUJ[BUJPO0WFSIFBE0 /
7
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 8
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 9 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • 2VBOUJ[BUJPOযରೖNBQQJOHਸযڌѱੜೞוջоޙઁ • '1UFOTPSEJTUSJCVUJPO_ */5UFOTPSEJTUSJCVUJPO • ߈ࠂ೧оݶࢲ*/5߸ജਸਤೠ0QUJNBM.JO .BYܳח •
0QUJNBM౸ױਸ,-%JWFSHFODF۽҅ • 7BMJEBUJPO%BUBTFUѐޙѐبSBOEPNTBNQMJOH • .JO .BY5ISFTIPMEفѐܳ೧ঠೞחؘ Ӓߑߨਸࣁоب۽ա־যࠆ 10 4ZNNFUSJD $POKVHBUF Ӓրٮ۽҅
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 11
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • /BJWF2VBOUJ[BUJPO45015PLFOਸյࣻহӝٸޙী/" • Ӕؘࣘب࢚0GGTFU;FSPоغחಞࣁೞѱࣘبо؊ࡅܰ • Ӓېࢲ4ZNNFUSJDࢎਊೣ 12
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 13 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO • 7//**OTUSVDUJPOࢎਊ • 0QFSBUJPO୨іࣻӝ • 3FPSEFS0QFSBUJPO • .,-۽0QUJNJ[BUJPO೯
• 1BSBMMFMJ[FCBUDIJOHFYFDVUJPO 14
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • "79 ࠺'."ো$ZDMFীоמ • '1 ѐ */5 ѐ • $BTDBEF-BLF$16ࠗఠ*/57//*ܳ؊୭ച
• 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ'1.BU.VMࠁYࡅܴ • 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ*/5.BU.VMࠁYࡅܴ 15
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 16
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • ೞ݅5FOTPS'MPXחJOUFHFS.BU.VMਊਵ۽(&..-081ۄחPQFOTPVSDFܳࢎਊೣ • (&..-081ח*/57//*ܳࢎਊೞঋҊ */5োद߸ജҗژೠਃೣ • Ӓېࢲ.,-#-"4ೣٜࣻ۽2VBOUJ[BUJPO4UFQࢿ •
/PO;FSP0GGTFUبহছ 17
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 18 • HPPHMFHFNNMPXQ • UFOTPSGMPXUFOTPSGMPXীࢲחইࢎਊೞחѪਸഛੋ • UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD •
UFOTPSGMPXMJUFLFSFOFMTDQV@CBDLFOE@HFNN@HFNNMPXQI
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 19 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 20 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 21 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 22 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 23 • /PO;FSP0GGTFUੌ҃HFNN@TVTো
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(BUIFS/% 24 • (BUIFS/%חݫݽܻ*0оਃೠোੋ݅ఀ */5۽؊ۄبোࣘبীחٙহ • ೞ݅'1ࠁ*/5EBUBTJ[Fоߓ पઁ۽חY ਵ۽*0ೱ࢚ਸӝೡࣻ
• (FOFSBUJPO-PPQীࢲױ҅Ѿҗীࢲ%FRVBOUJ[FEܳউೞҊ(BUIFS/%ܳ߄۽ࣻ೯ೡ ࣻਵ۽Yࢿמೱ࢚
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO*OQVU1JQFMJOF 25 • 1BE5PLFOҭ0WFSIFBEӝٸޙী೧షਸহগࠁ۰חदب • 5PSDI1BDLFE4FRVFODFܳࢤп೮חؘ ӒѤইצ٠ • ֤ޙীࢲחৈ۞ߓܳTPSUJOH೧ࢲQBEUPLFO୭ࣗ۽ٜযоب۾ೣ
• بࢿמೱ࢚ਸ • ࠗ࠙5PSDI1BDLFE4FRVFODFэোҗэѐ֛ਸҳഅ೧ࢲॳݶ۳ೡਃبহ ഻ঁࡅܳѪэ
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 26 • ীࢲࣿ೮٠,-%JWFSHFODFܳਊ೧ UISFTEIPMETܳחߑधNJO NBYܳোೞח दрਸহগળ $POTU0QFSBUJPOਵ۽߄Պ • 3FRVBOUJ[F৬3FRVBOUJ[BUJPO3BOHF
0QFSBUJPOਸ(SBQIীࢲহঞҊ */5ীࢲ '-0"5۽߄۽߄Բب۾োਸ߸҃೮ ӝઓ ো*/5 */5 '-0"5
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 27
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 28 • &YFDVUJPOUJNFCBUDIউTFOUFODFMFOHUIীઓ • -POHFSTFOUFODFח$16഻ܳঁബਯਵ۽ॳחѪਸҙೡࣻҊ • 4FSJBMFYFDVUJPOदীח഻ঁ࠺ബਯਵ۽ॳחѪਸࠅࣻ
• ѦQBSBMMFMFYFDVUJPOೞݶYࢿמೱ࢚ઓೣ • *NQMFNFOUBUJPO • '*'02VFVFܳҙܻೞח1BSFOU5FOTPS'MPX4FTTJPOࢿ • ౠ$16য৬MPDBMNFNPSZীBGGJOJUJ[FEػ /6." DIJMEQSPDFTTGPSL • $IJMEQSPDFTTח2VFVFীࢲ"TZODISPOPVTೞѱো೯
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 29
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 30
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 31
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 32
хࢎפ✌ ୶оޙژחҾӘೠݶઁٚইېোۅ۽োۅࣁਃ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH &NBJMVLKBF!TDBUUFSMBCDPLS 'BDFCPPL!KFPOHVLKBF -JOLFEJO!KFPOHVLKBF
scatterlab
resnant
sobamchan
n1n9
shunk031
pshiko
wangzwhu
issdu
shinyorke
omshivaprakash
bakanaouji
task4233
ysekky
mza
lynnandtonic
hatefulcrawdad
eileencodes
skipperchong
sferik
phodgson
holman
chrislema
scottboms
addyosmani
matthewcrist
