Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
8-bit Quantization of Transformer Model
Search
Scatter Lab Inc.
April 29, 2020
Research
0
2.3k
8-bit Quantization of Transformer Model
Scatter Lab Inc.
April 29, 2020
Tweet
Share
More Decks by Scatter Lab Inc.
See All by Scatter Lab Inc.
zeta introduction
scatterlab
0
1.7k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
scatterlab
0
4k
Adversarial Filters of Dataset Biases
scatterlab
0
2.2k
Sparse, Dense, and Attentional Representations for Text Retrieval
scatterlab
0
2.3k
Weight Poisoning Attacks on Pre-trained Models
scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
scatterlab
0
2.5k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
scatterlab
0
2.3k
Open-Retrieval Conversational Question Answering
scatterlab
0
2.3k
What Can Neural Networks Reason About?
scatterlab
0
2.2k
Other Decks in Research
See All in Research
学生向けアンケート<データサイエンティストについて>
datascientistsociety
PRO
0
3.7k
業界横断 副業・兼業者の実態調査
fkske
0
190
数理最適化に基づく制御
mickey_kubo
5
680
When Submarine Cables Go Dark: Examining the Web Services Resilience Amid Global Internet Disruptions
irvin
0
230
Cross-Media Information Spaces and Architectures
signer
PRO
0
230
MGDSS:慣性式モーションキャプチャを用いたジェスチャによるドローンの操作 / ec75-yamauchi
yumulab
0
260
90 分で学ぶ P 対 NP 問題
e869120
18
7.6k
大規模な2値整数計画問題に対する 効率的な重み付き局所探索法
mickey_kubo
1
270
NLP2025参加報告会 LT資料
hargon24
1
320
Pix2Poly: A Sequence Prediction Method for End-to-end Polygonal Building Footprint Extraction from Remote Sensing Imagery
satai
3
490
20250624_熊本経済同友会6月例会講演
trafficbrain
1
400
データxデジタルマップで拓く ミラノ発・地域共創最前線
mapconcierge4agu
0
190
Featured
See All Featured
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Documentation Writing (for coders)
carmenintech
72
4.9k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
740
YesSQL, Process and Tooling at Scale
rocio
173
14k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.7k
It's Worth the Effort
3n
185
28k
Speed Design
sergeychernyshev
32
1k
How to train your dragon (web standard)
notwaldorf
96
6.1k
Become a Pro
speakerdeck
PRO
29
5.4k
The Invisible Side of Design
smashingmag
301
51k
Navigating Team Friction
lara
187
15k
Transcript
#JU2VBOUJ[BUJPOPG5SBOTGPSNFS.PEFM .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH
ݾର ݾର &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM "CTUSBDU *OUSPEVDUJPO .FUIPE
3FTVMU
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS /FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM
"CTUSBDU &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • *$.- *OUFM • ߓࣘب UISPVHIQVUೱ࢚ਸਵݶࢲب#-&6TDPSFBDDVSBDZ݅ڄয • *OUFMDQVী୭ച
• 5FOTPS'MPX • '1*OU 4
*OUSPEVDUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ୭न*OUFM$16ٜWFDUPSJ[FEOFVSBMOFUXPSLJOTUSVDUJPO 7//* ٜਸನೣ • ѐCJUܳ'." 'VTFE.VMUJQMZBOE"EE 0QFSBUJPOਸجܻחѪਸ$ZDMF۽ࣻ೯
• .BJO$POUSJCVUJPO • '1*/5RVBOUJ[BUJPOਸ݅405"#-&64DPSFೞۅ݅ਵ۽ܖযն • 1FSGPSNBODF0QUJNJ[BUJPO • .BU.VM • 2VBOUJ[FE.BU.VM(SBQI0QUJNJ[BUJPO • *OQVU1JQFMJOF0QUJNJ[BUJPO • 1BSBMMFM&YFDVUJPO 5
.PEFM%FTDSJQUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ೨ब'1*/5۽QSFDJTJPOਸ൞ࢤೞ؊ۄب࠙ࢿמੜաৢஹನքܳ2VBOUJ[F • 4PGUNBY -BZFS/PSNBMJ[BUJPOҗэ҃ח*/5۽աఋղӝী൨ٚч݆ই ֫BDD൞ࢤ࢚ؽ • .FBO
7BSJBODF &YQ١҅*/5۽աఋղӝী൨ٝ 6
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ӝઓ݆ॳ؍-JOFBS2VBOUJ[BUJPOߑधࢎਊ • .BY .JOਸ҅೧ঠೞ۽-JOFBSTDBOਸਃ۽ೣ • 2VBOUJ[BUJPO0WFSIFBE0 /
7
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 8
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 9 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • 2VBOUJ[BUJPOযରೖNBQQJOHਸযڌѱੜೞוջоޙઁ • '1UFOTPSEJTUSJCVUJPO_*/5UFOTPSEJTUSJCVUJPO • ߈ࠂ೧оݶࢲ*/5߸ജਸਤೠ0QUJNBM.JO .BYܳח •
0QUJNBM౸ױਸ,-%JWFSHFODF۽҅ • 7BMJEBUJPO%BUBTFUѐޙѐبSBOEPNTBNQMJOH • .JO .BY5ISFTIPMEفѐܳ೧ঠೞחؘ Ӓߑߨਸࣁоب۽ա־যࠆ 10 4ZNNFUSJD $POKVHBUF Ӓրٮ۽҅
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 11
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • /BJWF2VBOUJ[BUJPO45015PLFOਸյࣻহӝٸޙী/" • Ӕؘࣘب࢚0GGTFU;FSPоغחಞࣁೞѱࣘبо؊ࡅܰ • Ӓېࢲ4ZNNFUSJDࢎਊೣ 12
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 13 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO • 7//**OTUSVDUJPOࢎਊ • 0QFSBUJPO୨іࣻӝ • 3FPSEFS0QFSBUJPO • .,-۽0QUJNJ[BUJPO೯
• 1BSBMMFMJ[FCBUDIJOHFYFDVUJPO 14
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • "79 ࠺'."ো$ZDMFীоמ • '1ѐ */5ѐ • $BTDBEF-BLF$16ࠗఠ*/57//*ܳ؊୭ച
• 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ'1.BU.VMࠁYࡅܴ • 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ*/5.BU.VMࠁYࡅܴ 15
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 16
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • ೞ݅5FOTPS'MPXחJOUFHFS.BU.VMਊਵ۽(&..-081ۄחPQFOTPVSDFܳࢎਊೣ • (&..-081ח*/57//*ܳࢎਊೞঋҊ */5োद߸ജҗژೠਃೣ • Ӓېࢲ.,-#-"4ೣٜࣻ۽2VBOUJ[BUJPO4UFQࢿ •
/PO;FSP0GGTFUبহছ 17
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 18 • HPPHMFHFNNMPXQ • UFOTPSGMPXUFOTPSGMPXীࢲחইࢎਊೞחѪਸഛੋ • UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD •
UFOTPSGMPXMJUFLFSFOFMTDQV@CBDLFOE@HFNN@HFNNMPXQI
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 19 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 20 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 21 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 22 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 23 • /PO;FSP0GGTFUੌ҃HFNN@TVTো
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(BUIFS/% 24 • (BUIFS/%חݫݽܻ*0оਃೠোੋ݅ఀ */5۽؊ۄبোࣘبীחٙহ • ೞ݅'1ࠁ*/5EBUBTJ[Fоߓ पઁ۽חY ਵ۽*0ೱ࢚ਸӝೡࣻ
• (FOFSBUJPO-PPQীࢲױ҅Ѿҗীࢲ%FRVBOUJ[FEܳউೞҊ(BUIFS/%ܳ߄۽ࣻ೯ೡ ࣻਵ۽Yࢿמೱ࢚
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO*OQVU1JQFMJOF 25 • 1BE5PLFOҭ0WFSIFBEӝٸޙী೧షਸহগࠁ۰חदب • 5PSDI1BDLFE4FRVFODFܳࢤп೮חؘ ӒѤইצ٠ • ֤ޙীࢲחৈ۞ߓܳTPSUJOH೧ࢲQBEUPLFO୭ࣗ۽ٜযоب۾ೣ
• بࢿמೱ࢚ਸ • ࠗ࠙5PSDI1BDLFE4FRVFODFэোҗэѐ֛ਸҳഅ೧ࢲॳݶ۳ೡਃبহ ഻ঁࡅܳѪэ
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 26 • ীࢲࣿ೮٠,-%JWFSHFODFܳਊ೧ UISFTEIPMETܳחߑधNJO NBYܳোೞח दрਸহগળ$POTU0QFSBUJPOਵ۽߄Պ • 3FRVBOUJ[F৬3FRVBOUJ[BUJPO3BOHF
0QFSBUJPOਸ(SBQIীࢲহঞҊ */5ীࢲ '-0"5۽߄۽߄Բب۾োਸ߸҃೮ ӝઓ ো*/5*/5'-0"5
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 27
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 28 • &YFDVUJPOUJNFCBUDIউTFOUFODFMFOHUIীઓ • -POHFSTFOUFODFח$16഻ܳঁബਯਵ۽ॳחѪਸҙೡࣻҊ • 4FSJBMFYFDVUJPOदীח഻ঁ࠺ബਯਵ۽ॳחѪਸࠅࣻ
• ѦQBSBMMFMFYFDVUJPOೞݶYࢿמೱ࢚ઓೣ • *NQMFNFOUBUJPO • '*'02VFVFܳҙܻೞח1BSFOU5FOTPS'MPX4FTTJPOࢿ • ౠ$16য৬MPDBMNFNPSZীBGGJOJUJ[FEػ /6." DIJMEQSPDFTTGPSL • $IJMEQSPDFTTח2VFVFীࢲ"TZODISPOPVTೞѱো೯
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 29
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 30
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 31
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 32
хࢎפ✌ ୶оޙژחҾӘೠݶઁٚইېোۅ۽োۅࣁਃ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH &NBJMVLKBF!TDBUUFSMBCDPLS 'BDFCPPL!KFPOHVLKBF -JOLFEJO!KFPOHVLKBF