Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
8-bit Quantization of Transformer Model
Search
Scatter Lab Inc.
April 29, 2020
Research
0
2.2k
8-bit Quantization of Transformer Model
Scatter Lab Inc.
April 29, 2020
Tweet
Share
More Decks by Scatter Lab Inc.
See All by Scatter Lab Inc.
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
scatterlab
0
3.6k
Adversarial Filters of Dataset Biases
scatterlab
0
2.2k
Sparse, Dense, and Attentional Representations for Text Retrieval
scatterlab
0
2.2k
Weight Poisoning Attacks on Pre-trained Models
scatterlab
0
2.1k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
scatterlab
0
2.3k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
scatterlab
0
2.2k
Open-Retrieval Conversational Question Answering
scatterlab
0
2.2k
What Can Neural Networks Reason About?
scatterlab
0
2.2k
Exploring the Limits of Transfer Learning with Unified Text-to-Text Transformer
scatterlab
0
2.1k
Other Decks in Research
See All in Research
ミニ四駆AI用制御装置の事例紹介
aks3g
0
160
メールからの名刺情報抽出におけるLLM活用 / Use of LLM in extracting business card information from e-mails
sansan_randd
2
130
The Fellowship of Trust in AI
tomzimmermann
0
130
システムから変える 自分と世界を変えるシステムチェンジの方法論 / Systems Change Approaches
dmattsun
3
860
[依頼講演] 適応的実験計画法に基づく効率的無線システム設計
k_sato
0
120
EBPMにおける生成AI活用について
daimoriwaki
0
150
クロスセクター効果研究会 熊本都市交通リノベーション～「車1割削減、渋滞半減、公共交通2倍」の実現へ～
trafficbrain
0
180
機械学習による言語パフォーマンスの評価
langstat
6
710
MetricSifter：クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
120
大規模言語モデルのバイアス
yukinobaba
PRO
4
690
Language is primarily a tool for communication rather than thought
ryou0634
4
730
RCEへの近道
kawakatz
1
820
Featured
See All Featured
The Invisible Side of Design
smashingmag
297
50k
BBQ
matthewcrist
85
9.3k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
Fontdeck: Realign not Redesign
paulrobertlloyd
82
5.2k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
109
49k
Fashionably flexible responsive web design (full day workshop)
malarkey
404
65k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
229
52k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
25
1.8k
Music & Morning Musume
bryan
46
6.2k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Mobile First: as difficult as doing things right
swwweet
222
8.9k
Transcript
#JU2VBOUJ[BUJPOPG5SBOTGPSNFS.PEFM .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH
ݾର ݾର &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM "CTUSBDU *OUSPEVDUJPO .FUIPE
3FTVMU
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS /FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM
"CTUSBDU &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • *$.- *OUFM • ߓࣘب UISPVHIQVUೱ࢚ਸਵݶࢲب#-&6TDPSFBDDVSBDZ݅ڄয • *OUFMDQVী୭ച
• 5FOTPS'MPX • '1 *OU 4
*OUSPEVDUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ୭न*OUFM$16ٜWFDUPSJ[FEOFVSBMOFUXPSLJOTUSVDUJPO 7//* ٜਸನೣ • ѐCJUܳ'." 'VTFE.VMUJQMZBOE"EE 0QFSBUJPOਸجܻחѪਸ$ZDMF۽ࣻ೯
• .BJO$POUSJCVUJPO • '1 */5RVBOUJ[BUJPOਸ݅405"#-&64DPSFೞۅ݅ਵ۽ܖযն • 1FSGPSNBODF0QUJNJ[BUJPO • .BU.VM • 2VBOUJ[FE.BU.VM(SBQI0QUJNJ[BUJPO • *OQVU1JQFMJOF0QUJNJ[BUJPO • 1BSBMMFM&YFDVUJPO 5
.PEFM%FTDSJQUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ೨ब'1 */5۽QSFDJTJPOਸ൞ࢤೞ؊ۄب࠙ࢿמੜաৢஹನքܳ2VBOUJ[F • 4PGUNBY -BZFS/PSNBMJ[BUJPOҗэ҃ח*/5۽աఋղӝী൨ٚч݆ই ֫BDD൞ࢤ࢚ؽ • .FBO
7BSJBODF &YQ١҅*/5۽աఋղӝী൨ٝ 6
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ӝઓ݆ॳ؍-JOFBS2VBOUJ[BUJPOߑधࢎਊ • .BY .JOਸ҅೧ঠೞ۽-JOFBSTDBOਸਃ۽ೣ • 2VBOUJ[BUJPO0WFSIFBE0 /
7
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 8
/BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 9 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • 2VBOUJ[BUJPOযରೖNBQQJOHਸযڌѱੜೞוջоޙઁ • '1UFOTPSEJTUSJCVUJPO_ */5UFOTPSEJTUSJCVUJPO • ߈ࠂ೧оݶࢲ*/5߸ജਸਤೠ0QUJNBM.JO .BYܳח •
0QUJNBM౸ױਸ,-%JWFSHFODF۽҅ • 7BMJEBUJPO%BUBTFUѐޙѐبSBOEPNTBNQMJOH • .JO .BY5ISFTIPMEفѐܳ೧ঠೞחؘ Ӓߑߨਸࣁоب۽ա־যࠆ 10 4ZNNFUSJD $POKVHBUF Ӓրٮ۽҅
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 11
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • /BJWF2VBOUJ[BUJPO45015PLFOਸյࣻহӝٸޙী/" • Ӕؘࣘب࢚0GGTFU;FSPоغחಞࣁೞѱࣘبо؊ࡅܰ • Ӓېࢲ4ZNNFUSJDࢎਊೣ 12
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 13 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU 63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO • 7//**OTUSVDUJPOࢎਊ • 0QFSBUJPO୨іࣻӝ • 3FPSEFS0QFSBUJPO • .,-۽0QUJNJ[BUJPO೯
• 1BSBMMFMJ[FCBUDIJOHFYFDVUJPO 14
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • "79 ࠺'."ো$ZDMFীоמ • '1 ѐ */5 ѐ • $BTDBEF-BLF$16ࠗఠ*/57//*ܳ؊୭ച
• 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ'1.BU.VMࠁYࡅܴ • 7//*ࢎਊೠ*/5.BU.VM"79ࢎਊೠ*/5.BU.VMࠁYࡅܴ 15
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 16
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • ೞ݅5FOTPS'MPXחJOUFHFS.BU.VMਊਵ۽(&..-081ۄחPQFOTPVSDFܳࢎਊೣ • (&..-081ח*/57//*ܳࢎਊೞঋҊ */5োद߸ജҗژೠਃೣ • Ӓېࢲ.,-#-"4ೣٜࣻ۽2VBOUJ[BUJPO4UFQࢿ •
/PO;FSP0GGTFUبহছ 17
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 18 • HPPHMFHFNNMPXQ • UFOTPSGMPXUFOTPSGMPXীࢲחইࢎਊೞחѪਸഛੋ • UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD •
UFOTPSGMPXMJUFLFSFOFMTDQV@CBDLFOE@HFNN@HFNNMPXQI
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 19 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 20 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 21 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 22 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 23 • /PO;FSP0GGTFUੌ҃HFNN@TVTো
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(BUIFS/% 24 • (BUIFS/%חݫݽܻ*0оਃೠোੋ݅ఀ */5۽؊ۄبোࣘبীחٙহ • ೞ݅'1ࠁ*/5EBUBTJ[Fоߓ पઁ۽חY ਵ۽*0ೱ࢚ਸӝೡࣻ
• (FOFSBUJPO-PPQীࢲױ҅Ѿҗীࢲ%FRVBOUJ[FEܳউೞҊ(BUIFS/%ܳ߄۽ࣻ೯ೡ ࣻਵ۽Yࢿמೱ࢚
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO*OQVU1JQFMJOF 25 • 1BE5PLFOҭ0WFSIFBEӝٸޙী೧షਸহগࠁ۰חदب • 5PSDI1BDLFE4FRVFODFܳࢤп೮חؘ ӒѤইצ٠ • ֤ޙীࢲחৈ۞ߓܳTPSUJOH೧ࢲQBEUPLFO୭ࣗ۽ٜযоب۾ೣ
• بࢿמೱ࢚ਸ • ࠗ࠙5PSDI1BDLFE4FRVFODFэোҗэѐ֛ਸҳഅ೧ࢲॳݶ۳ೡਃبহ ഻ঁࡅܳѪэ
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 26 • ীࢲࣿ೮٠,-%JWFSHFODFܳਊ೧ UISFTEIPMETܳחߑधNJO NBYܳোೞח दрਸহগળ $POTU0QFSBUJPOਵ۽߄Պ • 3FRVBOUJ[F৬3FRVBOUJ[BUJPO3BOHF
0QFSBUJPOਸ(SBQIীࢲহঞҊ */5ীࢲ '-0"5۽߄۽߄Բب۾োਸ߸҃೮ ӝઓ ো*/5 */5 '-0"5
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 27
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 28 • &YFDVUJPOUJNFCBUDIউTFOUFODFMFOHUIীઓ • -POHFSTFOUFODFח$16഻ܳঁബਯਵ۽ॳחѪਸҙೡࣻҊ • 4FSJBMFYFDVUJPOदীח഻ঁ࠺ബਯਵ۽ॳחѪਸࠅࣻ
• ѦQBSBMMFMFYFDVUJPOೞݶYࢿמೱ࢚ઓೣ • *NQMFNFOUBUJPO • '*'02VFVFܳҙܻೞח1BSFOU5FOTPS'MPX4FTTJPOࢿ • ౠ$16য৬MPDBMNFNPSZীBGGJOJUJ[FEػ /6." DIJMEQSPDFTTGPSL • $IJMEQSPDFTTח2VFVFীࢲ"TZODISPOPVTೞѱো೯
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 29
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 30
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 31
&GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 32
хࢎפ✌ ୶оޙژחҾӘೠݶઁٚইېোۅ۽োۅࣁਃ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH &NBJMVLKBF!TDBUUFSMBCDPLS 'BDFCPPL!KFPOHVLKBF -JOLFEJO!KFPOHVLKBF
scatterlab
aks3g
sansan_randd
tomzimmermann
dmattsun
k_sato
daimoriwaki
trafficbrain
langstat
yuukit
yukinobaba
ryou0634
kawakatz
smashingmag
matthewcrist
eileencodes
paulrobertlloyd
aki_iinuma
malarkey
destraynor
jnunemaker
marcduiker
bryan
samlambert
swwweet
