Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

8-bit Quantization of Transformer Model

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Scatter Lab Inc. Scatter Lab Inc.
April 29, 2020
Research
2.4k
0

8-bit Quantization of Transformer Model

Avatar for Scatter Lab Inc.

Scatter Lab Inc.

April 29, 2020

More Decks by Scatter Lab Inc.

See All by Scatter Lab Inc.
zeta introduction
Avatar for Scatter Lab Inc. scatterlab
0
2k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
Avatar for Scatter Lab Inc. scatterlab
0
4.5k
Adversarial Filters of Dataset Biases
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Sparse, Dense, and Attentional Representations for Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
Weight Poisoning Attacks on Pre-trained Models
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.6k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
Open-Retrieval Conversational Question Answering
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
What Can Neural Networks Reason About?
Avatar for Scatter Lab Inc. scatterlab
0
2.3k

Other Decks in Research

See All in Research
Claude Code × autoresearch 実践
Avatar for 数理の弾丸 mathbullet
0
270
超効率化への挑戦：1bit LLMの現状と展望
Avatar for Yuma Ichikawa yumaichikawa
0
690
【中間報告】国会議員の立法・政策実務を支える環境を巡る現状と課題
Avatar for 株式会社PoliPoli/PoliPoli.Inc polipoli
0
580
CDCL を用いた MILP の厳密解法
Avatar for 今井義弥 imai448
0
240
AIエージェント時代のLLM-jpモデルのあるべき姿
Avatar for Kouta Nakayama k141303
0
620
SLAMはどこまで解決されたのか？
Avatar for tomonom tomonom
0
1.3k
量子サマースクール2026「量子計算機アーキテクチャ分野の概観」
Avatar for Yosuke Ueno youten622
1
870
ふとした出会いで生まれたSkillが、 社内利用1位になるまで
Avatar for earlgrayMK mikimhk
21
24k
敵対生成プロンプト同時探索による内省型プロンプト最適化
Avatar for Kotaro Inoue kinoue_smarthr
0
410
[CV勉強会＠関東 CVPR2026] PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow / kantocv 67th CVPR 2026
Avatar for Shunsuke KITADA shunk031
0
290
Using our influence and power for patient safety
Avatar for Helen Bevan helenbevan
0
420
適応的スパムフィルタのための軽量な類似メッセージカウンタ / jsai2026-adaptive-spam-filter
Avatar for monochromegane monochromegane
0
5.4k

Featured

See All Featured
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
Avatar for Eileen M. Uchitelle eileencodes
28
3.6k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
39
3.4k
Lightning talk: Run Django tests with GitHub Actions
Avatar for Sarah Abderemane sabderemane
0
250
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
202
76k
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
710
Paper Plane (Part 1)
Avatar for Katie Cordina Lindsey katiecoart
PRO
1
11k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
10
1.3k
The Spectacular Lies of Maps
Avatar for Per Axbom axbom
PRO
1
990
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
Avatar for Tech SEO Connect techseoconnect
PRO
0
430
HDC tutorial
Avatar for Michiel Stock michielstock
2
860
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
174
15k
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
191
12k

Transcript

  1. #JU2VBOUJ[BUJPOPG5SBOTGPSNFS.PEFM ੿਌੤ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH

  2. ݾର ݾର  &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM  "CTUSBDU  *OUSPEVDUJPO  .FUIPE

     3FTVMU

  3. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS /FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM

  4. "CTUSBDU &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • *$.- *OUFM • ߓ੄ࣘب UISPVHIQVUೱ࢚ਸ঳ਵݶࢲ੿ب੄#-&6TDPSFBDDVSBDZ݅ڄয૗ • *OUFMDQVী୭੸ച

    • 5FOTPS'MPX • '1*OU 4

  5. *OUSPEVDUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ୭न੄*OUFM$16ٜ਷WFDUPSJ[FEOFVSBMOFUXPSLJOTUSVDUJPO 7//* ٜਸನೣ • ѐ੄CJUܳ'." 'VTFE.VMUJQMZBOE"EE 0QFSBUJPOਸجܻחѪਸ$ZDMF۽ࣻ೯

    • .BJO$POUSJCVUJPO • '1*/5RVBOUJ[BUJPOਸ޷݅੄405"#-&64DPSFೞۅ݅ਵ۽੉ܖযն • 1FSGPSNBODF0QUJNJ[BUJPO • .BU.VM • 2VBOUJ[FE.BU.VM(SBQI0QUJNJ[BUJPO • *OQVU1JQFMJOF0QUJNJ[BUJPO • 1BSBMMFM&YFDVUJPO 5

  6. .PEFM%FTDSJQUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ೨ब'1*/5۽QSFDJTJPOਸ൞ࢤೞ؊ۄب୽࠙൤ࢿמ੉ੜաৢஹನք౟ܳ2VBOUJ[F • 4PGUNBY -BZFS/PSNBMJ[BUJPOҗэ਷҃਋ח*/5۽աఋղӝী൨ٚч੉݆ই
 ֫਷BDD൞ࢤ੉৘࢚ؽ • .FBO

    7BSJBODF &YQ١੄҅࢑੉*/5۽աఋղӝী൨ٝ 6

  7. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ӝઓ݆੉ॳ੉؍-JOFBS2VBOUJ[BUJPOߑधࢎਊ • .BY .JOਸ҅࢑೧ঠೞ޲۽-JOFBSTDBOਸ೙ਃ۽ೣ • 2VBOUJ[BUJPO0WFSIFBE0 /

     7

  8. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 8

  9. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 9 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU  63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG

  10. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • 2VBOUJ[BUJPO਷যରೖNBQQJOHਸযڌѱੜೞוջоޙઁ • '1UFOTPSEJTUSJCVUJPO_*/5UFOTPSEJTUSJCVUJPO • ߈ࠂ೧оݶࢲ*/5߸ജਸਤೠ0QUJNBM.JO .BYܳ଺ח׮ •

    0QUJNBM౸ױਸ,-%JWFSHFODF۽҅࢑ • 7BMJEBUJPO%BUBTFUѐ੄ޙ੢઺ѐ੿بSBOEPNTBNQMJOH • .JO .BY5ISFTIPMEفѐܳ੿೧ঠೞחؘ Ӓߑߨਸࣁо૑੿ب۽ա־যࠆ 10 4ZNNFUSJD $POKVHBUF Ӓրٮ۽҅࢑

  11. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 11

  12. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • /BJWF2VBOUJ[BUJPO਷45015PLFOਸյࣻহ঻ӝٸޙী/" • Ӕؘࣘب࢚0GGTFU੉;FSPоغחಞ੉޷ࣁೞѱࣘبо؊ࡅܰ׮ • Ӓېࢲ4ZNNFUSJDࢎਊೣ 12

  13. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 13 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU  63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG

  14. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO • 7//**OTUSVDUJPOࢎਊ • 0QFSBUJPO୨іࣻ઴੉ӝ • 3FPSEFS0QFSBUJPO • .,-۽0QUJNJ[BUJPO૓೯

    • 1BSBMMFMJ[FCBUDIJOHFYFDVUJPO 14

  15. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • "79 ࠺౟੄'."ো࢑$ZDMFীоמ • '1ѐ */5ѐ • $BTDBEF-BLF$16ࠗఠ*/57//*ܳ؊୭੸ച

    • 7//*ࢎਊೠ*/5.BU.VM਷"79ࢎਊೠ'1.BU.VMࠁ׮Yࡅܴ • 7//*ࢎਊೠ*/5.BU.VM਷"79ࢎਊೠ*/5.BU.VMࠁ׮Yࡅܴ 15

  16. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 16

  17. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • ೞ૑݅5FOTPS'MPXחJOUFHFS.BU.VMਊਵ۽(&..-081ۄחPQFOTPVSDFܳࢎਊೣ • (&..-081ח*/57//*ܳࢎਊೞ૑ঋҊ */5ো࢑द߸ജҗ੿ژೠ೙ਃೣ • Ӓېࢲ.,-#-"4ೣٜࣻ۽૒੽2VBOUJ[BUJPO4UFQ੘ࢿ •

    /PO;FSP0GGTFUبহছ 17

  18. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 18 • HPPHMFHFNNMPXQ • UFOTPSGMPXUFOTPSGMPXীࢲחই૒ࢎਊೞחѪਸഛੋ • UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD •

    UFOTPSGMPXMJUFLFSFOFMTDQV@CBDLFOE@HFNN@HFNNMPXQI

  19. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 19 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD

  20. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 20 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD

  21. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 21 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD

  22. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 22 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD

  23. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 23 • /PO;FSP0GGTFUੌ҃਋HFNN@TVT੄ো࢑

  24. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(BUIFS/% 24 • (BUIFS/%חݫݽܻ*0о઺ਃೠো࢑ੋ݅ఀ */5۽઴੉؊ۄبো࢑ࣘبীח੉ٙ੉হ਺ • ೞ૑݅'1ࠁ׮*/5੉EBUBTJ[Fоߓ पઁ۽חY ੘ਵ޲۽*0੄ೱ࢚ਸӝ؀ೡࣻ੓਺

    • (FOFSBUJPO-PPQীࢲ੹ױ҅੄Ѿҗীࢲ%FRVBOUJ[FEܳউೞҊ(BUIFS/%ܳ߄۽ࣻ೯ೡ ࣻ੓ਵ޲۽Y੄ࢿמೱ࢚੉੓঻਺

  25. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO*OQVU1JQFMJOF 25 • 1BE5PLFO੉ҭ੢൤௾0WFSIFBE੉ӝٸޙী೧׼ష௾ਸহগࠁ۰חदب • 5PSDI੄1BDLFE4FRVFODFܳࢤп೮חؘ ӒѤইצ٠ • ੉֤ޙীࢲחৈ۞ߓ஖ܳTPSUJOH೧ࢲQBEUPLFO੉୭ࣗ۽ٜযоب۾ೣ

    • ੿ب੄ࢿמೱ࢚ਸ঳਺ • ੉ࠗ࠙਷5PSDI੄1BDLFE4FRVFODFэ਷ো࢑җэ਷ѐ֛ਸҳഅ೧ࢲॳݶ੿۳ೡ೙ਃبহ ੉഻ঁࡅܳѪэ׮

  26. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 26 • ੉੹ীࢲࣿ೮٠,-%JWFSHFODFܳ੉ਊ೧ UISFTEIPMETܳ଺חߑध਷NJO NBYܳো࢑ೞח दрਸহগળ׮$POTU0QFSBUJPOਵ۽߄Պ • 3FRVBOUJ[F৬3FRVBOUJ[BUJPO3BOHF

    0QFSBUJPOਸ(SBQIীࢲহঞҊ */5ীࢲ '-0"5۽߄۽߄Բب۾ো࢑ਸ߸҃೮׮ ӝઓ ো࢑਷*/5*/5'-0"5 

  27. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 27

  28. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 28 • &YFDVUJPOUJNF਷CBUDIউ੄TFOUFODFMFOHUIী੄ઓ੸੉׮ • -POHFSTFOUFODFח$16഻ܳঁബਯ੸ਵ۽ॳחѪਸҙ଴ೡࣻ੓঻Ҋ  • 4FSJBMFYFDVUJPOदীח഻ঁ࠺ബਯ੸ਵ۽ॳחѪਸࠅࣻ੓঻׮

    • ੉ѦQBSBMMFMFYFDVUJPOೞݶYࢿמೱ࢚੉ઓ੤ೣ • *NQMFNFOUBUJPO • '*'02VFVFܳҙܻೞח1BSFOU5FOTPS'MPX4FTTJPO੘ࢿ • ౠ੿$16௏য৬MPDBMNFNPSZীBGGJOJUJ[FEػ /6." DIJMEQSPDFTTGPSL • $IJMEQSPDFTTח2VFVFীࢲ"TZODISPOPVTೞѱো࢑૓೯

  29. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 29

  30. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 30

  31. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 31

  32. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 32

  33. хࢎ೤פ׮✌ ୶о૕ޙژחҾӘೠ੼੉੓׮ݶ঱ઁٚইېোۅ୊۽োۅ઱ࣁਃ ੿਌੤ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH  &NBJMVLKBF!TDBUUFSMBCDPLS 'BDFCPPL!KFPOHVLKBF -JOLFEJO!KFPOHVLKBF