Upgrade to Pro — share decks privately, control downloads, hide ads and more …

8-bit Quantization of Transformer Model

Avatar for Scatter Lab Inc. Scatter Lab Inc.
April 29, 2020
Research
0
2.3k

8-bit Quantization of Transformer Model

Avatar for Scatter Lab Inc.

Scatter Lab Inc.

April 29, 2020
Tweet

More Decks by Scatter Lab Inc.

See All by Scatter Lab Inc.
zeta introduction
Avatar for Scatter Lab Inc. scatterlab
0
1.7k
SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
Avatar for Scatter Lab Inc. scatterlab
0
4k
Adversarial Filters of Dataset Biases
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
Sparse, Dense, and Attentional Representations for Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Weight Poisoning Attacks on Pre-trained Models
Avatar for Scatter Lab Inc. scatterlab
0
2.2k
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Avatar for Scatter Lab Inc. scatterlab
0
2.4k
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
Open-Retrieval Conversational Question Answering
Avatar for Scatter Lab Inc. scatterlab
0
2.3k
What Can Neural Networks Reason About?
Avatar for Scatter Lab Inc. scatterlab
0
2.2k

Other Decks in Research

See All in Research
Vision And Languageモデルにおける異なるドメインでの継続事前学習が性能に与える影響の検証 / YANS2024
Avatar for Sansan R&D sansan_randd
1
110
2025年度人工知能学会全国大会チュートリアル講演「深層基盤モデルの数理」
Avatar for Taiji Suzuki taiji_suzuki
24
14k
言語モデルによるAI創薬の進展 / Advancements in AI-Driven Drug Discovery Using Language Models
Avatar for tsurubee tsurubee
2
370
NLP Colloquium
Avatar for Juno Kim junokim
1
150
ストレス計測方法の確立に向けたマルチモーダルデータの活用
Avatar for Yuriko Kikuchi yurikomium
0
580
データサイエンティストの就労意識～2015→2024 一般（個人）会員アンケートより
Avatar for The Japan DataScientist Society datascientistsociety
PRO
0
660
Ad-DS Paper Circle #1
Avatar for Yusuke Kaneko ykaneko1992
0
5.5k
言語モデルの内部機序：解析と解釈
Avatar for Sho Yokoi eumesy
PRO
48
18k
SSII2025 [SS1] レンズレスカメラ
Avatar for 画像センシングシンポジウム ssii
PRO
2
950
在庫管理のための機械学習と最適化の融合
Avatar for MIKIO KUBO mickey_kubo
3
1.1k
生成的推薦の人気バイアスの分析：暗記の観点から / JSAI2025
Avatar for Shotaro Ishihara upura
0
180
Combinatorial Search with Generators
Avatar for Keisuke Okumura | 奥村圭祐 kei18
0
300

Featured

See All Featured
JavaScript: Past, Present, and Future - NDC Porto 2020
Avatar for David Neal reverentgeek
48
5.4k
What’s in a name? Adding method to the madness
Avatar for Product Marketing Alliance productmarketing
PRO
23
3.5k
Building an army of robots
Avatar for Kyle Neath kneath
306
45k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
229
22k
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
29
2.7k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
694
190k
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
207
24k
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
72
4.9k
Designing for Performance
Avatar for Lara Hogan lara
609
69k
Adopting Sorbet at Scale
Avatar for Ufuk Kayserilioglu ufuk
77
9.4k
Building Adaptive Systems
Avatar for Chris Keathley keathley
43
2.6k
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
17
940

Transcript

  1. #JU2VBOUJ[BUJPOPG5SBOTGPSNFS.PEFM ੿਌੤ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH

  2. ݾର ݾର  &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM  "CTUSBDU  *OUSPEVDUJPO  .FUIPE

     3FTVMU

  3. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS /FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM

  4. "CTUSBDU &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • *$.- *OUFM • ߓ੄ࣘب UISPVHIQVUೱ࢚ਸ঳ਵݶࢲ੿ب੄#-&6TDPSFBDDVSBDZ݅ڄয૗ • *OUFMDQVী୭੸ച

    • 5FOTPS'MPX • '1*OU 4

  5. *OUSPEVDUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ୭न੄*OUFM$16ٜ਷WFDUPSJ[FEOFVSBMOFUXPSLJOTUSVDUJPO 7//* ٜਸನೣ • ѐ੄CJUܳ'." 'VTFE.VMUJQMZBOE"EE 0QFSBUJPOਸجܻחѪਸ$ZDMF۽ࣻ೯

    • .BJO$POUSJCVUJPO • '1*/5RVBOUJ[BUJPOਸ޷݅੄405"#-&64DPSFೞۅ݅ਵ۽੉ܖযն • 1FSGPSNBODF0QUJNJ[BUJPO • .BU.VM • 2VBOUJ[FE.BU.VM(SBQI0QUJNJ[BUJPO • *OQVU1JQFMJOF0QUJNJ[BUJPO • 1BSBMMFM&YFDVUJPO 5

  6. .PEFM%FTDSJQUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ೨ब'1*/5۽QSFDJTJPOਸ൞ࢤೞ؊ۄب୽࠙൤ࢿמ੉ੜաৢஹನք౟ܳ2VBOUJ[F • 4PGUNBY -BZFS/PSNBMJ[BUJPOҗэ਷҃਋ח*/5۽աఋղӝী൨ٚч੉݆ই
 ֫਷BDD൞ࢤ੉৘࢚ؽ • .FBO

    7BSJBODF &YQ١੄҅࢑੉*/5۽աఋղӝী൨ٝ 6

  7. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM • ӝઓ݆੉ॳ੉؍-JOFBS2VBOUJ[BUJPOߑधࢎਊ • .BY .JOਸ҅࢑೧ঠೞ޲۽-JOFBSTDBOਸ೙ਃ۽ೣ • 2VBOUJ[BUJPO0WFSIFBE0 /

     7

  8. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 8

  9. /BJWF2VBOUJ[BUJPO &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 9 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU  63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG

  10. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • 2VBOUJ[BUJPO਷যରೖNBQQJOHਸযڌѱੜೞוջоޙઁ • '1UFOTPSEJTUSJCVUJPO_*/5UFOTPSEJTUSJCVUJPO • ߈ࠂ೧оݶࢲ*/5߸ജਸਤೠ0QUJNBM.JO .BYܳ଺ח׮ •

    0QUJNBM౸ױਸ,-%JWFSHFODF۽҅࢑ • 7BMJEBUJPO%BUBTFUѐ੄ޙ੢઺ѐ੿بSBOEPNTBNQMJOH • .JO .BY5ISFTIPMEفѐܳ੿೧ঠೞחؘ Ӓߑߨਸࣁо૑੿ب۽ա־যࠆ 10 4ZNNFUSJD $POKVHBUF Ӓրٮ۽҅࢑

  11. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 11

  12. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME • /BJWF2VBOUJ[BUJPO਷45015PLFOਸյࣻহ঻ӝٸޙী/" • Ӕؘࣘب࢚0GGTFU੉;FSPоغחಞ੉޷ࣁೞѱࣘبо؊ࡅܰ׮ • Ӓېࢲ4ZNNFUSJDࢎਊೣ 12

  13. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM ,-%JWFSHFODFGPSPQUJNBMUISFTIPME 13 .JHBD[ 4CJUJOGFSFODFXJUIUFOTPSSU  63-IUUQPOEFNBOEHQVUFDIDPOGDPNHUDQSFTFOUBUJPOTCJUJOGFSFODFXJUIUFOTPSSUQEG

  14. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO • 7//**OTUSVDUJPOࢎਊ • 0QFSBUJPO୨іࣻ઴੉ӝ • 3FPSEFS0QFSBUJPO • .,-۽0QUJNJ[BUJPO૓೯

    • 1BSBMMFMJ[FCBUDIJOHFYFDVUJPO 14

  15. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • "79 ࠺౟੄'."ো࢑$ZDMFীоמ • '1ѐ */5ѐ • $BTDBEF-BLF$16ࠗఠ*/57//*ܳ؊୭੸ച

    • 7//*ࢎਊೠ*/5.BU.VM਷"79ࢎਊೠ'1.BU.VMࠁ׮Yࡅܴ • 7//*ࢎਊೠ*/5.BU.VM਷"79ࢎਊೠ*/5.BU.VMࠁ׮Yࡅܴ 15

  16. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 16

  17. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT • ೞ૑݅5FOTPS'MPXחJOUFHFS.BU.VMਊਵ۽(&..-081ۄחPQFOTPVSDFܳࢎਊೣ • (&..-081ח*/57//*ܳࢎਊೞ૑ঋҊ */5ো࢑द߸ജҗ੿ژೠ೙ਃೣ • Ӓېࢲ.,-#-"4ೣٜࣻ۽૒੽2VBOUJ[BUJPO4UFQ੘ࢿ •

    /PO;FSP0GGTFUبহছ 17

  18. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 18 • HPPHMFHFNNMPXQ • UFOTPSGMPXUFOTPSGMPXীࢲחই૒ࢎਊೞחѪਸഛੋ • UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD •

    UFOTPSGMPXMJUFLFSFOFMTDQV@CBDLFOE@HFNN@HFNNMPXQI

  19. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 19 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD

  20. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 20 UFOTPSGMPXDPSFLFSOFMTRVBOUJ[FE@NBUNVM@PQTDD

  21. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 21 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD

  22. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 22 UFOTPSGMPXDPSFLFSOFMTEFRVBOUJ[F@PQDD

  23. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO2VBOUJ[FE.BU.VMT 23 • /PO;FSP0GGTFUੌ҃਋HFNN@TVT੄ো࢑

  24. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(BUIFS/% 24 • (BUIFS/%חݫݽܻ*0о઺ਃೠো࢑ੋ݅ఀ */5۽઴੉؊ۄبো࢑ࣘبীח੉ٙ੉হ਺ • ೞ૑݅'1ࠁ׮*/5੉EBUBTJ[Fоߓ पઁ۽חY ੘ਵ޲۽*0੄ೱ࢚ਸӝ؀ೡࣻ੓਺

    • (FOFSBUJPO-PPQীࢲ੹ױ҅੄Ѿҗীࢲ%FRVBOUJ[FEܳউೞҊ(BUIFS/%ܳ߄۽ࣻ೯ೡ ࣻ੓ਵ޲۽Y੄ࢿמೱ࢚੉੓঻਺

  25. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO*OQVU1JQFMJOF 25 • 1BE5PLFO੉ҭ੢൤௾0WFSIFBE੉ӝٸޙী೧׼ష௾ਸহগࠁ۰חदب • 5PSDI੄1BDLFE4FRVFODFܳࢤп೮חؘ ӒѤইצ٠ • ੉֤ޙীࢲחৈ۞ߓ஖ܳTPSUJOH೧ࢲQBEUPLFO੉୭ࣗ۽ٜযоب۾ೣ

    • ੿ب੄ࢿמೱ࢚ਸ঳਺ • ੉ࠗ࠙਷5PSDI੄1BDLFE4FRVFODFэ਷ো࢑җэ਷ѐ֛ਸҳഅ೧ࢲॳݶ੿۳ೡ೙ਃبহ ੉഻ঁࡅܳѪэ׮

  26. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 26 • ੉੹ীࢲࣿ೮٠,-%JWFSHFODFܳ੉ਊ೧ UISFTEIPMETܳ଺חߑध਷NJO NBYܳো࢑ೞח दрਸহগળ׮$POTU0QFSBUJPOਵ۽߄Պ • 3FRVBOUJ[F৬3FRVBOUJ[BUJPO3BOHF

    0QFSBUJPOਸ(SBQIীࢲহঞҊ */5ীࢲ '-0"5۽߄۽߄Բب۾ো࢑ਸ߸҃೮׮ ӝઓ ো࢑਷*/5*/5'-0"5 

  27. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO(SBQI0QUJNJ[BUJPO 27

  28. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 28 • &YFDVUJPOUJNF਷CBUDIউ੄TFOUFODFMFOHUIী੄ઓ੸੉׮ • -POHFSTFOUFODFח$16഻ܳঁബਯ੸ਵ۽ॳחѪਸҙ଴ೡࣻ੓঻Ҋ  • 4FSJBMFYFDVUJPOदীח഻ঁ࠺ബਯ੸ਵ۽ॳחѪਸࠅࣻ੓঻׮

    • ੉ѦQBSBMMFMFYFDVUJPOೞݶYࢿמೱ࢚੉ઓ੤ೣ • *NQMFNFOUBUJPO • '*'02VFVFܳҙܻೞח1BSFOU5FOTPS'MPX4FTTJPO੘ࢿ • ౠ੿$16௏য৬MPDBMNFNPSZীBGGJOJUJ[FEػ /6." DIJMEQSPDFTTGPSL • $IJMEQSPDFTTח2VFVFীࢲ"TZODISPOPVTೞѱো࢑૓೯

  29. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO1BSBMMFM#BUDIJOH 29

  30. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 30

  31. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 31

  32. &GGJDJFOU#JU2VBOUJ[BUJPOPG5SBOTGPSNFS/FVSBM.BDIJOF-BOHVBHF5SBOTMBUJPO.PEFM 1FSGPSNBODF0QUJNJ[BUJPO$PODMVTJPO 32

  33. хࢎ೤פ׮✌ ୶о૕ޙژחҾӘೠ੼੉੓׮ݶ঱ઁٚইېোۅ୊۽োۅ઱ࣁਃ ੿਌੤ .BDIJOF-FBSOJOH4PGUXBSF&OHJOFFS 1JOHQPOH  &NBJMVLKBF!TDBUUFSMBCDPLS 'BDFCPPL!KFPOHVLKBF -JOLFEJO!KFPOHVLKBF