Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

Avatar for onizuka laboratory onizuka laboratory
July 11, 2018
Research
0
82

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

弊研究室で行なったNAACL読み会の発表資料です。

Avatar for onizuka laboratory

onizuka laboratory

July 11, 2018
Tweet

More Decks by onizuka laboratory

See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
Avatar for onizuka laboratory onilab
0
110
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Avatar for onizuka laboratory onilab
0
71
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
Avatar for onizuka laboratory onilab
0
33
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Avatar for onizuka laboratory onilab
0
120
Integrating Transformer and Paraphrase Rules for Sentence Simplification
Avatar for onizuka laboratory onilab
0
59
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Avatar for onizuka laboratory onilab
0
55
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Avatar for onizuka laboratory onilab
0
100
Modeling Multi-turn Conversation with Deep Utterance Aggregation
Avatar for onizuka laboratory onilab
0
95
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
Avatar for onizuka laboratory onilab
0
120

Other Decks in Research

See All in Research
Google Agent Development Kit (ADK) 入門 🚀
Avatar for MIKIO KUBO mickey_kubo
2
1.7k
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
Avatar for Shotaro Ishihara upura
0
180
When Submarine Cables Go Dark: Examining the Web Services Resilience Amid Global Internet Disruptions
Avatar for Irvin Chen irvin
0
300
Submeter-level land cover mapping of Japan
Avatar for SatAI.challenge satai
3
290
多言語カスタマーインタビューの“壁”を越える~PMと生成AIの共創~ 株式会社ジグザグ 松野 亘
Avatar for Wataru Matsuno watarumatsuno
0
120
AIスパコン「さくらONE」のLLM学習ベンチマークによる性能評価 / SAKURAONE LLM Training Benchmarking
Avatar for Yuuki Tsubouchi (yuuk1) yuukit
0
330
スキマバイトサービスにおける現場起点でのデザインアプローチ
Avatar for 眞行寺由郎 yoshioshingyouji
0
220
論文読み会 SNLP2025 Learning Dynamics of LLM Finetuning. In: ICLR 2025
Avatar for S s_mizuki_nlp
0
200
[RSJ25] Enhancing VLA Performance in Understanding and Executing Free-form Instructions via Visual Prompt-based Paraphrasing
Avatar for Semantic Machine Intelligence Lab., Keio Univ. keio_smilab
PRO
0
110
データｘデジタルマップで拓く
ミラノ発・地域共創最前線
Avatar for mapconcierge mapconcierge4agu
0
210
Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Avatar for SatAI.challenge satai
3
230
言語モデルの地図：確率分布と 情報幾何による類似性の可視化
Avatar for Hidetoshi Shimodaira shimosan
5
1.4k

Featured

See All Featured
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
95
14k
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
33
8.8k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
40
2k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.7k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
13k
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
184
22k
Music & Morning Musume
Avatar for Bryan Veloso bryan
46
6.8k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
26
1.9k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
53
2.9k
How to train your dragon (web standard)
Avatar for Monica Dinculescu notwaldorf
96
6.2k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
Avatar for Tammy Everts tammyeverts
8
520

Transcript

  1. /""$-:0.*,"* UI+VMZ  %&&1$0/5&956"-*;&% 803%3&13&4&/5"5*0/4 (3"%6"5&4$)00-PG*/'03."5*0/4$*&/$&BOE5&$)/0-0(:  04","6/*7 +6/:"5",":"."

  2. 1BQFS*/'0 .&1FUFST ./FVNBOO .*ZZFS .(BSEOFS  $$MBSL ,-FFBOE-;FUUMFNPZFS l%FFQDPOUFYUVBMJ[FEXPSESFQSFTFOUBUJPOTz *O1SPDFFEJOHTPG/""$-)-5

    #FTU1BQFS 

  3. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  4. 0WFSWJFX 1SPQPTFBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT 

  5. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  6. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 8PSESFQSFTFOUBUJPOGSPN#J-45.-BOHVBHF.PEFM  1SFUSBJOFE#J-45.-BOHVBHF.PEFM TNBMM DBU -45. -45. -45.

    DVUF ɾɾɾ #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. &-.P ! "# "$ "%

  7. #JEJSFDUJPOBMMBOHVBHFNPEFMT 5SBJOCJEJSFDUJPOBM-.CBTFEPO4UBDLFE#J-45.  TNBMM DBU -45. -45. -45. DVUF ɾɾɾ

    #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. DBU JT FPT !"#$%$&': ) *+, - log 1 2*|2,, … , 2*6,; 89, 8:;<=, 8> + log 1(2*A,, … , 2-; 89, 8:;<=, 8>) 89: Embedding matrix 8:;<=: parameters for LSTMs 8>: parameters for Softmax Layer

  8. &-.P 8PSE3FQSFTFOUBUJPOPGUPLFO!" JTCFMPX  !" -45. GPSXBSE -45. GPSXBSE #"

    = %& '(, *&,+ '(, *&,+ '(|- = 1, … , 0 = *&,+ '(|- = 0, … , 0 %& '( XPSEFNCFEEJOHT *&,+ '(PVUQVUPG KUI MBZFS LUI DFMM CBDLXBSE -45. CBDLXBSE *&,+ '( *&,+ '( %&,+ '(

  9. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 0QUJNJ[FUIFXFJHIUPGFBDIIJEEFOMBZFSGPSFBDIUBTL  -45. #J-45. -45. #J-45. &-.PL UBTL

    ! "# "$ "% &' ()*+' ,-.' = ( 0' ; 2,-.' = !,-.' 3 456 7 "4 ,-.'89,; <= >?@>9  TPGUNBYOPSNBMJ[FE XFJHIUT !,-.'  XFJHIUGPSFOUJSF &-.P WFDUPS

  10. "QQMZJOH&-.P UPTVQFSWJTFEUBTLT $PODBU &-.P BOEUBTL`TFNCFEEJOHIJEEFOMBZFSPVUQVU  &-.PL UBTL 8PSE&NCFEEJOHL &-.PL

    UBTL )JEEFO MBZFS PVUQVUL 3// $//

  11. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  12. 5SBJOJOHTFUUJOHT • /PGMBZFSTUBDLJOH MBZFST • /PGUSBJOJOHFQPDIT FQPDIT • -45. VOJUTJ[F

    VOJUT • -45. EJNFOTJPO  • $PSQVT #JMMJPO8PSE#FODINPSL • "QQMZJOHSFTJEVBMDPOOFDUJPOGPSGJSTUMBZFSUPTFDPOEMBZFS 

  13. &WBMVBUJPO3FTVMUT • "QQMZ&-.0UPBTJNQMFCBTFMJOFJOUBTLT 2" 35& 43- $PSFGFSFODF /&3 4FOUJNFOU 

    • "DIJFWFTDPSFTDPNQBSBCMFUPPSCFUUFSUIBO405"JOBMM 

  14. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  15. 8IFSFUPJODMVEF&-.P • *O43- UBTLTQFDJGJDSFQSFTFOUBUJPOJTNPSFJNQPSUBOU  UIVT&-.P JOUIFPVUQVUMBZFSJTMPXFGGFDU • 'PSNPEFMTXJUIBUUFOUJPOMBZFSBGUFS3//JUJTFGGFDUJWF UPBEE&-.P

    UPUIFPVUQVUPG3// 

  16. 8IBU*/'0TDBQUVSFECZ&-.P • $PNQBSFEXJUI(MPWF &-.P DBOXFMMDBQUVSFDPOUFYU JOGPSNBUJPO 

  17. 4BNQMFFGGJDJFODZ • 6TJOH&-.P NBLFT MFBSOJOHNPSFFGGJDJFOU UIBOFYDJTUJOH NPEFM 

  18. 7JTVBMJ[BUJPOPGMFBSOFEXFJHIUT • 7JTVBMJ[BUJPOPGXFJHIUJOH PG&-.P JOFBDIFOEUBTL • *OJOQVUMBZFS GJSTU#J-45. -BZFSJTNPTUFGGFDUJWF •

    *OPVUQVUMBZFS VTJOH JOGPSNBUJPOPGBMMMBZFSTPO BWFSBHF 

  19. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  20. $PODMVTJPO 1SPQPTFEBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT