Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

Avatar for onizuka laboratory onizuka laboratory
July 11, 2018
Research
0
81

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

弊研究室で行なったNAACL読み会の発表資料です。
Avatar for onizuka laboratory

onizuka laboratory

July 11, 2018
Tweet

More Decks by onizuka laboratory

See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
Avatar for onizuka laboratory onilab
0
110
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Avatar for onizuka laboratory onilab
0
71
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
Avatar for onizuka laboratory onilab
0
33
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Avatar for onizuka laboratory onilab
0
120
Integrating Transformer and Paraphrase Rules for Sentence Simplification
Avatar for onizuka laboratory onilab
0
59
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Avatar for onizuka laboratory onilab
0
55
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Avatar for onizuka laboratory onilab
0
100
Modeling Multi-turn Conversation with Deep Utterance Aggregation
Avatar for onizuka laboratory onilab
0
95
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
Avatar for onizuka laboratory onilab
0
120

Other Decks in Research

See All in Research
Agentic AIとMCPを利用したサービス作成入門
Avatar for MIKIO KUBO mickey_kubo
0
270
なめらかなシステムと運用維持の終わらぬ未来 / dicomo2025_coherently_fittable_system
Avatar for monochromegane monochromegane
0
400
線形判別分析のPU学習による 朝日歌壇短歌の分析
Avatar for MasaKat0 masakat0
0
130
90 分で学ぶ P 対 NP 問題
Avatar for E869120 e869120
17
7.5k
Adaptive fusion of multi-modal remote sensing data for optimal sub-field crop yield prediction
Avatar for SatAI.challenge satai
3
220
NLP2025参加報告会 LT資料
Avatar for Hayahide Yamagishi hargon24
1
320
近似動的計画入門
Avatar for MIKIO KUBO mickey_kubo
4
970
A multimodal data fusion model for accurate and interpretable urban land use mapping with uncertainty analysis
Avatar for SatAI.challenge satai
3
220
大規模な2値整数計画問題に対する 効率的な重み付き局所探索法
Avatar for MIKIO KUBO mickey_kubo
1
260
20250502_ABEJA_論文読み会_スライド
Avatar for Tomoki Fujihara flatton
0
170
EOGS: Gaussian Splatting for Efficient Satellite Image Photogrammetry
Avatar for SatAI.challenge satai
4
270
実行環境に中立なWebAssemblyライブマイグレーション機構/techtalk-2025spring
Avatar for Yuki Nakata chikuwait chikuwait
0
230

Featured

See All Featured
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
124
52k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
270
20k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
357
30k
The World Runs on Bad Software
Avatar for Brandon Keepers bkeepers
PRO
69
11k
Designing Experiences People Love
Avatar for Jonathan Moore moore
142
24k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
48
2.9k
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
53
7.7k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
331
22k
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
207
24k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
63
7.8k
Building Adaptive Systems
Avatar for Chris Keathley keathley
43
2.6k
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
184
22k

Transcript

  1. /""$-:0.*,"* UI+VMZ  %&&1$0/5&956"-*;&% 803%3&13&4&/5"5*0/4 (3"%6"5&4$)00-PG*/'03."5*0/4$*&/$&BOE5&$)/0-0(:  04","6/*7 +6/:"5",":"."

  2. 1BQFS*/'0 .&1FUFST ./FVNBOO .*ZZFS .(BSEOFS  $$MBSL ,-FFBOE-;FUUMFNPZFS l%FFQDPOUFYUVBMJ[FEXPSESFQSFTFOUBUJPOTz *O1SPDFFEJOHTPG/""$-)-5

    #FTU1BQFS 

  3. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  4. 0WFSWJFX 1SPQPTFBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT 

  5. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  6. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 8PSESFQSFTFOUBUJPOGSPN#J-45.-BOHVBHF.PEFM  1SFUSBJOFE#J-45.-BOHVBHF.PEFM TNBMM DBU -45. -45. -45.

    DVUF ɾɾɾ #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. &-.P ! "# "$ "%

  7. #JEJSFDUJPOBMMBOHVBHFNPEFMT 5SBJOCJEJSFDUJPOBM-.CBTFEPO4UBDLFE#J-45.  TNBMM DBU -45. -45. -45. DVUF ɾɾɾ

    #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. DBU JT FPT !"#$%$&': ) *+, - log 1 2*|2,, … , 2*6,; 89, 8:;<=, 8> + log 1(2*A,, … , 2-; 89, 8:;<=, 8>) 89: Embedding matrix 8:;<=: parameters for LSTMs 8>: parameters for Softmax Layer

  8. &-.P 8PSE3FQSFTFOUBUJPOPGUPLFO!" JTCFMPX  !" -45. GPSXBSE -45. GPSXBSE #"

    = %& '(, *&,+ '(, *&,+ '(|- = 1, … , 0 = *&,+ '(|- = 0, … , 0 %& '( XPSEFNCFEEJOHT *&,+ '(PVUQVUPG KUI MBZFS LUI DFMM CBDLXBSE -45. CBDLXBSE *&,+ '( *&,+ '( %&,+ '(

  9. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 0QUJNJ[FUIFXFJHIUPGFBDIIJEEFOMBZFSGPSFBDIUBTL  -45. #J-45. -45. #J-45. &-.PL UBTL

    ! "# "$ "% &' ()*+' ,-.' = ( 0' ; 2,-.' = !,-.' 3 456 7 "4 ,-.'89,; <= >?@>9  TPGUNBYOPSNBMJ[FE XFJHIUT !,-.'  XFJHIUGPSFOUJSF &-.P WFDUPS

  10. "QQMZJOH&-.P UPTVQFSWJTFEUBTLT $PODBU &-.P BOEUBTL`TFNCFEEJOHIJEEFOMBZFSPVUQVU  &-.PL UBTL 8PSE&NCFEEJOHL &-.PL

    UBTL )JEEFO MBZFS PVUQVUL 3// $//

  11. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  12. 5SBJOJOHTFUUJOHT • /PGMBZFSTUBDLJOH MBZFST • /PGUSBJOJOHFQPDIT FQPDIT • -45. VOJUTJ[F

    VOJUT • -45. EJNFOTJPO  • $PSQVT #JMMJPO8PSE#FODINPSL • "QQMZJOHSFTJEVBMDPOOFDUJPOGPSGJSTUMBZFSUPTFDPOEMBZFS 

  13. &WBMVBUJPO3FTVMUT • "QQMZ&-.0UPBTJNQMFCBTFMJOFJOUBTLT 2" 35& 43- $PSFGFSFODF /&3 4FOUJNFOU 

    • "DIJFWFTDPSFTDPNQBSBCMFUPPSCFUUFSUIBO405"JOBMM 

  14. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  15. 8IFSFUPJODMVEF&-.P • *O43- UBTLTQFDJGJDSFQSFTFOUBUJPOJTNPSFJNQPSUBOU  UIVT&-.P JOUIFPVUQVUMBZFSJTMPXFGGFDU • 'PSNPEFMTXJUIBUUFOUJPOMBZFSBGUFS3//JUJTFGGFDUJWF UPBEE&-.P

    UPUIFPVUQVUPG3// 

  16. 8IBU*/'0TDBQUVSFECZ&-.P • $PNQBSFEXJUI(MPWF &-.P DBOXFMMDBQUVSFDPOUFYU JOGPSNBUJPO 

  17. 4BNQMFFGGJDJFODZ • 6TJOH&-.P NBLFT MFBSOJOHNPSFFGGJDJFOU UIBOFYDJTUJOH NPEFM 

  18. 7JTVBMJ[BUJPOPGMFBSOFEXFJHIUT • 7JTVBMJ[BUJPOPGXFJHIUJOH PG&-.P JOFBDIFOEUBTL • *OJOQVUMBZFS GJSTU#J-45. -BZFSJTNPTUFGGFDUJWF •

    *OPVUQVUMBZFS VTJOH JOGPSNBUJPOPGBMMMBZFSTPO BWFSBHF 

  19. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  20. $PODMVTJPO 1SPQPTFEBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT