Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

onizuka laboratory
July 11, 2018
Research
0
77

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

弊研究室で行なったNAACL読み会の発表資料です。

onizuka laboratory

July 11, 2018
Tweet

More Decks by onizuka laboratory

See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
onilab
0
110
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
onilab
0
65
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
onilab
0
31
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
onilab
0
98
Integrating Transformer and Paraphrase Rules for Sentence Simplification
onilab
0
56
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
onilab
0
48
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
onilab
0
98
Modeling Multi-turn Conversation with Deep Utterance Aggregation
onilab
0
91
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
onilab
0
110

Other Decks in Research

See All in Research
ゼロからわかるリザバーコンピューティング
kurotaky
1
310
Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations
rudorudo11
0
160
Cross-Media Information Spaces and Architectures
signer
PRO
0
120
Rの機械学習フレームワークの紹介〜tidymodelsを中心に〜 / machine_learning_with_r2024
s_uryu
0
230
Weekly AI Agents News!
masatoto
12
3.6k
媒介分析と疫学
kingqwert
0
100
[2023 CCSE] ZOZOTOWN検索における 研究開発の取り組みについて
tomoyayama
0
130
第14回対話システムシンポジウム EMNLP 2023 参加報告
atsumoto
0
160
Introduction of NII S. Koyama's Lab (AY2024)
skoyamalab
0
130
Sosiaalisen median katsaus 02/2024
hponka
0
2.7k
方策の長期性能に対する 効率的なオフライン評価・学習 (Long-term Off-Policy Evaluation and Learning)
usaito
PRO
2
190
CASCON 2023 Most Influential Paper Award Talk
tsantalis
0
130

Featured

See All Featured
Side Projects
sachag
451
41k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
20
1.6k
Embracing the Ebb and Flow
colly
80
4.2k
Building a Scalable Design System with Sketch
lauravandoore
457
32k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
14
1.5k
In The Pink: A Labor of Love
frogandcode
138
21k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
660
120k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
41
4.4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
649
58k
Agile that works and the tools we love
rasmusluckow
325
20k
Building Effective Engineering Teams - LeadDev
addyosmani
32
1.9k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
19
6.9k

Transcript

  1. /""$-:0.*,"* UI+VMZ  %&&1$0/5&956"-*;&% 803%3&13&4&/5"5*0/4 (3"%6"5&4$)00-PG*/'03."5*0/4$*&/$&BOE5&$)/0-0(:  04","6/*7 +6/:"5",":"."

  2. 1BQFS*/'0 .&1FUFST ./FVNBOO .*ZZFS .(BSEOFS  $$MBSL ,-FFBOE-;FUUMFNPZFS l%FFQDPOUFYUVBMJ[FEXPSESFQSFTFOUBUJPOTz *O1SPDFFEJOHTPG/""$-)-5

    #FTU1BQFS 

  3. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  4. 0WFSWJFX 1SPQPTFBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT 

  5. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  6. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 8PSESFQSFTFOUBUJPOGSPN#J-45.-BOHVBHF.PEFM  1SFUSBJOFE#J-45.-BOHVBHF.PEFM TNBMM DBU -45. -45. -45.

    DVUF ɾɾɾ #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. &-.P ! "# "$ "%

  7. #JEJSFDUJPOBMMBOHVBHFNPEFMT 5SBJOCJEJSFDUJPOBM-.CBTFEPO4UBDLFE#J-45.  TNBMM DBU -45. -45. -45. DVUF ɾɾɾ

    #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. DBU JT FPT !"#$%$&': ) *+, - log 1 2*|2,, … , 2*6,; 89, 8:;<=, 8> + log 1(2*A,, … , 2-; 89, 8:;<=, 8>) 89: Embedding matrix 8:;<=: parameters for LSTMs 8>: parameters for Softmax Layer

  8. &-.P 8PSE3FQSFTFOUBUJPOPGUPLFO!" JTCFMPX  !" -45. GPSXBSE -45. GPSXBSE #"

    = %& '(, *&,+ '(, *&,+ '(|- = 1, … , 0 = *&,+ '(|- = 0, … , 0 %& '( XPSEFNCFEEJOHT *&,+ '(PVUQVUPG KUI MBZFS LUI DFMM CBDLXBSE -45. CBDLXBSE *&,+ '( *&,+ '( %&,+ '(

  9. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 0QUJNJ[FUIFXFJHIUPGFBDIIJEEFOMBZFSGPSFBDIUBTL  -45. #J-45. -45. #J-45. &-.PL UBTL

    ! "# "$ "% &' ()*+' ,-.' = ( 0' ; 2,-.' = !,-.' 3 456 7 "4 ,-.'89,; <= >?@>9  TPGUNBYOPSNBMJ[FE XFJHIUT !,-.'  XFJHIUGPSFOUJSF &-.P WFDUPS

  10. "QQMZJOH&-.P UPTVQFSWJTFEUBTLT $PODBU &-.P BOEUBTL`TFNCFEEJOHIJEEFOMBZFSPVUQVU  &-.PL UBTL 8PSE&NCFEEJOHL &-.PL

    UBTL )JEEFO MBZFS PVUQVUL 3// $//

  11. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  12. 5SBJOJOHTFUUJOHT • /PGMBZFSTUBDLJOH MBZFST • /PGUSBJOJOHFQPDIT FQPDIT • -45. VOJUTJ[F

    VOJUT • -45. EJNFOTJPO  • $PSQVT #JMMJPO8PSE#FODINPSL • "QQMZJOHSFTJEVBMDPOOFDUJPOGPSGJSTUMBZFSUPTFDPOEMBZFS 

  13. &WBMVBUJPO3FTVMUT • "QQMZ&-.0UPBTJNQMFCBTFMJOFJOUBTLT 2" 35& 43- $PSFGFSFODF /&3 4FOUJNFOU 

    • "DIJFWFTDPSFTDPNQBSBCMFUPPSCFUUFSUIBO405"JOBMM 

  14. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  15. 8IFSFUPJODMVEF&-.P • *O43- UBTLTQFDJGJDSFQSFTFOUBUJPOJTNPSFJNQPSUBOU  UIVT&-.P JOUIFPVUQVUMBZFSJTMPXFGGFDU • 'PSNPEFMTXJUIBUUFOUJPOMBZFSBGUFS3//JUJTFGGFDUJWF UPBEE&-.P

    UPUIFPVUQVUPG3// 

  16. 8IBU*/'0TDBQUVSFECZ&-.P • $PNQBSFEXJUI(MPWF &-.P DBOXFMMDBQUVSFDPOUFYU JOGPSNBUJPO 

  17. 4BNQMFFGGJDJFODZ • 6TJOH&-.P NBLFT MFBSOJOHNPSFFGGJDJFOU UIBOFYDJTUJOH NPEFM 

  18. 7JTVBMJ[BUJPOPGMFBSOFEXFJHIUT • 7JTVBMJ[BUJPOPGXFJHIUJOH PG&-.P JOFBDIFOEUBTL • *OJOQVUMBZFS GJSTU#J-45. -BZFSJTNPTUFGGFDUJWF •

    *OPVUQVUMBZFS VTJOH JOGPSNBUJPOPGBMMMBZFSTPO BWFSBHF 

  19. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  20. $PODMVTJPO 1SPQPTFEBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT