Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for onizuka laboratory onizuka laboratory
July 11, 2018
Research
0
86

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

弊研究室で行なったNAACL読み会の発表資料です。

Avatar for onizuka laboratory

onizuka laboratory

July 11, 2018
Tweet

More Decks by onizuka laboratory

See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
Avatar for onizuka laboratory onilab
0
120
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Avatar for onizuka laboratory onilab
0
72
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
Avatar for onizuka laboratory onilab
0
37
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Avatar for onizuka laboratory onilab
0
130
Integrating Transformer and Paraphrase Rules for Sentence Simplification
Avatar for onizuka laboratory onilab
0
61
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Avatar for onizuka laboratory onilab
0
57
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Avatar for onizuka laboratory onilab
0
100
Modeling Multi-turn Conversation with Deep Utterance Aggregation
Avatar for onizuka laboratory onilab
0
98
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
Avatar for onizuka laboratory onilab
0
120

Other Decks in Research

See All in Research
Earth AI: Unlocking Geospatial Insights with Foundation Models and Cross-Modal Reasoning
Avatar for SatAI.challenge satai
3
480
存立危機事態の再検討
Avatar for Ken Jimbo jimboken
0
240
病院向け生成AIプロダクト開発の実践と課題
Avatar for Takashi Nishibayashi hagino3000
0
530
自動運転におけるデータ駆動型AIに対する安全性の考え方 / Safety Engineering for Data-Driven AI in Autonomous Driving Systems
Avatar for Fuyuki Ishikawa / 石川 冬樹 ishikawafyu
0
130
生成AIとうまく付き合うためのプロンプトエンジニアリング
Avatar for yuri ohashi yuri_ohashi
0
140
令和最新技術で伝統掲示板を再構築: HonoX で作る型安全なスレッドフロート型掲示板 / かろっく@calloc134 - Hono Conference 2025
Avatar for calloc134 calloc134
0
550
An Open and Reproducible Deep Research Agent for Long-Form Question Answering
Avatar for Ikuya Yamada ikuyamada
0
270
SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensing
Avatar for SatAI.challenge satai
3
500
その推薦システムの評価指標、ユーザーの感覚とズレてるかも
Avatar for kuri8ive kuri8ive
1
310
教師あり学習と強化学習で作る 最強の数学特化LLM
Avatar for Hiroshi Y (RabotniKuma) analokmaus
2
890
離散凸解析に基づく予測付き離散最適化手法 (IBIS '25)
Avatar for Taihei OKI taihei_oki
PRO
1
690
都市交通マスタープランとその後への期待＠熊本商工会議所・熊本経済同友会
Avatar for Traffic Brain trafficbrain
0
120

Featured

See All Featured
Test your architecture with Archunit
Avatar for Yoan thirion
1
2.2k
Rebuilding a faster, lazier Slack
Avatar for samanthasiow samanthasiow
85
9.4k
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
640
Visualization
Avatar for Eitan Lees eitanlees
150
17k
Building AI with AI
Avatar for Ines Montani inesmontani
PRO
1
700
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
2.1k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
76
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
57
50k
What’s in a name? Adding method to the madness
Avatar for The Alliance productmarketing
PRO
24
3.9k
We Are The Robots
Avatar for Honza Javorek honzajavorek
0
160
svc-hook: hooking system calls on ARM64 by binary rewriting
Avatar for Akira Moroo retrage
1
100
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
120

Transcript

  1. /""$-:0.*,"* UI+VMZ  %&&1$0/5&956"-*;&% 803%3&13&4&/5"5*0/4 (3"%6"5&4$)00-PG*/'03."5*0/4$*&/$&BOE5&$)/0-0(:  04","6/*7 +6/:"5",":"."

  2. 1BQFS*/'0 .&1FUFST ./FVNBOO .*ZZFS .(BSEOFS  $$MBSL ,-FFBOE-;FUUMFNPZFS l%FFQDPOUFYUVBMJ[FEXPSESFQSFTFOUBUJPOTz *O1SPDFFEJOHTPG/""$-)-5

    #FTU1BQFS 

  3. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  4. 0WFSWJFX 1SPQPTFBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT 

  5. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  6. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 8PSESFQSFTFOUBUJPOGSPN#J-45.-BOHVBHF.PEFM  1SFUSBJOFE#J-45.-BOHVBHF.PEFM TNBMM DBU -45. -45. -45.

    DVUF ɾɾɾ #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. &-.P ! "# "$ "%

  7. #JEJSFDUJPOBMMBOHVBHFNPEFMT 5SBJOCJEJSFDUJPOBM-.CBTFEPO4UBDLFE#J-45.  TNBMM DBU -45. -45. -45. DVUF ɾɾɾ

    #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. DBU JT FPT !"#$%$&': ) *+, - log 1 2*|2,, … , 2*6,; 89, 8:;<=, 8> + log 1(2*A,, … , 2-; 89, 8:;<=, 8>) 89: Embedding matrix 8:;<=: parameters for LSTMs 8>: parameters for Softmax Layer

  8. &-.P 8PSE3FQSFTFOUBUJPOPGUPLFO!" JTCFMPX  !" -45. GPSXBSE -45. GPSXBSE #"

    = %& '(, *&,+ '(, *&,+ '(|- = 1, … , 0 = *&,+ '(|- = 0, … , 0 %& '( XPSEFNCFEEJOHT *&,+ '(PVUQVUPG KUI MBZFS LUI DFMM CBDLXBSE -45. CBDLXBSE *&,+ '( *&,+ '( %&,+ '(

  9. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 0QUJNJ[FUIFXFJHIUPGFBDIIJEEFOMBZFSGPSFBDIUBTL  -45. #J-45. -45. #J-45. &-.PL UBTL

    ! "# "$ "% &' ()*+' ,-.' = ( 0' ; 2,-.' = !,-.' 3 456 7 "4 ,-.'89,; <= >?@>9  TPGUNBYOPSNBMJ[FE XFJHIUT !,-.'  XFJHIUGPSFOUJSF &-.P WFDUPS

  10. "QQMZJOH&-.P UPTVQFSWJTFEUBTLT $PODBU &-.P BOEUBTL`TFNCFEEJOHIJEEFOMBZFSPVUQVU  &-.PL UBTL 8PSE&NCFEEJOHL &-.PL

    UBTL )JEEFO MBZFS PVUQVUL 3// $//

  11. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  12. 5SBJOJOHTFUUJOHT • /PGMBZFSTUBDLJOH MBZFST • /PGUSBJOJOHFQPDIT FQPDIT • -45. VOJUTJ[F

    VOJUT • -45. EJNFOTJPO  • $PSQVT #JMMJPO8PSE#FODINPSL • "QQMZJOHSFTJEVBMDPOOFDUJPOGPSGJSTUMBZFSUPTFDPOEMBZFS 

  13. &WBMVBUJPO3FTVMUT • "QQMZ&-.0UPBTJNQMFCBTFMJOFJOUBTLT 2" 35& 43- $PSFGFSFODF /&3 4FOUJNFOU 

    • "DIJFWFTDPSFTDPNQBSBCMFUPPSCFUUFSUIBO405"JOBMM 

  14. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  15. 8IFSFUPJODMVEF&-.P • *O43- UBTLTQFDJGJDSFQSFTFOUBUJPOJTNPSFJNQPSUBOU  UIVT&-.P JOUIFPVUQVUMBZFSJTMPXFGGFDU • 'PSNPEFMTXJUIBUUFOUJPOMBZFSBGUFS3//JUJTFGGFDUJWF UPBEE&-.P

    UPUIFPVUQVUPG3// 

  16. 8IBU*/'0TDBQUVSFECZ&-.P • $PNQBSFEXJUI(MPWF &-.P DBOXFMMDBQUVSFDPOUFYU JOGPSNBUJPO 

  17. 4BNQMFFGGJDJFODZ • 6TJOH&-.P NBLFT MFBSOJOHNPSFFGGJDJFOU UIBOFYDJTUJOH NPEFM 

  18. 7JTVBMJ[BUJPOPGMFBSOFEXFJHIUT • 7JTVBMJ[BUJPOPGXFJHIUJOH PG&-.P JOFBDIFOEUBTL • *OJOQVUMBZFS GJSTU#J-45. -BZFSJTNPTUFGGFDUJWF •

    *OPVUQVUMBZFS VTJOH JOGPSNBUJPOPGBMMMBZFSTPO BWFSBHF 

  19. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  20. $PODMVTJPO 1SPQPTFEBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT