Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

Avatar for onizuka laboratory onizuka laboratory
July 11, 2018
Research
0
85

[論紹] DEEP CONTEXTUALIZED WORD REPRESENTATIONS

弊研究室で行なったNAACL読み会の発表資料です。

Avatar for onizuka laboratory

onizuka laboratory

July 11, 2018
Tweet

More Decks by onizuka laboratory

See All by onizuka laboratory
Phrase-Based & Neural Unsupervised Machine Translation
Avatar for onizuka laboratory onilab
0
120
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Avatar for onizuka laboratory onilab
0
72
Card-660: A Reliable Evaluation Framework for Rare Word Representation Models
Avatar for onizuka laboratory onilab
0
36
A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification
Avatar for onizuka laboratory onilab
0
130
Integrating Transformer and Paraphrase Rules for Sentence Simplification
Avatar for onizuka laboratory onilab
0
61
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation
Avatar for onizuka laboratory onilab
0
57
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Avatar for onizuka laboratory onilab
0
100
Modeling Multi-turn Conversation with Deep Utterance Aggregation
Avatar for onizuka laboratory onilab
0
98
Learning Semantic Sentence Embeddings using Pair-wise Discriminator
Avatar for onizuka laboratory onilab
0
120

Other Decks in Research

See All in Research
国際論文を出そう！ICRA / IROS / RA-L への論文投稿の心構えとノウハウ / RSJ2025 Luncheon Seminar
Avatar for koide3 koide3
10
6.3k
Sat2City:3D City Generation from A Single Satellite Image with Cascaded Latent Diffusion
Avatar for SatAI.challenge satai
4
330
MIRU2025 チュートリアル講演「ロボット基盤モデルの最前線」
Avatar for Kento Kawaharazuka haraduka
15
11k
Combining Deep Learning and Street View Imagery to Map Smallholder Crop Types
Avatar for SatAI.challenge satai
3
270
Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation
Avatar for SatAI.challenge satai
3
400
VectorLLM: Human-like Extraction of Structured Building Contours via Multimodal LLMs
Avatar for SatAI.challenge satai
4
500
情報技術の社会実装に向けた応用と課題：ニュースメディアの事例から / appmech-jsce 2025
Avatar for Shotaro Ishihara upura
0
280
論文紹介: ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement
Avatar for Hisao Katsumi hisaokatsumi
0
140
生成AI による論文執筆サポート・ワークショップ ─ サーベイ／リサーチクエスチョン編 / Workshop on AI-Assisted Paper Writing Support: Survey/Research Question Edition
Avatar for Kenji Saito ks91
PRO
0
120
論文紹介：Safety Alignment Should be Made More Than Just a Few Tokens Deep
Avatar for Kazutoshi Shinoda kazutoshishinoda
0
150
MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generation
Avatar for SatAI.challenge satai
4
490
空間音響処理における物理法則に基づく機械学習
Avatar for NII S. Koyama's Lab skoyamalab
0
110

Featured

See All Featured
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
280
24k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
1.8k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
51k
Fireside Chat
Avatar for paigeccino paigeccino
41
3.7k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
510
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
249
1.3M
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.4k
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
432
66k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.6k
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
210
24k
Building an army of robots
Avatar for Kyle Neath kneath
306
46k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
48
9.8k

Transcript

  1. /""$-:0.*,"* UI+VMZ  %&&1$0/5&956"-*;&% 803%3&13&4&/5"5*0/4 (3"%6"5&4$)00-PG*/'03."5*0/4$*&/$&BOE5&$)/0-0(:  04","6/*7 +6/:"5",":"."

  2. 1BQFS*/'0 .&1FUFST ./FVNBOO .*ZZFS .(BSEOFS  $$MBSL ,-FFBOE-;FUUMFNPZFS l%FFQDPOUFYUVBMJ[FEXPSESFQSFTFOUBUJPOTz *O1SPDFFEJOHTPG/""$-)-5

    #FTU1BQFS 

  3. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  4. 0WFSWJFX 1SPQPTFBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT 

  5. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  6. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 8PSESFQSFTFOUBUJPOGSPN#J-45.-BOHVBHF.PEFM  1SFUSBJOFE#J-45.-BOHVBHF.PEFM TNBMM DBU -45. -45. -45.

    DVUF ɾɾɾ #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. &-.P ! "# "$ "%

  7. #JEJSFDUJPOBMMBOHVBHFNPEFMT 5SBJOCJEJSFDUJPOBM-.CBTFEPO4UBDLFE#J-45.  TNBMM DBU -45. -45. -45. DVUF ɾɾɾ

    #J-45. #J-45. #J-45. -45. -45. -45. #J-45. #J-45. #J-45. DBU JT FPT !"#$%$&': ) *+, - log 1 2*|2,, … , 2*6,; 89, 8:;<=, 8> + log 1(2*A,, … , 2-; 89, 8:;<=, 8>) 89: Embedding matrix 8:;<=: parameters for LSTMs 8>: parameters for Softmax Layer

  8. &-.P 8PSE3FQSFTFOUBUJPOPGUPLFO!" JTCFMPX  !" -45. GPSXBSE -45. GPSXBSE #"

    = %& '(, *&,+ '(, *&,+ '(|- = 1, … , 0 = *&,+ '(|- = 0, … , 0 %& '( XPSEFNCFEEJOHT *&,+ '(PVUQVUPG KUI MBZFS LUI DFMM CBDLXBSE -45. CBDLXBSE *&,+ '( *&,+ '( %&,+ '(

  9. &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT 0QUJNJ[FUIFXFJHIUPGFBDIIJEEFOMBZFSGPSFBDIUBTL  -45. #J-45. -45. #J-45. &-.PL UBTL

    ! "# "$ "% &' ()*+' ,-.' = ( 0' ; 2,-.' = !,-.' 3 456 7 "4 ,-.'89,; <= >?@>9  TPGUNBYOPSNBMJ[FE XFJHIUT !,-.'  XFJHIUGPSFOUJSF &-.P WFDUPS

  10. "QQMZJOH&-.P UPTVQFSWJTFEUBTLT $PODBU &-.P BOEUBTL`TFNCFEEJOHIJEEFOMBZFSPVUQVU  &-.PL UBTL 8PSE&NCFEEJOHL &-.PL

    UBTL )JEEFO MBZFS PVUQVUL 3// $//

  11. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  12. 5SBJOJOHTFUUJOHT • /PGMBZFSTUBDLJOH MBZFST • /PGUSBJOJOHFQPDIT FQPDIT • -45. VOJUTJ[F

    VOJUT • -45. EJNFOTJPO  • $PSQVT #JMMJPO8PSE#FODINPSL • "QQMZJOHSFTJEVBMDPOOFDUJPOGPSGJSTUMBZFSUPTFDPOEMBZFS 

  13. &WBMVBUJPO3FTVMUT • "QQMZ&-.0UPBTJNQMFCBTFMJOFJOUBTLT 2" 35& 43- $PSFGFSFODF /&3 4FOUJNFOU 

    • "DIJFWFTDPSFTDPNQBSBCMFUPPSCFUUFSUIBO405"JOBMM 

  14. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  15. 8IFSFUPJODMVEF&-.P • *O43- UBTLTQFDJGJDSFQSFTFOUBUJPOJTNPSFJNQPSUBOU  UIVT&-.P JOUIFPVUQVUMBZFSJTMPXFGGFDU • 'PSNPEFMTXJUIBUUFOUJPOMBZFSBGUFS3//JUJTFGGFDUJWF UPBEE&-.P

    UPUIFPVUQVUPG3// 

  16. 8IBU*/'0TDBQUVSFECZ&-.P • $PNQBSFEXJUI(MPWF &-.P DBOXFMMDBQUVSFDPOUFYU JOGPSNBUJPO 

  17. 4BNQMFFGGJDJFODZ • 6TJOH&-.P NBLFT MFBSOJOHNPSFFGGJDJFOU UIBOFYDJTUJOH NPEFM 

  18. 7JTVBMJ[BUJPOPGMFBSOFEXFJHIUT • 7JTVBMJ[BUJPOPGXFJHIUJOH PG&-.P JOFBDIFOEUBTL • *OJOQVUMBZFS GJSTU#J-45. -BZFSJTNPTUFGGFDUJWF •

    *OPVUQVUMBZFS VTJOH JOGPSNBUJPOPGBMMMBZFSTPO BWFSBHF 

  19. $POUFOUT  0WFSWJFX  &-.P&NCFEEJOHT GSPN-BOHVBHF.PEFMT  &WBMVBUJPO  "OBMZTJT

     $PODMVTJPO 

  20. $PODMVTJPO 1SPQPTFEBOFXUZQFPGXPSESFQSFTFOUBUJPO&-.P • .PEFMTCPUIPG ØDPNQMFYDIBSBDUFSJTUJDTPGXPSEVTF FH TZOUBYBOETFNBOUJDT Ø)PXUIFTFVTFTWBSZBDSPTTMJOHVJTUJDDPOUFYUT JF UPNPEFMQPMZTFNZ

    • #BTFEPOJOUFSOBMTUBUFTPGB#J-45.-BOHVBHFNPEFM • $BOCFFBTJMZUPBEEFEUPFYJTUJOHNPEFMT BOEJNQSPWFUIF405"BDSPTTNBOZUBTLT