$30 off During Our Annual Pro Sale. View Details »

文献紹介: Similarity-Based Reconstruction Loss for Meaning Representation

Yumeto Inaoka
May 26, 2019
Research
1
140

文献紹介: Similarity-Based Reconstruction Loss for Meaning Representation

2019/05/28の文献紹介で発表

Yumeto Inaoka

May 26, 2019
Tweet

More Decks by Yumeto Inaoka

See All by Yumeto Inaoka
文献紹介: Quantity doesn’t buy quality syntax with neural language models
yumeto
1
100
文献紹介: Open Domain Web Keyphrase Extraction Beyond Language Modeling
yumeto
0
130
文献紹介: Self-Supervised_Neural_Machine_Translation
yumeto
0
110
文献紹介: Comparing and Developing Tools to Measure the Readability of Domain-Specific Texts
yumeto
0
110
文献紹介: PAWS: Paraphrase Adversaries from Word Scrambling
yumeto
0
76
文献紹介: Beyond BLEU: Training Neural Machine Translation with Semantic Similarity
yumeto
0
190
文献紹介: EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing
yumeto
0
240
文献紹介: Decomposable Neural Paraphrase Generation
yumeto
0
170
文献紹介: Analyzing the Limitations of Cross-lingual Word Embedding Mappings
yumeto
0
160

Other Decks in Research

See All in Research
遺伝的アルゴリズムの簡単な紹介
kentaitakura
0
310
[CVPR 2023 論文紹介]Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentation
elith
0
140
自作パケット処理系の性能測定と可視化&改善のPDCAを回して最強のパケット処理系の作り方を学ぼう / Let's Measure the Performance of Packet Processing System with Python Tools.
takehaya
3
2k
Google Cloud Next’23「Title：出店ブースで10社以上におすすめ機能を質問してきた」
yasumuusan
0
380
20230923_Kaggler_meetup_LT_ya9do.pdf
ya9do
2
1.2k
Elix, CBI 2023, フォーカストセッション, 生成モデルを中心としたElixにおけるAI創薬
elix
0
130
[NLPコロキウム 2023/9/13] Unbalanced Optimal Transport for Unbalanced Word Alignment
yukiar
2
200
DroidKaigi2023 突撃！隣のコードレビュー
satotaichi
0
2.1k
Hyena Hierarchy: Towards Larger Convolutional Language Models
hpprc
3
860
マルチリンガルな言語モデル入門：これまでとこれから
ryou0634
2
2.1k
論文紹介 / Llama 2: Open Foundation and Fine-Tuned Chat Models
kyoun
4
4.5k
【論文紹介】gSASRec_Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling / recsys2023-gsasrec
yuya4
0
300

Featured

See All Featured
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
16
1.5k
Designing with Data
zakiwarfel
93
4.6k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
17k
10 Git Anti Patterns You Should be Aware of
lemiorhan
644
57k
The Mythical Team-Month
searls
214
41k
From Idea to $5000 a Month in 5 Months
shpigford
376
45k
Practical Orchestrator
shlominoach
179
9.4k
In The Pink: A Labor of Love
frogandcode
135
21k
The Power of CSS Pseudo Elements
geoffreycrofte
56
4.7k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
1
880
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
44
12k
Imperfection Machines: The Place of Print at Facebook
scottboms
257
12k

Transcript

  1. Similarity-Based Reconstruction Loss
    for Meaning Representation

    View Slide

  2. Literature
    2

    View Slide

  3. Abstract
    •
    •
    •
    3

    View Slide

  4. Introduction
    •
    •
    4

    View Slide

  5. Related Work
    •
    •
    •
    •
    5

    View Slide

  6. Related Work
    •
    •
    6

    View Slide

  7. Auto-Encoder
    •ℒ ,
    •
    •
    •
    •
    7

    View Slide

  8. Weighted similarity loss
    •ℒ = − σ
    =1
    sim
    ,

    •
    •
    •
    :
    •
    • sim()
    •
    8

    View Slide

  9. Weighted cross-entropy loss
    •ℒ = − σ
    =1
    sim
    ,
    log(
    )
    •
    •
    9

    View Slide

  10. Soft label loss
    •ℒ = − σ
    =1

    ∗log
    •
    ∗ = ൞
    sim ,
    σ
    =1
    sim(,)
    ,
    ∈ top N
    0 ,
    ∉ top N
    •
    •
    10

    View Slide

  11. True-label encoding
    11

    View Slide

  12. Tasks & Datasets
    •
    •
    •
    12

    View Slide

  13. Results
    13

    View Slide

  14. Results
    14

    View Slide

  15. Additional Experiments
    •
    •
    15

    View Slide

  16. Results
    •
    •
    16

    View Slide

  17. Results
    17

    View Slide

  18. Results
    18

    View Slide

  19. Results
    19

    View Slide

  20. Discussion
    •
    •
    20

    View Slide

  21. Conclusion
    •
    •
    •
    •
    21

    View Slide