Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Searching for Needles in a Haystack: On the Rol...

Avatar for Hiroyuki Deguchi Hiroyuki Deguchi
October 03, 2023
Research
0
130

Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM’s Translation Capability

Avatar for Hiroyuki Deguchi

Hiroyuki Deguchi

October 03, 2023
Tweet

More Decks by Hiroyuki Deguchi

See All by Hiroyuki Deguchi
20250226 NLP colloquium: "SoftMatcha: 10億単語規模コーパス検索のための柔らかくも高速なパターンマッチャー"
Avatar for Hiroyuki Deguchi de9uch1
0
600
20240820: Minimum Bayes Risk Decoding for High-Quality Text Generation Beyond High-Probability Text
Avatar for Hiroyuki Deguchi de9uch1
0
300
サブセット探索を用いた高速なkNNニューラル機械翻訳
Avatar for Hiroyuki Deguchi de9uch1
0
140
20240226_AAMT-Japio
Avatar for Hiroyuki Deguchi de9uch1
0
160
Paper Reading: Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translation
Avatar for Hiroyuki Deguchi de9uch1
0
190
My Research Environmental Setup
Avatar for Hiroyuki Deguchi de9uch1
0
310
Nearest Neighbor Machine Translation
Avatar for Hiroyuki Deguchi de9uch1
0
260
Paper Reading - Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
Avatar for Hiroyuki Deguchi de9uch1
0
290
paper reading - Tree Transformer
Avatar for Hiroyuki Deguchi de9uch1
0
260

Other Decks in Research

See All in Research
Sat2City:3D City Generation from A Single Satellite Image with Cascaded Latent Diffusion
Avatar for SatAI.challenge satai
4
260
日本語新聞記事を用いた大規模言語モデルの暗記定量化 / LLMC2025
Avatar for Shotaro Ishihara upura
0
340
生成AI による論文執筆サポート・ワークショップ ─ サーベイ／リサーチクエスチョン編 / Workshop on AI-Assisted Paper Writing Support: Survey/Research Question Edition
Avatar for Kenji Saito ks91
PRO
0
110
Adaptive Experimental Design for Efficient Average Treatment Effect Estimation and Treatment Choice
Avatar for MasaKat0 masakat0
0
140
SREのためのテレメトリー技術の探究 / Telemetry for SRE
Avatar for Yuuki Tsubouchi (yuuk1) yuukit
12
1.9k
論文紹介: ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement
Avatar for Hisao Katsumi hisaokatsumi
0
130
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
Avatar for Akihiro Kokubo akihiro_kokubo
PRO
63
34k
とあるSREの博士「過程」 / A Certain SRE’s Ph.D. Journey
Avatar for Yuuki Tsubouchi (yuuk1) yuukit
11
4.9k
問いを起点に、社会と共鳴する知を育む場へ
Avatar for MATSUMOTO Ryosuke matsumoto_r
PRO
0
710
カスタマーサクセスの視点からAWS Summitの展示を考える～製品開発で活用できる勘所～
Avatar for モブエンジニア（Masaki Okuda） masakiokuda
2
230
心理言語学の視点から再考する言語モデルの学習過程
Avatar for Masato Mita chemical_tree
2
680
機械学習と数理最適化の融合 (MOAI) による革新
Avatar for MIKIO KUBO mickey_kubo
1
420

Featured

See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
Avatar for Vitaly Friedman smashingmag
253
22k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
671
58k
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
659
61k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
410
Music & Morning Musume
Avatar for Bryan Veloso bryan
46
7k
Rebuilding a faster, lazier Slack
Avatar for samanthasiow samanthasiow
84
9.3k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
25
1.6k
Optimizing for Happiness
Avatar for Tom Preston-Werner mojombo
379
70k
How GitHub (no longer) Works
Avatar for Zach Holman holman
315
140k
Large-scale JavaScript Application Architecture
Avatar for Addy Osmani addyosmani
514
110k
Code Reviewing Like a Champion
Avatar for maltzj maltzj
527
40k

Transcript

  1. None

  2. ◼ [1] ◼ ◼ ◼ [1] PaLM: Scaling Language Modeling

    with Pathways (Chowdhery+, JMLR vol.24, no.240, 2023)

  3. [1] ◼ ◼ ⚫ ⚫ ⚫ ⚫ ◼ ◼ [1]

    PaLM: Scaling Language Modeling with Pathways (Chowdhery+, JMLR vol.24, no.240, 2023)

  4. ◼ ⚫ ⚫ ◼ ⚫ ⚫ ▶ ▶

  5. ◼ ⚫ ⚫ [2] ⚫ [2] ◼ ⚫ ⚫ [2]

    Language-agnostic BERT Sentence Embedding (Feng+, ACL2022)

  6. ◼ ⚫ ⚫

  7. ◼ ⚫ ◼ ◼

  8. ◼ ◼ ◼

  9. ◼ ⚫ ▶ ▶ ⚫ ⚫ ◼ ⚫ ▶

  10. ◼ ⚫ ⚫ ⚫ ▶ ▶ ▶

  11. ◼ ⚫ ⚫ ▶ ▶

  12. ◼ ⚫ ▶ ▶ ◼ ⚫ ⚫ ▶ ⚫