Upgrade to Pro — share decks privately, control downloads, hide ads and more …

論文紹介：Not All Tokens Are What You Need for Pretr...

Avatar for Kosuke Nishida Kosuke Nishida
August 21, 2025
Research
0
1

論文紹介：Not All Tokens Are What You Need for Pretraining

Avatar for Kosuke Nishida

Kosuke Nishida

August 21, 2025
Tweet

More Decks by Kosuke Nishida

See All by Kosuke Nishida
⼤規模⾔語モデルとVision-and-Language
Avatar for Kosuke Nishida kosuken
6
2.3k
論文紹介：What Learning Algorithm is In-Context Learning? Investigation with Linear Models
Avatar for Kosuke Nishida kosuken
0
1k
論文紹介: Memorisation versus Generalisation in Pre-trained Language Models
Avatar for Kosuke Nishida kosuken
2
1.2k

Other Decks in Research

See All in Research
NLP2025参加報告会 LT資料
Avatar for Hayahide Yamagishi hargon24
1
350
経済学と機械学習：因果推論と密度比推定を中心に
Avatar for MasaKat0 masakat0
0
130
When Submarine Cables Go Dark: Examining the Web Services Resilience Amid Global Internet Disruptions
Avatar for Irvin Chen irvin
0
280
cvpaper.challenge 10年の軌跡 / cvpaper.challenge a decade-long journey
Avatar for gatheluck gatheluck
2
290
SSII2025 [SS2] 横浜DeNAベイスターズの躍進を支えたAIプロダクト
Avatar for 画像センシングシンポジウム ssii
PRO
7
3.9k
利用シーンを意識した推薦システム 〜SpotifyとAmazonの事例から〜
Avatar for kuri8ive kuri8ive
1
240
研究テーマのデザインと研究遂行の方法論
Avatar for Hisashi Ishihara hisashiishihara
5
1.6k
【緊急警告】日本の未来設計図 ～沈没か、再生か。国民と断行するラストチャンス～
Avatar for 高木祐介 yuutakasan
0
150
Submeter-level land cover mapping of Japan
Avatar for SatAI.challenge satai
3
200
2021年度-基盤研究B-研究計画調書
Avatar for Y. Yamamoto trycycle
PRO
0
220
不確実性下における目的と手段の統合的探索に向けた連続腕バンディットの応用 / iot70_gp_rff_mab
Avatar for monochromegane monochromegane
1
110
業界横断 副業・兼業者の実態調査
Avatar for fkske fkske
0
220

Featured

See All Featured
Code Reviewing Like a Champion
Avatar for maltzj maltzj
525
40k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.4k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
25
1.8k
Fireside Chat
Avatar for paigeccino paigeccino
39
3.6k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
7
810
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
328
39k
GraphQLの誤解/rethinking-graphql
Avatar for sonatard sonatard
71
11k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
183
54k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
9
770
Rebuilding a faster, lazier Slack
Avatar for samanthasiow samanthasiow
83
9.1k
How GitHub (no longer) Works
Avatar for Zach Holman holman
314
140k

Transcript

  1. © NTT, Inc. 2025 紹介者：西田光甫（NTT人間研） 最先端NLP勉強会2025