Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[Journal club] Scalable Diffusion Models with T...
Search
Semantic Machine Intelligence Lab., Keio Univ.
PRO
July 22, 2024
Technology
0
110
[Journal club] Scalable Diffusion Models with Transformers
Semantic Machine Intelligence Lab., Keio Univ.
PRO
July 22, 2024
Tweet
Share
More Decks by Semantic Machine Intelligence Lab., Keio Univ.
See All by Semantic Machine Intelligence Lab., Keio Univ.
[Journal club] Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking
keio_smilab
PRO
0
24
[Journal club] Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance
keio_smilab
PRO
0
38
[Journal club] Influence-Balanced Loss for Imbalanced Visual Classification
keio_smilab
PRO
0
0
[Journal club] Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval
keio_smilab
PRO
0
2
[Journal club] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
keio_smilab
PRO
0
2
[MIRU25] NaiLIA: Multimodal Retrieval of Nail Designs Based on Dense Intent Descriptions
keio_smilab
PRO
1
84
[MIRU25] An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions
keio_smilab
PRO
1
59
[MIRU2025]Preference Optimization for Multimodal Large Language Models for Image Captioning Tasks
keio_smilab
PRO
0
100
Semantic Machine Intelligence for Vision, Language, and Actions
keio_smilab
PRO
2
410
Other Decks in Technology
See All in Technology
claude codeでPrompt Engineering
iori0311
0
530
Vision Language Modelと自動運転AIの最前線_20250730
yuyamaguchi
1
550
robocopy の怖い話/scary-story-about-robocopy
emiki
0
400
激動の時代、新卒エンジニアはAIツールにどう向き合うか。 [LayerX Bet AI Day Countdown LT Day1 ツールの選択]
tak848
0
610
地域コミュニティへの「感謝」と「恩返し」 / 20250726jawsug-tochigi
kasacchiful
0
100
ユーザー理解の爆速化とPdMの価値
kakehashi
PRO
1
110
AWS表彰プログラムとキャリアについて
naoki_0531
1
140
メモ整理が苦手な者による頑張らないObsidian活用術
optim
0
150
CSPヘッダー導入で実現するWebサイトの多層防御:今すぐ試せる設定例と運用知見
llamakko
1
260
RapidPen: AIエージェントによる高度なペネトレーションテスト自動化の研究開発
laysakura
1
410
新規事業におけるAIリサーチの活用例
ranxxx
0
170
[TechNight #91] Oracle Database 最新パフォーマンス分析手法
oracle4engineer
PRO
3
150
Featured
See All Featured
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.3k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
850
Agile that works and the tools we love
rasmusluckow
329
21k
GraphQLとの向き合い方2022年版
quramy
49
14k
The Language of Interfaces
destraynor
158
25k
Navigating Team Friction
lara
187
15k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
44
2.4k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
283
13k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Bash Introduction
62gerente
613
210k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Transcript
4DBMBCMF%JGGVTJPO.PEFMTXJUI 5SBOTGPSNFST ܚጯٛक़େֶ ਿӜ໌ݚڀࣨ#ീౡେ 8JMMJBN1FFCMFT 4BJOJOH9JF 6$#FSLFMFZ /FX:PSL6OJWFSTJUZ *$$7 8JMMJBN1FFCMFT
4BJOJOH9JFl4DBMBCMF%JGGVTJPO.PEFMTXJUI5SBOTGPSNFSTzJO*$$7 QQ
എܠɿ֦ࢄϞσϧʹΑΔಈը૾ੜ FH 4PSB ͷൃల IUUQTXXXZPVUVCFDPNXBUDI W),Z%"1/@
ؔ࿈ݚڀɿ֦ࢄϞσϧͷόοΫϘʔϯͱͯ͠6/FU͕ଟ༻ • 6/FUͷ.VMUJTDBMFTLJQDPOOFDUJPOTˠ ෆཁͳܭࢉࢿݯͷ༻ 手法 概要 DALL-E 2 [Ramesh+,
22] CLIPを用いてテキストと画像のAlignmentを行う Stable Diffusion [Rombach+, CVPR22] 潜在拡散モデル 6/FU<3POOFCFSHFS .*$$"*> 4UBCMF%JGGVTJPO<3PNCBDI $713>
ఏҊख๏ɿ%JGGVTJPO5SBOTGPSNFS %J5 • જࡏ֦ࢄϞσϧ -%. <3PNCBDI $713>Λϕʔεʹߏங • 7JTJPO5SBOTGPSNFS 7J5
<%PTPWJUTLJZ *$-3>ػߏΛಋೖ • $POEJUJPOJOHʹΑΔ݅ใͷೖྗ
ఏҊख๏ ɿજࡏ֦ࢄϞσϧͱֶͯ͠शͤ͞Δ • ߴ࣍ݩͷըૉۭؒͰ֦ࢄϞσϧΛֶश ͤ͞Δ͜ͱܭࢉྔతʹࠔ • -%.ͱֶͯ͠शͤ͞Δ͜ͱͰ ܭࢉྔΛݮ
• ըૉۭؒͷ֦ࢄϞσϧͰ͋Δ "%.<%IBSJXBM /FVS*14> ͷͷͷ(GMPQTͰֶशՄೳ
ఏҊख๏ ɿೖྗϊΠζΛQBUDIʹղ • "VUPFODPEFS͔ΒಘΒΕͨ /PJTFE-BUFOU YY Λ 7J5ͱಉ༷ʹE࣍ݩͷτʔΫϯ5ʹม •
1BUDIαΠζQΛʹ͢Δͱ5ഒ ʹͳΓUSBOTGPSNFSͷ (GMPQTগͳ͘ͱഒҎ্
ఏҊख๏ ɿ͖݅ೖྗ $POEJUJPOJOH ͷॲཧ • ͖֦݅ࢄϞσϧͰϊΠζΛؚΉը૾ͱͱʹՃใ͕Ճ͑ΒΕΔ FH UJNFTUFQɼΫϥεϥϕϧɼࣗવݴޠ FUD
• ຊݚڀͰ͜ΕΒͷ͖݅ೖྗΛॲཧ͢ΔͨΊʹҎԼͷͭͷҟͳΔઃܭΛఏҊ • *ODPOUFYUDPOEJUJPOJOH • $SPTT"UUFOUJPOCMPDL • "EBQUJWFMBZFSOPSN BEB-/ CMPDL • BEB-/;FSPCMPDL
ఏҊख๏ ɿBEB-/;FSPCMPDL • 7J5ͷTFMGBUUFOUJPOCMPDLʹରͯ͠"EB-/ػߏΛಋೖ • "EB-/ͷεέʔϧ ͓Αͼ ࠩଓͷલͷεέʔϧ
Λύϥϝʔλͱͯ͠Ճ ˠ݅ใΛը૾ʹΑΓڧ͘ө • "EB-/;FSPCMPDLͰͦΕΒΛθϩʹॳظԽ ˠֶशͷॳظஈ֊߃ؔʹ͍ۙಇ͖ ˠ ֶशͷ҆ఆԽ
࣮ݧઃఆ • σʔληοτ • $MBTT$POEJUJPOBM*NBHF/FUY Y<%FOH $713> • ΞʔΩςΫνϟ
• 7J5ͱಉ༷ʹͭͷϞσϧͷେ͖͞ 4 # - 9- Λ༻ҙ • QBUDITJ[FQ • %%1.TBNQMJOHTUFQT • ධՁई • '*% T'*% *4 1SFDJTJPO 3FDBMM • (GMPQT • ֶश • 516WQPE #BUDITJ[F
ఆྔత݁Ռɿ6/FUϕʔεͷख๏Λ্ճͬͨ
ఆੑత݁Ռ • 1BUDITJ[FΛখ͘͞ɼϞσϧΛେ͖͘͢ΔͱΑΓࣗવͳը૾͕ग़ྗ͞ΕΔ ˠ%J5Ͱ(GMPQT͕େ͖͍΄Ͳग़ྗը૾ͷ্࣭͕͕Δ
ࢼ͓ΑͼΤϥʔੳ ఆੑత݁Ռ ɿࣦഊྫ • ಛఆͷMBCFMʹରͯ͠ෆࣗવͳը૾͕ੜ͞ΕΔ • ྫɿJOQVUMBCFM UPZQPPEMF %%1.TBNQMJOHTUFQ
• ϥϕϧʹΑͬͯTUFQͰੜը૾͕ෆ҆ఆˠ ਪ࣌ͷTUFQΛಈతʹมߋ
ॴײ • 4USFOHUI • ֦ࢄϞσϧʹUSBOTGPSNFSΛಋೖ • ܭࢉࢿݯͱग़ྗը૾ͷ࣭ʹ͍ͭͯͷߟ • 8FBLOFTT
• ͕ࣜগͳ͔ͬͨ • Τϥʔੳ͕ͳ͍
·ͱΊ • എܠ • ֦ࢄϞσϧʹΑΔಈը૾ੜ FH 4PSB ͷൃల • ֦ࢄϞσϧʹ͓͚ΔUSBOTGPSNFSͷར༻͕গͳ͍
• ఏҊख๏ • USBOTGPSNFSϕʔεͷ֦ࢄϞσϧͰ͋Δ%JGGVTJPO5SBOTGPSNFS %J5 ΛఏҊ • ݁Ռ • %J5εέʔϥϏϦςΟ͕ߴ͘ɼ(GMPQT͕େ͖͍΄Ͳ'*%͕Լ ˠ ܭࢉࢿݯͱग़ྗը૾ͷ࣭ʹڧ͍૬ؔؔ • %J59-Ϟσϧ͕ɼ$MBTT$POEJUJPOBM*NBHF/FUʹ͓͍ͯ ैདྷͷ6/FUϕʔεͷ֦ࢄϞσϧΛ্ճͬͨ
"QQFOEJYɿ%FOPJTJOH%JGGVTJPO1SPCBCJMJTUJD.PEFM %%1. ֶश
"QQFOEJYɿ$MBTTJGJFSGSFFHVJEBODF • ͖֦݅ࢄϞσϧͰΫϥεϥϕϧΛϥϯμϜʹυϩοϓ ˠ αϯϓϦϯάͷਫ਼Λ্ • #BZFTͷఆཧΑΓ • ֦ࢄϞσϧͷग़ྗΛείΞͱͯ͠ղऍ͢Δͱਪఆ͢ΔϊΠζҎԼͷΑ͏ʹͳΔ
TɿΨΠμϯεεέʔϧ
"QQFOEJYɿ*ODPOUFYUDPOEJUJPOJOH • $POEJUJPOJOHͰ݅ͱͯ͠ೖྗ͞ΕͨτʔΫϯΛ ը૾τʔΫϯͷઌ಄ʹՃ • ͜ΕΒͷτʔΫϯը૾τʔΫϯͱಉ༷ʹѻΘΕɺ 7J5ʣʹ͓͚ΔDMTτʔΫϯͱࣅׂͨΛ࣋ͭ
"QQFOEJYɿ$SPTT"UUFOUJPOCMPDL • 4FMG"UUFOUJPOϒϩοΫͷޙʹ$SPTT"UUFOUJPOΛ Ճͨ͠ઃܭ • <7BTXBOJ /*14>-%.ͱྨࣅͨ͠ΞʔΩςΫνϟ
"QQFOEJYɿ%J5CMPDLEFTJHO • %J59-Ϟσϧʹ͓͍ͯBEB-/;FSPΛ༻͍ͨ ߹͕࠷গͳ͍ܭࢉࢿݯͰ࠷ྑ͍ '*%,είΞΛୡ
"QQFOEJYɿ7JTJPO5SBOTGPSNFS <%PTPWJUTLJZ *$-3>
"QQFOEJYɿ*ODFQUJPO4DPSF *4 • *NBHF/FUͰࣄલֶशࡁΈͷ*ODFQUJPOOFUXPSLΛ༻͍ͨධՁࢦඪ • *ODFQUJPOOFUXPSL͕ࣝผ͘͢͠ɼࣝผ͞ΕΔϥϕϧͷଟ༷ੑ͕͋Δ΄Ͳ େ͖͘ͳΔࢦඪ <4[FHFEZ $713>
"QQFOEJYɿ'SFDIFU*ODFQUJPO%JTUBODF '*% • *NBHF/FUͰࣄલֶशࡁΈͷ*ODFQUJPOOFUXPSLΛ༻͍ͨධՁࢦඪ • ੜ͞Εͨը૾ͷಛ͕(5ը૾ͷಛͱͲͷఔ ࣅ͍ͯΔ͔ΛධՁ͢Δࢦඪ • '*%͕খ͍͞΄Ͳੜ͞Εͨը૾ͷ࣭͕(5ը૾ʹ͍ۙͱߟ͑ΒΕΔ
"QQFOEJYɿ1SFDJTJPO3FDBMM • *NBHF/FUͰࣄલֶशࡁΈͷ7((<4JNPOZBO *$-3>Λ༻͍ͯ ಛϕΫτϧू߹ΛಘΔ
"QQFOEJYɿ(GMPQT • 'MPQTɿුಈখԋࢉͷճ • (GMPQT 'MPQT • ը૾ੜλεΫͰΞʔΩςΫνϟͷෳࡶ͞ΛධՁ͢ΔࡍύϥϝʔλΛ༻͍Δͷ ͕Ұൠత
• ੑೳʹେ͖͘Өڹ͢Δը૾ղ૾ΛҰߟྀ͍ͯ͠ͳ͍ • Ϟσϧͷෳࡶ͞Λද͢ࢦඪͱͯ͠ෆेͳ߹͕͋Δ
"QQFOEJYɿఆྔత݁Ռ
"QQFOEJY(GMPQTͱ'*%ͷ૬ؔ • ΑΓଟ͘ͷ(GMPQTΛͭϞσϧ'*%͕͘ͳΔ
"QQFOEJYɿϞσϧαΠζͱύοναΠζͷݕ౼