Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
[Journal club] Scalable Diffusion Models with T...
Search
Semantic Machine Intelligence Lab., Keio Univ.
PRO
July 22, 2024
Technology
0
83
[Journal club] Scalable Diffusion Models with Transformers
Semantic Machine Intelligence Lab., Keio Univ.
PRO
July 22, 2024
Tweet
Share
More Decks by Semantic Machine Intelligence Lab., Keio Univ.
See All by Semantic Machine Intelligence Lab., Keio Univ.
Will multimodal language processing change the world?
keio_smilab
PRO
3
410
[Journal club] MOKA: Open-Vocabulary Robotic Manipulation through Mark-Based Visual Prompting
keio_smilab
PRO
0
82
[Journal club] Seeing the Unseen: Visual Common Sense for Semantic Placement
keio_smilab
PRO
0
82
[Journal club] Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile Robot
keio_smilab
PRO
0
84
[Journal club] RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation
keio_smilab
PRO
1
100
[Journal club] Simplified State Space Layers for Sequence Modeling
keio_smilab
PRO
0
93
[Journal club] Detecting and Preventing Hallucinations in Large Vision Language Models
keio_smilab
PRO
1
110
[IROS24] Object Segmentation from Open-Vocabulary Manipulation Instructions Based on Optimal Transport Polygon Matching with Multimodal Foundation Models
keio_smilab
PRO
0
78
[IROS24] Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Engine
keio_smilab
PRO
0
130
Other Decks in Technology
See All in Technology
30分でわかる「リスクから学ぶKubernetesコンテナセキュリティ」/30min-k8s-container-sec
mochizuki875
3
450
ドメイン駆動設計の実践により事業の成長スピードと保守性を両立するショッピングクーポン
lycorptech_jp
PRO
14
2.5k
.NET 最新アップデート ~ AI とクラウド時代のアプリモダナイゼーション
chack411
0
200
Azureの開発で辛いところ
re3turn
0
240
新卒1年目、はじめてのアプリケーションサーバー【IBM WebSphere Liberty】
ktgrryt
0
150
PaaSの歴史と、 アプリケーションプラットフォームのこれから
jacopen
7
1.5k
色々なAWSサービス名の由来を調べてみた
iriikeita
0
110
WantedlyでのKotlin Multiplatformの導入と課題 / Kotlin Multiplatform Implementation and Challenges at Wantedly
kubode
0
250
Reactフレームワークプロダクトを モバイルアプリにして、もっと便利に。 ユーザに価値を届けよう。/React Framework with Capacitor
rdlabo
0
140
Visual StudioとかIDE関連小ネタ話
kosmosebi
1
380
AIアプリケーション開発でAzure AI Searchを使いこなすためには
isidaitc
1
130
生成AI × 旅行 LLMを活用した旅行プラン生成・チャットボット
kominet_ava
0
160
Featured
See All Featured
Git: the NoSQL Database
bkeepers
PRO
427
64k
Automating Front-end Workflow
addyosmani
1366
200k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Optimizing for Happiness
mojombo
376
70k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Embracing the Ebb and Flow
colly
84
4.5k
Into the Great Unknown - MozCon
thekraken
34
1.6k
Making the Leap to Tech Lead
cromwellryan
133
9k
Navigating Team Friction
lara
183
15k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
10
870
Statistics for Hackers
jakevdp
797
220k
Transcript
4DBMBCMF%JGGVTJPO.PEFMTXJUI 5SBOTGPSNFST ܚጯٛक़େֶ ਿӜ໌ݚڀࣨ#ീౡେ 8JMMJBN1FFCMFT 4BJOJOH9JF 6$#FSLFMFZ /FX:PSL6OJWFSTJUZ *$$7 8JMMJBN1FFCMFT
4BJOJOH9JFl4DBMBCMF%JGGVTJPO.PEFMTXJUI5SBOTGPSNFSTzJO*$$7 QQ
എܠɿ֦ࢄϞσϧʹΑΔಈը૾ੜ FH 4PSB ͷൃల IUUQTXXXZPVUVCFDPNXBUDI W),Z%"1/@
ؔ࿈ݚڀɿ֦ࢄϞσϧͷόοΫϘʔϯͱͯ͠6/FU͕ଟ༻ • 6/FUͷ.VMUJTDBMFTLJQDPOOFDUJPOTˠ ෆཁͳܭࢉࢿݯͷ༻ 手法 概要 DALL-E 2 [Ramesh+,
22] CLIPを用いてテキストと画像のAlignmentを行う Stable Diffusion [Rombach+, CVPR22] 潜在拡散モデル 6/FU<3POOFCFSHFS .*$$"*> 4UBCMF%JGGVTJPO<3PNCBDI $713>
ఏҊख๏ɿ%JGGVTJPO5SBOTGPSNFS %J5 • જࡏ֦ࢄϞσϧ -%. <3PNCBDI $713>Λϕʔεʹߏங • 7JTJPO5SBOTGPSNFS 7J5
<%PTPWJUTLJZ *$-3>ػߏΛಋೖ • $POEJUJPOJOHʹΑΔ݅ใͷೖྗ
ఏҊख๏ ɿજࡏ֦ࢄϞσϧͱֶͯ͠शͤ͞Δ • ߴ࣍ݩͷըૉۭؒͰ֦ࢄϞσϧΛֶश ͤ͞Δ͜ͱܭࢉྔతʹࠔ • -%.ͱֶͯ͠शͤ͞Δ͜ͱͰ ܭࢉྔΛݮ
• ըૉۭؒͷ֦ࢄϞσϧͰ͋Δ "%.<%IBSJXBM /FVS*14> ͷͷͷ(GMPQTͰֶशՄೳ
ఏҊख๏ ɿೖྗϊΠζΛQBUDIʹղ • "VUPFODPEFS͔ΒಘΒΕͨ /PJTFE-BUFOU YY Λ 7J5ͱಉ༷ʹE࣍ݩͷτʔΫϯ5ʹม •
1BUDIαΠζQΛʹ͢Δͱ5ഒ ʹͳΓUSBOTGPSNFSͷ (GMPQTগͳ͘ͱഒҎ্
ఏҊख๏ ɿ͖݅ೖྗ $POEJUJPOJOH ͷॲཧ • ͖֦݅ࢄϞσϧͰϊΠζΛؚΉը૾ͱͱʹՃใ͕Ճ͑ΒΕΔ FH UJNFTUFQɼΫϥεϥϕϧɼࣗવݴޠ FUD
• ຊݚڀͰ͜ΕΒͷ͖݅ೖྗΛॲཧ͢ΔͨΊʹҎԼͷͭͷҟͳΔઃܭΛఏҊ • *ODPOUFYUDPOEJUJPOJOH • $SPTT"UUFOUJPOCMPDL • "EBQUJWFMBZFSOPSN BEB-/ CMPDL • BEB-/;FSPCMPDL
ఏҊख๏ ɿBEB-/;FSPCMPDL • 7J5ͷTFMGBUUFOUJPOCMPDLʹରͯ͠"EB-/ػߏΛಋೖ • "EB-/ͷεέʔϧ ͓Αͼ ࠩଓͷલͷεέʔϧ
Λύϥϝʔλͱͯ͠Ճ ˠ݅ใΛը૾ʹΑΓڧ͘ө • "EB-/;FSPCMPDLͰͦΕΒΛθϩʹॳظԽ ˠֶशͷॳظஈ֊߃ؔʹ͍ۙಇ͖ ˠ ֶशͷ҆ఆԽ
࣮ݧઃఆ • σʔληοτ • $MBTT$POEJUJPOBM*NBHF/FUY Y<%FOH $713> • ΞʔΩςΫνϟ
• 7J5ͱಉ༷ʹͭͷϞσϧͷେ͖͞ 4 # - 9- Λ༻ҙ • QBUDITJ[FQ • %%1.TBNQMJOHTUFQT • ධՁई • '*% T'*% *4 1SFDJTJPO 3FDBMM • (GMPQT • ֶश • 516WQPE #BUDITJ[F
ఆྔత݁Ռɿ6/FUϕʔεͷख๏Λ্ճͬͨ
ఆੑత݁Ռ • 1BUDITJ[FΛখ͘͞ɼϞσϧΛେ͖͘͢ΔͱΑΓࣗવͳը૾͕ग़ྗ͞ΕΔ ˠ%J5Ͱ(GMPQT͕େ͖͍΄Ͳग़ྗը૾ͷ্࣭͕͕Δ
ࢼ͓ΑͼΤϥʔੳ ఆੑత݁Ռ ɿࣦഊྫ • ಛఆͷMBCFMʹରͯ͠ෆࣗવͳը૾͕ੜ͞ΕΔ • ྫɿJOQVUMBCFM UPZQPPEMF %%1.TBNQMJOHTUFQ
• ϥϕϧʹΑͬͯTUFQͰੜը૾͕ෆ҆ఆˠ ਪ࣌ͷTUFQΛಈతʹมߋ
ॴײ • 4USFOHUI • ֦ࢄϞσϧʹUSBOTGPSNFSΛಋೖ • ܭࢉࢿݯͱग़ྗը૾ͷ࣭ʹ͍ͭͯͷߟ • 8FBLOFTT
• ͕ࣜগͳ͔ͬͨ • Τϥʔੳ͕ͳ͍
·ͱΊ • എܠ • ֦ࢄϞσϧʹΑΔಈը૾ੜ FH 4PSB ͷൃల • ֦ࢄϞσϧʹ͓͚ΔUSBOTGPSNFSͷར༻͕গͳ͍
• ఏҊख๏ • USBOTGPSNFSϕʔεͷ֦ࢄϞσϧͰ͋Δ%JGGVTJPO5SBOTGPSNFS %J5 ΛఏҊ • ݁Ռ • %J5εέʔϥϏϦςΟ͕ߴ͘ɼ(GMPQT͕େ͖͍΄Ͳ'*%͕Լ ˠ ܭࢉࢿݯͱग़ྗը૾ͷ࣭ʹڧ͍૬ؔؔ • %J59-Ϟσϧ͕ɼ$MBTT$POEJUJPOBM*NBHF/FUʹ͓͍ͯ ैདྷͷ6/FUϕʔεͷ֦ࢄϞσϧΛ্ճͬͨ
"QQFOEJYɿ%FOPJTJOH%JGGVTJPO1SPCBCJMJTUJD.PEFM %%1. ֶश
"QQFOEJYɿ$MBTTJGJFSGSFFHVJEBODF • ͖֦݅ࢄϞσϧͰΫϥεϥϕϧΛϥϯμϜʹυϩοϓ ˠ αϯϓϦϯάͷਫ਼Λ্ • #BZFTͷఆཧΑΓ • ֦ࢄϞσϧͷग़ྗΛείΞͱͯ͠ղऍ͢Δͱਪఆ͢ΔϊΠζҎԼͷΑ͏ʹͳΔ
TɿΨΠμϯεεέʔϧ
"QQFOEJYɿ*ODPOUFYUDPOEJUJPOJOH • $POEJUJPOJOHͰ݅ͱͯ͠ೖྗ͞ΕͨτʔΫϯΛ ը૾τʔΫϯͷઌ಄ʹՃ • ͜ΕΒͷτʔΫϯը૾τʔΫϯͱಉ༷ʹѻΘΕɺ 7J5ʣʹ͓͚ΔDMTτʔΫϯͱࣅׂͨΛ࣋ͭ
"QQFOEJYɿ$SPTT"UUFOUJPOCMPDL • 4FMG"UUFOUJPOϒϩοΫͷޙʹ$SPTT"UUFOUJPOΛ Ճͨ͠ઃܭ • <7BTXBOJ /*14>-%.ͱྨࣅͨ͠ΞʔΩςΫνϟ
"QQFOEJYɿ%J5CMPDLEFTJHO • %J59-Ϟσϧʹ͓͍ͯBEB-/;FSPΛ༻͍ͨ ߹͕࠷গͳ͍ܭࢉࢿݯͰ࠷ྑ͍ '*%,είΞΛୡ
"QQFOEJYɿ7JTJPO5SBOTGPSNFS <%PTPWJUTLJZ *$-3>
"QQFOEJYɿ*ODFQUJPO4DPSF *4 • *NBHF/FUͰࣄલֶशࡁΈͷ*ODFQUJPOOFUXPSLΛ༻͍ͨධՁࢦඪ • *ODFQUJPOOFUXPSL͕ࣝผ͘͢͠ɼࣝผ͞ΕΔϥϕϧͷଟ༷ੑ͕͋Δ΄Ͳ େ͖͘ͳΔࢦඪ <4[FHFEZ $713>
"QQFOEJYɿ'SFDIFU*ODFQUJPO%JTUBODF '*% • *NBHF/FUͰࣄલֶशࡁΈͷ*ODFQUJPOOFUXPSLΛ༻͍ͨධՁࢦඪ • ੜ͞Εͨը૾ͷಛ͕(5ը૾ͷಛͱͲͷఔ ࣅ͍ͯΔ͔ΛධՁ͢Δࢦඪ • '*%͕খ͍͞΄Ͳੜ͞Εͨը૾ͷ࣭͕(5ը૾ʹ͍ۙͱߟ͑ΒΕΔ
"QQFOEJYɿ1SFDJTJPO3FDBMM • *NBHF/FUͰࣄલֶशࡁΈͷ7((<4JNPOZBO *$-3>Λ༻͍ͯ ಛϕΫτϧू߹ΛಘΔ
"QQFOEJYɿ(GMPQT • 'MPQTɿුಈখԋࢉͷճ • (GMPQT 'MPQT • ը૾ੜλεΫͰΞʔΩςΫνϟͷෳࡶ͞ΛධՁ͢ΔࡍύϥϝʔλΛ༻͍Δͷ ͕Ұൠత
• ੑೳʹେ͖͘Өڹ͢Δը૾ղ૾ΛҰߟྀ͍ͯ͠ͳ͍ • Ϟσϧͷෳࡶ͞Λද͢ࢦඪͱͯ͠ෆेͳ߹͕͋Δ
"QQFOEJYɿఆྔత݁Ռ
"QQFOEJY(GMPQTͱ'*%ͷ૬ؔ • ΑΓଟ͘ͷ(GMPQTΛͭϞσϧ'*%͕͘ͳΔ
"QQFOEJYɿϞσϧαΠζͱύοναΠζͷݕ౼