Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
アバターミーティングを支える技術
Search
ねおりん
January 13, 2021
Technology
0
620
アバターミーティングを支える技術
Gotanda.unity #16
https://meetup.unity3d.jp/jp/events/1268
https://youtu.be/FTHfVpBfpo0
ねおりん
January 13, 2021
Tweet
Share
More Decks by ねおりん
See All by ねおりん
UIElements 最高! 一番好きなUIフレームワークです!
noir_neo
0
620
24時間くらいで作ったゲーム 2019年
noir_neo
0
1.3k
24時間でゲームをつくるための Unity
noir_neo
2
1.9k
Multiuser AR Experience を ARKit + Unity で実装する
noir_neo
0
2.6k
これで安心! hubot 沼の堕ち方〜娘がいると開発が楽しいね!
noir_neo
0
360
MDLT#004
noir_neo
0
1.5k
FU2 展示反省
noir_neo
0
130
FU2進捗141004
noir_neo
0
150
FU2進捗140920
noir_neo
0
140
Other Decks in Technology
See All in Technology
Geminiとv0による高速プロトタイピング
shinya337
0
210
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
4.3k
Beyond Kaniko: Navigating Unprivileged Container Image Creation
f30
0
120
製造業からパッケージ製品まで、あらゆる領域をカバー!生成AIを利用したテストシナリオ生成 / 20250627 Suguru Ishii
shift_evolve
PRO
1
160
AI専用のリンターを作る #yumemi_patch
bengo4com
5
3k
OpenHands🤲にContributeしてみた
kotauchisunsun
1
510
CursorによるPMO業務の代替 / Automating PMO Tasks with Cursor
motoyoshi_kakaku
2
860
Yamla: Rustでつくるリアルタイム性を追求した機械学習基盤 / Yamla: A Rust-Based Machine Learning Platform Pursuing Real-Time Capabilities
lycorptech_jp
PRO
4
200
生成AIで小説を書くためにプロンプトの制約や原則について学ぶ / prompt-engineering-for-ai-fiction
nwiizo
6
3.9k
Fabric + Databricks 2025.6 の最新情報ピックアップ
ryomaru0825
1
160
ドメイン特化なCLIPモデルとデータセットの紹介
tattaka
2
540
Node-REDのFunctionノードでMCPサーバーの実装を試してみた / Node-RED × MCP 勉強会 vol.1
you
PRO
0
130
Featured
See All Featured
Statistics for Hackers
jakevdp
799
220k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
Designing Experiences People Love
moore
142
24k
The World Runs on Bad Software
bkeepers
PRO
69
11k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Transcript
ΞόλʔϛʔςΟϯάΛࢧ͑ Δٕज़ 2021/1/13 Gotanda.unity #16 Ͷ͓ΓΜ 1
noir_neo • Ͷ͓ΓΜ • Ϋϥελʔגࣜձࣾ • ιϑτΣΞΤϯδχΞ • Ϋϥελʔͷϋτ •
2021ͷඪʮ͓෩࿊ͷ࣌ܭͷి Λ͑Δʯ 2
ͱ͜ΖͰΈͳ͞ΜɺϏ σΦϛʔςΟϯά͠· ͢ΑͶ? 3
ฐࣾɺVRϛʔςΟϯά વΓ·͕͢ɺϏσΦϛʔ ςΟϯά͠·͢ VRHMDඃͬͯϛʔςΟϯάϊʔτऔΔͷେมͳͨΊɻ 4
͜Μͳײ͡ → 5
6
ҰൠతͳϏσΦϛʔςΟϯά ͷޫܠͰ͢Ͷ? (ͦ͏Ͱ͋ͬͯ΄͍͠) 7
͍ΘΏΔ VTuber ΞϓϦ ͍Ζ͍Ζ͋Δɻ 8
͍Ζ͍Ζࢼͨ݁͠ՌɺࣗͰ ࡞Γ࢝Ίͨ ͦ͏͍͏͜ͱ͋Δɻ 9
͢͜ͱ • ͍ΘΏΔ VTuber ΞϓϦͷߏྫ • ݟɺͩ͜ΘΓϙΠϯτͳͲ 10
ݟग़͠ • γεςϜ֓ཁ • إͱखͷτϥοΩϯά • ΞόλʔͷಡΈࠐΈɾදࣔɾΞχϝʔγϣϯ • ө૾ͷૹग़ 11
γεςϜ֓ཁ 12
13
إͱखͷτϥοΩϯά • ARKit • Leap Motion 14
ARKit Face Tracking • iOS σόΠεͷϑϩϯτΧϝϥͰϑΣΠετϥοΩϯά͢Δͭ • Unity Ͱ ARFoundation,
ARKit Face Tracking plugin Λ͏ 15
ARKit Face Tracking • Ϋϥογϡ͢Δ • σʔλڸ • දͷϚοϐϯά 16
Ϋϥογϡ͢Δ • ͠Β͘ಈ͔ͯ͠Δͱ EXC_BAD_ACCESS ͰམͪΔ • ARFoundation 3.x ͳΒͳ͍ •
͜Ε͕͋ͬͯ ARFoundation 4.x ʹ্͛ΒΕͣɺ Unity2020.2 ʹ ্͛ΒΕͳ͍ " • Issue ͰiOS14ͷͱݴΘΕ͍ͯΔ(मਖ਼Λ͔ͭ͠ͳͦ͞͏) > github.com/Unity-Technologies/arfoundation-samples/issues/ 707 17
Face Tracking ͷσʔλڸ • ARKit ͷ Face Tracking ͰऔಘͰ͖Δσʔλࠨӈసͯ͠ೖͬͯ ͘Δ
• ΧϝϥͷલͰӈΛ͘ͱࠨճసΛɺӈΛด͡ΔͱࠨΛด ͡ΔΑ͏ͳσʔλΛฦ͢ 18
إͷ pose Λڸࣸ͠ʹ͢Δ rotation.x *= -1; rotation.y *= -1; position.z
*= -1; 19
BlendShapeLocation ͷࠨӈΛೖΕସ ͑Δ 20
{ BlendShapePreset.Blink_L, new [] { ARKitBlendShapeLocation.EyeBlinkRight } }, { BlendShapePreset.Blink_R,
new [] { ARKitBlendShapeLocation.EyeBlinkLeft } } 21
ʮࠨΛݟ͍ͯΔʯɺʮ(͔ͬͯ)ࠨͷ͕ଆΛݟ͍ͯΔʯͱ ʮ(͔ͬͯ)ӈͷ͕֎ଆΛݟ͍ͯΔʯ { BlendShapePreset.LookLeft, new [] { ARKitBlendShapeLocation.EyeLookInLeft, ARKitBlendShapeLocation.EyeLookOutRight }
}, { BlendShapePreset.LookRight, new [] { ARKitBlendShapeLocation.EyeLookOutLeft, ARKitBlendShapeLocation.EyeLookInRight } }, 22
දͷϚοϐϯά • ARKitͰऔಘͰ͖Δ Blend Shape Ωʔ(ଟ͘ͷ)Ξόλʔͷ࣋ͭͦ ΕͱҟͳΔ • ಉ͡ Blend
Shape ΛΞόλʔʹ༻ҙͯ͠ɺͦͷ··ύϥϝʔ λʔΛөͤ͞ΔʮύʔϑΣΫτγϯΫʯͱݺΕΔ֓೦ ͋Δ 23
දͷϚοϐϯά • aiueo ͷαϯϓϧΛऔ͓͍ͬͯͯ࠷ۙࣅ͢ΔͷΛ࠾༻ɺͰͩ ͍͍͍͍ͨײ͡ʹͳͬͨ • ۃʹֻ͚ͨΓɺ2൪ʹࣅͯΔͷΛࠞͥͨΓɺͪΌΜͱޱ Λด͡ΔΑ͏ʹ͢Δͷʹຊͬͱ͍Ζ͍ΖͬͯΔ • ͜͜ͷڍಈਓʹΑͬͯΈ͔ΕΔͱࢥ͏ͷͰࣗ࡞ͯͯ͠
໘ന͍ͱ͜Ζ 24
Leap Motion • ϋϯυδΣενϟʔίϯτϩʔϥʔ • 2012ͷσόΠε • ڈ(20209݄) IR 170
ͱ͍͏݁ߏੑೳ্͕͕ͬͨධՁΩοτ͕ ൃച͞ΕͯΞπ͍ • Unity ϓϥάΠϯ͕ఏڙ͞Ε͍ͯΔ 25
Leap Motion • جຊతʹ LeapServiceProvider or LeapXRServiceProvider ίϯϙʔωϯτΛ࣮͑؆୯ • LeapServiceProvider
صஔ͖ • LeapXRServiceProvider टͱ͔಄ʹϚϯτ͢Δ༻ • ࣗͰ࣮͢Δ(pure class ʹ͍ͨ͠ͱ͔ɺصஔ͖ͱटϚϯτ ΛΓସ͑ΒΕΔΑ͏ʹ͍ͨ͠)߹ɺ PolicyFlag ΛΓସ͑ Δͷͱɺ࠷దͳ Interpolate ͷΦϓγϣϯ͕ҧ͏ͬΆ͍ʹҙ 26
ΞόλʔͷಡΈࠐΈɾදࣔɾ Ξχϝʔγϣϯ • VRM • Custom UTI • Animation Rigging
27
VRM • VRΞϓϦέʔγϣϯ͚ͷਓܕ3DΞόλʔσʔλΛѻ͏ͨΊͷ ϑΝΠϧϑΥʔϚοτ • UniVRM (C#ʹΑΔඪ४࣮)Λ͏ • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊࠐΜͰྑ͍͕ɺͲ͏ͤΞ όλʔΑ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹରԠ͓ͯ͘͠ͱ
ָ) 28
Custom UTI • iOS ͷڞ༗ϘλϯΛԡͨ͠ͱ͖ʹग़ͯ ͘ΔϦετʹ Unity ΞϓϦΛฒΔ • ͜ΕͰVRMϑΝΠϧΛ։͚Δͱຯʹ
ศར • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊ ࠐΜͰྑ͍͕ɺͲ͏ͤΞόλʔΑ ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹର Ԡ͓ͯ͘͠ͱָ) 29
Info.plist Λ͕Μͬͯॻ͘ [PostProcessBuild] public static void OnPostProcessBuild(BuildTarget target, string path)
{ if (target != BuildTarget.iOS) return; var plistPath = path + "/Info.plist"; var plist = new PlistDocument(); plist.ReadFromString(File.ReadAllText(plistPath)); var rootDict = plist.root; var exportedTypeDeclarations = rootDict.CreateArray("UTExportedTypeDeclarations"); var exportedTypeDeclarationsDict = exportedTypeDeclarations.AddDict(); var typeConformsTo = exportedTypeDeclarationsDict.CreateArray("UTTypeConformsTo"); typeConformsTo.AddString("public.data"); exportedTypeDeclarationsDict.SetString("UTTypeDescription", "VRM"); exportedTypeDeclarationsDict.CreateArray("UTTypeIconFiles"); var typeIdentifier = $"{PlayerSettings.applicationIdentifier}.custom"; exportedTypeDeclarationsDict.SetString("UTTypeIdentifier", typeIdentifier); var typeTagSpecification = exportedTypeDeclarationsDict.CreateDict("UTTypeTagSpecification"); var typeTagSpecificationArray = typeTagSpecification.CreateArray("public.filename-extension"); typeTagSpecificationArray.AddString("vrm"); var documentTypes = rootDict.CreateArray("CFBundleDocumentTypes"); var documentTypesDict = documentTypes.AddDict(); documentTypesDict.SetString("CFBundleTypeName", "VRM"); documentTypesDict.SetString("CFBundleTypeRole", "Viewer"); documentTypesDict.SetString("LSHandlerRank", "Alternate"); var itemContentTypes = documentTypesDict.CreateArray("LSItemContentTypes"); itemContentTypes.AddString(typeIdentifier); File.WriteAllText(plistPath, plist.WriteToString()); } https:/ /gist.github.com/noir-neo/ 91e7a09e87e15280c1b2c70648b833f1 30
͋ͱURLͷϋϯυϦϯάͱಉ͡ @interface UrlHandlerAppController : UnityAppController @end @implementation UrlHandlerAppController - (BOOL)application:(UIApplication
*)application openURL:(NSURL *)url options:(NSDictionary<UIApplicationOpenURLOptionsKey, id> *)options { // TODO: Handling return YES; } @end ͨͩ͜͜͠Ͱड͚औͬͨURI࣍ճىಈ࣌ʹ͑ͳ͍ͷͰΞό λʔΛΩϟογϡ͓ͯ͘͠ΈͳͲ͕ผ్ඞཁ 31
Animation Rigging • "ΞχϝʔγϣϯԽ͞Ε͍ͯΔεέϧτϯʹର࣮ͯ͠ߦ࣌ʹϓϩ γʔδϟϧͳϞʔγϣϯΛηοτΞοϓͰ͖ΔΑ͏ʹ͢Δ" • ཁΞχϝʔγϣϯΛϥϯλΠϜͰ͍͍ײ͡ʹ্ॻ͖Ͱ͖Δ • Unity2020.2 Ͱ
verified ʹͳΓ·ͨ͠Ͷ • σϑΥϧτͷΞχϝʔγϣϯʹରͯ͠(τϥοΩϯάͨ͠)֤Ϙʔ ϯͷճసΛ্ॻ͖͢Δͷʹศར 32
Animation Rigging • TwoBoneIK ͕ඪ४Ͱೖ͍ͬͯΔͷͰͷ IK ʹ͑Δ(݁ߏ͍͍ ײ͡) • (಄ͱख͔ΒମશମΛΑ͠ͳʹIK͢ΔɺΈ͍ͨͳͷ
FinalIK ͱ͔͏ํָ͕) • มͳճసΛೖΕͯ muscle ͷൣғͰಈ͔ͯ͘͠ΕΔͷͰ࠷ѱͷ ഁͷํ͠ͳ͍ 33
ө૾ͷૹग़ • ө૾ͷϛϥʔ • Spout / Syphon / NDI 34
ࣗͰݟΔө૾ڸʹ͢Δ • Unity Ͱ Camera ͷө૾ΛϑϦοϓ͢Δʹ projectionMatrix Λ͍ ͡ΔΒ͍͠ •
culling ͱ͔ΊΜͲͦ͘͞͏ • Render Texture ʹඳ͍ͨͷΛ Canvas ͷ UV Ͱసͤ͞Δͷ͕ ָ • ͍ͭͰʹɺૹग़͢Δө૾ʹ UI ΛೖΕͨ͘ͳ͍ͱ͔͋Δͷ Ͱແବͳ͍ 35
Spout / Syphon / NDI • ө૾(ͷϑϨʔϜ)Λڞ༗͢Δϓϩτίϧ • Spout(Windows) /
Syphon(mac) Ϛγϯ • NDI ϩʔΧϧωοτϫʔΫ • Unity Ͱ Klak{Spout|Syphon|NDI} Λ͏ • keijiroਆ ! 36
Spout / Syphon / NDI • ಁաө૾ѻ͑ͯ৴ιϑτʹೖΕΒΕΔͷͰάϦʔϯ όοΫͰൈ͍ͨΓ͠ͳͯ͘Α͍ͷͰΑ͍ 37
·ͱΊ • Unity ͰΩϛ͚ͩͷόʔνϟϧΧϝϥΛ࡞ͬͯɺϏσΦϛʔςΟ ϯάͰࠩΛ͚Α͏ʂ 38
39