Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
アバターミーティングを支える技術
Search
ねおりん
January 13, 2021
Technology
0
630
アバターミーティングを支える技術
Gotanda.unity #16
https://meetup.unity3d.jp/jp/events/1268
https://youtu.be/FTHfVpBfpo0
ねおりん
January 13, 2021
Tweet
Share
More Decks by ねおりん
See All by ねおりん
UIElements 最高! 一番好きなUIフレームワークです!
noir_neo
0
630
24時間くらいで作ったゲーム 2019年
noir_neo
0
1.3k
24時間でゲームをつくるための Unity
noir_neo
2
1.9k
Multiuser AR Experience を ARKit + Unity で実装する
noir_neo
0
2.7k
これで安心! hubot 沼の堕ち方〜娘がいると開発が楽しいね!
noir_neo
0
360
MDLT#004
noir_neo
0
1.5k
FU2 展示反省
noir_neo
0
130
FU2進捗141004
noir_neo
0
160
FU2進捗140920
noir_neo
0
140
Other Decks in Technology
See All in Technology
生成AI時代のデータ基盤
shibuiwilliam
0
260
[CV勉強会@関東 CVPR2025 読み会] MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos (Li+, CVPR2025)
abemii
0
200
イオン店舗一覧ページのパフォーマンスチューニング事例 / Performance tuning example for AEON store list page
aeonpeople
2
370
進捗
ydah
1
210
「守る」から「進化させる」セキュリティへ ~AWS re:Inforce 2025参加報告~ / AWS re:Inforce 2025 Participation Report
yuj1osm
1
170
生成AI時代に必要な価値ある意思決定を育てる「開発プロセス定義」を用いた中期戦略
kakehashi
PRO
1
190
人と組織に偏重したEMへのアンチテーゼ──なぜ、EMに設計力が必要なのか/An antithesis to the overemphasis of people and organizations in EM
dskst
7
730
絶対に失敗できないキャンペーンページの高速かつ安全な開発、WINTICKET × microCMS の開発事例
microcms
0
250
Goss: New Production-Ready Go Binding for Faiss #coefl_go_jp
bengo4com
1
1.1k
kintone開発チームの紹介
cybozuinsideout
PRO
0
73k
クラウドセキュリティを支える技術と運用の最前線 / Cutting-edge Technologies and Operations Supporting Cloud Security
yuj1osm
2
150
Postman MCP 関連機能アップデート / Postman MCP feature updates
yokawasa
1
220
Featured
See All Featured
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
110
20k
How to Ace a Technical Interview
jacobian
279
23k
Thoughts on Productivity
jonyablonski
69
4.8k
Faster Mobile Websites
deanohume
309
31k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
Visualization
eitanlees
147
16k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
185
54k
Optimizing for Happiness
mojombo
379
70k
A designer walks into a library…
pauljervisheath
207
24k
Side Projects
sachag
455
43k
Transcript
ΞόλʔϛʔςΟϯάΛࢧ͑ Δٕज़ 2021/1/13 Gotanda.unity #16 Ͷ͓ΓΜ 1
noir_neo • Ͷ͓ΓΜ • Ϋϥελʔגࣜձࣾ • ιϑτΣΞΤϯδχΞ • Ϋϥελʔͷϋτ •
2021ͷඪʮ͓෩࿊ͷ࣌ܭͷి Λ͑Δʯ 2
ͱ͜ΖͰΈͳ͞ΜɺϏ σΦϛʔςΟϯά͠· ͢ΑͶ? 3
ฐࣾɺVRϛʔςΟϯά વΓ·͕͢ɺϏσΦϛʔ ςΟϯά͠·͢ VRHMDඃͬͯϛʔςΟϯάϊʔτऔΔͷେมͳͨΊɻ 4
͜Μͳײ͡ → 5
6
ҰൠతͳϏσΦϛʔςΟϯά ͷޫܠͰ͢Ͷ? (ͦ͏Ͱ͋ͬͯ΄͍͠) 7
͍ΘΏΔ VTuber ΞϓϦ ͍Ζ͍Ζ͋Δɻ 8
͍Ζ͍Ζࢼͨ݁͠ՌɺࣗͰ ࡞Γ࢝Ίͨ ͦ͏͍͏͜ͱ͋Δɻ 9
͢͜ͱ • ͍ΘΏΔ VTuber ΞϓϦͷߏྫ • ݟɺͩ͜ΘΓϙΠϯτͳͲ 10
ݟग़͠ • γεςϜ֓ཁ • إͱखͷτϥοΩϯά • ΞόλʔͷಡΈࠐΈɾදࣔɾΞχϝʔγϣϯ • ө૾ͷૹग़ 11
γεςϜ֓ཁ 12
13
إͱखͷτϥοΩϯά • ARKit • Leap Motion 14
ARKit Face Tracking • iOS σόΠεͷϑϩϯτΧϝϥͰϑΣΠετϥοΩϯά͢Δͭ • Unity Ͱ ARFoundation,
ARKit Face Tracking plugin Λ͏ 15
ARKit Face Tracking • Ϋϥογϡ͢Δ • σʔλڸ • දͷϚοϐϯά 16
Ϋϥογϡ͢Δ • ͠Β͘ಈ͔ͯ͠Δͱ EXC_BAD_ACCESS ͰམͪΔ • ARFoundation 3.x ͳΒͳ͍ •
͜Ε͕͋ͬͯ ARFoundation 4.x ʹ্͛ΒΕͣɺ Unity2020.2 ʹ ্͛ΒΕͳ͍ " • Issue ͰiOS14ͷͱݴΘΕ͍ͯΔ(मਖ਼Λ͔ͭ͠ͳͦ͞͏) > github.com/Unity-Technologies/arfoundation-samples/issues/ 707 17
Face Tracking ͷσʔλڸ • ARKit ͷ Face Tracking ͰऔಘͰ͖Δσʔλࠨӈసͯ͠ೖͬͯ ͘Δ
• ΧϝϥͷલͰӈΛ͘ͱࠨճసΛɺӈΛด͡ΔͱࠨΛด ͡ΔΑ͏ͳσʔλΛฦ͢ 18
إͷ pose Λڸࣸ͠ʹ͢Δ rotation.x *= -1; rotation.y *= -1; position.z
*= -1; 19
BlendShapeLocation ͷࠨӈΛೖΕସ ͑Δ 20
{ BlendShapePreset.Blink_L, new [] { ARKitBlendShapeLocation.EyeBlinkRight } }, { BlendShapePreset.Blink_R,
new [] { ARKitBlendShapeLocation.EyeBlinkLeft } } 21
ʮࠨΛݟ͍ͯΔʯɺʮ(͔ͬͯ)ࠨͷ͕ଆΛݟ͍ͯΔʯͱ ʮ(͔ͬͯ)ӈͷ͕֎ଆΛݟ͍ͯΔʯ { BlendShapePreset.LookLeft, new [] { ARKitBlendShapeLocation.EyeLookInLeft, ARKitBlendShapeLocation.EyeLookOutRight }
}, { BlendShapePreset.LookRight, new [] { ARKitBlendShapeLocation.EyeLookOutLeft, ARKitBlendShapeLocation.EyeLookInRight } }, 22
දͷϚοϐϯά • ARKitͰऔಘͰ͖Δ Blend Shape Ωʔ(ଟ͘ͷ)Ξόλʔͷ࣋ͭͦ ΕͱҟͳΔ • ಉ͡ Blend
Shape ΛΞόλʔʹ༻ҙͯ͠ɺͦͷ··ύϥϝʔ λʔΛөͤ͞ΔʮύʔϑΣΫτγϯΫʯͱݺΕΔ֓೦ ͋Δ 23
දͷϚοϐϯά • aiueo ͷαϯϓϧΛऔ͓͍ͬͯͯ࠷ۙࣅ͢ΔͷΛ࠾༻ɺͰͩ ͍͍͍͍ͨײ͡ʹͳͬͨ • ۃʹֻ͚ͨΓɺ2൪ʹࣅͯΔͷΛࠞͥͨΓɺͪΌΜͱޱ Λด͡ΔΑ͏ʹ͢Δͷʹຊͬͱ͍Ζ͍ΖͬͯΔ • ͜͜ͷڍಈਓʹΑͬͯΈ͔ΕΔͱࢥ͏ͷͰࣗ࡞ͯͯ͠
໘ന͍ͱ͜Ζ 24
Leap Motion • ϋϯυδΣενϟʔίϯτϩʔϥʔ • 2012ͷσόΠε • ڈ(20209݄) IR 170
ͱ͍͏݁ߏੑೳ্͕͕ͬͨධՁΩοτ͕ ൃച͞ΕͯΞπ͍ • Unity ϓϥάΠϯ͕ఏڙ͞Ε͍ͯΔ 25
Leap Motion • جຊతʹ LeapServiceProvider or LeapXRServiceProvider ίϯϙʔωϯτΛ࣮͑؆୯ • LeapServiceProvider
صஔ͖ • LeapXRServiceProvider टͱ͔಄ʹϚϯτ͢Δ༻ • ࣗͰ࣮͢Δ(pure class ʹ͍ͨ͠ͱ͔ɺصஔ͖ͱटϚϯτ ΛΓସ͑ΒΕΔΑ͏ʹ͍ͨ͠)߹ɺ PolicyFlag ΛΓସ͑ Δͷͱɺ࠷దͳ Interpolate ͷΦϓγϣϯ͕ҧ͏ͬΆ͍ʹҙ 26
ΞόλʔͷಡΈࠐΈɾදࣔɾ Ξχϝʔγϣϯ • VRM • Custom UTI • Animation Rigging
27
VRM • VRΞϓϦέʔγϣϯ͚ͷਓܕ3DΞόλʔσʔλΛѻ͏ͨΊͷ ϑΝΠϧϑΥʔϚοτ • UniVRM (C#ʹΑΔඪ४࣮)Λ͏ • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊࠐΜͰྑ͍͕ɺͲ͏ͤΞ όλʔΑ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹରԠ͓ͯ͘͠ͱ
ָ) 28
Custom UTI • iOS ͷڞ༗ϘλϯΛԡͨ͠ͱ͖ʹग़ͯ ͘ΔϦετʹ Unity ΞϓϦΛฒΔ • ͜ΕͰVRMϑΝΠϧΛ։͚Δͱຯʹ
ศར • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊ ࠐΜͰྑ͍͕ɺͲ͏ͤΞόλʔΑ ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹର Ԡ͓ͯ͘͠ͱָ) 29
Info.plist Λ͕Μͬͯॻ͘ [PostProcessBuild] public static void OnPostProcessBuild(BuildTarget target, string path)
{ if (target != BuildTarget.iOS) return; var plistPath = path + "/Info.plist"; var plist = new PlistDocument(); plist.ReadFromString(File.ReadAllText(plistPath)); var rootDict = plist.root; var exportedTypeDeclarations = rootDict.CreateArray("UTExportedTypeDeclarations"); var exportedTypeDeclarationsDict = exportedTypeDeclarations.AddDict(); var typeConformsTo = exportedTypeDeclarationsDict.CreateArray("UTTypeConformsTo"); typeConformsTo.AddString("public.data"); exportedTypeDeclarationsDict.SetString("UTTypeDescription", "VRM"); exportedTypeDeclarationsDict.CreateArray("UTTypeIconFiles"); var typeIdentifier = $"{PlayerSettings.applicationIdentifier}.custom"; exportedTypeDeclarationsDict.SetString("UTTypeIdentifier", typeIdentifier); var typeTagSpecification = exportedTypeDeclarationsDict.CreateDict("UTTypeTagSpecification"); var typeTagSpecificationArray = typeTagSpecification.CreateArray("public.filename-extension"); typeTagSpecificationArray.AddString("vrm"); var documentTypes = rootDict.CreateArray("CFBundleDocumentTypes"); var documentTypesDict = documentTypes.AddDict(); documentTypesDict.SetString("CFBundleTypeName", "VRM"); documentTypesDict.SetString("CFBundleTypeRole", "Viewer"); documentTypesDict.SetString("LSHandlerRank", "Alternate"); var itemContentTypes = documentTypesDict.CreateArray("LSItemContentTypes"); itemContentTypes.AddString(typeIdentifier); File.WriteAllText(plistPath, plist.WriteToString()); } https:/ /gist.github.com/noir-neo/ 91e7a09e87e15280c1b2c70648b833f1 30
͋ͱURLͷϋϯυϦϯάͱಉ͡ @interface UrlHandlerAppController : UnityAppController @end @implementation UrlHandlerAppController - (BOOL)application:(UIApplication
*)application openURL:(NSURL *)url options:(NSDictionary<UIApplicationOpenURLOptionsKey, id> *)options { // TODO: Handling return YES; } @end ͨͩ͜͜͠Ͱड͚औͬͨURI࣍ճىಈ࣌ʹ͑ͳ͍ͷͰΞό λʔΛΩϟογϡ͓ͯ͘͠ΈͳͲ͕ผ్ඞཁ 31
Animation Rigging • "ΞχϝʔγϣϯԽ͞Ε͍ͯΔεέϧτϯʹର࣮ͯ͠ߦ࣌ʹϓϩ γʔδϟϧͳϞʔγϣϯΛηοτΞοϓͰ͖ΔΑ͏ʹ͢Δ" • ཁΞχϝʔγϣϯΛϥϯλΠϜͰ͍͍ײ͡ʹ্ॻ͖Ͱ͖Δ • Unity2020.2 Ͱ
verified ʹͳΓ·ͨ͠Ͷ • σϑΥϧτͷΞχϝʔγϣϯʹରͯ͠(τϥοΩϯάͨ͠)֤Ϙʔ ϯͷճసΛ্ॻ͖͢Δͷʹศར 32
Animation Rigging • TwoBoneIK ͕ඪ४Ͱೖ͍ͬͯΔͷͰͷ IK ʹ͑Δ(݁ߏ͍͍ ײ͡) • (಄ͱख͔ΒମશମΛΑ͠ͳʹIK͢ΔɺΈ͍ͨͳͷ
FinalIK ͱ͔͏ํָ͕) • มͳճసΛೖΕͯ muscle ͷൣғͰಈ͔ͯ͘͠ΕΔͷͰ࠷ѱͷ ഁͷํ͠ͳ͍ 33
ө૾ͷૹग़ • ө૾ͷϛϥʔ • Spout / Syphon / NDI 34
ࣗͰݟΔө૾ڸʹ͢Δ • Unity Ͱ Camera ͷө૾ΛϑϦοϓ͢Δʹ projectionMatrix Λ͍ ͡ΔΒ͍͠ •
culling ͱ͔ΊΜͲͦ͘͞͏ • Render Texture ʹඳ͍ͨͷΛ Canvas ͷ UV Ͱసͤ͞Δͷ͕ ָ • ͍ͭͰʹɺૹग़͢Δө૾ʹ UI ΛೖΕͨ͘ͳ͍ͱ͔͋Δͷ Ͱແବͳ͍ 35
Spout / Syphon / NDI • ө૾(ͷϑϨʔϜ)Λڞ༗͢Δϓϩτίϧ • Spout(Windows) /
Syphon(mac) Ϛγϯ • NDI ϩʔΧϧωοτϫʔΫ • Unity Ͱ Klak{Spout|Syphon|NDI} Λ͏ • keijiroਆ ! 36
Spout / Syphon / NDI • ಁաө૾ѻ͑ͯ৴ιϑτʹೖΕΒΕΔͷͰάϦʔϯ όοΫͰൈ͍ͨΓ͠ͳͯ͘Α͍ͷͰΑ͍ 37
·ͱΊ • Unity ͰΩϛ͚ͩͷόʔνϟϧΧϝϥΛ࡞ͬͯɺϏσΦϛʔςΟ ϯάͰࠩΛ͚Α͏ʂ 38
39