Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
アバターミーティングを支える技術
Search
ねおりん
January 13, 2021
Technology
0
480
アバターミーティングを支える技術
Gotanda.unity #16
https://meetup.unity3d.jp/jp/events/1268
https://youtu.be/FTHfVpBfpo0
ねおりん
January 13, 2021
Tweet
Share
More Decks by ねおりん
See All by ねおりん
UIElements 最高! 一番好きなUIフレームワークです!
noir_neo
0
540
24時間くらいで作ったゲーム 2019年
noir_neo
0
1.2k
24時間でゲームをつくるための Unity
noir_neo
2
1.7k
Multiuser AR Experience を ARKit + Unity で実装する
noir_neo
0
2.5k
これで安心! hubot 沼の堕ち方〜娘がいると開発が楽しいね!
noir_neo
0
330
MDLT#004
noir_neo
0
1.4k
FU2 展示反省
noir_neo
0
100
FU2進捗141004
noir_neo
0
130
FU2進捗140920
noir_neo
0
120
Other Decks in Technology
See All in Technology
SwiftUIのpropertyWrapperをふんわり理解する
jambo_develop_team
0
110
滑空スポーツ講習会2023 航空安全講習会 第4回 日常整備に役立ちそうな雑情報 / JSA Safety Seminar 2023 glider maintenance
jsaseminar
0
110
SecurityCamp2023基板作るコース講義資料/Security Camp 2023 Lecture Materials
tnishinaga
5
1.3k
『QAという人』が必要ではなく、『QAという技術』が必要
sadonosake
2
270
進化する事業とデータ構造 ~Cloudbaseの場合~
tockn
2
230
Beginner's Guide to Partitioning vs. Sharding in Postgres | Claire Giordano | Nordic PGDay 2024
clairegiordano
0
210
AWSアカウント74件をバックアップした方法と課題
chiharutsuboi
0
100
スクラムガイドに載っていないスクラムのはじめかた - チームでスクラムをはじめるときに知っておきたい5個のコツ - / How to start Scrum that is not written in the Scrum Guide
takaking22
14
5.3k
AWS IAM の結果整合性を避けるためセッションポリシーを用いてポリシーの動作確認を行う、を解説する
yukihirochiba
0
380
技育祭2024春 LT Finatextホールディングス
kevinrobot34
1
120
DevOps Topologies 10 years on: what have we learned about silos, collaboration, and flow? - Matthew Skelton, Conflux
matthewskelton
PRO
2
420
【Cyber-sec+】ログの森で出会ったCloudTrail との奇妙な旅
hssh2_bin
1
230
Featured
See All Featured
Typedesign – Prime Four
hannesfritz
36
2k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Designing for Performance
lara
601
67k
Infographics Made Easy
chrislema
237
17k
Keith and Marios Guide to Fast Websites
keithpitt
407
22k
Build your cross-platform service in a week with App Engine
jlugia
223
17k
Rebuilding a faster, lazier Slack
samanthasiow
72
8.1k
Building a Modern Day E-commerce SEO Strategy
aleyda
15
6.3k
Building an army of robots
kneath
300
41k
For a Future-Friendly Web
brad_frost
170
8.8k
Designing on Purpose - Digital PM Summit 2013
jponch
109
6.4k
Robots, Beer and Maslow
schacon
PRO
154
7.9k
Transcript
ΞόλʔϛʔςΟϯάΛࢧ͑ Δٕज़ 2021/1/13 Gotanda.unity #16 Ͷ͓ΓΜ 1
noir_neo • Ͷ͓ΓΜ • Ϋϥελʔגࣜձࣾ • ιϑτΣΞΤϯδχΞ • Ϋϥελʔͷϋτ •
2021ͷඪʮ͓෩࿊ͷ࣌ܭͷి Λ͑Δʯ 2
ͱ͜ΖͰΈͳ͞ΜɺϏ σΦϛʔςΟϯά͠· ͢ΑͶ? 3
ฐࣾɺVRϛʔςΟϯά વΓ·͕͢ɺϏσΦϛʔ ςΟϯά͠·͢ VRHMDඃͬͯϛʔςΟϯάϊʔτऔΔͷେมͳͨΊɻ 4
͜Μͳײ͡ → 5
6
ҰൠతͳϏσΦϛʔςΟϯά ͷޫܠͰ͢Ͷ? (ͦ͏Ͱ͋ͬͯ΄͍͠) 7
͍ΘΏΔ VTuber ΞϓϦ ͍Ζ͍Ζ͋Δɻ 8
͍Ζ͍Ζࢼͨ݁͠ՌɺࣗͰ ࡞Γ࢝Ίͨ ͦ͏͍͏͜ͱ͋Δɻ 9
͢͜ͱ • ͍ΘΏΔ VTuber ΞϓϦͷߏྫ • ݟɺͩ͜ΘΓϙΠϯτͳͲ 10
ݟग़͠ • γεςϜ֓ཁ • إͱखͷτϥοΩϯά • ΞόλʔͷಡΈࠐΈɾදࣔɾΞχϝʔγϣϯ • ө૾ͷૹग़ 11
γεςϜ֓ཁ 12
13
إͱखͷτϥοΩϯά • ARKit • Leap Motion 14
ARKit Face Tracking • iOS σόΠεͷϑϩϯτΧϝϥͰϑΣΠετϥοΩϯά͢Δͭ • Unity Ͱ ARFoundation,
ARKit Face Tracking plugin Λ͏ 15
ARKit Face Tracking • Ϋϥογϡ͢Δ • σʔλڸ • දͷϚοϐϯά 16
Ϋϥογϡ͢Δ • ͠Β͘ಈ͔ͯ͠Δͱ EXC_BAD_ACCESS ͰམͪΔ • ARFoundation 3.x ͳΒͳ͍ •
͜Ε͕͋ͬͯ ARFoundation 4.x ʹ্͛ΒΕͣɺ Unity2020.2 ʹ ্͛ΒΕͳ͍ " • Issue ͰiOS14ͷͱݴΘΕ͍ͯΔ(मਖ਼Λ͔ͭ͠ͳͦ͞͏) > github.com/Unity-Technologies/arfoundation-samples/issues/ 707 17
Face Tracking ͷσʔλڸ • ARKit ͷ Face Tracking ͰऔಘͰ͖Δσʔλࠨӈసͯ͠ೖͬͯ ͘Δ
• ΧϝϥͷલͰӈΛ͘ͱࠨճసΛɺӈΛด͡ΔͱࠨΛด ͡ΔΑ͏ͳσʔλΛฦ͢ 18
إͷ pose Λڸࣸ͠ʹ͢Δ rotation.x *= -1; rotation.y *= -1; position.z
*= -1; 19
BlendShapeLocation ͷࠨӈΛೖΕସ ͑Δ 20
{ BlendShapePreset.Blink_L, new [] { ARKitBlendShapeLocation.EyeBlinkRight } }, { BlendShapePreset.Blink_R,
new [] { ARKitBlendShapeLocation.EyeBlinkLeft } } 21
ʮࠨΛݟ͍ͯΔʯɺʮ(͔ͬͯ)ࠨͷ͕ଆΛݟ͍ͯΔʯͱ ʮ(͔ͬͯ)ӈͷ͕֎ଆΛݟ͍ͯΔʯ { BlendShapePreset.LookLeft, new [] { ARKitBlendShapeLocation.EyeLookInLeft, ARKitBlendShapeLocation.EyeLookOutRight }
}, { BlendShapePreset.LookRight, new [] { ARKitBlendShapeLocation.EyeLookOutLeft, ARKitBlendShapeLocation.EyeLookInRight } }, 22
දͷϚοϐϯά • ARKitͰऔಘͰ͖Δ Blend Shape Ωʔ(ଟ͘ͷ)Ξόλʔͷ࣋ͭͦ ΕͱҟͳΔ • ಉ͡ Blend
Shape ΛΞόλʔʹ༻ҙͯ͠ɺͦͷ··ύϥϝʔ λʔΛөͤ͞ΔʮύʔϑΣΫτγϯΫʯͱݺΕΔ֓೦ ͋Δ 23
දͷϚοϐϯά • aiueo ͷαϯϓϧΛऔ͓͍ͬͯͯ࠷ۙࣅ͢ΔͷΛ࠾༻ɺͰͩ ͍͍͍͍ͨײ͡ʹͳͬͨ • ۃʹֻ͚ͨΓɺ2൪ʹࣅͯΔͷΛࠞͥͨΓɺͪΌΜͱޱ Λด͡ΔΑ͏ʹ͢Δͷʹຊͬͱ͍Ζ͍ΖͬͯΔ • ͜͜ͷڍಈਓʹΑͬͯΈ͔ΕΔͱࢥ͏ͷͰࣗ࡞ͯͯ͠
໘ന͍ͱ͜Ζ 24
Leap Motion • ϋϯυδΣενϟʔίϯτϩʔϥʔ • 2012ͷσόΠε • ڈ(20209݄) IR 170
ͱ͍͏݁ߏੑೳ্͕͕ͬͨධՁΩοτ͕ ൃച͞ΕͯΞπ͍ • Unity ϓϥάΠϯ͕ఏڙ͞Ε͍ͯΔ 25
Leap Motion • جຊతʹ LeapServiceProvider or LeapXRServiceProvider ίϯϙʔωϯτΛ࣮͑؆୯ • LeapServiceProvider
صஔ͖ • LeapXRServiceProvider टͱ͔಄ʹϚϯτ͢Δ༻ • ࣗͰ࣮͢Δ(pure class ʹ͍ͨ͠ͱ͔ɺصஔ͖ͱटϚϯτ ΛΓସ͑ΒΕΔΑ͏ʹ͍ͨ͠)߹ɺ PolicyFlag ΛΓସ͑ Δͷͱɺ࠷దͳ Interpolate ͷΦϓγϣϯ͕ҧ͏ͬΆ͍ʹҙ 26
ΞόλʔͷಡΈࠐΈɾදࣔɾ Ξχϝʔγϣϯ • VRM • Custom UTI • Animation Rigging
27
VRM • VRΞϓϦέʔγϣϯ͚ͷਓܕ3DΞόλʔσʔλΛѻ͏ͨΊͷ ϑΝΠϧϑΥʔϚοτ • UniVRM (C#ʹΑΔඪ४࣮)Λ͏ • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊࠐΜͰྑ͍͕ɺͲ͏ͤΞ όλʔΑ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹରԠ͓ͯ͘͠ͱ
ָ) 28
Custom UTI • iOS ͷڞ༗ϘλϯΛԡͨ͠ͱ͖ʹग़ͯ ͘ΔϦετʹ Unity ΞϓϦΛฒΔ • ͜ΕͰVRMϑΝΠϧΛ։͚Δͱຯʹ
ศར • (શʹࣗ༻ͩͬͨΒΞϓϦʹຒΊ ࠐΜͰྑ͍͕ɺͲ͏ͤΞόλʔΑ ͘मਖ਼͢ΔͷͰϥϯλΠϜϩʔυʹର Ԡ͓ͯ͘͠ͱָ) 29
Info.plist Λ͕Μͬͯॻ͘ [PostProcessBuild] public static void OnPostProcessBuild(BuildTarget target, string path)
{ if (target != BuildTarget.iOS) return; var plistPath = path + "/Info.plist"; var plist = new PlistDocument(); plist.ReadFromString(File.ReadAllText(plistPath)); var rootDict = plist.root; var exportedTypeDeclarations = rootDict.CreateArray("UTExportedTypeDeclarations"); var exportedTypeDeclarationsDict = exportedTypeDeclarations.AddDict(); var typeConformsTo = exportedTypeDeclarationsDict.CreateArray("UTTypeConformsTo"); typeConformsTo.AddString("public.data"); exportedTypeDeclarationsDict.SetString("UTTypeDescription", "VRM"); exportedTypeDeclarationsDict.CreateArray("UTTypeIconFiles"); var typeIdentifier = $"{PlayerSettings.applicationIdentifier}.custom"; exportedTypeDeclarationsDict.SetString("UTTypeIdentifier", typeIdentifier); var typeTagSpecification = exportedTypeDeclarationsDict.CreateDict("UTTypeTagSpecification"); var typeTagSpecificationArray = typeTagSpecification.CreateArray("public.filename-extension"); typeTagSpecificationArray.AddString("vrm"); var documentTypes = rootDict.CreateArray("CFBundleDocumentTypes"); var documentTypesDict = documentTypes.AddDict(); documentTypesDict.SetString("CFBundleTypeName", "VRM"); documentTypesDict.SetString("CFBundleTypeRole", "Viewer"); documentTypesDict.SetString("LSHandlerRank", "Alternate"); var itemContentTypes = documentTypesDict.CreateArray("LSItemContentTypes"); itemContentTypes.AddString(typeIdentifier); File.WriteAllText(plistPath, plist.WriteToString()); } https:/ /gist.github.com/noir-neo/ 91e7a09e87e15280c1b2c70648b833f1 30
͋ͱURLͷϋϯυϦϯάͱಉ͡ @interface UrlHandlerAppController : UnityAppController @end @implementation UrlHandlerAppController - (BOOL)application:(UIApplication
*)application openURL:(NSURL *)url options:(NSDictionary<UIApplicationOpenURLOptionsKey, id> *)options { // TODO: Handling return YES; } @end ͨͩ͜͜͠Ͱड͚औͬͨURI࣍ճىಈ࣌ʹ͑ͳ͍ͷͰΞό λʔΛΩϟογϡ͓ͯ͘͠ΈͳͲ͕ผ్ඞཁ 31
Animation Rigging • "ΞχϝʔγϣϯԽ͞Ε͍ͯΔεέϧτϯʹର࣮ͯ͠ߦ࣌ʹϓϩ γʔδϟϧͳϞʔγϣϯΛηοτΞοϓͰ͖ΔΑ͏ʹ͢Δ" • ཁΞχϝʔγϣϯΛϥϯλΠϜͰ͍͍ײ͡ʹ্ॻ͖Ͱ͖Δ • Unity2020.2 Ͱ
verified ʹͳΓ·ͨ͠Ͷ • σϑΥϧτͷΞχϝʔγϣϯʹରͯ͠(τϥοΩϯάͨ͠)֤Ϙʔ ϯͷճసΛ্ॻ͖͢Δͷʹศར 32
Animation Rigging • TwoBoneIK ͕ඪ४Ͱೖ͍ͬͯΔͷͰͷ IK ʹ͑Δ(݁ߏ͍͍ ײ͡) • (಄ͱख͔ΒମશମΛΑ͠ͳʹIK͢ΔɺΈ͍ͨͳͷ
FinalIK ͱ͔͏ํָ͕) • มͳճసΛೖΕͯ muscle ͷൣғͰಈ͔ͯ͘͠ΕΔͷͰ࠷ѱͷ ഁͷํ͠ͳ͍ 33
ө૾ͷૹग़ • ө૾ͷϛϥʔ • Spout / Syphon / NDI 34
ࣗͰݟΔө૾ڸʹ͢Δ • Unity Ͱ Camera ͷө૾ΛϑϦοϓ͢Δʹ projectionMatrix Λ͍ ͡ΔΒ͍͠ •
culling ͱ͔ΊΜͲͦ͘͞͏ • Render Texture ʹඳ͍ͨͷΛ Canvas ͷ UV Ͱసͤ͞Δͷ͕ ָ • ͍ͭͰʹɺૹग़͢Δө૾ʹ UI ΛೖΕͨ͘ͳ͍ͱ͔͋Δͷ Ͱແବͳ͍ 35
Spout / Syphon / NDI • ө૾(ͷϑϨʔϜ)Λڞ༗͢Δϓϩτίϧ • Spout(Windows) /
Syphon(mac) Ϛγϯ • NDI ϩʔΧϧωοτϫʔΫ • Unity Ͱ Klak{Spout|Syphon|NDI} Λ͏ • keijiroਆ ! 36
Spout / Syphon / NDI • ಁաө૾ѻ͑ͯ৴ιϑτʹೖΕΒΕΔͷͰάϦʔϯ όοΫͰൈ͍ͨΓ͠ͳͯ͘Α͍ͷͰΑ͍ 37
·ͱΊ • Unity ͰΩϛ͚ͩͷόʔνϟϧΧϝϥΛ࡞ͬͯɺϏσΦϛʔςΟ ϯάͰࠩΛ͚Α͏ʂ 38
39