remote controlling Oculus Quest even with macOS

It's hard to develop Oculus Quest app with non-Windows pc. but the A-npanRemote enables it.

Transcript

1. macOSͱ͔Ͱ΋ QuestΛϦϞʔτ͍ͨ͠ΜͩΑ ΞϚνϡΞϦϯάϑΟολʔ @toru_inoue (ג)KISSAKI
2. macOSͱ͔Ͱ΋ QuestΛϦϞʔτ͍ͨ͠ΜͩΑ [J[BJ͞Μͷ͓ख఻͍ @toru_inoue (ג)KISSAKI
3. ͕Μ͹Γͨ͘ͳ͍ ֓ཁ 4NBMM
   ͍ͬͯ͏2VFTUΞϓϦΛ࡞͍ͬͯͯɻ
   0DVMVT
   2VFTUͰಈ͘%ϞσϦϯάπʔϧɻ
   ࠓ·ͩQSJWBUFͳΜ͚ͩͲɻ
   ͰɺNBDΛ࢖ͬͯ2VFTUΞϓϦΛ࡞͍ͬͯΔͱɺ·͋πʔϧ͕
   ଍Γͳ͍ɻओʹ։ൃɺσόοάπʔϧɻ
4. ͕Μ͹Γͨ͘ͳ͍ ֓ཁ ਐḿ͢ΔͨΊʹຖճ࣮ػϏϧυ͢Δͷ͸ਓੜͷແବ͗͢Δɻ
   ϦϞʔτͰૹΓࠐΊΑ͏͕Ϗϧυ͕ΊͲ͍ɻ୤ࣈͰ͸ͳ͍ɻ
   ͦΜͳΘ͚Ͱ։ൃͨ͠
   "OQBO3FNPUF
   ͱ͍͏ϦϞʔτΠϯϓοτπʔϧͷ࿩Λ͢Δɻ
   IUUQTHJUIVCDPNTBTTFNCMB"OQBO3FNPUF
5. A-npanRemoteͱ͸ Unity Editor΁ͱ͍Ζ͍Ζͳ֎෦πʔϧͷΠϯϓοτΛೖྗ͢Δ΍ͭ ͜ΕΒ
   UP
   6OJUZ
   &EJUPS͕Մೳɻ
   ‣ "3,JU
   J04࣮ػ
   ‣ 0DVMVT
   2VFTU
   ‣ 0DVMVT
   3JGU
   ‣ )PMPMFOTͱ͔ ΍ͬͨ͜ͱͳ͍ ͨͿΜͰ͖Δ
   ‣ "3ܥͱ͔ଞ ·ͩݟ͵Πϯϓοτ΋ରԠՄೳ
6. ͜ΕΒ
   UP
   6OJUZ
   &EJUPS͕Մೳɻ
   ‣ "3,JU
   ‣ 0DVMVT
   2VFTU
   ‣ 0DVMVT
   3JGU
   ‣ )PMPMFOTͱ͔ ΍ͬͨ͜ͱͳ͍ ͨͿΜͰ͖Δ
   ‣ "3ܥͱ͔ଞ ·ͩݟ͵Πϯϓοτ΋ରԠՄೳ A-npanRemoteͱ͸ Unity Editor΁ͱ͍Ζ͍Ζͳ֎෦πʔϧͷΠϯϓοτΛೖྗ͢Δ΍ͭ ͜Ε͕͋Δͱͳʹ͕͍͍ͬͯ
   ࣮ػϏϧυͳ͠Ͱ
   ΤσΟλ͚ͩͰΞϓϦ։ൃ͕Ͱ͖ΔΜͩΑͳʂ
   ΠςϨʔγϣϯͩΜͪʂʂ
7. A-npanRemote ߏ੒ ▸ ࣮ػ → Unity Editor͸WebSocketͰ઀ଓ ▸ ͋ΒΏΔೖྗσʔλ͕EditorʹདྷΔͷͰɺ
 EditorͰੜͷΠϯϓοτΛड͚ͳ͕ΒΞϓϦΛ࡞ ੒Ͱ͖Δ ▸ ࣮ػϏϧυجຊతʹෆཁ Unity Editor Quest ӈखɺࠨखɺ಄ͷ܏͖ etc WS Client → WS Server
8. ͜Μͳײ͡ʹϚωΩϯϔουʹஔ͘ͱ ಄ͷ܏͖ͱ͔Ґஔͱ͔ͷτϥοΩϯά৘ใ͕ௐ੔Ͱָ͖ͯɻ ίϯτϩʔϥ΋௻Δͨ͠Γͯ͠ྑ͍ײ͡ͷҐஔʹݻఆָ͕ 2VFTUͰͷ։ൃͷ
   Α͏͢
9. A-npanRemote ૊ΈࠐΈ طଘͷίʔυΛ΄ͱΜͲ͍͡Βͣʹߦ͚Δɻָ͍͠ͷͰ঺հ͢Δɻ
    ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
    సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ
 ͍͕ͭ͜σʔλͷѹॖͱల։Λࣗಈతʹ΍ͬͯ͘ΕΔ
    "@OQBO3FNPUF4FUVQؔ਺Λ࣮ߦ
    3&.05&ϑϥάΛ͚࣮ͭͯػϏϧυ
   ͜ͷͭΛຬͨͤ͹0,ɻ
10. A-npanRemote iOSͷFaceTrackingΛA-npanRemoteରԠͯ͠ΈΑ͏ʂ ૊ΈࠐΈBefore // ී௨ʹإೝࣝ(FaceTracking)Λ։࢝ͤ͞Δ fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot) => { // إೝࣝͷUpdate } );
11. // ී௨ʹإೝࣝ(FaceTracking)Λ։࢝ͤ͞Δ fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot) => { // إೝࣝͷUpdate } ); A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote ૊ΈࠐΈAfter
12. // ී௨ʹإೝࣝ(FaceTracking)Λ։࢝ͤ͞Δ fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot) => { // إೝࣝͷUpdate } ); A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote ૊ΈࠐΈAfter ͜Ε͚ͩͰ
    ࣮ػ
    ˠ
    &EJUPSసૹʂ
13. // ී௨ʹإೝࣝ(FaceTracking)Λ։࢝ͤ͞Δ fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot) => { // إೝࣝͷUpdate } ); A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote ૊ΈࠐΈAfter ͜Ε͚ͩͰ
    ࣮ػ
    ˠ
    &EJUPSసૹʂ ࣮࣭ 1 ߦॻ͍ͯऴΘΓɻͶɺ؆୯Ͱ͠ΐ͏ʁ
14. ARKitͰA_npanRemote͸͜Μͳײ͡ ͱ͍͏ ࣮ԋ(਺ඵ)
15. fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote࣮ߦ࣌ͷग़དྷࣄ Unity Editor peerͷน ͜͏͍͏؀ڥΛ࡞ͬͨͱͯ͠ɺ fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); ࣮ػ
16. fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote࣮ߦ࣌ͷग़དྷࣄ Unity Editor peerͷน fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); ಧ͘ʂ ࣮ػ ಉ͡ίʔυͷಉ͡ϝιου΁ͱσʔλ͕ಧ͘ɻૉఢɻ
17. A-npanRemote ৄࡉ:૊ΈࠐΈ  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
 
 'BDF5SBDLJOH಺ʹ<إͷVQEBUF͕ݕग़Ͱ͖ͨ࣌ʹ࣮ߦ͞ΕΔ>
 0O6QEBUFͱ͍͏"DUJPO͕͋ͬͯɻ
 
 ͦΕΛQVCMJDʹͯ͠ɺ֎෦͔Βॻ͖׵͑Մೳʹ͢Δɻ
18. A-npanRemote ৄࡉ:૊ΈࠐΈ  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ ✔ Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot public class FTPayload : IRemotePayload3 { [SerializeField] public Matrix4x4 facePosAndRot; [SerializeField] public string[] keys; [SerializeField] public float[] values; [SerializeField] public Quaternion cameraRot; …
19. Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot A-npanRemote ৄࡉ:૊ΈࠐΈ  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ ✔ public class FTPayload : IRemotePayload3 { [SerializeField] public Matrix4x4 facePosAndRot; [SerializeField] public string[] keys; [SerializeField] public float[] values; [SerializeField] public Quaternion cameraRot; … ࡾͭͷύϥϝʔλΛ ಛఆܕʹม׵
20. Matrix4x4 facePosAndRot, Dictionary<string, float> faceBlendShapes, Quaternion cameraRot A-npanRemote ৄࡉ:૊ΈࠐΈ  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ ✔ public class FTPayload : IRemotePayload3 { [SerializeField] public Matrix4x4 facePosAndRot; [SerializeField] public string[] keys; [SerializeField] public float[] values; [SerializeField] public Quaternion cameraRot; … ͜ͷϕʔεܕ͕ࣗಈతʹ৭ʑ΍ͬͯ͘ΕΔɻ ఆٛ͑͢͞Ε͹༏উɻ
21.  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ
     "@OQBO3FNPUF4FUVQΛߦ͏ A-npanRemote ৄࡉ:૊ΈࠐΈ ✔ ✔ ipΛೖΕͯSetupɻ 1Ͱpublicʹͨؔ͠਺͕ड͚औΔܕ + 2Ͱ࡞ͬͨܕΛηοτ͠ɺ ؔ਺ࣗମΛrefͰ౉͢ɻ A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate );
22.  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ
     "@OQBO3FNPUF4FUVQΛߦ͏ A-npanRemote ৄࡉ:૊ΈࠐΈ ✔ ✔ ipΛೖΕͯSetupɻ 1Ͱpublicʹͨؔ͠਺͕ड͚औΔܕ + 2Ͱ࡞ͬͨܕΛηοτ͠ɺ ؔ਺ࣗମΛrefͰ౉͢ɻ A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate );
23.  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ
     "@OQBO3FNPUF4FUVQΛߦ͏ A-npanRemote ৄࡉ:૊ΈࠐΈ ✔ ✔ ipΛೖΕͯSetupɻ 1Ͱpublicʹͨؔ͠਺͕ड͚औΔܕ + 2Ͱ࡞ͬͨܕΛηοτ͠ɺ ؔ਺ࣗମΛrefͰ౉͢ɻ A_npanRemote.Setup<Matrix4x4, Dictionary<string, float>, Quaternion, FTPayload>( ipText, ref fTrack.OnTrackingUpdate );
24.  ࣮ػͰσʔλΛड͚औΔؔ਺ΛQVCMJDʹͯ͠࿐ग़ͤ͞Δ
     సૹ͍ͨ͠σʔλΛ4USFBNʹม׵͢ΔܕΛ༻ҙ͢Δ
     "@OQBO3FNPUF4FUVQΛߦ͏
     3&.05&
    4DSJQUJOH%FpOF4ZNCPMΛ͚ͭΔ A-npanRemote ৄࡉ:૊ΈࠐΈ ✔ ✔ ✔ Ҏ্ʂ
    ͜Ε͚ͩͰ ✔
25. fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); A-npanRemote࣮ߦ࣌ͷग़དྷࣄ Unity Editor peerͷน fTrack.StartTracking( () => { // إೝࣝ։࢝ }, (Matrix4x4 facePosAndRot, … { } ); A_npanRemote.Setup<Matrix4x4, … ipText, ref fTrack.OnTrackingUpdate ); ಧ͘ʂ ΍ͬͨͥʂ ࣮ػ
26. A-npanRemote in ZIZAI ZIZAIͷͲ͜Ͱ࢖ͬͯΔͷʁ ▸ Unity iOSͷARKitRemote͕҆ఆੑɺ؆қʹ઀ଓͰ͖ΔతͳҙຯͰ͋Μ·Γʹ΋ ࢖͍ͮΒ͍ͷͰɺARKit͕ΒΈͷ։ൃʹA-npanΛ࢖ͬͯΔɻ ▸ ࠶઀ଓ͠΍͘͢Ϩεϙϯε΋ྑ͘ɺI/OΛࣗ෼ͰσβΠϯͰ͖ΔͷͰָɻ
27. ͦͷଞͷ࣮༻ྫ @tomori_hikage IUUQTUXJUUFSDPNUPNPSJ@IJLBHFTUBUVT ද৘͸"OQBO3FNPUF 
    'BDF
    5SBDLJOH
    XJUI
    "3,JU
    
    ࢦ͸7PY)BOETΛσʔλάϩʔϒରԠ͢ΔͨΊʹ
    ͝ʹΐ͝ʹΐ͠·ͨ͠
    ͔Θ͍͔Ζʙ