DroidKnights 2025 - Jetpack XR 살펴보기: XR 개발은 어떻게 이루어지는가?

https://www.droidknights.dev

Transcript

2. Heesung Youn at Samsung heesung6701 heesung-youn-quokka Quokkaman OneUI 라이브러리 및 프레임워크 개발
3. 0 Android XR SDK
4. Android XR SDK Project Moohan
5. Android XR SDK Android XR SDK 공개 12월 24, 2024 Reference
6. Android XR 기본 개념 Android XR SDK Reference
7. Android XR SDK 홈 공간 모드 및 전체 공간 모드 Reference Home Space Mode Full Space Mode
8. Android XR SDK 공간 환경의 요소 Reference Skybox Geometry
9. Android XR SDK Passthrough 가상 환경 실제 환경
10. Android XR SDK 180° / 360° video Reference 180° video 360° video
11. Android XR SDK Google IO 2025 Preview 2 Building differentiated apps for Android XR with 3D content The future is now, with Compose and AI on Android XR
12. App + Jetpack XR = XR App Android XR SDK How
14. 1 Extensions 구조
15. Extensions 구조 컴파일 타임과 런타임 &" Android SDK C" Android Library
16. Extensions 구조 Android SDK Application Android Android SDK Project Developer System Image Android Partner 실행될 수 있는 최소 API 레벨 Min SDK Compile SDK 참조 가능한 최대 API Level
17. Application Android Android SDK Project Developer System Image Android Partner compile time Extensions 구조 Android SDK 실행될 수 있는 최소 API 레벨 Min SDK Compile SDK 참조 가능한 최대 API Level
18. Extensions 구조 Android SDK Application Android Android SDK Project Developer System Image Android Partner runtime 실행될 수 있는 최저 SDK 버전 Min SDK Compile SDK 참조 가능한 최대 API Level
19. Extensions 구조 Android Library AAR Jar R AndroidManifest Res JNI Assets Proguard AAR = JAR + 안드로이드 구성 파일
20. Extensions 구조 Android Library AAR Jar R AndroidManifest Res JNI Assets Proguard Project Jar R AndroidManifest Res JNI Assets Proguard APK dex JNI resources.arsc Res Assets APK = Project + AAR + AAR + …
21. Extensions 구조 Android SDK & Library Source Code Compile .class .class Android SDK App / Project R8 D8 .class Library / AAR Source Code Library / Project Compile .class Android SDK BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device
22. Source Code Compile .class .class Android SDK App / Project R8 D8 .class Library / AAR Source Code Library / Project Compile .class Android SDK BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Extensions 구조 Android SDK & Library
23. Source Code Compile .class .class Android SDK App / Project R8 D8 .class Library / AAR Source Code Library / Project Compile .class Android SDK BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Extensions 구조 Android SDK & Library
24. Source Code Compile .class .class Android SDK App / Project R8 D8 .class Library / AAR Source Code Library / Project Compile .class Android SDK BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Extensions 구조 Android SDK & Library
25. Extensions 구조 Android Wear SDK Android Foldable SDK Android XR SDK 등장 배경 (1)
26. 등장 배경 (2) Extensions 구조 Android SDK
27. Extensions 구조 신규 단말 지원 단일 Android SDK Android SDK extensions.jar extensions.jar extensions.jar 등장 배경 (3)
28. Extensions 구조 <uses-library> <manifest> <application> <uses-library /> </application> </manifest> android android :name :required ="androidx.window.extensions" ="false" 시스템 라이브러리 의존성 명시 Device /system_ext/framework extensions.jar .dex APK .dex
29. Extensions 구조 Extensions 원리 동일한 심볼의 라이브러리 컴파일 참조: stub.jar 런타임 참조: <uses-library> Developer Project Device Impl.jar Application Library Stub.jar compile runtime same symbol generate
30. Extensions 구조 Extensions 원리 동일한 심볼의 라이브러리 컴파일 참조: stub.jar 런타임 참조: <uses-library> Developer Project Device Impl.jar Application Library Stub.jar compile runtime same symbol generate
31. Extensions 구조 Extensions 원리 동일한 심볼의 라이브러리 컴파일 참조: stub.jar 런타임 참조: <uses-library> Developer Project Device Impl.jar Application Library Stub.jar compile runtime same symbol generate
32. Extensions 구조 Extensions 원리 Developer Project Device Impl.jar Application Library Stub.jar compile runtime same symbol generate 동일한 심볼의 라이브러리 컴파일 참조: stub.jar 런타임 참조: <uses-library>
33. 단점 Extensions 구조 Developer Project Device A(a, b, c) Application Library A(a, b) A(a, b) A(a, b) miss match generate 런타임 에러 발생 코드 최적화 제약 API 변경 어려움 복잡한 환경 설정
34. 단점 Extensions 구조 런타임 에러 발생 코드 최적화 제약 API 변경 어려움 복잡한 환경 설정 Impl.dex class Node class Consumer {
 fun accecpt() } Stub.class class Node class Consumer {
 fun accecpt() } R8 Stub.dex class aa class Consumer { } miss match
35. 단점 Extensions 구조 런타임 에러 발생 코드 최적화 제약 API 변경 어려움 복잡한 환경 설정 Device Impl.jar Library Stub.jar Same
 Symbol
36. 단점 Extensions 구조 런타임 에러 발생 코드 최적화 제약 API 변경 어려움 복잡한 환경 설정 Developer AndroidManifest build.gradle compileOnly uses-library
37. Extensions 구조 고수준 라이브러리 독립적인 API 직관적인 API 쉬운 개발 설정 Developer Project Device Impl.jar Application Library stub.jar Library compile runtime same symbol generate
38. Extensions 구조 고수준 라이브러리 독립적인 API 직관적인 API 쉬운 환경 설정 Use Library Developer AndroidManifest build.gradle compileOnly uses-library
39. .dex extensions.jar /system_ext/framework BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Source Code Compile .class .class Android SDK App / Project D8 R8 .class extensions-stub.jar .class Library / AAR Source Code Library / Project Compile .class Android SDK Extensions 구조
40. .dex extensions.jar /system_ext/framework BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Source Code Compile .class .class Android SDK App / Project D8 R8 .class extensions-stub.jar .class Library / AAR Source Code Library / Project Compile .class Android SDK Extensions 구조
41. .dex extensions.jar /system_ext/framework BootClassLoader PathClassLoader Application Process .dex APK .dex framework.jar /system/framework Device Source Code Compile .class .class Android SDK App / Project D8 R8 .class extensions-stub.jar .class Library / AAR Source Code Library / Project Compile .class Android SDK Extensions 구조
42. Extensions 구조 Summary 컴파일·런타임에 다른 라이브러리 사용 Android SDK 수정 없이 기능 확장 고수준 라이브러리로 안정성 확보
43. 2 SceneCore
44. XrExtensions의 추상화 SceneCore androidx.xr.scenecore XrExtensions SceneCore Public Maven
45. XrExtension 여부 AndroidManifest.xml <manifest> <application> <uses-library /> </application> </manifest> android android :name :required ="com.android.extensions.xr" ="false" build.gradle // For com.android.xr.extensions.
 compileOnly(files( File(AndroidXConfig. ( ) ))) new , getPrebuiltsRoot project "androidx/xr/extensions/ com.android.extensions.xr.jar" SceneCore
46. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
47. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
48. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
49. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
50. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
51. Entity Component System SceneCore Activity Session Scene Entity Component N N Y PanelEntitR Y GltfEntiR Y SurfaceEntiR Y ... Y Movablp Y Resizablp Y Interactablp Y ...
52. SceneCore :Integration-test private val by this as session session lazy { (Session.create( ) SessionCreateSuccess). } Session.kt @JvmOverloads
 @JvmStatic
 public fun false create(
 activity: Activity,
 coroutineContext: CoroutineContext = CoroutineContexts. ,
 ): SessionCreateResult = create(activity, coroutineContext, ) Lightweight Session 생성 과정 (1)
53. SceneCore Session.kt val val jxrPlatformAdapter = jxrPlatformAdapterFactory?.createPlatformAdapter(activity, session =
 Session(
 activity,
 runtime,
 jxrPlatformAdapter,
 stateExtenders,
 sessionConnectors,
 CoroutineScope( coroutineContext),
 ) ...

 context = Session 생성 과정 (2)
54. Session 생성 과정 (3) JxrPlatformAdapterAxr.kt import import import import import import com.android.extensions.xr.XrExtensions;
 com.android.extensions.xr.node.Node;
 com.android.extensions.xr.node.NodeTransaction;
 com.android.extensions.xr.space.ActivityPanel;
 com.android.extensions.xr.space.ActivityPanelLaunchParameters;
 com.android.extensions.xr.space.SpatialState; SceneCore
55. JxrPlatformAdapter SceneCore xr.runtime JxrPlatformAdapter com.android.extension XrExtensions xr.scenecore JxrPlatformAdapterAxr
56. SceneCore :integration-tests val panelEntity = PanelEntity.create(session!!, panel, IntSize2d( , ), ) panelEntity.setPose(Pose(Vector3( , , ))) panelEntity.parent = session!!.scene.activitySpace 640 480 0f -0.5f 0.5f "panel" PanelEntity.kt public fun create(session: Session, view: View, dimens: IntSize2d, name: String, pose: Pose ): PanelEntity = PanelEntity.create( session. , session. , session. . , ... ... activity platformAdapter scene entityManager Entity 생성 과정 (1)
57. internal fun import public open class internal constructor create(
 context: Context,
 adapter: JxrPlatformAdapter, ): PanelEntity = PanelEntity(adapter.createPanelEntity( ), entityManager,)

 androidx.xr.runtime.internal.PanelEntity as RtPanelEntity PanelEntity (
 rtEntity: RtPanelEntity,
 entityManager: EntityManager, ) : BasePanelEntity<RtPanelEntity>(rtEntity, entityManager) { ...
 ... ...
 ... SceneCore PanelEntity.kt Entity 생성 과정 (2)
58. Entity 생성 과정 (3) public new return PanelEntity createPanelEntity( Context context, ) {
 Node node = mExtensions.createNode();
 PanelEntity panelEntity = PanelEntityImpl(
 context,
 node,
 view,
 ,
 ,
 dimensions,
 name,
 );
 panelEntity.setParent(parent);
 panelEntity.setPose(pose, Space. );
 panelEntity;
 } @NonNull ... mExtensions mEntityManager mExecutor PARENT
59. Entity 와 EntityImpl SceneCore xr.runtime Entity (interface) Node com.android.extension.xr Entity Adapter EntityImpl xr.scenecore
60. Entity 와 EntityImpl SceneCore xr.runtime Entity (interface) Node com.android.extension.xr Entity Adapter EntityImpl xr.scenecore
61. Entity 와 EntityImpl SceneCore xr.runtime Entity (interface) Node com.android.extension.xr Entity Adapter EntityImpl xr.scenecore
62. Entity 와 EntityImpl SceneCore xr.runtime Entity (interface) Node com.android.extension.xr Entity Adapter EntityImpl xr.scenecore
63. Entity 와 EntityImpl SceneCore xr.runtime Entity (interface) Node com.android.extension.xr Entity Adapter EntityImpl xr.scenecore
64. Entity Behavior - setScale @Override
 @NonNull @SpaceValue public void super super try setScale( Vector3 scale, int relativeTo) {
 .setScale(scale, relativeTo);
 Vector3 localScale = .getScale(Space. );
 (NodeTransaction transaction = .createNodeTransaction()) {
 transaction
 .setScale( , localScale.getX(), localScale.getY(), localScale.getZ())
 .apply();
 }
 } PARENT mExtensions mNode SceneCore AndroidXrEntity
65. Entity Behavior - setPose @Override
 @NonNull @SpaceValue public void try ( Pose pose, int relativeTo) {
 (NodeTransaction transaction = .createNodeTransaction()) {
 transaction
 .setPosition( , , , )
 .setOrientation( , , , , )
 .apply();
 }
 } setPose ...
 x y z x y z w mExtensions mNode mNode SceneCore AndroidXrEntity
66. Entity 와 Node SceneCore Scencore 2 다수의 Entity 제공 2 책임별 클래스 분리 2 상속을 통한 확장 Node NodeTransaction Entity Entity XrExtension 2 최소 클래스 세트 2 트랜잭션 활용한 업데이트
67. SceneCore EntityManager Activity Session Entity EntityImpl Component Scene N N PlatformAdapter XrExtensions Node NodeTransaction XrExtensions SceneCore
68. SceneCore EntityManager Activity Session Entity EntityImpl Component Scene N N PlatformAdapter XrExtensions Node NodeTransaction XrExtensions SceneCore
69. SceneCore EntityManager Activity Session Entity EntityImpl Component Scene N N PlatformAdapter XrExtensions Node NodeTransaction XrExtensions SceneCore
70. SceneCore EntityManager Activity Session Entity EntityImpl Component Scene N N PlatformAdapter XrExtensions Node NodeTransaction XrExtensions SceneCore
71. SceneCore EntityManager Activity Session Entity EntityImpl Component Scene N N PlatformAdapter XrExtensions Node NodeTransaction XrExtensions SceneCore
72. SceneCore Summary 고수준 API 제공을 위한 추상화 계층 platformAdapter를 이용한 간접 연결 Wrapping 구조: Entity → EntityImpl → Node
73. 3 XR Compose
74. XR Compose XR Compose 구성 Spatialized Components @Composable ` OrbiteV ` SpatialDialoH ` SpatialElevatioE ` SpatialPopup Subspace Components @SubspaceComposable ` SpatialBos ` SpatialColumn / SpatialRoe ` SpatialLayoutSpaceV ` SpatialPaneo ` Volum— ` SpatialExternalSpace
75. XR Compose Spatialized 컴포넌트 Orbiter SpatialDialog SpatialPopup SpatialElevation
76. XR Compose ElevatedPanel 기반 Orbiter SpatialDialog SpatialElevation SpatialPopup ElevatedPanel ├── spatial
 │ ├── ConstrainToModifier.kt
 │ ├── ElevatedPanel.kt
 │ ├── Orbiter.kt
 │ ├── OutsideInputHandler.kt
 │ ├── RememberCalculatePose.kt
 │ ├── SpatialDialog.kt
 │ ├── SpatialElevation.kt
 │ ├── SpatialElevationLevel.kt
 │ ├── SpatialPopup.kt
 │ └── Subspace.kt
77. @Composable @Composable fun val val val val (contentSize: IntSize, shape: SpatialShape, pose: Pose?, content: () -> Unit) { session = checkNotNull( . ) { } parentEntity = . ?: session. view = () panelEntity = ( shape) { PanelEntity.create( session, view, contentSize. { PixelDimensions( , ) }, ${view. } , ) } ElevatedPanel run LocalSession LocalCoreEntity width height id current "session must be initialized" current coreMainPanelEntity rememberComposeView rememberCorePanelEntity "ElevatedPanel: " shape = session = view = pixelDimensions = name = ElevatedPanel 들여보기 (1)
78. @Composable fun ElevatedPanel(contentSize: IntSize, shape: SpatialShape, pose: Pose?, content: @Composable () -> Unit) { val session = checkNotNull(LocalSession.current) { "session must be initialized" } val parentEntity = LocalCoreEntity.current ?: session.coreMainPanelEntity val panelEntity = rememberCorePanelEntity(shape = shape) { PanelEntity.create( session = session, view = view, pixelDimensions = contentSize.run { PixelDimensions(width, height) }, name = "ElevatedPanel:${view.id}", ) } view = () val rememberComposeView ElevatedPanel 들여보기 (2)
79. ElevatedPanel 들여보기 (3) @Composable
 internal fun val by return (): ComposeView {
 composeView {
 disposableValueOf(
 ComposeView(context). { ... }
 ) { it.disposeComposition() }
 }
 composeView
 } rememberComposeView apply ...
 remember XR Compose subspace/RememberComposeView.kt
80. @Composable fun ElevatedPanel(contentSize: IntSize, shape: SpatialShape, pose: Pose?, content: @Composable () -> Unit) { val session = checkNotNull(LocalSession.current) { "session must be initialized" } val parentEntity = LocalCoreEntity.current ?: session.coreMainPanelEntity val view = rememberComposeView() panelEntity = ( shape) { PanelEntity.create( session, view, contentSize. { PixelDimensions( , ) }, ${view. } , ) } val rememberCorePanelEntity "ElevatedPanel: " shape = session = view = pixelDimensions = name = run width height id ElevatedPanel 들여보기 (4)
81. ...
 LaunchedEffect(pose) {
 if (pose != null) {
 panelEntity.entity.setPose(pose)
 }
 } view.setContent {
 ( provides panelEntity) {
 (Modifier. ( (pose == ) 0.0f 1.0f)) { () }
 }
 }

 CompositionLocalProvider content LocalOpaqueEntity Box alpha if null else XR Compose ElevatedPanel 들여보기 (5)
82. ...
 ... LaunchedEffect LaunchedEffect (pose) {
 (pose != ) {
 panelEntity. .setPose(pose)
 }
 } (contentSize) {
 width = contentSize. height = contentSize. panelEntity. = IntVolumeSize( width, height, 0)
 } if null val val entity width
 height
 size width = height = depth = XR Compose ElevatedPanel 들여보기 (6)
83. XR Compose Spatialized 컴포넌트 Orbiter SpatialDialog SpatialElevation SpatialPopup ElevatedPanel PanelEntity ComposeView content
84. XR Compose Subspace 컴포넌트 SpatialRow SpatialColumn SpatialBox SpatialPanel
85. XR Compose @SubspaceComposable @Composable @SubspaceComposable
 @Composable @SubspaceComposable private fun SpatialRow(modifier: SubspaceModifier, alignment: SpatialAlignment,
 content: SpatialRowScope.() -> Unit) @Composable @SubspaceComposable
 @Composable @SubspaceComposable private fun SpatialColumn(modifier: SubspaceModifier, alignment: SpatialAlignment,
 content: SpatialColumnScope.() -> Unit) @Composable @SubspaceComposable
 @Composable @SubspaceComposable public fun SpatialBox(modifier: SubspaceModifier, alignment: SpatialAlignment,
 content: SpatialBoxScope.() -> Unit)
86. XR Compose SubspaceLayout 기반 SpatialRow/Column SpatialBox SpatialPanel SpatialSpacer Volume SubspaceLayout
87. XR Compose @SubspaceComposable @Composable
 @Composable @SubspaceComposable public inline fun crossinline this val SubspaceLayout( content: () -> Unit, modifier: SubspaceModifier, coreEntity: CoreEntity = { ContentlessEntity.create( , entityName( )) }, measurePolicy: MeasurePolicy ) {
 compositionLocalMap = . rememberCoreContentlessEntity "Entity" currentComposer session = name = currentCompositionLocalMap
 ... SubspaceLayout 들여보기 (1)
88. SubspaceLayout 들여보기 (2) XR Compose ...
 set(compositionLocalMap, SetCompositionLocalMap)
 set(measurePolicy, SetMeasurePolicy) ... <ComposeSubspaceNode, Applier<Any>>(
 ComposeSubspaceNode. ,
 {
 set(coreEntity, )
 set(modifier, ) } { ( provides coreEntity) { () } },
 ComposeNode CompositionLocalProvider content factory = update = content = Constructor SetCoreEntity SetModifier LocalOpaqueEntity
89. SubspaceLayout 들여보기 (3) XR Compose public override fun this placeAt(pose: Pose) {
 layoutPose = pose

 ?. ( .getAll<CoreEntityNode>())
 ?.updateEntityPose()
 ?. = IntVolumeSize( , , )
 .getAll<LayoutCoordinatesAwareModifierNode>(). {
 it.onLayoutCoordinates( )
 }
 } coreEntity nodes coreEntity coreEntity size measuredWidth measuredHeight measuredDepth nodes applyCoreEntityNodes forEach ...
 subspace/node/SubspaceLayoutNode.kt
90. SubspaceLayout 들여보기 (4) XR Compose internal sealed class public val internal fun val if CoreEntity( : Entity) : OpaqueEntity { updateEntityPose() {
 corePose =
 ?. ?. ?. ( )
 ?: Pose. ( .getPose() != corePose) {
 .setPose(corePose)
 }
 } } entity layout measurableLayout poseInParentEntity density Identity
 entity entity ... convertPixelsToMeters subspace/layout/CoreEntity.kt
91. XR Compose @Composable
 @Composable @SubspaceComposable public fun val if is else if else (content: SpatialBoxScope.() -> Unit) {
 activity = . . ()
 ( . SubspaceNodeApplier) {
 ( content)
 } ( .current) {
 (activity, content)
 } {
 ( activity, )
 }
 } Subspace getActivity LocalContext applier LocalIsInApplicationSubspace current currentComposer SpatialBox NestedSubspace ApplicationSubspace ...
 ... content = activity = 진입점 1. Composable (1)
92. XR Compose @Composable
 private fun val by (activity: ComponentActivity, ) {
 scene (rootConstraints) {
 SpatialComposeScene(activity, session, compositionContext, rootConstraints) }
 scene.setContent { ( content) } ApplicationSubspace ... ...
 session.scene.mainPanelEntity.setHidden(true)
 disposableValueOf( ) {
 it.dispose()
 session.scene.mainPanelEntity.setHidden(false)
 }
 ... ... ... remember SpatialBox content = 진입점 1. Composable (2)
93. XR Compose public fun ComponentActivity. (session: Session, content: ) {
 (session).setContent { 
 (
 provides . ,
 content,
 )
 }
 } setSubspaceContent getOrCreateSpatialSceneForActivity ... ...
 ...
 CompositionLocalProvider LocalView window decorView content = platform/Activity.kt 진입점 2. Activity 확장 함수 (1)
94. XR Compose 진입점 2. Activity 확장 함수 (2) private fun return this this ComponentActivity. (
 session: Session = ): SpatialComposeScene {
 .computeIfAbsent( ) {
 SpatialComposeScene( , session). { subspace -> (subspace) }
 }
 } getOrCreateSpatialSceneForActivity also setUpSubspace defaultSession
 activityToSpatialComposeScene platform/Activity.kt
95. XR Compose 진입점 - SpatialComposeScene SubspaceComposable Activity::setSubspaceContent SpatialComposeScene
96. Compose UI 확장 XR Compose Composable Layout Modifier ComposeUiNode LayoutNode setContent SubspaceComposable SubspaceLayout SubspaceModifier ComposeSubspaceNode SubspaceLayoutNode setSubspaceContent
97. XR Compose SpatialPanel 들여보기 @Composable @SubspaceComposable
 public fun val val val this SpatialPanel( ) {
 view = ()
 dialogManager = . corePanelEntity = ( shape) {
 PanelEntity.create( , view, )
 }
 ( modifier, corePanelEntity) { ... ... ...

 ... rememberComposeView current
 rememberCorePanelEntity SubspaceLayout LocalDialogManager shape = session = view = modifier = coreEntity = subspace/SpatialPanel.kt
98. XR Compose SpatialPanel SpatialPanel ComposeView PanelEntity SubspaceLayout
99. XR Compose Summary 두 가지 컴포넌트: Spatialized vs Subspace SceneCore를 이용한 확장 Compose UI 확장: Compose 스타일로 Layout 지원
100. Recap XR Extensions으로 XR SDK 확장 SceneCore로 직관적인 API 제공 XR Compose를 통해 Compose 스타일 지원 High Level xr compose scenecore xr-extensoins Low Level