Kotlin DSL Magic for Augmented Reality

Kotlin DSL magic for Augmented Reality Giovanni Laquidara @joaolaq

AR can bring anything to you. AR can *bring* anything
to you. It adds computer-generated information and objects to your everyday world. @joaolaq

https://developers.google.com/ar/

MOTION TRACKING ENVIRONMENTAL UNDERSTANDING LIGHT ESTIMATION

ARCore provides SDKs for many of the most popular development
environments. These SDKs provide native APIs for all of the essential AR features like motion tracking, environmental understanding, and light estimation. With these capabilities you can build entirely new AR experiences or enhance existing apps with AR features. Use your favourite environment @joaolaq

Sceneform Sceneform is a 3D framework that makes it easy
for Java developers to build augmented reality apps. • A high-level scene graph API • A realistic physically based renderer provided by Filament • An Android Studio plugin for importing, viewing, and building 3D assets https://developers.google.com/ar/develop/java/sceneform/

Development integration

Android sample integration

Can we make it more readable?

Inspiration <html> <head> <title> A Simple HTML Document </title> </head>
<body> <p>This is a very simple HTML document</p> <p>It only has two paragraphs</p> </body> </html>

Domain Specific Languages by Martin Fowler (with Rebecca Parsons) Domain
Specific Languages (DSLs) have been around since I've been in computing, but it's hard to find much information about how to work with them. DSLs are small languages, focused on a particular aspect of a software system. You can't build a whole program with a DSL, but you often use multiple DSLs in a system mainly written in a general purpose language.

Internal DSL • Internal DSLs are particular ways of using
a host language to give the host language the feel of a particular language. • External DSLs have their own custom syntax and you write a full parser to process them.

Kotlin can help us with • Use of lambdas outside
of method parentheses • Lambdas with receivers • Builders • Infix • DSL Marker

Data class Scene data class Scene(var nodes: List<Node>? = null)

lambdas outside of method parentheses. fun scene(block: (Scene) -> Unit)
: Scene { val s = Scene() block(s) return s }

A scene now val scene = scene { it.nodes =
listOf(Node(), Node(), Node()) }

Lambda with receiver fun scene(block: Scene.() -> Unit): Scene {
val s = Scene() s.block() return s } // Or better fun scene(block: Scene.() -> Unit): Scene = Scene().apply { block }

A scene now val scene = scene { nodes =
listOf(Node(), Node(), Node()) }

I Want immutability data class Scene(val nodes: List<Node>)

Builders class SceneBuilder { private val nodes = mutableListOf<Node>() fun
build(): Scene { return Scene(nodes) } }

Lambda with receiver ( Builder version) fun scene(block: SceneBuilder.() ->
Unit): Scene = SceneBuilder().apply { block }

Add an Anchor Node // SceneBuilder fun anchorNode(anchor: Anchor, setup:
AnchorNodeBuilder.() -> Unit = {}) { val nodeBuilder = AnchorNodeBuilder(anchor) nodeBuilder.setup() nodes += nodeBuilder.build() }

AnchorNodeBuilder // SceneBuilder class AnchorNodeBuilder(var anchor: Anchor?) { private val
nodes = mutableListOf<Node>() fun build(): AnchorNode { anchor?.let { return AnchorNode(anchor).apply { nodes.forEach { it.setParent(this)} } } ?: throw IllegalArgumentException("Anchor cannot be null") }

Our Scene now val scene = scene { anchorNode(anchora) {
node { transformationSystem = fragment.transformationSystem model = renderable } } }

Add a Transformable Node

Add a Transformable Node // AnchorNodeBuilder fun node(transformationSystem: TransformationSystem? =
null, model: Renderable? = null, setup: NodeBuilder.() -> Unit = {}) { val nodeBuilder = NodeBuilder(transformationSystem, model) nodeBuilder.setup() nodes += nodeBuilder.build() }

How to build a Node class NodeBuilder(var transformationSystem: TransformationSystem?, var
model: Renderable?) { fun build(): Node { transformationSystem?.let { return TransformableNode(transformationSystem).apply { renderable = model } } ?: throw IllegalArgumentException("TransformationSystem cannot be null") } }

Generalize open class NodeBuilder(var position: Vector3?, var scale: Vector3?, var
model: Renderable?) { protected open val nodes = mutableListOf<Node>() open fun build(): Node { return Node().apply { if (position != null) localPosition = position if (scale != null) localScale = scale if (model != null) renderable = model nodes.forEach { it.setParent(this) } } }

Generalize …….. fun node(position: Vector3? = null, scale: Vector3? =
null, model: Renderable? = null, setup: NodeBuilder.() -> Unit = {}) { val nodeBuilder = NodeBuilder(position, scale, model) nodeBuilder.setup() nodes += nodeBuilder.build() } }

Generalize class AnchorNodeBuilder(var anchor: Anchor?) : NodeBuilder(null, null, null) {
override fun build(): AnchorNode { anchor?.let { return AnchorNode(anchor).apply { nodes.forEach { it.setParent(this) } } } ?: throw IllegalArgumentException("Anchor cannot be null") } }

DSL Marker @DslMarker annotation class ArDsl @ArDsl class NodeBuilder(var transformationSystem:
TransformationSystem?, var model: Renderable?) {

DSL Marker ( Context control ) @DslMarker annotation class ArDsl
@ArDsl class NodeBuilder(var transformationSystem: TransformationSystem?, var model: Renderable?) {

Infix arSceneView!!.scene setTo scene

Infix and Extension function private infix fun com.google.ar.sceneform.Scene.setTo(scene: Scene) {
this.addChild(scene.nodes.first()) }

Future to couroutines import kotlinx.coroutines.future.await private suspend fun placeObject(fragment: ArFragment,
anchor: Anchor, model: Uri) { val renderableFuture = ModelRenderable.builder() .setSource(fragment.context, model) .build() try { addNodeToScene(fragment, anchor, renderableFuture.await())

Complex sample val scene = scene { anchorNode { anchor
= anchora node { node { position = Vector3(0.0f, 0.5f, 0.0f) scale = Vector3(0.5f, 0.5f, 0.5f) model = sunRenderable node { position = Vector3(1.0f * AU_TO_METERS, 0.0f, 0.0f) scale = Vector3(0.05f, 0.05f, 0.05f) model = earthRenderable } }

How to create a DSL with Kotlin and your IDE

With Kotlin super powers • Use of lambdas outside of
method parentheses • Lambdas with receivers • Builders • Infix • DSL Marker • Operator Overloading • Extension functions

Next stop, commands https://youtu.be/JzTeAM8N1-o

Give me the source! https://github.com/joaobiriba/sceneform_ar_dsl https://github.com/joaobiriba/ARSample https://github.com/google-ar/sceneform-android-sdk https://github.com/google-ar/arcore-android-sdk/tree/master/samples/hello_ar_java

? https://bit.ly/2PbaKt9 @joaolaq [email protected]

Kotlin DSL Magic for Augmented Reality

Kotlin DSL Magic for Augmented Reality

More Decks by Giovanni Laquidara

Other Decks in Science

Featured

Transcript