Slide 1

Slide 1 text

David WURSTEISEN - DroidKaigi 2024 Crafting Cross-Platform Adventures: Building a Game Engine with Kotlin Multiplatform

Slide 2

Slide 2 text

Disclaimer ࢲ͸೔ຊޠΛ࿩ͤ·ͤΜ ͔ͩΒ͸ӳޠ࿩ͤ·͢

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Game loop

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }

Slide 8

Slide 8 text

fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }

Slide 9

Slide 9 text

fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }

Slide 10

Slide 10 text

fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 } 60x seconds

Slide 11

Slide 11 text

Kotlin Multiplatform

Slide 12

Slide 12 text

Common iOS JS JVM Android

Slide 13

Slide 13 text

interface InputHandler { fun isTouch(): Boolean } Common

Slide 14

Slide 14 text

interface InputHandler { fun isTouch(): Boolean } class JVMInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } Common JVM

Slide 15

Slide 15 text

interface InputHandler { fun isTouch(): Boolean } class JVMInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } class JSInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } Common JVM JS

Slide 16

Slide 16 text

fun main() { Engine( JVMInputHandler() ) } fun main() { Engine( JSInputHandler() ) } JVM JS Application entry point

Slide 17

Slide 17 text

expect fun createInputHandler(): InputHandler actual fun createInputHandler(): InputHandler { return LwjglInput() } actual fun createInputHandler(): InputHandler { return JsInputHandler() } JVM JS Factory method Common

Slide 18

Slide 18 text

Common iOS JS JVM Android Playcanvas LibGDX Raylib

Slide 19

Slide 19 text

JS JVM N ative

Slide 20

Slide 20 text

Common iOS JS JVM Android OpenGL ES OpenGL ES WebGL OpenGL

Slide 21

Slide 21 text

OpenGL

Slide 22

Slide 22 text

OpenGL Pipeline Vertex Shader Fragment Shader Run on the GPU !

Slide 23

Slide 23 text

void main() { vec4 sum = vec4(0); float step = iResolution.y; // y for(float i = -area ; i <= area ; i += 1.) { // x for(float j = -area ; j <= area ; j += 1.) { float x = v_texCoords.x + i / iResolution.x; float y = v_texCoords.y + j / iResolution.y; sum += texture2D(u_texture, vec2(x, y)) * 0.005; } } gl_FragColor = texture2D(u_texture, v_texCoords) + sum; } OpenGL Shading Language (GLSL)

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

+ +

Slide 26

Slide 26 text

+ +

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

LWJGL if (!GLFW.glfwInit()) { throw IllegalStateException("Unable to initialize GLFW") } val window = GLFW.glfwCreateWindow(/* … */) ) // More of configuration code // [...] while (!GLFW.glfwWindowShouldClose(window)) { // Render the frame GLFW.glfwSwapBuffers(window) GLFW.glfwPollEvents() } GLFW.glfwTerminate() Render

Slide 29

Slide 29 text

LWJGL (Full code) if (!GLFW.glfwInit()) { throw IllegalStateException("Unable to initialize GLFW") } GLFW.glfwWindowHint(GLFW.GLFW_CONTEXT_VERSION_MAJOR, 2) GLFW.glfwWindowHint(GLFW.GLFW_CONTEXT_VERSION_MINOR, 0) GLFW.glfwWindowHint(GLFW.GLFW_OPENGL_PROFILE, GLFW.GLFW_OPENGL_CORE_PROFILE) GLFW.glfwWindowHint(GLFW.GLFW_OPENGL_DEBUG_CONTEXT, GLFW.GLFW_TRUE) GLFW.glfwDefaultWindowHints() // optional, the current window hints are already the default GLFW.glfwWindowHint( GLFW.GLFW_VISIBLE, GLFW.GLFW_FALSE, ) // the window will stay hidden after creation GLFW.glfwWindowHint( GLFW.GLFW_RESIZABLE, GLFW.GLFW_FALSE, ) // the window will be resizable // Create the window val windowWidth = (gameOptions.width + gameOptions.gutter.first * 2) * gameOptions.zoom val windowHeight = (gameOptions.height + gameOptions.gutter.first * 2) * gameOptions.zoom val window = GLFW.glfwCreateWindow( windowWidth, windowHeight, "Tiny", MemoryUtil.NULL, MemoryUtil.NULL, ) // Make the OpenGL context current GLFW.glfwMakeContextCurrent(window) // Enable v-sync GLFW.glfwSwapInterval(1) // Make the window visible GLFW.glfwShowWindow(window) // Get the size of the device window val tmpWidth = MemoryUtil.memAllocInt(1) val tmpHeight = MemoryUtil.memAllocInt(1) GLFW.glfwGetWindowSize(window, tmpWidth, tmpHeight) val tmpFrameBufferWidth = MemoryUtil.memAllocInt(1) val tmpFrameBufferHeight = MemoryUtil.memAllocInt(1) GLFW.glfwGetFramebufferSize(window, tmpFrameBufferWidth, tmpFrameBufferHeight) GL.createCapabilities(true) while (!GLFW.glfwWindowShouldClose(window)) { // Render the frame GLFW.glfwSwapBuffers(window) GLFW.glfwPollEvents() } GLFW.glfwTerminate()

Slide 30

Slide 30 text

WebGL val canvas = document.createElement("canvas") val gl = canvas.getContext("webgl2") as? WebGL2RenderingContext window.requestAnimationFrame { now -> // Render frame } Render

Slide 31

Slide 31 text

Android class MiniGdxSurfaceView(/* ... */) : GLSurfaceView(context) { init { setEGLContextClientVersion(2) setRenderer(object : Renderer { override fun onDrawFrame(gl: GL10?) { // Render the frame } } ) // Listen for input events setOnTouchListener(gameContext.input as OnTouchListener) } } val activity = configuration.activity!! // [...] val surfaceView = MiniGdxSurfaceView(this, gameContext, gameFactory, activity) activity.setContentView(surfaceView) Render

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

OpenGL val vertexData = floatArrayOf( 3f, -1f, -1f, 3f, -1f, -1f, ) val positionBuffer = gl.createBuffer() gl.bindBuffer(GL_ARRAY_BUFFER, positionBuffer) gl.bufferData(GL_ARRAY_BUFFER, FloatBuffer(vertexData), vertexData.size, GL_STATIC_DRAW) val position = gl.getAttribLocation(shaderProgram, "position") gl.vertexAttribPointer( location = position, size = 2, type = GL_FLOAT, normalized = false, stride = 0, offset = 0, ) gl.enableVertexAttribArray(position) gl.drawArrays(GL_TRIANGLES, 0, 3) Send vertices Assign it to an attribute of the shader Draw !

Slide 34

Slide 34 text

val position = gl.getAttribLocation(shaderProgram, "position") gl.vertexAttribPointer( location = position, size = 2, type = GL_FLOAT, normalized = false, stride = 0, offset = 0, ) gl.enableVertexAttribArray(position) OpenGL attribute vec3 position; void main() { gl_Position = vec4(position, 1.0); } Assign it to an attribute of the shader

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Matrix [ 1 | 0 | 0 | 0 ] [ 0 | 1 | 0 | 0 ] [ 0 | 0 | 1 | 0 ] [ 0 | 0 | 0 | 1 ]

Slide 37

Slide 37 text

Matrix Scale [ 1 | 0 | 0 | 0 ] [ 0 | 1 | 0 | 0 ] [ 0 | 0 | 1 | 0 ] [ 0 | 0 | 0 | 1 ] X Z Y Translation Rotation

Slide 38

Slide 38 text

TRANSFORMATION = IDENTITY

Slide 39

Slide 39 text

TRANSFORMATION = IDENTITY * TRANSLATION

Slide 40

Slide 40 text

TRANSFORMATION = IDENTITY * TRANSLATION * TRANSLATION

Slide 41

Slide 41 text

TRANSFORMATION = IDENTITY * TRANSLATION * TRANSLATION * ROTATION

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

data class Mat4(…) { // ... operator fun times(m: Mat4): Mat4 { val t = transpose(this) return Mat4( Float4(dot(t.x, m.x), dot(t.y, m.x), dot(t.z, m.x), dot(t.w, m.x)), Float4(dot(t.x, m.y), dot(t.y, m.y), dot(t.z, m.y), dot(t.w, m.y)), Float4(dot(t.x, m.z), dot(t.y, m.z), dot(t.z, m.z), dot(t.w, m.z)), Float4(dot(t.x, m.w), dot(t.y, m.w), dot(t.z, m.w), dot(t.w, m.w)) ) } }

Slide 44

Slide 44 text

val result: Mat4 = identity * Mat4(…) * Mat4(…) * Mat4(…)

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Group of vertices

Slide 47

Slide 47 text

Reference position Another group of vertices Apply transformation

Slide 48

Slide 48 text

Reference position Apply transformation

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

OpenGL Shading Language (GLSL)

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

iOS

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

usePinned override fun getShaderParameter(shader: Shader, mask: ByteMask): Any { val params = IntArray(1) params.usePinned { glGetShaderiv(shader.address, mask.toUInt(), it.addressOf(0)) } return params[0] }

Slide 57

Slide 57 text

usePinned override fun getShaderParameter(shader: Shader, mask: ByteMask): Any { val params = IntArray(1) params.usePinned { glGetShaderiv(shader.address, mask.toUInt(), it.addressOf(0)) } return params[0] } void glGetShaderiv( GLuint shader, GLenum pname, GLint *params);

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

Rewrite all shaders? 😱

Slide 63

Slide 63 text

3D is not easy Modeling Level Of Detail Perspective Camera Orthographic Camera 💥 Collision management Lightning

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

It’s time to try again (But in 2D this time)

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

OpenGL Pipeline Vertex Shader Fragment Shader

Slide 68

Slide 68 text

Common OpenGL ES OpenGL WebGL Metal iOS JS JVM Android

Slide 69

Slide 69 text

WebGL JS WebGPU

Slide 70

Slide 70 text

Frame generation

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

1 1 2 4 5 3 3 7 1 2 3 4 5 6 7 8 0 256 0 0 0 256

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

1 1 1 1 0 2 2 0 3 3 3 3 1 2 2 1 0 3 3 0

Slide 76

Slide 76 text

Lua Here Comes a New Challenger!

Slide 77

Slide 77 text

function draw(content) -- clear the screen gfx.cls() -- print content on the screen print(content[1] .. " " .. content[2]) end local array = {"hello", "world"} draw(array) Function declaration Array starting at 1 Variable declaration

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

Lua Kotlin Game Engine Game Script Game Lib Sprite Ctrl Sound

Slide 81

Slide 81 text

function _draw() gfx.cls() shape.line(x1, y1, x2, y2) shape.rectf(x1, y1, width, height, color) spr.draw(1, x, y) end

Slide 82

Slide 82 text

local x = 128 local y = 128 local pad_default = 32 local length = 48 function _draw() gfx.cls() local step = 2 * math.pi / 56 for i = 1, 56 do local s = i * step local pad = pad_default + math.perlin(0.1, 0.2 * i, tiny.frame / 20) * 24 local x1 = x + math.cos(s + tiny.t) * pad local x2 = x + math.cos(s + tiny.t) * (pad + length) local y1 = x + math.sin(s + tiny.t) * pad local y2 = x + math.sin(s + tiny.t) * (pad + length) shape.line(x1, y1, x2, y2, math.max(2, i % 16)) end end

Slide 83

Slide 83 text

local x = 128 local y = 128 local pad_default = 32 local length = 48 function _draw() gfx.cls() local step = 2 * math.pi / 56 for i = 1, 56 do local s = i * step local pad = pad_default + math.perlin(0.1, 0.2 * i, tiny.frame / 20) * 24 local x1 = x + math.cos(s + tiny.t) * pad local x2 = x + math.cos(s + tiny.t) * (pad + length) local y1 = x + math.sin(s + tiny.t) * pad local y2 = x + math.sin(s + tiny.t) * (pad + length) shape.line(x1, y1, x2, y2, math.max(2, i % 16)) end end

Slide 84

Slide 84 text

local x,y,speed = 128,128,4 function _draw() gfx.cls(1) if(ctrl.pressing(keys.left)) then x = x - speed elseif ctrl.pressing(keys.right) then x = x + speed end if(ctrl.pressing(keys.up)) then y = y - speed elseif ctrl.pressing(keys.down) then y = y + speed end spr.draw(1, x, y) end

Slide 85

Slide 85 text

No content

Slide 86

Slide 86 text

Sound

Slide 87

Slide 87 text

MIDI Musical Instrument Digital Interface

Slide 88

Slide 88 text

🎹 🎵 Synthesizer 2F 3A 5D 📄 1A 5F FE Synthesizer 🎶

Slide 89

Slide 89 text

Baguette (όήοτ) 🥖

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

No content

Slide 94

Slide 94 text

https://en.wikipedia.org/wiki/Waveform

Slide 95

Slide 95 text

No content

Slide 96

Slide 96 text

Generate sample

Slide 97

Slide 97 text

class Sine2( override val note: Note, override val modulation: Modulation? = null, override val envelope: Envelope? = null, override val volume: Float, ) : SoundGenerator { override val index: Int = 1 override val name: String = "Sine" override val frequency: Float = note.frequency override fun apply(index: Int): Float { return sin(angle(index)) * 0.7f * volume } }

Slide 98

Slide 98 text

class Sine2( override val note: Note, override val modulation: Modulation? = null, override val envelope: Envelope? = null, override val volume: Float, ) : SoundGenerator { override val index: Int = 1 override val name: String = "Sine" override val frequency: Float = note.frequency override fun apply(index: Int): Float { return sin(angle(index)) * 0.7f * volume } } Generate one sample

Slide 99

Slide 99 text

class Square2( override val note: Note, override val modulation: Modulation?, override val envelope: Envelope?, override val volume: Float, ) : SoundGenerator { override val index: Int = 2 override val name: String = "Square" override val frequency: Float = note.frequency override fun apply(index: Int): Float { val value = sin(angle(index)) return if (value > 0f) { 0.7f } else { -0.7f } } }

Slide 100

Slide 100 text

class Square2( override val note: Note, override val modulation: Modulation?, override val envelope: Envelope?, override val volume: Float, ) : SoundGenerator { override val index: Int = 2 override val name: String = "Square" override val frequency: Float = note.frequency override fun apply(index: Int): Float { val value = sin(angle(index)) return if (value > 0f) { 0.7f } else { -0.7f } } } Generate one sample

Slide 101

Slide 101 text

val buffer = FloatArray(nbSample) val generator = Sine2(/* ... */) for (0 until nbSample).forEach { i -> buffer[i] = generator.generate(i) } Generate sound

Slide 102

Slide 102 text

Play a sound Using Kotlin/JS external class AudioContext { val destination: AudioNode fun createOscillator(): OscillatorNode fun createBuffer(numOfChannels: Int, length: Int, sampleRate: Int): AudioBuffer } open external class AudioNode { fun connect( destination: AudioNode, output: Int = definedExternally, input: Int = definedExternally ): AudioNode } Describe JS class Describe default JS Args

Slide 103

Slide 103 text

Play a sound Using Kotlin/JS internal fun playSfxBuffer(result: Float32Array, loop: Boolean = false): AudioBufferSourceNode { val sfxBuffer = audioContext.createBuffer( 1, result.length, SAMPLE_RATE, ) val channel = sfxBuffer.getChannelData(0) channel.set(result) val source = audioContext.createBufferSource() source.buffer = sfxBuffer source.connect(audioContext.destination) source.loop = loop source.start() return source } Set the sound buffer Play the sound!

Slide 104

Slide 104 text

https://en.wikipedia.org/wiki/Envelope_(music)

Slide 105

Slide 105 text

Enveloppe fun apply(sample: Float, index: Int, nbSample: Int): Float { // attack phase if (index <= endOfAttackIndex) { val percentAttack = index / endOfAttackIndex.toFloat() return sample * percentAttack } else if (index > endOfAttackIndex && index <= endOfDecay) { // decay phase val percentDecay = (index - endOfAttackIndex) / decayDuration.toFloat() return sample * (1f - (1f - sustain) * percentDecay) } else if (index > endOfDecay && index <= nbSample - releaseDuration) { // sustain phase return sample * sustain } else { // release phase val percentRelease = (index - (nbSample - releaseDuration)) / releaseDuration.toFloat() return sample * (sustain * (1f - percentRelease)) } } Attack Decay Substain Release

Slide 106

Slide 106 text

Performance

Slide 107

Slide 107 text

Map versus Array class ColorPalette(colors: List) { private val rgba: Map private val rgb: Map private val rgbForGif: Map }

Slide 108

Slide 108 text

Map versus Array class ColorPalette(colors: List) { private val rgba: Map private val rgb: Map private val rgbForGif: Map } class ColorPalette(colors: List) { private val rgba: Array private val rgb: Array private val rgbForGif: Array } 1 2 3 4 5 6 7 8

Slide 109

Slide 109 text

Triangle 1 Triangle 2

Slide 110

Slide 110 text

Only 1 Triangle

Slide 111

Slide 111 text

Common OpenGL ES OpenGL WebGL Metal iOS JS JVM Android CPU

Slide 112

Slide 112 text

OpenGL ES OpenGL WebGL Metal iOS JS JVM Android GPU

Slide 113

Slide 113 text

1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8

Slide 114

Slide 114 text

Index to color conversion private fun convert(data: ByteArray): IntArray { val result = IntArray(data.size) val colorPalette = gameOptions.colors() data.forEachIndexed { index, byte -> result[index] = colorPalette.getRGAasInt(byte.toInt()) } return result }

Slide 115

Slide 115 text

1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8 CPU

Slide 116

Slide 116 text

1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8 CPU GPU

Slide 117

Slide 117 text

Index to color conversion varying vec2 texture; uniform sampler2D image; uniform sampler2D colors; void main() { vec4 point = texture2D(image, texture); vec4 color = texture2D(colors, vec2(point.r, 1.0)); gl_FragColor = color; } Get the color index Get the color from the index Set the color

Slide 118

Slide 118 text

&TLDR

Slide 119

Slide 119 text

👏 Kotlin Multiplatform 👏 🦾 You might prefer 2D over 3D 🦾 🔍 Looking for a Multiplatform graphic layer 🔍

Slide 120

Slide 120 text

No content

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

No content

Slide 123

Slide 123 text

fun makeGames() On the Web On Android On the JVM On iOS

Slide 124

Slide 124 text

Questions? Try a game Check the slides Check the source code

Slide 125

Slide 125 text

Links • Tiny: https://minigdx.github.io/tiny/index.html • Korge: https://korge.org/ • LittleKT: https://littlekt.com/ • OpenGL: https://www.opengl.org/ • Metal: https://developer.apple.com/metal/ • Kotlin Multiplatform: https://kotlinlang.org/docs/multiplatform.html • MIDI: https://fr.wikipedia.org/wiki/Musical_Instrument_Digital_Interface • Tiny Jumper (Sample game): https://dwursteisen.itch.io/tiny-jumper • Blog Adrien Courreges (DOOM shaders): https://www.adriancourreges.com/