Crafting Cross-Platform Adventures: Building a Game Engine with Kotlin Multiplatform

Building a game engine is an adventure. Building a multiplatform game engine in Kotlin Multiplatform is an even more exciting adventure.

In this session I will share what I learned from creating game engines with Kotlin Multiplatform. From mastering Graphic APIs like OpenGL and WebGL to crafting sounds using waveform synthesis, we'll explore the essentials of game development. Learn how to design cross-platform APIs, what are the specificities of each platform and what were the issues I had to create my game engine. Join me as we embark on this thrilling journey of creativity, innovation, and endless possibilities in game engine development.

(DroidKaigi 2024 - Tokyo - https://2024.droidkaigi.jp/en/timetable/683368/)

- https://dwursteisen.itch.io/tiny-jumper
- https://github.com/minigdx/tiny-jumper
- https://minigdx.github.io/tiny/index.html
- https://github.com/minigdx/tiny

Transcript

1. David WURSTEISEN - DroidKaigi 2024 Crafting Cross-Platform Adventures: Building a Game Engine with Kotlin Multiplatform
2. Disclaimer ࢲ͸೔ຊޠΛ࿩ͤ·ͤΜ ͔ͩΒ͸ӳޠ࿩ͤ·͢
4. Game loop
7. fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }
8. fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }
9. fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 }
10. fun render(delta: Float) {
 movePlayer(delta)
 moveEnemies(delta)
 moveWorld(delta)
 
 renderWorld()
 renderEnemies()
 renderPlayer()
 } 60x seconds
11. Kotlin Multiplatform
12. Common iOS JS JVM Android
13. interface InputHandler { fun isTouch(): Boolean } Common
14. interface InputHandler { fun isTouch(): Boolean } class JVMInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } Common JVM
15. interface InputHandler { fun isTouch(): Boolean } class JVMInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } class JSInputHandler : InputHandler { override fun isTouch(): Boolean { // ... } } Common JVM JS
16. fun main() { Engine( JVMInputHandler() ) } fun main() { Engine( JSInputHandler() ) } JVM JS Application entry point
17. expect fun createInputHandler(): InputHandler actual fun createInputHandler(): InputHandler { return LwjglInput() } actual fun createInputHandler(): InputHandler { return JsInputHandler() } JVM JS Factory method Common
18. Common iOS JS JVM Android Playcanvas LibGDX Raylib
19. JS JVM N ative
20. Common iOS JS JVM Android OpenGL ES OpenGL ES WebGL OpenGL
21. OpenGL
22. OpenGL Pipeline Vertex Shader Fragment Shader Run on the GPU !
23. void main() { vec4 sum = vec4(0); float step = iResolution.y; // y for(float i = -area ; i <= area ; i += 1.) { // x for(float j = -area ; j <= area ; j += 1.) { float x = v_texCoords.x + i / iResolution.x; float y = v_texCoords.y + j / iResolution.y; sum += texture2D(u_texture, vec2(x, y)) * 0.005; } } gl_FragColor = texture2D(u_texture, v_texCoords) + sum; } OpenGL Shading Language (GLSL)
25. + +
26. + +
28. LWJGL if (!GLFW.glfwInit()) { throw IllegalStateException("Unable to initialize GLFW") } val window = GLFW.glfwCreateWindow(/* … */) ) // More of configuration code // [...] while (!GLFW.glfwWindowShouldClose(window)) { // Render the frame GLFW.glfwSwapBuffers(window) GLFW.glfwPollEvents() } GLFW.glfwTerminate() Render
29. LWJGL (Full code) if (!GLFW.glfwInit()) { throw IllegalStateException("Unable to initialize GLFW") } GLFW.glfwWindowHint(GLFW.GLFW_CONTEXT_VERSION_MAJOR, 2) GLFW.glfwWindowHint(GLFW.GLFW_CONTEXT_VERSION_MINOR, 0) GLFW.glfwWindowHint(GLFW.GLFW_OPENGL_PROFILE, GLFW.GLFW_OPENGL_CORE_PROFILE) GLFW.glfwWindowHint(GLFW.GLFW_OPENGL_DEBUG_CONTEXT, GLFW.GLFW_TRUE) GLFW.glfwDefaultWindowHints() // optional, the current window hints are already the default GLFW.glfwWindowHint( GLFW.GLFW_VISIBLE, GLFW.GLFW_FALSE, ) // the window will stay hidden after creation GLFW.glfwWindowHint( GLFW.GLFW_RESIZABLE, GLFW.GLFW_FALSE, ) // the window will be resizable // Create the window val windowWidth = (gameOptions.width + gameOptions.gutter.first * 2) * gameOptions.zoom val windowHeight = (gameOptions.height + gameOptions.gutter.first * 2) * gameOptions.zoom val window = GLFW.glfwCreateWindow( windowWidth, windowHeight, "Tiny", MemoryUtil.NULL, MemoryUtil.NULL, ) // Make the OpenGL context current GLFW.glfwMakeContextCurrent(window) // Enable v-sync GLFW.glfwSwapInterval(1) // Make the window visible GLFW.glfwShowWindow(window) // Get the size of the device window val tmpWidth = MemoryUtil.memAllocInt(1) val tmpHeight = MemoryUtil.memAllocInt(1) GLFW.glfwGetWindowSize(window, tmpWidth, tmpHeight) val tmpFrameBufferWidth = MemoryUtil.memAllocInt(1) val tmpFrameBufferHeight = MemoryUtil.memAllocInt(1) GLFW.glfwGetFramebufferSize(window, tmpFrameBufferWidth, tmpFrameBufferHeight) GL.createCapabilities(true) while (!GLFW.glfwWindowShouldClose(window)) { // Render the frame GLFW.glfwSwapBuffers(window) GLFW.glfwPollEvents() } GLFW.glfwTerminate()
30. WebGL val canvas = document.createElement("canvas") val gl = canvas.getContext("webgl2") as? WebGL2RenderingContext window.requestAnimationFrame { now -> // Render frame } Render
31. Android class MiniGdxSurfaceView(/* ... */) : GLSurfaceView(context) { init { setEGLContextClientVersion(2) setRenderer(object : Renderer { override fun onDrawFrame(gl: GL10?) { // Render the frame } } ) // Listen for input events setOnTouchListener(gameContext.input as OnTouchListener) } } val activity = configuration.activity!! // [...] val surfaceView = MiniGdxSurfaceView(this, gameContext, gameFactory, activity) activity.setContentView(surfaceView) Render
33. OpenGL val vertexData = floatArrayOf( 3f, -1f, -1f, 3f, -1f, -1f, ) val positionBuffer = gl.createBuffer() gl.bindBuffer(GL_ARRAY_BUFFER, positionBuffer) gl.bufferData(GL_ARRAY_BUFFER, FloatBuffer(vertexData), vertexData.size, GL_STATIC_DRAW) val position = gl.getAttribLocation(shaderProgram, "position") gl.vertexAttribPointer( location = position, size = 2, type = GL_FLOAT, normalized = false, stride = 0, offset = 0, ) gl.enableVertexAttribArray(position) gl.drawArrays(GL_TRIANGLES, 0, 3) Send vertices Assign it to an attribute of the shader Draw !
34. val position = gl.getAttribLocation(shaderProgram, "position") gl.vertexAttribPointer( location = position, size = 2, type = GL_FLOAT, normalized = false, stride = 0, offset = 0, ) gl.enableVertexAttribArray(position) OpenGL attribute vec3 position; void main() { gl_Position = vec4(position, 1.0); } Assign it to an attribute of the shader
36. Matrix [ 1 | 0 | 0 | 0 ] [ 0 | 1 | 0 | 0 ] [ 0 | 0 | 1 | 0 ] [ 0 | 0 | 0 | 1 ]
37. Matrix Scale [ 1 | 0 | 0 | 0 ] [ 0 | 1 | 0 | 0 ] [ 0 | 0 | 1 | 0 ] [ 0 | 0 | 0 | 1 ] X Z Y Translation Rotation
38. TRANSFORMATION = IDENTITY
39. TRANSFORMATION = IDENTITY * TRANSLATION
40. TRANSFORMATION = IDENTITY * TRANSLATION * TRANSLATION
41. TRANSFORMATION = IDENTITY * TRANSLATION * TRANSLATION * ROTATION
43. data class Mat4(…) { // ... operator fun times(m: Mat4): Mat4 { val t = transpose(this) return Mat4( Float4(dot(t.x, m.x), dot(t.y, m.x), dot(t.z, m.x), dot(t.w, m.x)), Float4(dot(t.x, m.y), dot(t.y, m.y), dot(t.z, m.y), dot(t.w, m.y)), Float4(dot(t.x, m.z), dot(t.y, m.z), dot(t.z, m.z), dot(t.w, m.z)), Float4(dot(t.x, m.w), dot(t.y, m.w), dot(t.z, m.w), dot(t.w, m.w)) ) } }
44. val result: Mat4 = identity * Mat4(…) * Mat4(…) * Mat4(…)
46. Group of vertices
47. Reference position Another group of vertices Apply transformation
48. Reference position Apply transformation
50. OpenGL Shading Language (GLSL)
53. iOS
56. usePinned override fun getShaderParameter(shader: Shader, mask: ByteMask): Any { val params = IntArray(1) params.usePinned { glGetShaderiv(shader.address, mask.toUInt(), it.addressOf(0)) } return params[0] }
57. usePinned override fun getShaderParameter(shader: Shader, mask: ByteMask): Any { val params = IntArray(1) params.usePinned { glGetShaderiv(shader.address, mask.toUInt(), it.addressOf(0)) } return params[0] } void glGetShaderiv( GLuint shader, GLenum pname, GLint *params);
62. Rewrite all shaders? 😱
63. 3D is not easy Modeling Level Of Detail Perspective Camera Orthographic Camera 💥 Collision management Lightning
65. It’s time to try again (But in 2D this time)
67. OpenGL Pipeline Vertex Shader Fragment Shader
68. Common OpenGL ES OpenGL WebGL Metal iOS JS JVM Android
69. WebGL JS WebGPU
70. Frame generation
73. 1 1 2 4 5 3 3 7 1 2 3 4 5 6 7 8 0 256 0 0 0 256
75. 1 1 1 1 0 2 2 0 3 3 3 3 1 2 2 1 0 3 3 0
76. Lua Here Comes a New Challenger!
77. function draw(content) -- clear the screen gfx.cls() -- print content on the screen print(content[1] .. " " .. content[2]) end local array = {"hello", "world"} draw(array) Function declaration Array starting at 1 Variable declaration
80. Lua Kotlin Game Engine Game Script Game Lib Sprite Ctrl Sound
81. function _draw() gfx.cls() shape.line(x1, y1, x2, y2) shape.rectf(x1, y1, width, height, color) spr.draw(1, x, y) end
82. local x = 128 local y = 128 local pad_default = 32 local length = 48 function _draw() gfx.cls() local step = 2 * math.pi / 56 for i = 1, 56 do local s = i * step local pad = pad_default + math.perlin(0.1, 0.2 * i, tiny.frame / 20) * 24 local x1 = x + math.cos(s + tiny.t) * pad local x2 = x + math.cos(s + tiny.t) * (pad + length) local y1 = x + math.sin(s + tiny.t) * pad local y2 = x + math.sin(s + tiny.t) * (pad + length) shape.line(x1, y1, x2, y2, math.max(2, i % 16)) end end
83. local x = 128 local y = 128 local pad_default = 32 local length = 48 function _draw() gfx.cls() local step = 2 * math.pi / 56 for i = 1, 56 do local s = i * step local pad = pad_default + math.perlin(0.1, 0.2 * i, tiny.frame / 20) * 24 local x1 = x + math.cos(s + tiny.t) * pad local x2 = x + math.cos(s + tiny.t) * (pad + length) local y1 = x + math.sin(s + tiny.t) * pad local y2 = x + math.sin(s + tiny.t) * (pad + length) shape.line(x1, y1, x2, y2, math.max(2, i % 16)) end end
84. local x,y,speed = 128,128,4 function _draw() gfx.cls(1) if(ctrl.pressing(keys.left)) then x = x - speed elseif ctrl.pressing(keys.right) then x = x + speed end if(ctrl.pressing(keys.up)) then y = y - speed elseif ctrl.pressing(keys.down) then y = y + speed end spr.draw(1, x, y) end
86. Sound
87. MIDI Musical Instrument Digital Interface
88. 🎹 🎵 Synthesizer 2F 3A 5D 📄 1A 5F FE Synthesizer 🎶
89. Baguette (όήοτ) 🥖
94. https://en.wikipedia.org/wiki/Waveform
96. Generate sample
97. class Sine2( override val note: Note, override val modulation: Modulation? = null, override val envelope: Envelope? = null, override val volume: Float, ) : SoundGenerator { override val index: Int = 1 override val name: String = "Sine" override val frequency: Float = note.frequency override fun apply(index: Int): Float { return sin(angle(index)) * 0.7f * volume } }
98. class Sine2( override val note: Note, override val modulation: Modulation? = null, override val envelope: Envelope? = null, override val volume: Float, ) : SoundGenerator { override val index: Int = 1 override val name: String = "Sine" override val frequency: Float = note.frequency override fun apply(index: Int): Float { return sin(angle(index)) * 0.7f * volume } } Generate one sample
99. class Square2( override val note: Note, override val modulation: Modulation?, override val envelope: Envelope?, override val volume: Float, ) : SoundGenerator { override val index: Int = 2 override val name: String = "Square" override val frequency: Float = note.frequency override fun apply(index: Int): Float { val value = sin(angle(index)) return if (value > 0f) { 0.7f } else { -0.7f } } }
100. class Square2( override val note: Note, override val modulation: Modulation?, override val envelope: Envelope?, override val volume: Float, ) : SoundGenerator { override val index: Int = 2 override val name: String = "Square" override val frequency: Float = note.frequency override fun apply(index: Int): Float { val value = sin(angle(index)) return if (value > 0f) { 0.7f } else { -0.7f } } } Generate one sample
101. val buffer = FloatArray(nbSample) val generator = Sine2(/* ... */) for (0 until nbSample).forEach { i -> buffer[i] = generator.generate(i) } Generate sound
102. Play a sound Using Kotlin/JS external class AudioContext { val destination: AudioNode fun createOscillator(): OscillatorNode fun createBuffer(numOfChannels: Int, length: Int, sampleRate: Int): AudioBuffer } open external class AudioNode { fun connect( destination: AudioNode, output: Int = definedExternally, input: Int = definedExternally ): AudioNode } Describe JS class Describe default JS Args
103. Play a sound Using Kotlin/JS internal fun playSfxBuffer(result: Float32Array, loop: Boolean = false): AudioBufferSourceNode { val sfxBuffer = audioContext.createBuffer( 1, result.length, SAMPLE_RATE, ) val channel = sfxBuffer.getChannelData(0) channel.set(result) val source = audioContext.createBufferSource() source.buffer = sfxBuffer source.connect(audioContext.destination) source.loop = loop source.start() return source } Set the sound buffer Play the sound!
104. https://en.wikipedia.org/wiki/Envelope_(music)
105. Enveloppe fun apply(sample: Float, index: Int, nbSample: Int): Float { // attack phase if (index <= endOfAttackIndex) { val percentAttack = index / endOfAttackIndex.toFloat() return sample * percentAttack } else if (index > endOfAttackIndex && index <= endOfDecay) { // decay phase val percentDecay = (index - endOfAttackIndex) / decayDuration.toFloat() return sample * (1f - (1f - sustain) * percentDecay) } else if (index > endOfDecay && index <= nbSample - releaseDuration) { // sustain phase return sample * sustain } else { // release phase val percentRelease = (index - (nbSample - releaseDuration)) / releaseDuration.toFloat() return sample * (sustain * (1f - percentRelease)) } } Attack Decay Substain Release
106. Performance
107. Map versus Array class ColorPalette(colors: List<HexColor>) { private val rgba: Map<Int, ByteArray> private val rgb: Map<Int, ByteArray> private val rgbForGif: Map<Int, Int> }
108. Map versus Array class ColorPalette(colors: List<HexColor>) { private val rgba: Map<Int, ByteArray> private val rgb: Map<Int, ByteArray> private val rgbForGif: Map<Int, Int> } class ColorPalette(colors: List<HexColor>) { private val rgba: Array<ByteArray> private val rgb: Array<ByteArray> private val rgbForGif: Array<Int> } 1 2 3 4 5 6 7 8
109. Triangle 1 Triangle 2
110. Only 1 Triangle
111. Common OpenGL ES OpenGL WebGL Metal iOS JS JVM Android CPU
112. OpenGL ES OpenGL WebGL Metal iOS JS JVM Android GPU
113. 1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8
114. Index to color conversion private fun convert(data: ByteArray): IntArray { val result = IntArray(data.size) val colorPalette = gameOptions.colors() data.forEachIndexed { index, byte -> result[index] = colorPalette.getRGAasInt(byte.toInt()) } return result }
115. 1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8 CPU
116. 1 1 2 4 5 2 2 7 1 2 3 4 5 6 7 8 CPU GPU
117. Index to color conversion varying vec2 texture; uniform sampler2D image; uniform sampler2D colors; void main() { vec4 point = texture2D(image, texture); vec4 color = texture2D(colors, vec2(point.r, 1.0)); gl_FragColor = color; } Get the color index Get the color from the index Set the color
118. &TLDR
119. 👏 Kotlin Multiplatform 👏 🦾 You might prefer 2D over 3D 🦾 🔍 Looking for a Multiplatform graphic layer 🔍
123. fun makeGames() On the Web On Android On the JVM On iOS
124. Questions? Try a game Check the slides Check the source code
125. Links • Tiny: https://minigdx.github.io/tiny/index.html • Korge: https://korge.org/ • LittleKT: https://littlekt.com/ • OpenGL: https://www.opengl.org/ • Metal: https://developer.apple.com/metal/ • Kotlin Multiplatform: https://kotlinlang.org/docs/multiplatform.html • MIDI: https://fr.wikipedia.org/wiki/Musical_Instrument_Digital_Interface • Tiny Jumper (Sample game): https://dwursteisen.itch.io/tiny-jumper • Blog Adrien Courreges (DOOM shaders): https://www.adriancourreges.com/