How to Build a Rhythm Game in (Almost) Any Game Engine

I’ve been building a set of utilities for creating rhythm games, but I didn’t choose just one game engine to build this plugin for; I built it for many of them. Currently, it works in Unity, Unreal, Godot, SDL, and MonoGame, with Love2D and Bevy to follow.

This talk will walk through the steps to create a rhythm game from scratch using the plugin.

- Choosing how to import songs and how to chart the notes
- How to load the notes and render them in sync with an audio track
- How to detect user input
- How to resolve the user input and the notes in the song

A working example will be made available on GitHub for those who want to test it out after the talk.

Transcript

1. How to Build a Rhythm Game in (Almost) Any Game Engine Boston Game Dev Week – MiniCon 2026 1
2. Who am I? • Web / Game Developer • Worked with Unity, Unreal, and Godot • Built games as well as consumer based products in game engines • Release a lot of what I do as open source projects • I love data-in-data-out systems, writing unit tests and regular expressions (seriously) 2
3. What Have I Built (Open Source Tools) And many, many more tools. 3
4. What Have I Built (Games) https://flipjacksgame.com/ 4
5. My Favorite Rhythm Games • PaRappa the Rapper (Playstation 1997) • Guitar Hero (PS2 - 2005) • Rock Band (XBOX - 2007) • Patapon (PSP - 2008) • DJ Hero (XBOX - 2009) • Amplitude (PS4 - 2016) • Audica (VR - 2019) • Hi-Fi Rush (PC - 2023) 5
6. My Favorite Rhythm Games • PaRappa the Rapper (Playstation 1997) • Guitar Hero (PS2 - 2005) • Rock Band (XBOX - 2007) • Patapon (PSP - 2008) • DJ Hero (XBOX - 2009) • Amplitude (PS4 - 2016) • Audica (VR - 2019) • Hi-Fi Rush (PC - 2023) 6
7. Disclaimer: I’m awful at rhythm games 7
8. Disclaimer: I’m awful at rhythm games & I’m not a musician. 😬 8
9. But I’ve talked to people who are: 1. Good at rhythm games 2. Actual musicians 9
10. 10 https://rhythmgameutilities.com/
11. Let’s Break Down Rhythm Games 11
12. Rhythm Games Have Complicated Systems 1. Charting Notes 2. Loading Notes 3. Rendering Notes 4. Syncing Notes With Audio 5. Calculating Accuracy From User Input 6. Beat Matching 7. (Bonus) Input Devices (Midi/Custom) 8. (Bonus) Handling Input Latency 12
13. 1. Charting Notes 13
14. 1. Charting Notes (.chart) https://github.com/FireFox2000000/Moonscraper-Chart-Editor 14
15. 1. Charting Notes (.mid) https://signalmidi.app/ 15
16. 1. Charting Notes (In Game) Rift of the NecroDancer Level Editor 16
17. 2. Loading Notes 17
18. 2. Loading Notes (.chart) song.chart 18
19. 2. Loading Notes (.chart) 19
20. 2. Loading Notes (.chart) 20
21. 2. Loading Notes (.chart) 21
22. 2. Loading Notes (.mid) 22
23. 2. Loading Notes (.mid) 23
24. 2. Loading Notes (.chart) song.chart 24
25. 2. Loading Notes (.chart) 25 Yes, I actually wrote these by hand because I’m nuts.
26. 2. Loading Notes • Song.FromChartData • Song.FromMidiData • ReadNotesFromChartData • ReadNotesFromMidiData npx chart-to-json ./notes.chart > notes.json 26
27. 3. Rendering Notes 27
28. 3. Rendering Notes Rift of the NecroDancer Amplitude (PS4) 28
29. 3. Rendering Notes 29
31. 3. Rendering Notes 31
33. 3. Rendering Notes 33
35. 4. Syncing Notes With Audio 35
36. So this section gets a bit … 36
38. Disclaimer: I’m awful at rhythm games & I’m not a musician. 😬 38
39. Disclaimer: I’m awful at rhythm games & I’m not a musician. 😬 39 (and I’m not great at math)
40. But I’ve talked to people who are: 1. Good at rhythm games 2. Actual musicians 40
41. 41 But I’ve talked to people who are: 1. Good at rhythm games 2. Actual musicians 3. Good at math
42. 4. Syncing Notes With Audio 42
44. 4. Syncing Notes With Audio 44
47. 4. Syncing Notes With Audio 47
49. 5. Calculating Accuracy From User Input 49
50. 5. Calculating Accuracy of Hit Notes From User Input 50
51. 5. Calculating Accuracy of Hit Notes From User Input 51
52. 5. Calculating Accuracy of Hit Notes From User Input 52
53. 5. Calculating Accuracy of Hit Notes From User Input 53
54. 5. Calculating Accuracy of Hit Notes From User Input 54
55. 5. Calculating Accuracy of Hit Notes From User Input 55
56. 5. Calculating Accuracy of Hit Notes From User Input 56
57. 6. Beat Matching 57
58. 6. Beat Matching Hi-Fi Rush Crypt of the NecroDancer 58
59. 6. Beat Matching 59
60. 6. Beat Matching 60
61. Upcoming Features • Stepmania Import • Love2D (Lua) Plugin • Bevy (Rust) Plugin • GameMaker (GML) Plugin • Desktop/Web Charting Tool 61
62. Examples Unity Godot 62
63. Examples SDL 63
64. Questions? 64
65. Thank you! 65