iOSDJ2025 - Stream Deck Plugin using Swift

by h.crane

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Stream Deck

Slide 3

Slide 3 text

Stream Deck Quate: https://www.elgato.com/jp/en/explorer/products/stream-deck/stream-deck-vs-steam-deck/

Slide 4

Slide 4 text

Stream Deck Steam Quate: https://www.elgato.com/jp/en/explorer/products/stream-deck/stream-deck-vs-steam-deck/

Slide 5

Slide 5 text

Quate: https://www.elgato.com/jp/en/explorer/products/stream-deck/stream-deck-vs-steam-deck/

Slide 6

Slide 6 text

Stream Deck + Quate: https://www.elgato.com/jp/ja/p/stream-deck-plus

Slide 7

Slide 7 text

Stream Deck + Quate: https://www.elgato.com/jp/ja/p/stream-deck-plus • Touch Screenɿ̐ • Toggleɿ̐ • LED Buttonɿ̔

Slide 8

Slide 8 text

I thought it might be possible to use Stream Deck like a device for DJs.

Slide 9

Slide 9 text

How Stream Deck works

Slide 10

Slide 10 text

Quate: https://www.elgato.com/jp/ja/p/stream-deck-plus USB PC Stream Deck App Application How Stream Deck works The Stream Deck commands are executed by the host app on the PC.

Slide 11

Slide 11 text

PC Stream Deck App Action Application Action Action Plugin WebSocket HTTP, Socket, gRPC etc. How Stream Deck works Stream Deck App and Plugin communicate internally via WebSocket.

Slide 12

Slide 12 text

How to create a plugin

Slide 13

Slide 13 text

Quate: https://github.com/elgatosf/streamdeck How to create a plugin The fastest way to create a plugin is by using the o ff i cial SDK. Note: However, it only supports TypeScript.

Slide 14

Slide 14 text

How to create a plugin Typescript Javascript Node.js Transpile Quate: https://www.typescriptlang.org/branding/, https://github.com/voodootikigod/logo.js/, https://nodejs.org/en/about/resources/ Transpile TypeScript with Node and use JavaScript as the executable fi le.

Slide 15

Slide 15 text

Stream Deck App Application Plugin Javascript How to create a plugin Quate: https://github.com/voodootikigod/logo.js/ The Stream Deck host application on the PC acts as the interpreter that runs the JavaScript code.

Slide 16

Slide 16 text

How to create a plugin using Swift Quate: https://developer.apple.com/swift/

Slide 17

Slide 17 text

How to create a plugin using Swift WebSocket Stream Deck App Plugin Application If WebSocket communication is implemented in Swift, a plugin can be created using Swift.

Slide 18

Slide 18 text

How to create a plugin using Swift WebSocket JSON message example

Slide 19

Slide 19 text

How to create a plugin using Swift Quate: https://docs.elgato.com/streamdeck/sdk/guides/actions#handling-events Event Lifecycle Since various event cycles are available,   the task is simply to cover them.

Slide 20

Slide 20 text

0 1 2 3 1 How to create a plugin using Swift 0 1 2 3 0 1 2 3 Also, button coordinates can be obtained from Stream Deck.

Slide 21

Slide 21 text

How to create a plugin using Swift Quate: https://github.com/emorydunn/StreamDeckPlugin There’s also uno ff i cial open-source Swift code available.

Slide 22

Slide 22 text

Mach-O Compile Quate: https://developer.apple.com/swift/, Swift swift build -c release How to create a plugin using Swift Swift code cannot be run directly, so it must be compiled into an executable using the Swift build command.

Slide 23

Slide 23 text

How to create a plugin using Swift Stream Deck App Application Plugin Mach-O Additionally, other compiled languages can be used as plugins if they implement WebSocket protocol correctly and produce an executable.

Slide 24

Slide 24 text

How DJ application works

Slide 25

Slide 25 text

Stream Deck App Plugin DJ application Socket Server Sound Player Unix Domain Socket .  . How DJ application works On Mac app side, it runs as a socket server while also implementing functions to control audio.

Slide 26

Slide 26 text

How DJ application works - Network Framework - Foundation Stream - CFSocket - Darwin API Low High API Level - Darwin API Options for Socket Communication Socket Server

Slide 27

Slide 27 text

How DJ application works Sound Player - AVAudioEngineɿ Controlling audio nodes - AVAudioNodeɿ Generates and effects sound The audio implementation uses AVFoundation APIs.

Slide 28

Slide 28 text

How DJ application works Sound Player Loop Delay Reverb Flanger Isolator Pitch Rate Stutter Sound Effect Scratch

Slide 29

Slide 29 text

Let's try this !

Slide 30

Slide 30 text

GitHub - StreamDeckWorkspace Quate: https://github.com/crane-hiromu/StreamDeckWorkspace

Slide 31

Slide 31 text

Stream Deck Mobile Quate: https://apps.apple.com/us/app/elgato-stream-deck-mobile/id1440014184

Slide 32

Slide 32 text

Virtual Stream Deck (BETA) Quate: https://www.elgato.com/us/en/s/virtual-stream-deck