Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fun with Native Code

Eric Butler
November 06, 2017

Fun with Native Code

Presented at Droidcon San Francisco, November 2017

Eric Butler

November 06, 2017
Tweet

More Decks by Eric Butler

Other Decks in Programming

Transcript

  1. Eric Butler
    @codebutler
    Droidcon San Francisco
    November 2017
    FUN WITH NATIVE CODE

    View Slide

  2. How This All Started

    View Slide

  3. Problems with Existing Apps
    Complicated Setup
    Broken Controller Support
    Touchscreen Required
    Ugly UI
    Intrusive Ads
    Bad Performance
    etc…

    View Slide

  4. Designing My Dream App
    Androidy TV Interface
    Works with game controllers
    Loads games from cloud
    Supports all retro consoles

    Leanback Support Library

    Android Input APIs

    Google Drive/Dropbox SDKs
    ??

    View Slide

  5. open source retro video game emulator library
    Google Search I’m Feeling Unlucky

    View Slide

  6. View Slide

  7. • Single API that abstracts many retro console emulators.
    • Specifically designed to make writing frontends easy.
    • Free, open-source.
    • Well documented API.
    • Lightweight cross-platform C library.
    • Perfect?

    View Slide

  8. • Single API that abstracts many retro console emulators
    • Specifically designed to make writing frontends easy.
    • Free, open-source.
    • Well documented API.
    • Lightweight cross-platform C library.
    • Perfect?
    h no…

    View Slide

  9. Foreign Function Interface
    ?
    FFI
    High-level
    language
    Low-level
    language

    View Slide

  10. Foreign Function Interface
    Python CTypes
    C# P/Invoke
    Java Native Interface

    View Slide

  11. JVM CODE

    View Slide

  12. JVM CODE
    NATIVE CODE
    Image: http://www.cakesandcomics.com/#/stranger-things/

    View Slide

  13. View Slide

  14. $ cat test.c
    #include
    void say_hello(const char* name) {
    printf("Hello %s!\n", name);
    }

    $ clang -shared -o libtest.dylib test.c
    A simple C shared library

    View Slide

  15. void say_hello(const char* name);
    $ cat test.py
    from ctypes import *
    lib = cdll.LoadLibrary("libtest.dylib")
    lib.say_hello("Droidcon")
    $ python test.py
    Hello Droidcon!
    Calling our native function with Python CTypes

    View Slide

  16. void say_hello(const char* name);
    Calling our native function with C# PInvoke
    $ cat Test.cs
    using System.Runtime.InteropServices;
    static class LibTest {
    [DllImport("libtest.dylib")]
    public static extern void say_hello(string name);
    }
    LibTest.say_hello("Droidcon")
    $ csharp Test.cs
    Hello Droidcon!

    View Slide

  17. void say_hello(const char* name);
    Calling our native function with Kotlin JNI…?
    $ cat test.kts

    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    $ kotlin -script test.kts
    java.lang.UnsatisfiedLinkError: Test.say_hello(Ljava/lang/String;)V
    at Test.say_hello(Native Method)
    at Test.(Unknown Source)

    View Slide

  18. void say_hello(const char* name);

    View Slide

  19. A simple C shared library
    $ cat test.c Z1Z1Z1Z1

    $ clang -shared -o libtest.dylib test.c Z5Z5Z5Z5
    #include Z2Z2Z2Z2
    void say_hello(const char* name) {Z3Z3Z3Z3
    printf("Hello %s!\n", name);
    }Z4Z4Z4Z4

    View Slide

  20. $ cat test.cpp Z1Z1Z1Z1

    #include Z2Z2Z2Z2
    #include
    extern "C" {
    void say_hello(const char* name) {Z3Z3Z3Z3
    printf("Hello %s!\n", name);
    }Z4Z4Z4Z4
    JNIEXPORT void JNICALL Java_Test_sayHello(
    JNIEnv *env, jclass cls, jstring name) {
    const char *nativeHello = env->GetStringUTFChars(name, JNI_FALSE);
    say_hello(nativeHello);
    env->ReleaseStringUTFChars(name, nativeHello);
    }
    }
    $ clang++ -shared -o libtest.dylib \
    -I${JAVA_HOME}/include -I${JAVA_HOME}/include/darwin test.cpp Z5Z5Z5Z5
    A (not so) simple JNI-compatible C++ shared library

    View Slide

  21. JNIEXPORT void JNICALL Java_Test_sayHello(
    JNIEnv *env, jclass cls, jstring name);
    Calling our native function with Kotlin JNI
    $ cat test.kts

    external fun sayHello(name: String)
    System.loadLibrary("test")
    sayHello("Droidcon")
    $ kotlin -script test.kts
    Hello Droidcon!

    View Slide

  22. View Slide

  23. call native code from java without jni
    Google Search I’m Feeling Hopeless

    View Slide

  24. JNA: Java Native Access

    View Slide

  25. “JNA provides Java programs easy access
    to native shared libraries without writing
    anything but Java code - no JNI or native
    code is required.

    This functionality is comparable to C#’s
    PInvoke and Python's ctypes.”
    github.com/java-native-access/jna
    JNA: Java Native Access
    Kotlin
    Kotlin

    View Slide

  26. JNA Features
    Functions Methods
    C JVM
    Data Types Primitives
    Structs Classes
    Function Pointers Callback Interfaces

    View Slide

  27. package com.sun.jna
    Library
    Native
    Structure
    Callback
    Pointer

    View Slide

  28. Base type for interfaces containing native library
    function definitions.
    interface Library
    interface TestLibrary : Library {
    fun some_method()
    }

    View Slide

  29. Static methods for loading native libraries, JNA
    configuration, and other utility tasks.
    final class Native
    val lib = Native.loadLibrary(
    "libName",
    TestLibrary::class.java)
    lib.some_method()

    View Slide

  30. Base type for classes that are automatically
    mapped to/from native structs.
    abstract class Structure
    class example_struct : Structure() {
    }
    @JvmField var field1: String? = null
    @JvmField var field2: String? = null
    override fun getFieldOrder()
    = listOf("field1", “field2")

    View Slide

  31. Base type for callback interfaces to be used as
    function pointers. Requires one public method.
    interface Callback
    interface my_callback : Callback {
    fun invoke()
    }
    interface TestLibrary : Library {
    fun set_callback(cb: my_callback)
    }
    val lib = Native.loadLibrary(/* … */)
    lib.set_callback(object : my_callback {
    override fun invoke() { /* TODO */ }
    })

    View Slide

  32. Represents a native pointer,
    used for accessing arbitrary memory.
    class Pointer
    interface TestLibrary : Library {
    fun set_memory(pointer: Pointer)
    fun get_memory(): Pointer
    }
    interface my_callback : Callback {
    fun invoke(data: Pointer)
    }

    View Slide

  33. Represents a native pointer,
    used for accessing arbitrary memory.
    class Pointer
    Pointer#getString(offset)
    Pointer#getInt(offset)
    Pointer#getLong(offset)
    Pointer#getByteArray(offset, length)
    Pointer#read(offset, buffer, index, length)
    // etc…

    View Slide

  34. JNA Performance
    https://github.com/java-native-access/jna/blob/master/www/FrequentlyAskedQuestions.md
    • Call overhead up to 10x slower

    View Slide

  35. JNA Performance
    https://github.com/java-native-access/jna/blob/master/www/FrequentlyAskedQuestions.md
    • Call overhead up to 10x slower
    • Ease of programing outweighs difference
    • Build, measure, optimize
    • Incrementally rewrite using JNI if needed

    View Slide

  36. $ cat test.c
    #include
    void say_hello(const char* name) {
    printf("Hello %s!\n", name);
    }

    $ clang -shared -o libtest.dylib test.c
    A simple C shared library, one last time

    View Slide

  37. void say_hello(const char* name)
    Calling our function from Kotlin with JNA!

    View Slide

  38. Calling our function from Kotlin with JNI…?
    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)

    View Slide

  39. Calling our function from Kotlin with JNA!
    import com.sun.jna.*

    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)

    View Slide

  40. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }

    View Slide

  41. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }

    View Slide

  42. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    val lib = Native.loadLibrary("test", TestLib::class.java)ZZZ
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }

    View Slide

  43. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    val lib = Native.loadLibrary("test", TestLib::class.java)ZZZ
    lib.say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }

    View Slide

  44. void say_hello(const char* name)
    Calling our function from Kotlin with JNA!
    $ cat test-jna.kts

    import com.sun.jna.*
    interface TestLib: Library {
    fun say_hello(name: String)
    }
    val lib = Native.loadLibrary("test", TestLib::class.java)
    lib.say_hello("Droidcon")
    $ mvn dependency:get \
    -Dartifact=net.java.dev.jna:jna:4.5.0 \
    -Ddest=jna.jar
    $ kotlinc -cp jna.jar -script test-jna.kts
    Hello Droidcon!

    View Slide

  45. My Dream App
    Androidy TV Interface
    Works with game controllers
    Loads games from cloud
    Supports all retro consoles

    Leanback Support Library

    Android Input APIs

    Google Drive/Dropbox SDKs
    ??

    View Slide

  46. My Dream App
    Androidy TV Interface
    Works with game controllers
    Loads games from cloud
    Supports all retro consoles

    Leanback Support Library

    Android Input APIs

    Google Drive/Dropbox SDKs
    libretro + JNA

    View Slide

  47. Learning Libretro
    Documentation
    C Headers
    Core Implementations
    Existing Frontends

    View Slide

  48. Libretro Documentation
    bit.ly/libretro-pdf doc.libretro.com/specs/api

    View Slide

  49. libretro.h
    • Single header file

    • 2178 lines

    • Mostly documentation comments!

    • 25 functions, 31 structs

    View Slide

  50. Libretro Cores
    • “core”: An emulator that has been ported to the common libretro API.

    • Useful to see which APIs are actually needed.

    • https://github.com/libretro/snes9x/blob/master/libretro/libretro.cpp

    • https://github.com/libretro/Genesis-Plus-GX/blob/master/libretro/libretro.c

    View Slide

  51. Libretro: Existing Frontends
    RetroArch

    C++

    retroarch.com
    Phoenix

    Qt/C++

    phoenix.vg
    GNOME Games

    C/Vala

    wiki.gnome.org/
    Apps/Games

    View Slide

  52. Libretro Architecture
    Init Callbacks Load Game ROM Run Game Loop
    Waveform https://thenounproject.com/icon/1345324/

    NES Cartridge https://thenounproject.com/icon/1127992/

    Refresh https://thenounproject.com/icon/974170/

    View Slide

  53. Libretro Architecture: Callbacks
    Video Callback: Called once per frame with a pointer to video data.
    Input Callback: Called for every supported button on every frame.
    Return true if button is pressed.
    Audio Callback: Called once per frame with a pointer to audio data.
    “Environment” Callback: Generic callback for obscure functions.
    Mostly unused, per documentation.

    View Slide

  54. Libretro Architecture: Loading Games
    Two Options
    1. Core loads game, specify file path.
    2. Frontend loads game, specify memory pointer.

    View Slide

  55. Libretro Architecture: Game Loop
    Libretro runs one frame at a time.
    Frontend maintains the framerate.

    View Slide

  56. libretro + JNA
    1. Write library bindings.

    2. Implement callbacks.

    3. Call functions.

    View Slide

  57. JNAerator
    github.com/nativelibs4java/JNAerator

    View Slide

  58. build.gradle
    dependencies {
    implementation 'net.java.dev.jna:jna:[email protected]'
    }

    View Slide

  59. Native Libraries Go Here

    View Slide

  60. Writing Bindings: Initialize
    void retro_init(void);

    View Slide

  61. Writing Bindings: Initialize
    void retro_init(void);
    interface LibRetro : Library {
    fun retro_init()
    }
    LibRetro.kt
    libretro.h

    View Slide

  62. Writing Bindings: Load Game
    bool retro_load_game(retro_game_info);

    View Slide

  63. Writing Bindings: Load Game
    struct retro_game_info
    {AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);

    View Slide

  64. Writing Bindings: Load Game
    struct retro_game_info
    {AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    LibRetro.kt
    libretro.h

    View Slide

  65. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    }AA
    struct retro_game_info

    };
    LibRetro.kt
    libretro.h

    View Slide

  66. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    }AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    LibRetro.kt
    libretro.h

    View Slide

  67. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    }AAA
    override fun getFieldOrder()
    = listOf("path", "data", "size", "meta")
    LibRetro.kt
    libretro.h

    View Slide

  68. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder()
    = listOf("path", "data", "size", "meta")
    }AAA
    bool retro_load_game(retro_game_info);
    interface LibRetro : Library {
    fun retro_load_game(info: retro_game_info): Boolean
    }
    LibRetro.kt
    libretro.h

    View Slide

  69. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);

    View Slide

  70. Writing Bindings: Audio Callback
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);

    View Slide

  71. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }AA
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    LibRetro.kt
    libretro.h

    View Slide

  72. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }AA
    interface LibRetro : Library {
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    }
    LibRetro.kt
    libretro.h

    View Slide

  73. Writing Bindings: Video Callback
    void retro_set_video_refresh(retro_video_refresh_t);

    View Slide

  74. Writing Bindings: Video Callback
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    void retro_set_video_refresh(retro_video_refresh_t);

    View Slide

  75. Writing Bindings: Video Callback
    void retro_set_video_refresh(retro_video_refresh_t);
    interface retro_video_refresh : Callback {
    fun invoke(
    data: Pointer,
    width: UnsignedInt,
    height: UnsignedInt,

    pitch: SizeT)
    }
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    LibRetro.kt
    libretro.h

    View Slide

  76. Writing Bindings: Video Callback
    interface retro_video_refresh : Callback {
    fun invoke(
    data: Pointer,
    width: UnsignedInt,
    height: UnsignedInt,

    pitch: SizeT)
    }
    void retro_set_video_refresh(retro_video_refresh_t);
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    interface LibRetro : Library {
    fun retro_set_video_refresh(cb: retro_video_refresh)
    }
    LibRetro.kt
    libretro.h

    View Slide

  77. void retro_set_input_state(retro_input_state_t);
    Writing Bindings: Input State Callback

    View Slide

  78. void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    Writing Bindings: Input State Callback

    View Slide

  79. interface retro_input_state : Callback {
    fun invoke(
    port: UInt,
    device: UInt,
    index: UInt,
    id: UInt): Short
    }
    void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    interface LibRetro : Library {
    fun retro_set_input_state(cb: retro_input_state)
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback

    View Slide

  80. interface retro_input_state : Callback {
    fun invoke(
    port: UInt,
    device: UInt,
    index: UInt,
    id: UInt): Short
    }
    void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    interface LibRetro : Library {
    fun retro_set_input_state(cb: retro_input_state)
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback

    View Slide

  81. #define RETRO_DEVICE_JOYPAD 1
    #define RETRO_DEVICE_ID_JOYPAD_START 3
    #define RETRO_DEVICE_ID_JOYPAD_UP 4
    #define RETRO_DEVICE_ID_JOYPAD_DOWN 5
    #define RETRO_DEVICE_ID_JOYPAD_LEFT 6
    #define RETRO_DEVICE_ID_JOYPAD_RIGHT 7
    const val RETRO_DEVICE_JOYPAD = 1
    const val RETRO_DEVICE_ID_JOYPAD_START = 3
    const val RETRO_DEVICE_ID_JOYPAD_UP = 4
    const val RETRO_DEVICE_ID_JOYPAD_DOWN = 5
    const val RETRO_DEVICE_ID_JOYPAD_LEFT = 6
    const val RETRO_DEVICE_ID_JOYPAD_RIGHT = 7
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback

    View Slide

  82. void retro_run(void);
    interface LibRetro : Library {
    fun retro_run()
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Run Game Loop

    View Slide

  83. class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder() = listOf("path", "data", "size", "meta")
    }
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }
    interface retro_video_refresh : Callback {
    fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
    }
    interface retro_input_state : Callback {
    fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
    }
    interface retro_input_poll : Callback {
    fun invoke()
    }
    interface LibRetro : Library {AA
    fun retro_init()
    fun retro_load_game(game: retro_game_info): Boolean
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    fun retro_set_video_refresh(cb: retro_video_refresh)
    fun retro_set_input_state(cb: retro_input_state)
    fun retro_set_input_poll(cb: retro_input_poll)
    fun retro_run()
    }BB

    View Slide

  84. libretro + JNA
    1. Write library bindings.

    2. Implement callbacks.

    3. Call functions.

    View Slide

  85. Implementing Callbacks: Audio
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    }AA
    }BB

    View Slide

  86. Implementing Callbacks: Audio
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    }AA
    }BB

    View Slide

  87. Implementing Callbacks: Audio
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    }AA
    }BB

    View Slide

  88. Implementing Callbacks: Audio
    import android.media.AudioTrack
    private val audioTrack = AudioTrack.Builder()
    .setAudioFormat(AudioFormat.Builder()
    .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
    .setSampleRate(32040)
    .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
    .build())
    .build()
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    }AA
    }BB

    View Slide

  89. Implementing Callbacks: Audio
    import android.media.AudioTrack
    private val audioTrack = AudioTrack.Builder()
    .setAudioFormat(AudioFormat.Builder()
    .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
    .setSampleRate(32040)
    .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
    .build())
    .build()
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    val written = audioTrack.write(buffer, 0, buffer.size)
    audioTrack.play()
    return SizeT(written)
    }AA
    }BB

    View Slide

  90. Implementing Callbacks: Video
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    }AA
    }BB

    View Slide

  91. Implementing Callbacks: Video
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    }AA
    }BB

    View Slide

  92. Implementing Callbacks: Video
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    }AA
    }BB

    View Slide

  93. Implementing Callbacks: Video
    import android.graphics.Bitmap
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    val bitmap = Bitmap.createBitmap(
    width.toInt(),
    height.toInt(),
    Bitmap.Config.RGB_565)
    bitmap.copyPixelsFromBuffer(ByteBuffer.wrap(buffer))
    }AA
    }BB

    View Slide

  94. Implementing Callbacks: Video
    import android.graphics.Bitmap
    private val imageView by lazy {
    findViewById(R.id.imageView)
    }
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    val bitmap = Bitmap.createBitmap(
    width.toInt(),
    height.toInt(),
    Bitmap.Config.RGB_565)
    bitmap.copyPixelsFromBuffer(ByteBuffer.wrap(buffer))
    imageView.post {
    imageView.setImageBitmap(bitmap)
    }
    }AA
    }BB

    View Slide

  95. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    }BB
    }ZZ

    View Slide

  96. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    }BB
    }ZZ
    // TODO: Multiplayer support
    if (port.toInt() != 0) {
    return 0AA
    }AA

    View Slide

  97. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0AA
    }AA
    }BB
    }ZZ
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0JJ
    }DD

    View Slide

  98. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0
    }AA
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0JJ
    }DD
    }BB
    }ZZ
    private val pressedButtons = mutableSetOf()

    View Slide

  99. Implementing Callbacks: Input State
    private val pressedButtons = mutableSetOf()
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0 RR
    }AA
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0 GG
    }DD
    }BB
    }ZZ
    val isPressed = when (id.toInt()) {
    RETRO_DEVICE_ID_JOYPAD_START ->
    pressedButtons.contains(KeyEvent.KEYCODE_BUTTON_START)
    RETRO_DEVICE_ID_JOYPAD_UP ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_UP)
    RETRO_DEVICE_ID_JOYPAD_DOWN ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_DOWN)
    RETRO_DEVICE_ID_JOYPAD_LEFT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_LEFT)
    RETRO_DEVICE_ID_JOYPAD_RIGHT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_RIGHT)
    else -> false
    }
    return if (isPressed) 1 else 0
    import android.view.KeyEvent

    View Slide

  100. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0 RR
    }AA
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0 GG
    }DD
    }BB
    }ZZ
    val isPressed = when (id.toInt()) {
    RETRO_DEVICE_ID_JOYPAD_START ->
    pressedButtons.contains(KeyEvent.KEYCODE_BUTTON_START)
    RETRO_DEVICE_ID_JOYPAD_UP ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_UP)
    RETRO_DEVICE_ID_JOYPAD_DOWN ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_DOWN)
    RETRO_DEVICE_ID_JOYPAD_LEFT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_LEFT)
    RETRO_DEVICE_ID_JOYPAD_RIGHT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_RIGHT)
    else -> false
    }
    return if (isPressed) 1 else 0
    import android.view.KeyEvent
    private val pressedButtons = mutableSetOf()

    View Slide

  101. Implementing Callbacks: Input State
    import android.view.KeyEvent
    private val pressedButtons = mutableSetOf()
    override fun dispatchKeyEvent(event: KeyEvent): Boolean {
    }ZZ

    View Slide

  102. Implementing Callbacks: Input State
    private val pressedButtons = mutableSetOf()
    override fun dispatchKeyEvent(event: KeyEvent): Boolean {
    }ZZ
    import android.view.KeyEvent
    super.dispatchKeyEvent(event)
    when (event.action) {
    KeyEvent.ACTION_DOWN -> pressedButtons.add(event.keyCode)
    KeyEvent.ACTION_UP -> pressedButtons.remove(event.keyCode)
    }
    return true

    View Slide

  103. libretro + JNA
    1. Write library bindings.

    2. Implement callbacks.

    3. Call functions.

    View Slide

  104. Calling Functions
    val lib = Native.loadLibrary(

    "snes9x_libretro_android",

    LibRetro::class.java)
    lib.retro_set_audio_sample_batch(audioSampleBatch)

    lib.retro_set_video_refresh(videoRefresh)

    lib.retro_set_input_poll(inputPoll)

    lib.retro_set_input_state(inputState)

    lib.retro_init()
    val gameInfo = retro_game_info()

    gameInfo.path = "/sdcard/allstars.smc"

    lib.retro_load_game(gameInfo)
    fixedRateTimer(period = 1000L / 60L) {

    lib.retro_run()

    }

    View Slide

  105. View Slide

  106. wat
    libc F Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 12676 (
    DEBUG F #00 pc 00000000
    F #01 pc 00010fc9 /data/app/com.codebutler.funwithnativecode-n1T9
    F #02 pc 00010c40 /data/app/com.codebutler.funwithnativecode-n1T9
    F #03 pc 000051e0 /data/app/com.codebutler.funwithnativecode-n1T9
    F #04 pc 00007c8f /data/app/com.codebutler.funwithnativecode-n1T9
    F #05 pc 0063ec67 /system/lib/libart.so (art_quick_generic_jni_tr
    F #06 pc 00638ea2 /system/lib/libart.so (art_quick_invoke_static_
    F #07 pc 00112b92 /system/lib/libart.so (_ZN3art9ArtMethod6Invoke
    F #08 pc 003231ff /system/lib/libart.so (_ZN3art11interpreter34Ar
    F #09 pc 0031bde1 /system/lib/libart.so (_ZN3art11interpreter6DoC
    F #10 pc 0061faf4 /system/lib/libart.so (MterpInvokeStatic+484)

    View Slide

  107. wat

    View Slide

  108. wat

    View Slide

  109. interface retro_environment : Callback {
    fun invoke(cmd: UInt, data: Pointer): Boolean
    }EE
    LibRetro.kt
    libretro.h
    interface LibRetro : Library {
    }
    void retro_set_environment(retro_environment_t);
    typedef bool (*retro_environment_t)(unsigned cmd, void *data);
    Writing Bindings: Environment Callback
    fun retro_set_environment(cb: retro_environment)

    View Slide

  110. class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder() = listOf("path", "data", "size", "meta")
    }
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }
    interface retro_video_refresh : Callback {
    fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
    }
    interface retro_input_state : Callback {
    fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
    }
    interface retro_input_poll : Callback {
    fun invoke()
    }
    interface LibRetro : Library {AA
    fun retro_init()
    fun retro_load_game(game: retro_game_info): Boolean
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    fun retro_set_video_refresh(cb: retro_video_refresh)
    fun retro_set_input_state(cb: retro_input_state)
    fun retro_set_input_poll(cb: retro_input_poll)
    fun retro_run()
    }BB
    fun retro_set_environment(cb: retro_environment)
    interface retro_environment : Callback {
    fun invoke(cmd: UInt, data: Pointer): Boolean
    }EE

    View Slide

  111. class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder() = listOf("path", "data", "size", "meta")
    }
    interface retro_environment : Callback {
    fun invoke(cmd: UInt, data: Pointer): Boolean
    }
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }
    interface retro_video_refresh : Callback {
    fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
    }
    interface retro_input_state : Callback {
    fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
    }
    interface retro_input_poll : Callback {
    fun invoke()
    }
    interface LibRetro : Library {AA
    fun retro_init()
    fun retro_load_game(game: retro_game_info): Boolean
    fun retro_set_environment(cb: retro_input_poll)
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    fun retro_set_video_refresh(cb: retro_video_refresh)
    fun retro_set_input_state(cb: retro_input_state)
    fun retro_set_input_poll(cb: retro_input_poll)
    fun retro_run()
    }BB

    View Slide

  112. private val environment = object : retro_environment {
    override fun invoke(cmd: UInt, data: Pointer): Boolean = false
    }
    Implementing Callbacks: Environment

    View Slide

  113. val lib = Native.loadLibrary(
    "snes9x_libretro_android",
    LibRetro::class.java)
    lib.retro_set_audio_sample_batch(audioSampleBatch)
    lib.retro_set_video_refresh(videoRefresh)
    lib.retro_set_input_poll(inputPoll)
    lib.retro_set_input_state(inputState)
    lib.retro_init()
    val gameInfo = retro_game_info()
    gameInfo.path = "/sdcard/allstars.smc"
    lib.retro_load_game(gameInfo)
    fixedRateTimer(period = 1000L / 60L) {
    lib.retro_run()
    }AA

    View Slide

  114. val lib = Native.loadLibrary(
    "snes9x_libretro_android",
    LibRetro::class.java)
    lib.retro_set_audio_sample_batch(audioSampleBatch)
    lib.retro_set_video_refresh(videoRefresh)
    lib.retro_set_input_poll(inputPoll)
    lib.retro_set_input_state(inputState)
    lib.retro_init()
    val gameInfo = retro_game_info()
    gameInfo.path = "/sdcard/allstars.smc"
    lib.retro_load_game(gameInfo)
    fixedRateTimer(period = 1000L / 60L) {
    lib.retro_run()
    }AA
    lib.retro_set_environment(environment)

    View Slide

  115. View Slide

  116. View Slide

  117. val lib = Native.loadLibrary(
    "snes9x_libretro_android",
    LibRetro::class.java)
    lib.retro_set_environment(environment)
    lib.retro_set_audio_sample_batch(audioSampleBatch)
    lib.retro_set_video_refresh(videoRefresh)
    lib.retro_set_input_poll(inputPoll)
    lib.retro_set_input_state(inputState)
    lib.retro_init()
    val gameInfo = retro_game_info()
    gameInfo.path = "/sdcard/allstars.smc"
    lib.retro_load_game(gameInfo)
    fixedRateTimer(period = 1000L / 60L) {
    lib.retro_run()
    }

    View Slide

  118. val lib = Native.loadLibrary(
    LibRetro::class.java)
    lib.retro_set_environment(environment)
    lib.retro_set_audio_sample_batch(audioSampleBatch)
    lib.retro_set_video_refresh(videoRefresh)
    lib.retro_set_input_poll(inputPoll)
    lib.retro_set_input_state(inputState)
    lib.retro_init()
    val gameInfo = retro_game_info()
    lib.retro_load_game(gameInfo)
    fixedRateTimer(period = 1000L / 60L) {
    lib.retro_run()
    }
    "genesis_plus_gx_libretro_android",
    gameInfo.path = "/sdcard/sonic3.md"

    View Slide

  119. View Slide

  120. MY DREAM APP...

    View Slide

  121. MY APP...!

    View Slide

  122. View Slide

  123. ODYSSEY
    github.com/codebutler/odyssey
    Current Features
    7 Console Systems
    Early Leanback UI
    Controller Support
    Game Covert Art
    Game Save/Restore
    WebDAV
    Future Features
    More Console Systems
    Better UI
    Local Multiplayer
    Game State Save/Restore
    Dropbox/GDrive
    Cheat Codes
    Internet Multiplayer
    Twitch/YouTube
    (Named after the Magnavox Odyssey, the first commercial home video game
    console released in 1972)

    View Slide

  124. THANKS!
    Slides & Code bit.ly/FunWithNativeCode
    Follow Me @codebutler

    View Slide