Fun with Native Code

Presented at Droidcon San Francisco, November 2017

Transcript

1. Eric Butler
   @codebutler
   Droidcon San Francisco
   November 2017
   FUN WITH NATIVE CODE
   

   View Slide

2. How This All Started
   

   View Slide

3. Problems with Existing Apps
   Complicated Setup
   Broken Controller Support
   Touchscreen Required
   Ugly UI
   Intrusive Ads
   Bad Performance
   etc…
   

   View Slide

4. Designing My Dream App
   Androidy TV Interface
   Works with game controllers
   Loads games from cloud
   Supports all retro consoles

   Leanback Support Library
   
   Android Input APIs
   
   Google Drive/Dropbox SDKs
   ??
   

   View Slide

5. open source retro video game emulator library
   Google Search I’m Feeling Unlucky
   

   View Slide

6. View Slide

7. • Single API that abstracts many retro console emulators.
   • Specifically designed to make writing frontends easy.
   • Free, open-source.
   • Well documented API.
   • Lightweight cross-platform C library.
   • Perfect?
   

   View Slide

8. • Single API that abstracts many retro console emulators
   • Specifically designed to make writing frontends easy.
   • Free, open-source.
   • Well documented API.
   • Lightweight cross-platform C library.
   • Perfect?
   h no…
   

   View Slide

9. Foreign Function Interface
   ?
   FFI
   High-level
   language
   Low-level
   language
   

   View Slide

10. Foreign Function Interface
    Python CTypes
    C# P/Invoke
    Java Native Interface
    

    View Slide

11. JVM CODE
    

    View Slide

12. JVM CODE
    NATIVE CODE
    Image: http://www.cakesandcomics.com/#/stranger-things/
    

    View Slide

13. View Slide

14. $ cat test.c
    #include
    void say_hello(const char* name) {
    printf("Hello %s!\n", name);
    }
    

    $ clang -shared -o libtest.dylib test.c
    A simple C shared library
    

    View Slide

15. void say_hello(const char* name);
    $ cat test.py
    from ctypes import *
    lib = cdll.LoadLibrary("libtest.dylib")
    lib.say_hello("Droidcon")
    $ python test.py
    Hello Droidcon!
    Calling our native function with Python CTypes
    

    View Slide

16. void say_hello(const char* name);
    Calling our native function with C# PInvoke
    $ cat Test.cs
    using System.Runtime.InteropServices;
    static class LibTest {
    [DllImport("libtest.dylib")]
    public static extern void say_hello(string name);
    }
    LibTest.say_hello("Droidcon")
    $ csharp Test.cs
    Hello Droidcon!
    

    View Slide

17. void say_hello(const char* name);
    Calling our native function with Kotlin JNI…?
    $ cat test.kts

    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    $ kotlin -script test.kts
    java.lang.UnsatisfiedLinkError: Test.say_hello(Ljava/lang/String;)V
    at Test.say_hello(Native Method)
    at Test.(Unknown Source)
    

    View Slide

18. void say_hello(const char* name);
    

    View Slide

19. A simple C shared library
    $ cat test.c Z1Z1Z1Z1
    

    $ clang -shared -o libtest.dylib test.c Z5Z5Z5Z5
    #include Z2Z2Z2Z2
    void say_hello(const char* name) {Z3Z3Z3Z3
    printf("Hello %s!\n", name);
    }Z4Z4Z4Z4
    

    View Slide

20. $ cat test.cpp Z1Z1Z1Z1

    #include Z2Z2Z2Z2
    #include
    extern "C" {
    void say_hello(const char* name) {Z3Z3Z3Z3
    printf("Hello %s!\n", name);
    }Z4Z4Z4Z4
    JNIEXPORT void JNICALL Java_Test_sayHello(
    JNIEnv *env, jclass cls, jstring name) {
    const char *nativeHello = env->GetStringUTFChars(name, JNI_FALSE);
    say_hello(nativeHello);
    env->ReleaseStringUTFChars(name, nativeHello);
    }
    }
    $ clang++ -shared -o libtest.dylib \
    -I${JAVA_HOME}/include -I${JAVA_HOME}/include/darwin test.cpp Z5Z5Z5Z5
    A (not so) simple JNI-compatible C++ shared library
    

    View Slide

21. JNIEXPORT void JNICALL Java_Test_sayHello(
    JNIEnv *env, jclass cls, jstring name);
    Calling our native function with Kotlin JNI
    $ cat test.kts

    external fun sayHello(name: String)
    System.loadLibrary("test")
    sayHello("Droidcon")
    $ kotlin -script test.kts
    Hello Droidcon!
    

    View Slide

22. View Slide

23. call native code from java without jni
    Google Search I’m Feeling Hopeless
    

    View Slide

24. JNA: Java Native Access
    

    View Slide

25. “JNA provides Java programs easy access
    to native shared libraries without writing
    anything but Java code - no JNI or native
    code is required.

    This functionality is comparable to C#’s
    PInvoke and Python's ctypes.”
    github.com/java-native-access/jna
    JNA: Java Native Access
    Kotlin
    Kotlin
    

    View Slide

26. JNA Features
    Functions Methods
    C JVM
    Data Types Primitives
    Structs Classes
    Function Pointers Callback Interfaces
    

    View Slide

27. package com.sun.jna
    Library
    Native
    Structure
    Callback
    Pointer
    

    View Slide

28. Base type for interfaces containing native library
    function definitions.
    interface Library
    interface TestLibrary : Library {
    fun some_method()
    }
    

    View Slide

29. Static methods for loading native libraries, JNA
    configuration, and other utility tasks.
    final class Native
    val lib = Native.loadLibrary(
    "libName",
    TestLibrary::class.java)
    lib.some_method()
    

    View Slide

30. Base type for classes that are automatically
    mapped to/from native structs.
    abstract class Structure
    class example_struct : Structure() {
    }
    @JvmField var field1: String? = null
    @JvmField var field2: String? = null
    override fun getFieldOrder()
    = listOf("field1", “field2")
    

    View Slide

31. Base type for callback interfaces to be used as
    function pointers. Requires one public method.
    interface Callback
    interface my_callback : Callback {
    fun invoke()
    }
    interface TestLibrary : Library {
    fun set_callback(cb: my_callback)
    }
    val lib = Native.loadLibrary(/* … */)
    lib.set_callback(object : my_callback {
    override fun invoke() { /* TODO */ }
    })
    

    View Slide

32. Represents a native pointer,
    used for accessing arbitrary memory.
    class Pointer
    interface TestLibrary : Library {
    fun set_memory(pointer: Pointer)
    fun get_memory(): Pointer
    }
    interface my_callback : Callback {
    fun invoke(data: Pointer)
    }
    

    View Slide

33. Represents a native pointer,
    used for accessing arbitrary memory.
    class Pointer
    Pointer#getString(offset)
    Pointer#getInt(offset)
    Pointer#getLong(offset)
    Pointer#getByteArray(offset, length)
    Pointer#read(offset, buffer, index, length)
    // etc…
    

    View Slide

34. JNA Performance
    https://github.com/java-native-access/jna/blob/master/www/FrequentlyAskedQuestions.md
    • Call overhead up to 10x slower
    

    View Slide

35. JNA Performance
    https://github.com/java-native-access/jna/blob/master/www/FrequentlyAskedQuestions.md
    • Call overhead up to 10x slower
    • Ease of programing outweighs difference
    • Build, measure, optimize
    • Incrementally rewrite using JNI if needed
    

    View Slide

36. $ cat test.c
    #include
    void say_hello(const char* name) {
    printf("Hello %s!\n", name);
    }
    

    $ clang -shared -o libtest.dylib test.c
    A simple C shared library, one last time
    

    View Slide

37. void say_hello(const char* name)
    Calling our function from Kotlin with JNA!
    

    View Slide

38. Calling our function from Kotlin with JNI…?
    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    

    View Slide

39. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    

    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    

    View Slide

40. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    external fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }
    

    View Slide

41. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    System.loadLibrary("test")
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }
    

    View Slide

42. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    val lib = Native.loadLibrary("test", TestLib::class.java)ZZZ
    say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }
    

    View Slide

43. Calling our function from Kotlin with JNA!
    import com.sun.jna.*
    fun say_hello(name: String)
    val lib = Native.loadLibrary("test", TestLib::class.java)ZZZ
    lib.say_hello("Droidcon")
    void say_hello(const char* name)
    interface TestLib: Library {
    }
    

    View Slide

44. void say_hello(const char* name)
    Calling our function from Kotlin with JNA!
    $ cat test-jna.kts
    

    import com.sun.jna.*
    interface TestLib: Library {
    fun say_hello(name: String)
    }
    val lib = Native.loadLibrary("test", TestLib::class.java)
    lib.say_hello("Droidcon")
    $ mvn dependency:get \
    -Dartifact=net.java.dev.jna:jna:4.5.0 \
    -Ddest=jna.jar
    $ kotlinc -cp jna.jar -script test-jna.kts
    Hello Droidcon!
    

    View Slide

45. My Dream App
    Androidy TV Interface
    Works with game controllers
    Loads games from cloud
    Supports all retro consoles

    Leanback Support Library
    
    Android Input APIs
    
    Google Drive/Dropbox SDKs
    ??
    

    View Slide

46. My Dream App
    Androidy TV Interface
    Works with game controllers
    Loads games from cloud
    Supports all retro consoles

    Leanback Support Library
    
    Android Input APIs
    
    Google Drive/Dropbox SDKs
    libretro + JNA
    

    View Slide

47. Learning Libretro
    Documentation
    C Headers
    Core Implementations
    Existing Frontends
    

    View Slide

48. Libretro Documentation
    bit.ly/libretro-pdf doc.libretro.com/specs/api
    

    View Slide

49. libretro.h
    • Single header file
    
    • 2178 lines
    
    • Mostly documentation comments!
    
    • 25 functions, 31 structs
    

    View Slide

50. Libretro Cores
    • “core”: An emulator that has been ported to the common libretro API.
    
    • Useful to see which APIs are actually needed.
    
    • https://github.com/libretro/snes9x/blob/master/libretro/libretro.cpp
    
    • https://github.com/libretro/Genesis-Plus-GX/blob/master/libretro/libretro.c
    

    View Slide

51. Libretro: Existing Frontends
    RetroArch
    
    C++
    
    retroarch.com
    Phoenix
    
    Qt/C++
    
    phoenix.vg
    GNOME Games
    
    C/Vala
    
    wiki.gnome.org/
    Apps/Games
    

    View Slide

52. Libretro Architecture
    Init Callbacks Load Game ROM Run Game Loop
    Waveform https://thenounproject.com/icon/1345324/

    NES Cartridge https://thenounproject.com/icon/1127992/

    Refresh https://thenounproject.com/icon/974170/
    

    View Slide

53. Libretro Architecture: Callbacks
    Video Callback: Called once per frame with a pointer to video data.
    Input Callback: Called for every supported button on every frame.
    Return true if button is pressed.
    Audio Callback: Called once per frame with a pointer to audio data.
    “Environment” Callback: Generic callback for obscure functions.
    Mostly unused, per documentation.
    

    View Slide

54. Libretro Architecture: Loading Games
    Two Options
    1. Core loads game, specify file path.
    2. Frontend loads game, specify memory pointer.
    

    View Slide

55. Libretro Architecture: Game Loop
    Libretro runs one frame at a time.
    Frontend maintains the framerate.
    

    View Slide

56. libretro + JNA
    1. Write library bindings.
    
    2. Implement callbacks.
    
    3. Call functions.
    

    View Slide

57. JNAerator
    github.com/nativelibs4java/JNAerator
    

    View Slide

58. build.gradle
    dependencies {
    implementation 'net.java.dev.jna:jna:4.5.0@aar'
    }
    

    View Slide

59. Native Libraries Go Here
    

    View Slide

60. Writing Bindings: Initialize
    void retro_init(void);
    

    View Slide

61. Writing Bindings: Initialize
    void retro_init(void);
    interface LibRetro : Library {
    fun retro_init()
    }
    LibRetro.kt
    libretro.h
    

    View Slide

62. Writing Bindings: Load Game
    bool retro_load_game(retro_game_info);
    

    View Slide

63. Writing Bindings: Load Game
    struct retro_game_info
    {AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    

    View Slide

64. Writing Bindings: Load Game
    struct retro_game_info
    {AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    LibRetro.kt
    libretro.h
    

    View Slide

65. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    }AA
    struct retro_game_info
    

    };
    LibRetro.kt
    libretro.h
    

    View Slide

66. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    }AA
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    LibRetro.kt
    libretro.h
    

    View Slide

67. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    }AAA
    override fun getFieldOrder()
    = listOf("path", "data", "size", "meta")
    LibRetro.kt
    libretro.h
    

    View Slide

68. Writing Bindings: Load Game
    struct retro_game_info
    {
    const char *path;
    const void *data;
    size_t size;
    const char *meta;
    };
    bool retro_load_game(retro_game_info);
    class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder()
    = listOf("path", "data", "size", "meta")
    }AAA
    bool retro_load_game(retro_game_info);
    interface LibRetro : Library {
    fun retro_load_game(info: retro_game_info): Boolean
    }
    LibRetro.kt
    libretro.h
    

    View Slide

69. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    

    View Slide

70. Writing Bindings: Audio Callback
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    

    View Slide

71. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }AA
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    LibRetro.kt
    libretro.h
    

    View Slide

72. Writing Bindings: Audio Callback
    void retro_set_audio_sample_batch(retro_audio_sample_batch_t);
    typedef size_t (*retro_audio_sample_batch_t)(
    const int16_t *data,
    size_t frames);
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }AA
    interface LibRetro : Library {
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    }
    LibRetro.kt
    libretro.h
    

    View Slide

73. Writing Bindings: Video Callback
    void retro_set_video_refresh(retro_video_refresh_t);
    

    View Slide

74. Writing Bindings: Video Callback
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    void retro_set_video_refresh(retro_video_refresh_t);
    

    View Slide

75. Writing Bindings: Video Callback
    void retro_set_video_refresh(retro_video_refresh_t);
    interface retro_video_refresh : Callback {
    fun invoke(
    data: Pointer,
    width: UnsignedInt,
    height: UnsignedInt,

    pitch: SizeT)
    }
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    LibRetro.kt
    libretro.h
    

    View Slide

76. Writing Bindings: Video Callback
    interface retro_video_refresh : Callback {
    fun invoke(
    data: Pointer,
    width: UnsignedInt,
    height: UnsignedInt,

    pitch: SizeT)
    }
    void retro_set_video_refresh(retro_video_refresh_t);
    typedef void (*retro_video_refresh_t) (
    const void *data,
    unsigned width,
    unsigned height,
    size_t pitch);
    interface LibRetro : Library {
    fun retro_set_video_refresh(cb: retro_video_refresh)
    }
    LibRetro.kt
    libretro.h
    

    View Slide

77. void retro_set_input_state(retro_input_state_t);
    Writing Bindings: Input State Callback
    

    View Slide

78. void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    Writing Bindings: Input State Callback
    

    View Slide

79. interface retro_input_state : Callback {
    fun invoke(
    port: UInt,
    device: UInt,
    index: UInt,
    id: UInt): Short
    }
    void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    interface LibRetro : Library {
    fun retro_set_input_state(cb: retro_input_state)
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback
    

    View Slide

80. interface retro_input_state : Callback {
    fun invoke(
    port: UInt,
    device: UInt,
    index: UInt,
    id: UInt): Short
    }
    void retro_set_input_state(retro_input_state_t);
    typedef int16_t (*retro_input_state_t)(
    unsigned port,
    unsigned device,
    unsigned index,
    unsigned id);
    interface LibRetro : Library {
    fun retro_set_input_state(cb: retro_input_state)
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback
    

    View Slide

81. #define RETRO_DEVICE_JOYPAD 1
    #define RETRO_DEVICE_ID_JOYPAD_START 3
    #define RETRO_DEVICE_ID_JOYPAD_UP 4
    #define RETRO_DEVICE_ID_JOYPAD_DOWN 5
    #define RETRO_DEVICE_ID_JOYPAD_LEFT 6
    #define RETRO_DEVICE_ID_JOYPAD_RIGHT 7
    const val RETRO_DEVICE_JOYPAD = 1
    const val RETRO_DEVICE_ID_JOYPAD_START = 3
    const val RETRO_DEVICE_ID_JOYPAD_UP = 4
    const val RETRO_DEVICE_ID_JOYPAD_DOWN = 5
    const val RETRO_DEVICE_ID_JOYPAD_LEFT = 6
    const val RETRO_DEVICE_ID_JOYPAD_RIGHT = 7
    LibRetro.kt
    libretro.h
    Writing Bindings: Input State Callback
    

    View Slide

82. void retro_run(void);
    interface LibRetro : Library {
    fun retro_run()
    }
    LibRetro.kt
    libretro.h
    Writing Bindings: Run Game Loop
    

    View Slide

83. class retro_game_info : Structure() {
    @JvmField var path: String? = null
    @JvmField var data: Pointer? = null
    @JvmField var size: SizeT = SizeT()
    @JvmField var meta: String? = null
    override fun getFieldOrder() = listOf("path", "data", "size", "meta")
    }
    interface retro_audio_sample_batch : Callback {
    fun invoke(data: Pointer, frames: SizeT): SizeT
    }
    interface retro_video_refresh : Callback {
    fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
    }
    interface retro_input_state : Callback {
    fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
    }
    interface retro_input_poll : Callback {
    fun invoke()
    }
    interface LibRetro : Library {AA
    fun retro_init()
    fun retro_load_game(game: retro_game_info): Boolean
    fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
    fun retro_set_video_refresh(cb: retro_video_refresh)
    fun retro_set_input_state(cb: retro_input_state)
    fun retro_set_input_poll(cb: retro_input_poll)
    fun retro_run()
    }BB
    

    View Slide

84. libretro + JNA
    1. Write library bindings.
    
    2. Implement callbacks.
    
    3. Call functions.
    

    View Slide

85. Implementing Callbacks: Audio
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    }AA
    }BB
    

    View Slide

86. Implementing Callbacks: Audio
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    }AA
    }BB
    

    View Slide

87. Implementing Callbacks: Audio
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    }AA
    }BB
    

    View Slide

88. Implementing Callbacks: Audio
    import android.media.AudioTrack
    private val audioTrack = AudioTrack.Builder()
    .setAudioFormat(AudioFormat.Builder()
    .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
    .setSampleRate(32040)
    .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
    .build())
    .build()
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    }AA
    }BB
    

    View Slide

89. Implementing Callbacks: Audio
    import android.media.AudioTrack
    private val audioTrack = AudioTrack.Builder()
    .setAudioFormat(AudioFormat.Builder()
    .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
    .setSampleRate(32040)
    .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO)
    .build())
    .build()
    private val audioBytesPerFrame = 4 // 16-bit stereo
    private val audioSampleBatch = object : retro_audio_sample_batch {
    override fun invoke(data: Pointer, frames: SizeT): SizeT {
    val length = frames * audioBytesPerFrame
    val buffer = data.getByteArray(0, length)
    val written = audioTrack.write(buffer, 0, buffer.size)
    audioTrack.play()
    return SizeT(written)
    }AA
    }BB
    

    View Slide

90. Implementing Callbacks: Video
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    }AA
    }BB
    

    View Slide

91. Implementing Callbacks: Video
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    }AA
    }BB
    

    View Slide

92. Implementing Callbacks: Video
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    }AA
    }BB
    

    View Slide

93. Implementing Callbacks: Video
    import android.graphics.Bitmap
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    val bitmap = Bitmap.createBitmap(
    width.toInt(),
    height.toInt(),
    Bitmap.Config.RGB_565)
    bitmap.copyPixelsFromBuffer(ByteBuffer.wrap(buffer))
    }AA
    }BB
    

    View Slide

94. Implementing Callbacks: Video
    import android.graphics.Bitmap
    private val imageView by lazy {
    findViewById(R.id.imageView)
    }
    private val videoBytesPerPixel = 2 // RGB 565
    private val videoRefresh = object : retro_video_refresh {
    override fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT) {
    val widthBytes = width * videoBytesPerPixel
    val totalBytes = width * height * videoBytesPerPixel
    val buffer = ByteArray(totalBytes)
    for (i in 0 until height.toInt()) {
    data.read(i * pitch.toLong(), buffer, i * widthBytes, widthBytes)
    }FF
    val bitmap = Bitmap.createBitmap(
    width.toInt(),
    height.toInt(),
    Bitmap.Config.RGB_565)
    bitmap.copyPixelsFromBuffer(ByteBuffer.wrap(buffer))
    imageView.post {
    imageView.setImageBitmap(bitmap)
    }
    }AA
    }BB
    

    View Slide

95. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    }BB
    }ZZ
    

    View Slide

96. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    }BB
    }ZZ
    // TODO: Multiplayer support
    if (port.toInt() != 0) {
    return 0AA
    }AA
    

    View Slide

97. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0AA
    }AA
    }BB
    }ZZ
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0JJ
    }DD
    

    View Slide

98. Implementing Callbacks: Input State
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0
    }AA
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0JJ
    }DD
    }BB
    }ZZ
    private val pressedButtons = mutableSetOf()
    

    View Slide

99. Implementing Callbacks: Input State
    private val pressedButtons = mutableSetOf()
    private val inputState = object : retro_input_state {
    override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
    if (port.toInt() != 0) {
    return 0 RR
    }AA
    if (device.toInt() != RETRO_DEVICE_JOYPAD) {
    return 0 GG
    }DD
    }BB
    }ZZ
    val isPressed = when (id.toInt()) {
    RETRO_DEVICE_ID_JOYPAD_START ->
    pressedButtons.contains(KeyEvent.KEYCODE_BUTTON_START)
    RETRO_DEVICE_ID_JOYPAD_UP ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_UP)
    RETRO_DEVICE_ID_JOYPAD_DOWN ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_DOWN)
    RETRO_DEVICE_ID_JOYPAD_LEFT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_LEFT)
    RETRO_DEVICE_ID_JOYPAD_RIGHT ->
    pressedButtons.contains(KeyEvent.KEYCODE_DPAD_RIGHT)
    else -> false
    }
    return if (isPressed) 1 else 0
    import android.view.KeyEvent
    

    View Slide

100. Implementing Callbacks: Input State
     private val inputState = object : retro_input_state {
     override fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short {
     if (port.toInt() != 0) {
     return 0 RR
     }AA
     if (device.toInt() != RETRO_DEVICE_JOYPAD) {
     return 0 GG
     }DD
     }BB
     }ZZ
     val isPressed = when (id.toInt()) {
     RETRO_DEVICE_ID_JOYPAD_START ->
     pressedButtons.contains(KeyEvent.KEYCODE_BUTTON_START)
     RETRO_DEVICE_ID_JOYPAD_UP ->
     pressedButtons.contains(KeyEvent.KEYCODE_DPAD_UP)
     RETRO_DEVICE_ID_JOYPAD_DOWN ->
     pressedButtons.contains(KeyEvent.KEYCODE_DPAD_DOWN)
     RETRO_DEVICE_ID_JOYPAD_LEFT ->
     pressedButtons.contains(KeyEvent.KEYCODE_DPAD_LEFT)
     RETRO_DEVICE_ID_JOYPAD_RIGHT ->
     pressedButtons.contains(KeyEvent.KEYCODE_DPAD_RIGHT)
     else -> false
     }
     return if (isPressed) 1 else 0
     import android.view.KeyEvent
     private val pressedButtons = mutableSetOf()
     

     View Slide

101. Implementing Callbacks: Input State
     import android.view.KeyEvent
     private val pressedButtons = mutableSetOf()
     override fun dispatchKeyEvent(event: KeyEvent): Boolean {
     }ZZ
     

     View Slide

102. Implementing Callbacks: Input State
     private val pressedButtons = mutableSetOf()
     override fun dispatchKeyEvent(event: KeyEvent): Boolean {
     }ZZ
     import android.view.KeyEvent
     super.dispatchKeyEvent(event)
     when (event.action) {
     KeyEvent.ACTION_DOWN -> pressedButtons.add(event.keyCode)
     KeyEvent.ACTION_UP -> pressedButtons.remove(event.keyCode)
     }
     return true
     

     View Slide

103. libretro + JNA
     1. Write library bindings.
     
     2. Implement callbacks.
     
     3. Call functions.
     

     View Slide

104. Calling Functions
     val lib = Native.loadLibrary(

     "snes9x_libretro_android",

     LibRetro::class.java)
     lib.retro_set_audio_sample_batch(audioSampleBatch)

     lib.retro_set_video_refresh(videoRefresh)

     lib.retro_set_input_poll(inputPoll)

     lib.retro_set_input_state(inputState)

     lib.retro_init()
     val gameInfo = retro_game_info()

     gameInfo.path = "/sdcard/allstars.smc"

     lib.retro_load_game(gameInfo)
     fixedRateTimer(period = 1000L / 60L) {

     lib.retro_run()

     }
     

     View Slide

105. View Slide

106. wat
     libc F Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 12676 (
     DEBUG F #00 pc 00000000
     F #01 pc 00010fc9 /data/app/com.codebutler.funwithnativecode-n1T9
     F #02 pc 00010c40 /data/app/com.codebutler.funwithnativecode-n1T9
     F #03 pc 000051e0 /data/app/com.codebutler.funwithnativecode-n1T9
     F #04 pc 00007c8f /data/app/com.codebutler.funwithnativecode-n1T9
     F #05 pc 0063ec67 /system/lib/libart.so (art_quick_generic_jni_tr
     F #06 pc 00638ea2 /system/lib/libart.so (art_quick_invoke_static_
     F #07 pc 00112b92 /system/lib/libart.so (_ZN3art9ArtMethod6Invoke
     F #08 pc 003231ff /system/lib/libart.so (_ZN3art11interpreter34Ar
     F #09 pc 0031bde1 /system/lib/libart.so (_ZN3art11interpreter6DoC
     F #10 pc 0061faf4 /system/lib/libart.so (MterpInvokeStatic+484)
     

     View Slide

107. wat
     

     View Slide

108. wat
     

     View Slide

109. interface retro_environment : Callback {
     fun invoke(cmd: UInt, data: Pointer): Boolean
     }EE
     LibRetro.kt
     libretro.h
     interface LibRetro : Library {
     }
     void retro_set_environment(retro_environment_t);
     typedef bool (*retro_environment_t)(unsigned cmd, void *data);
     Writing Bindings: Environment Callback
     fun retro_set_environment(cb: retro_environment)
     

     View Slide

110. class retro_game_info : Structure() {
     @JvmField var path: String? = null
     @JvmField var data: Pointer? = null
     @JvmField var size: SizeT = SizeT()
     @JvmField var meta: String? = null
     override fun getFieldOrder() = listOf("path", "data", "size", "meta")
     }
     interface retro_audio_sample_batch : Callback {
     fun invoke(data: Pointer, frames: SizeT): SizeT
     }
     interface retro_video_refresh : Callback {
     fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
     }
     interface retro_input_state : Callback {
     fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
     }
     interface retro_input_poll : Callback {
     fun invoke()
     }
     interface LibRetro : Library {AA
     fun retro_init()
     fun retro_load_game(game: retro_game_info): Boolean
     fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
     fun retro_set_video_refresh(cb: retro_video_refresh)
     fun retro_set_input_state(cb: retro_input_state)
     fun retro_set_input_poll(cb: retro_input_poll)
     fun retro_run()
     }BB
     fun retro_set_environment(cb: retro_environment)
     interface retro_environment : Callback {
     fun invoke(cmd: UInt, data: Pointer): Boolean
     }EE
     

     View Slide

111. class retro_game_info : Structure() {
     @JvmField var path: String? = null
     @JvmField var data: Pointer? = null
     @JvmField var size: SizeT = SizeT()
     @JvmField var meta: String? = null
     override fun getFieldOrder() = listOf("path", "data", "size", "meta")
     }
     interface retro_environment : Callback {
     fun invoke(cmd: UInt, data: Pointer): Boolean
     }
     interface retro_audio_sample_batch : Callback {
     fun invoke(data: Pointer, frames: SizeT): SizeT
     }
     interface retro_video_refresh : Callback {
     fun invoke(data: Pointer, width: UInt, height: UInt, pitch: SizeT)
     }
     interface retro_input_state : Callback {
     fun invoke(port: UInt, device: UInt, index: UInt, id: UInt): Short
     }
     interface retro_input_poll : Callback {
     fun invoke()
     }
     interface LibRetro : Library {AA
     fun retro_init()
     fun retro_load_game(game: retro_game_info): Boolean
     fun retro_set_environment(cb: retro_input_poll)
     fun retro_set_audio_sample_batch(cb: retro_audio_sample_batch)
     fun retro_set_video_refresh(cb: retro_video_refresh)
     fun retro_set_input_state(cb: retro_input_state)
     fun retro_set_input_poll(cb: retro_input_poll)
     fun retro_run()
     }BB
     

     View Slide

112. private val environment = object : retro_environment {
     override fun invoke(cmd: UInt, data: Pointer): Boolean = false
     }
     Implementing Callbacks: Environment
     

     View Slide

113. val lib = Native.loadLibrary(
     "snes9x_libretro_android",
     LibRetro::class.java)
     lib.retro_set_audio_sample_batch(audioSampleBatch)
     lib.retro_set_video_refresh(videoRefresh)
     lib.retro_set_input_poll(inputPoll)
     lib.retro_set_input_state(inputState)
     lib.retro_init()
     val gameInfo = retro_game_info()
     gameInfo.path = "/sdcard/allstars.smc"
     lib.retro_load_game(gameInfo)
     fixedRateTimer(period = 1000L / 60L) {
     lib.retro_run()
     }AA
     

     View Slide

114. val lib = Native.loadLibrary(
     "snes9x_libretro_android",
     LibRetro::class.java)
     lib.retro_set_audio_sample_batch(audioSampleBatch)
     lib.retro_set_video_refresh(videoRefresh)
     lib.retro_set_input_poll(inputPoll)
     lib.retro_set_input_state(inputState)
     lib.retro_init()
     val gameInfo = retro_game_info()
     gameInfo.path = "/sdcard/allstars.smc"
     lib.retro_load_game(gameInfo)
     fixedRateTimer(period = 1000L / 60L) {
     lib.retro_run()
     }AA
     lib.retro_set_environment(environment)
     

     View Slide

115. View Slide

116. View Slide

117. val lib = Native.loadLibrary(
     "snes9x_libretro_android",
     LibRetro::class.java)
     lib.retro_set_environment(environment)
     lib.retro_set_audio_sample_batch(audioSampleBatch)
     lib.retro_set_video_refresh(videoRefresh)
     lib.retro_set_input_poll(inputPoll)
     lib.retro_set_input_state(inputState)
     lib.retro_init()
     val gameInfo = retro_game_info()
     gameInfo.path = "/sdcard/allstars.smc"
     lib.retro_load_game(gameInfo)
     fixedRateTimer(period = 1000L / 60L) {
     lib.retro_run()
     }
     

     View Slide

118. val lib = Native.loadLibrary(
     LibRetro::class.java)
     lib.retro_set_environment(environment)
     lib.retro_set_audio_sample_batch(audioSampleBatch)
     lib.retro_set_video_refresh(videoRefresh)
     lib.retro_set_input_poll(inputPoll)
     lib.retro_set_input_state(inputState)
     lib.retro_init()
     val gameInfo = retro_game_info()
     lib.retro_load_game(gameInfo)
     fixedRateTimer(period = 1000L / 60L) {
     lib.retro_run()
     }
     "genesis_plus_gx_libretro_android",
     gameInfo.path = "/sdcard/sonic3.md"
     

     View Slide

119. View Slide

120. MY DREAM APP...
     

     View Slide

121. MY APP...!
     

     View Slide

122. View Slide

123. ODYSSEY
     github.com/codebutler/odyssey
     Current Features
     7 Console Systems
     Early Leanback UI
     Controller Support
     Game Covert Art
     Game Save/Restore
     WebDAV
     Future Features
     More Console Systems
     Better UI
     Local Multiplayer
     Game State Save/Restore
     Dropbox/GDrive
     Cheat Codes
     Internet Multiplayer
     Twitch/YouTube
     (Named after the Magnavox Odyssey, the first commercial home video game
     console released in 1972)
     

     View Slide

124. THANKS!
     Slides & Code bit.ly/FunWithNativeCode
     Follow Me @codebutler
     

     View Slide