NIFs in MoDern Languages

LT @ Ruby x Elixir Conf TW 2018

Transcript

1. NIFs in MoDern
   Languages
   Ryo Kajiwara, 4/27/2018 @
   RubyxElixirConf Taiwan 2018
   

   View Slide

2. Self-Intro
   Ryo Kajiwara (ֿݪ ཾ)
   the IDIOT (ID + IoT)
   engineer
   Twitter: @s01
   

   View Slide

3. Elixir Web
   Framework
   developed by
   ACCESS,
   releasing today!
   Make your own
   PaaS
   

   View Slide

4. View Slide

5. TL;DR
   

   View Slide

6. View Slide

7. meets
   

   View Slide

8. View Slide

9. Okay, actually what
   • Played around with NIFs(Native Implemented
   Functions) in Dlang
   • Implemented SHA-3 (esp. SHA3-256) in D, called
   from Elixir
   • one of NIFs' important use is cryptographic
   libraries
   

   View Slide

10. NIFs are dangerous
    • When NIFs fail, the whole VM dies
    • So, we want to write NIFs in a relatively "safer"
    language
    • One of the more popular options is Rust, which is
    supported by Rustler
    • Using ports is a safer way to interact with foreign
    languages
    

    View Slide

11. Why Dlang?
    • Fast execution
    • Memory management made simpler
    • Dynamic arrays
    • No pointers until you really need it
    • Strong compile-time metaprogramming
    

    View Slide

12. View Slide

13. Okay, Let's
    Start Playing
    With Fire
    

    View Slide

14. Code at:
    github.com/sylph01/
    nifs_in_d
    (code is not intended for production)
    

    View Slide

15. How to compile
    {_result, _errcode} = System.cmd("dmd",
    [
    "native_lib/sha3_d.d",
    "-c",
    "-fPIC",
    "-of=native_obj/sha3_d.o"
    ], stderr_to_stdout: true)
    • -c: Compile only
    • -fPIC: Position-independent-code
    

    View Slide

16. How to compile
    {_result, _errcode} = System.cmd("g++",
    [
    "native_lib/sha3.cpp",
    "native_obj/sha3_d.o",
    "-fPIC",
    "-shared",
    "-L/home/sylph01/.asdf/installs/dmd/2.079.1/dmd2/linux/lib64",
    "-I/home/sylph01/.asdf/installs/erlang/20.3.4/usr/include",
    "-lphobos2",
    "-pthread",
    "-o", "native_obj/sha3.so"
    ], stderr_to_stdout: true)
    (Paths can be omitted when included in env vars)
    

    View Slide

17. Example 1: Simple Add
    extern (C++) int add(int a, int b){
    return a + b;
    }
    Here we use C++ as the interface between D and
    Erlang.
    

    View Slide

18. Example 1: Simple Add
    ERL_NIF_TERM add_nif(ErlNifEnv* env, int argc,
    const ERL_NIF_TERM argv[]){
    int a = 0;
    int b = 0;
    if (!enif_get_int(env, argv[0], &a)) {
    return enif_make_badarg(env);
    }
    if (!enif_get_int(env, argv[1], &b)) {
    return enif_make_badarg(env);
    }
    

    View Slide

19. Example 1: Simple Add
    int result = add(a, b);
    return enif_make_int(env, result);
    }
    The function does:
    • Extract integer value from ERL_NIF_TERM
    • Call the function in D
    • Then make an ERL_NIF_TERM with the result
    

    View Slide

20. Example 1: Simple Add
    ErlNifFunc nif_funcs[] = {
    {"add", 2, add_nif}
    };
    int load(ErlNifEnv* env, void** priv_data,
    ERL_NIF_TERM load_info) {
    rt_init();
    return 0;
    }
    ERL_NIF_INIT(Elixir.Add, nif_funcs, load,
    NULL, NULL, NULL);
    

    View Slide

21. Example 2: SHA-3
    extern (C++) immutable(char)* sha3_256_c(const char *s) {
    import std.string;
    return toStringz(toHex(sha3_256(fromStringz(s))));
    }
    Different from Example 1: we are getting a string =
    binary in Erlang world
    

    View Slide

22. Example 2: SHA-3
    inside sha3_256_nif:
    ERL_NIF_TERM term;
    unsigned char* ret =
    enif_make_new_binary(env, 64, &term);
    ...
    return term;
    }
    • Make a ERL_NIF_TERM, and assign a char* to
    indicate its binary content
    

    View Slide

23. Example 2: SHA-3
    if (!enif_inspect_iolist_as_binary(env, argv[0], &input)){
    return enif_make_badarg(env);
    }
    memcpy(ret,
    sha3_256_c(reinterpret_cast(input.data)),
    64);
    • Extract binary from ERL_NIF_TERM argv[0]
    • Then use its data to call sha3_256_c, copy its value
    to ret
    

    View Slide

24. View Slide

25. View Slide

26. c93d30bc856...d96edb
    matches!
    

    View Slide

27. Thoughts
    • Tons of boilerplate
    • Why use D when C++ is used as interface?
    • Increases boilerplate even more...
    • Variable-length lists in D are hard to leverage
    when C++ is interfacing
    • Next: Explore possibilities of direct NIFs in D?
    

    View Slide

28. Feedbacks welcome!
    GitHub: sylph01/nifs_in_d
    Twitter: @s01
    

    View Slide