NIFs in MoDern Languages

404139d782ec666acea93dffc86e089f?s=47 sylph01
April 27, 2018

NIFs in MoDern Languages

LT @ Ruby x Elixir Conf TW 2018

404139d782ec666acea93dffc86e089f?s=128

sylph01

April 27, 2018
Tweet

Transcript

  1. NIFs in MoDern Languages Ryo Kajiwara, 4/27/2018 @ RubyxElixirConf Taiwan

    2018
  2. Self-Intro Ryo Kajiwara (ֿݪ ཾ) the IDIOT (ID + IoT)

    engineer Twitter: @s01
  3. Elixir Web Framework developed by ACCESS, releasing today! Make your

    own PaaS
  4. None
  5. TL;DR

  6. None
  7. meets

  8. None
  9. Okay, actually what • Played around with NIFs(Native Implemented Functions)

    in Dlang • Implemented SHA-3 (esp. SHA3-256) in D, called from Elixir • one of NIFs' important use is cryptographic libraries
  10. NIFs are dangerous • When NIFs fail, the whole VM

    dies • So, we want to write NIFs in a relatively "safer" language • One of the more popular options is Rust, which is supported by Rustler • Using ports is a safer way to interact with foreign languages
  11. Why Dlang? • Fast execution • Memory management made simpler

    • Dynamic arrays • No pointers until you really need it • Strong compile-time metaprogramming
  12. None
  13. Okay, Let's Start Playing With Fire

  14. Code at: github.com/sylph01/ nifs_in_d (code is not intended for production)

  15. How to compile {_result, _errcode} = System.cmd("dmd", [ "native_lib/sha3_d.d", "-c",

    "-fPIC", "-of=native_obj/sha3_d.o" ], stderr_to_stdout: true) • -c: Compile only • -fPIC: Position-independent-code
  16. How to compile {_result, _errcode} = System.cmd("g++", [ "native_lib/sha3.cpp", "native_obj/sha3_d.o",

    "-fPIC", "-shared", "-L/home/sylph01/.asdf/installs/dmd/2.079.1/dmd2/linux/lib64", "-I/home/sylph01/.asdf/installs/erlang/20.3.4/usr/include", "-lphobos2", "-pthread", "-o", "native_obj/sha3.so" ], stderr_to_stdout: true) (Paths can be omitted when included in env vars)
  17. Example 1: Simple Add extern (C++) int add(int a, int

    b){ return a + b; } Here we use C++ as the interface between D and Erlang.
  18. Example 1: Simple Add ERL_NIF_TERM add_nif(ErlNifEnv* env, int argc, const

    ERL_NIF_TERM argv[]){ int a = 0; int b = 0; if (!enif_get_int(env, argv[0], &a)) { return enif_make_badarg(env); } if (!enif_get_int(env, argv[1], &b)) { return enif_make_badarg(env); }
  19. Example 1: Simple Add int result = add(a, b); return

    enif_make_int(env, result); } The function does: • Extract integer value from ERL_NIF_TERM • Call the function in D • Then make an ERL_NIF_TERM with the result
  20. Example 1: Simple Add ErlNifFunc nif_funcs[] = { {"add", 2,

    add_nif} }; int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) { rt_init(); return 0; } ERL_NIF_INIT(Elixir.Add, nif_funcs, load, NULL, NULL, NULL);
  21. Example 2: SHA-3 extern (C++) immutable(char)* sha3_256_c(const char *s) {

    import std.string; return toStringz(toHex(sha3_256(fromStringz(s)))); } Different from Example 1: we are getting a string = binary in Erlang world
  22. Example 2: SHA-3 inside sha3_256_nif: ERL_NIF_TERM term; unsigned char* ret

    = enif_make_new_binary(env, 64, &term); ... return term; } • Make a ERL_NIF_TERM, and assign a char* to indicate its binary content
  23. Example 2: SHA-3 if (!enif_inspect_iolist_as_binary(env, argv[0], &input)){ return enif_make_badarg(env); }

    memcpy(ret, sha3_256_c(reinterpret_cast<const char* >(input.data)), 64); • Extract binary from ERL_NIF_TERM argv[0] • Then use its data to call sha3_256_c, copy its value to ret
  24. None
  25. None
  26. c93d30bc856...d96edb matches!

  27. Thoughts • Tons of boilerplate • Why use D when

    C++ is used as interface? • Increases boilerplate even more... • Variable-length lists in D are hard to leverage when C++ is interfacing • Next: Explore possibilities of direct NIFs in D?
  28. Feedbacks welcome! GitHub: sylph01/nifs_in_d Twitter: @s01