NIFs in MoDern Languages

LT @ Ruby x Elixir Conf TW 2018

Transcript

1. NIFs in MoDern Languages Ryo Kajiwara, 4/27/2018 @ RubyxElixirConf Taiwan 2018
2. Self-Intro Ryo Kajiwara (ֿݪ ཾ) the IDIOT (ID + IoT) engineer Twitter: @s01
3. Elixir Web Framework developed by ACCESS, releasing today! Make your own PaaS
5. TL;DR
7. meets
9. Okay, actually what • Played around with NIFs(Native Implemented Functions) in Dlang • Implemented SHA-3 (esp. SHA3-256) in D, called from Elixir • one of NIFs' important use is cryptographic libraries
10. NIFs are dangerous • When NIFs fail, the whole VM dies • So, we want to write NIFs in a relatively "safer" language • One of the more popular options is Rust, which is supported by Rustler • Using ports is a safer way to interact with foreign languages
11. Why Dlang? • Fast execution • Memory management made simpler • Dynamic arrays • No pointers until you really need it • Strong compile-time metaprogramming
13. Okay, Let's Start Playing With Fire
14. Code at: github.com/sylph01/ nifs_in_d (code is not intended for production)
15. How to compile {_result, _errcode} = System.cmd("dmd", [ "native_lib/sha3_d.d", "-c", "-fPIC", "-of=native_obj/sha3_d.o" ], stderr_to_stdout: true) • -c: Compile only • -fPIC: Position-independent-code
16. How to compile {_result, _errcode} = System.cmd("g++", [ "native_lib/sha3.cpp", "native_obj/sha3_d.o", "-fPIC", "-shared", "-L/home/sylph01/.asdf/installs/dmd/2.079.1/dmd2/linux/lib64", "-I/home/sylph01/.asdf/installs/erlang/20.3.4/usr/include", "-lphobos2", "-pthread", "-o", "native_obj/sha3.so" ], stderr_to_stdout: true) (Paths can be omitted when included in env vars)
17. Example 1: Simple Add extern (C++) int add(int a, int b){ return a + b; } Here we use C++ as the interface between D and Erlang.
18. Example 1: Simple Add ERL_NIF_TERM add_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]){ int a = 0; int b = 0; if (!enif_get_int(env, argv[0], &a)) { return enif_make_badarg(env); } if (!enif_get_int(env, argv[1], &b)) { return enif_make_badarg(env); }
19. Example 1: Simple Add int result = add(a, b); return enif_make_int(env, result); } The function does: • Extract integer value from ERL_NIF_TERM • Call the function in D • Then make an ERL_NIF_TERM with the result
20. Example 1: Simple Add ErlNifFunc nif_funcs[] = { {"add", 2, add_nif} }; int load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) { rt_init(); return 0; } ERL_NIF_INIT(Elixir.Add, nif_funcs, load, NULL, NULL, NULL);
21. Example 2: SHA-3 extern (C++) immutable(char)* sha3_256_c(const char *s) { import std.string; return toStringz(toHex(sha3_256(fromStringz(s)))); } Different from Example 1: we are getting a string = binary in Erlang world
22. Example 2: SHA-3 inside sha3_256_nif: ERL_NIF_TERM term; unsigned char* ret = enif_make_new_binary(env, 64, &term); ... return term; } • Make a ERL_NIF_TERM, and assign a char* to indicate its binary content
23. Example 2: SHA-3 if (!enif_inspect_iolist_as_binary(env, argv[0], &input)){ return enif_make_badarg(env); } memcpy(ret, sha3_256_c(reinterpret_cast<const char* >(input.data)), 64); • Extract binary from ERL_NIF_TERM argv[0] • Then use its data to call sha3_256_c, copy its value to ret
26. c93d30bc856...d96edb matches!
27. Thoughts • Tons of boilerplate • Why use D when C++ is used as interface? • Increases boilerplate even more... • Variable-length lists in D are hard to leverage when C++ is interfacing • Next: Explore possibilities of direct NIFs in D?
28. Feedbacks welcome! GitHub: sylph01/nifs_in_d Twitter: @s01