What actually is the BEAM?

At this conference, we're surrounded by the word "BEAM". A quick exploration will reveal it has to do with how Erlang code is executed and is the secret sauce uniting all of us - Erlang, Elixir, LFE, Efene and others. But do you really know what it is? In this talk, Michal will carefully explore the BEAM, both the runtime and the compiler. We'll see how they relate, what particular strengths this gives to our code and how we can leverage it from the applications we write every day in Erlang and Elixir. Finally, we'll see how the BEAM is similar and different to other environments and languages like Java's JVM, C#'s CLR or Lua.

Transcript

1. @michalmuskala
   WHAT ACTUALLY IS THE BEAM?
   Amsterdam, 30.11.2018
   

   View Slide

2. MICHAŁ MUSKAŁA
   Elixir & Ecto core team
   http://michal.muskala.eu/
   https://github.com/michalmuskala/
   @michalmuskala
   

   View Slide

3. @michalmuskala
   GLOSSARY
   • Erlang
   • Erlang/OTP
   • BEAM
   • ERTS
   • Elixir
   

   View Slide

4. @michalmuskala
   1. THE VM
   

   View Slide

5. @michalmuskala
   QUICK HISTORY OF ERLANG
   • Prolog library (1986)
   • Joe’s Abstract Machine (1989)
   • Turbo Erlang Abstract Machine
   • Bogdan’s Erlang Abstract Machine
   • Virding’s Erlang Engine
   • Björn’s Erlang Abstract Machine
   

   View Slide

6. @michalmuskala
   OTHER IMPLEMENTATIONS
   • https://github.com/trifork/erjang
   • https://github.com/cloudozer/ling - Erlang on Xen
   

   View Slide

7. @michalmuskala
   WHAT IS A VIRTUAL MACHINE?
   

   View Slide

8. @michalmuskala
   STACK VM VS REGISTER VM
   Virtual Machine Showdown: Stack vs Registers

   https://www.usenix.org/legacy/events/vee05/full_papers/p153-yunhe.pdf
   

   View Slide

9. https://markfaction.wordpress.com/2012/07/15/stack-based-vs-register-based-virtual-machine-architecture-and-the-dalvik-vm/
   STACK VS REGISTER VM
   PUSH 20
   PUSH 7
   ADD
   

   View Slide

10. https://markfaction.wordpress.com/2012/07/15/stack-based-vs-register-based-virtual-machine-architecture-and-the-dalvik-vm/
    STACK VS REGISTER VM
    ADD R3 R1 R2
    ADD R3 20 7
    or
    

    View Slide

11. @michalmuskala
    BEAM
    • Register-based VM
    • Threaded emulator
    • Per-process GC with shared 

    large-object heap
    • Tail recursion
    • First-class closures
    • Immutable data
    • No custom data types
    • low-level messaging primitives
    • NIFs/BIFs
    • Tracing
    

    View Slide

12. View Slide

13. @michalmuskala
    PAM
    

    View Slide

14. @michalmuskala
    2. THE BYTECODE
    

    View Slide

15. @michalmuskala
    BYTECODE
    • 1024 general X registers
    • stack slots referred to as Y registers
    • exchange format between the compiler and the runtime system
    

    View Slide

16. • erlc -S module.erl
    • mix decompile 

    https: //github.com/michalmuskala/decompile
    • :erts_debug.df/1,2,3
    

    View Slide

17. LITERALS
    test() -> [1, 2, #{a => 5}, foo].
    {function, test, 0, 2}.
    {label,2}.
    {move,{literal,[1,2, #{a => 5},foo]},{x,0}}.
    return.
    

    View Slide

18. STACK ALLOCATION & TAIL RECURSION
    defp id(x),
    do: x
    def test(a, b),
    do: id(id(a) + id(b))
    {function, test, 2, 12}.
    {label,12}.
    {allocate,1,2}.
    {move,{x,1},{y,0}}.
    {call,1,{f,10}}.
    {move,{x,0},{x,1}}.
    {move,{y,0},{x,0}}.
    {move,{x,1},{y,0}}.
    {call,1,{f,10}}.
    {gc_bif,'+',{f,0},1,[{y,0},{x,0}],{x,0}}.
    {call_last,1,{f,10},1}.
    

    View Slide

19. MULTI-WAY BRANCH
    def test(x) do
    case x do
    0 -> "zero"
    1 -> "one"
    2 -> "two"
    end
    end
    {function, test, 1, 10}.
    {label,10}.
    {test,is_integer,{f,14},[{x,0}]}.
    {select_val,{x,0},{f,14},
    {list,[{integer,0},{f,11},
    {integer,1},{f,12},
    {integer,2},{f,13}]}}.
    {label,11}.
    {move,{literal, <<"zero" >>},{x,0}}.
    return.
    {label,12}.
    {move,{literal, <<"one" >>},{x,0}}.
    return.
    {label,13}.
    {move,{literal, <<"two" >>},{x,0}}.
    return.
    {label,14}.
    {case_end,{x,0}}.
    

    View Slide

20. @michalmuskala
    THE LOADER
    • maps .beam instructions to VM instructions
    • fuses some common instruction sequences
    • specialises instructions working on constants or registers
    • ops.tab and beam_makeops
    

    View Slide

21. @michalmuskala
    THE LOADER
    move S=cxy x ==0 | call Ar P=f => move_call S P
    i_increment rxy W t d
    i_plus x xy j? t d
    i_plus s s j? t d
    new_map d t I
    i_new_small_map_lit d t q
    update_map_assoc s d t I
    update_map_exact j? s d t I
    

    View Slide

22. LITERALS
    test() -> [1, 2, #{a => 5}, foo].
    00000000582FB1E8: i_func_info_IaaI 0 test test 0
    00000000582FB210: move_return_c [1,2, #{a =>5},foo]
    

    View Slide

23. STACK ALLOCATION & TAIL RECURSION
    defp id(x),
    do: x
    def test(a, b),
    do: id(id(a) + id(b))
    000000001DEBA7B8: i_func_info_IaaI 0 'Elixir.Test' test 2
    000000001DEBA7E0: allocate_tt 1 2
    000000001DEBA7E8: move_xy x(1) y(0)
    000000001DEBA7F0: i_call_f 'Elixir.Test':id/1
    000000001DEBA7F8: swap_xy x(0) y(0)
    000000001DEBA800: i_call_f 'Elixir.Test':id/1
    000000001DEBA808: i_plus_ssjtd y(0) x(0) j(0) 1 x(0)
    000000001DEBA830: i_call_last_fQ 'Elixir.Test':id/1 1
    

    View Slide

24. MULTI-WAY BRANCH
    def test(x) do
    case x do
    0 -> "zero"
    1 -> "one"
    2 -> "two"
    end
    end 0000000014758100: i_func_info_IaaI 0 'Elixir.Test' test 1
    0000000014758128: i_jump_on_val_zero_xfI x(0) f(0000000014758178) 3
    f(0000000014758148) f(0000000014758168) f(0000000014758158)
    0000000014758148: move_return_c <<"zero" >>
    0000000014758158: move_return_c <<"two" >>
    0000000014758168: move_return_c <<"one" >>
    0000000014758178: case_end_x x(0)
    

    View Slide

25. @michalmuskala
    3. THE COMPILER
    

    View Slide

26. @michalmuskala
    THE ROAD FROM .EX TO .BEAM
    Elixir Source
    Elixir AST
    Erlang Abstract Format
    Core Erlang
    Kernel Erlang
    BEAM SSA BEAM
    Erlang Source
    

    View Slide

27. ERLANG
    -module(test).
    -compile(export_all).
    test(sum, X, Y) -> X + Y;
    test(diff, X, Y) -> X - Y.
    

    View Slide

28. CORE ERLANG
    module 'test' ['module_info'/0, 'module_info'/1,'test'/3]
    attributes [
    'file' = [{[116|[101|[115|[116|[46|[101|[114|[108]]]]]]]],1}],
    'compile' = ['export_all']]
    'test'/3 =
    fun (_2,_1,_0) ->
    case <_2,_1,_0> of
    <'sum',X,Y> when 'true' -> call 'erlang':'+'(X, Y)
    <'diff',X,Y> when 'true' -> call 'erlang':'-'(X, Y)
    <_5,_4,_3> when 'true' -> primop ‘match_fail'({'function_clause',_5,_4,_3})
    end
    'module_info'/0 =
    fun () -> call 'erlang':'get_module_info'('test')
    'module_info'/1 =
    fun (_0) -> call 'erlang':'get_module_info'('test', _0)
    end
    

    View Slide

29. KERNEL ERLANG
    fdef 'test'/3(_2, _1, _0) =
    match _2,_1,_0
    alt
    select _2
    type k_atom
    'diff' ->
    do bif (remote 'erlang':'-'/2)(_1, _0) >> <_12>
    then <<_12 >>
    'sum' ->
    do bif (remote 'erlang':'+'/2)(_1, _0) >> <_11>
    then <<_11 >>
    do [_0] >> <_8> then do [_1|_8] >> <_9> then do [_2|_9] >> <_10>
    then enter (remote 'erlang':'error'/2)('function_clause', _10)
    end >> <>
    end
    

    View Slide

30. BEAM
    {function, test, 3, 2}.
    {label,1}.
    {func_info,{atom,test},{atom,test},3}.
    {label,2}.
    {test,is_atom,{f,1},[{x,0}]}.
    {select_val,{x,0},{f,1},{list,[{atom,diff},{f,3},{atom,sum},{f,4}]}}.
    {label,3}.
    {gc_bif,’-',{f,0},3,[{x,1},{x,2}],{x,0}}.
    return.
    {label,4}.
    {gc_bif,'+',{f,0},3,[{x,1},{x,2}],{x,0}}.
    return.
    

    View Slide

31. @michalmuskala
    BINARY PATTERN MATCHING
    89: bs_put_integer/5
    90: bs_put_binary/5
    91: bs_put_float/5
    92: bs_put_string/2
    109: bs_init2/6
    111: bs_add/5
    116: bs_start_match2/5
    117: bs_get_integer2/7
    118: bs_get_float2/7
    119: bs_get_binary2/7
    120: bs_skip_bits2/5
    121: bs_test_tail2/3
    122: bs_save2/2
    123: bs_restore2/2
    130: bs_context_to_binary/1
    131: bs_test_unit/3
    132: bs_match_string/4
    133: bs_init_writable/0
    134: bs_append/8
    135: bs_private_append/6
    137: bs_init_bits/6
    138: bs_get_utf8/5
    139: bs_skip_utf8/4
    140: bs_get_utf16/5
    141: bs_skip_utf16/4
    142: bs_get_utf32/5
    143: bs_skip_utf32/4
    144: bs_utf8_size/3
    145: bs_put_utf8/3
    146: bs_utf16_size/3
    147: bs_put_utf16/3
    148: bs_put_utf32/3
    165: bs_get_tail/3
    166: bs_start_match3/4
    167: bs_get_position/3
    168: bs_set_position/2
    

    View Slide

32. @michalmuskala
    HIPE
    

    View Slide

33. @michalmuskala
    4. THE COMMUNITY
    

    View Slide

34. @michalmuskala
    THE FUTURE
    

    View Slide

35. @michalmuskala
    WHAT DOES IT MEAN FOR ME?
    

    View Slide

36. View Slide

37. @michalmuskala
    YOU CAN PLAY WITH IT TOO
    • http://blog.erlang.org/
    • http://blog.erlang.org/compiler-time-option/ - “Exploring the Compiler Using the 'time' Option”
    • http://blog.erlang.org/compiler-lost-in-translation/ - “Lost in Translation (Exploring the Compiler's Front End)”
    • http://blog.erlang.org/core-erlang-by-example/ - “Core Erlang by Example”
    • https://github.com/happi/theBeamBook - “The BEAM Book”
    • http://beam-wisdoms.clau.se/en/latest/ - “BEAM Wisdoms”

    

    View Slide

38. @michalmuskala
    WHAT ACTUALLY IS THE BEAM?
    Amsterdam, 30.11.2018
    

    View Slide