$30 off During Our Annual Pro Sale. View Details »

Who supervises supervisors?

Who supervises supervisors?

Introduction to using systemd with Erlang

Łukasz Niemier

June 21, 2020
Tweet

More Decks by Łukasz Niemier

Other Decks in Programming

Transcript

  1. @hauleth 2021
    Who supervises supervisors?
    Introduction to using systemd with Erlang
    1

    View Slide

  2. • Hauleth - https://hauleth.dev

    • EEF Observability WG Member

    • OpenTelemetry Erlang Maintainer

    • Author of systemd Erlang library
    (and few other libraries)
    Who am I?
    2

    View Slide

  3. Some knowledge
    needed
    I will not explain everything, there is no time for that
    3

    View Slide

  4. systemd is ok
    but it has
    fl
    aws
    4

    View Slide

  5. • What is init system

    • How it works on the high level

    • What systemd features are interesting for us

    • How we can use these features
    5

    View Slide

  6. Erlang init
    App2
    App1 App3
    ?
    6

    View Slide

  7. *nix init system
    How your system starts
    • First process ran by your OS

    • Run with PID 1

    • When it exits, everything goes down, either by shutdown or errors out

    • Responsible for starting everything else

    • Responsible for reaping zombies (in most cases)

    • In systemd it is the system supervisor (not all inits does that)
    7

    View Slide

  8. Similarities between system init and Erlang init
    • When it dies, (virtual) machine dies

    • It is responsible for starting everything else

    • It reads job con
    fi
    guration and then starts required jobs

    • Listen on messages when to stop the (virtual) machine
    8

    View Slide

  9. Why bother?
    9

    View Slide

  10. "Top-level" process management
    • Ordering process startup

    • Restarting process when it died

    • Startup and shutdown hooks

    • Process isolation and hardening

    • Resource limits

    • System state monitoring
    10

    View Slide

  11. Notifications and watchdog
    • Informing administrator about state of the process

    • Health checks whether the system is still alive

    • Triggering restarts from within the application

    • Storing opened
    fi
    le descriptors (like sockets) between application starts
    11

    View Slide

  12. How to use it?
    12
    defmodule MyApp.Application do


    use Application


    def start(_, _) do


    children = [


    # …


    :systemd.ready()


    ]


    # …


    Supervisor.start_link(


    children,


    opts


    )


    end


    end
    [Service]


    # …


    Type=notify


    ExecStart=/path/to/rel/bin start

    View Slide

  13. Centralised log management
    journald
    • All systems send logs to single place

    • No need for manual management of log rotation

    • Keeping metadata together with logs

    • Built-in log dispatching tooling

    • Supports both stdout and direct logging via socket

    • Logs stored in binary format
    13

    View Slide

  14. How to use it?
    Version simple (well, in Erlang)
    14
    {deps, [systemd]}.

    View Slide

  15. How to use it?
    Version hard
    15
    defp deps do


    [{:systemd, "~> 0.0"}]


    end
    defmodule MyApp.Application do


    use Application


    def start(_, _) do


    :logger.add_handlers(:systemd)


    end


    end

    View Slide

  16. Lazy startup
    socket activation
    • Start application just when the request arrives

    • Keep sockets open as soon as system starts

    • Keep sockets open between application restarts

    • Use sockets from the privileged scope without superuser
    16

    View Slide

  17. How to use it?
    17
    fds = :systemd.listen_fds()


    option =


    case fds do


    [{fd, _} | _] -> {:fd, fd}


    [fd | _] -> {:fd, fd}


    _ -> {:port, default_port}


    end


    {:ok, socket} = :gen_tcp.listen(0, [option, :inet6])

    View Slide

  18. Demo time
    18

    View Slide

  19. Questions?
    19

    View Slide