Who supervises supervisors?

Introduction to using systemd with Erlang

Transcript

1. @hauleth 2021 Who supervises supervisors? Introduction to using systemd with Erlang 1
2. • Hauleth - https://hauleth.dev • EEF Observability WG Member • OpenTelemetry Erlang Maintainer • Author of systemd Erlang library (and few other libraries) Who am I? 2
3. Some knowledge needed I will not explain everything, there is no time for that 3
4. systemd is ok but it has fl aws 4
5. • What is init system • How it works on the high level • What systemd features are interesting for us • How we can use these features 5
6. Erlang init App2 App1 App3 ? 6
7. *nix init system How your system starts • First process ran by your OS • Run with PID 1 • When it exits, everything goes down, either by shutdown or errors out • Responsible for starting everything else • Responsible for reaping zombies (in most cases) • In systemd it is the system supervisor (not all inits does that) 7
8. Similarities between system init and Erlang init • When it dies, (virtual) machine dies • It is responsible for starting everything else • It reads job con fi guration and then starts required jobs • Listen on messages when to stop the (virtual) machine 8
9. Why bother? 9
10. "Top-level" process management • Ordering process startup • Restarting process when it died • Startup and shutdown hooks • Process isolation and hardening • Resource limits • System state monitoring 10
11. Notifications and watchdog • Informing administrator about state of the process • Health checks whether the system is still alive • Triggering restarts from within the application • Storing opened fi le descriptors (like sockets) between application starts 11
12. How to use it? 12 defmodule MyApp.Application do use Application def start(_, _) do children = [ # … :systemd.ready() ] # … Supervisor.start_link( children, opts ) end end [Service] # … Type=notify ExecStart=/path/to/rel/bin start
13. Centralised log management journald • All systems send logs to single place • No need for manual management of log rotation • Keeping metadata together with logs • Built-in log dispatching tooling • Supports both stdout and direct logging via socket • Logs stored in binary format 13
14. How to use it? Version simple (well, in Erlang) 14 {deps, [systemd]}.
15. How to use it? Version hard 15 defp deps do [{:systemd, "~> 0.0"}] end defmodule MyApp.Application do use Application def start(_, _) do :logger.add_handlers(:systemd) end end
16. Lazy startup socket activation • Start application just when the request arrives • Keep sockets open as soon as system starts • Keep sockets open between application restarts • Use sockets from the privileged scope without superuser 16
17. How to use it? 17 fds = :systemd.listen_fds() option = case fds do [{fd, _} | _] -> {:fd, fd} [fd | _] -> {:fd, fd} _ -> {:port, default_port} end {:ok, socket} = :gen_tcp.listen(0, [option, :inet6])
18. Demo time 18
19. Questions? 19