Init: then and now

Init: then and now

A story of starting systems up

2fcc875f98607b3007909fe4be99160d?s=128

Pierre-Yves Ritschard

November 07, 2017
Tweet

Transcript

  1. Init: then and now A story of starting systems up

    1
  2. Grüß Gott Pierre-Yves Ritschard CTO & Co-founder at Unix since

    96 Open-Source developer at OpenBSD, Riemann, Collectd, and more @pyr Exoscale 2 . 1
  3. Exoscale Infrastructure as a service Part of A1 Digital Zones

    in Frankfurt, Vienna, Zürich, Geneva 3 . 1
  4. How do systems start? 4 . 1

  5. The rmware 5 . 1

  6. The rmware 6 . 1

  7. The rmware Exposes BUS access Rudimentary way to communicate list

    devices Looks for a bootloader in on devices Usually very basic environment (sometimes 16- bit) 7 . 1
  8. The bootloader 8 . 1

  9. The bootloader 9 . 1

  10. The bootloader An intermediate mini-OS Rudimentary le-system support Prepares environment

    to start kernel 10 . 1
  11. The kernel 11 . 1

  12. The kernel 12 . 1

  13. The kernel Discovers devices on Buses Prepares environment to run

    process(es) using a common format Exposes standard facilities for programs: Abstracted I/O: open(2) Memory management: mmap(2), malloc(3) Process handling: fork(2), signal(2) Network abstractions: socket(2), bind(2), connect(2) 13 . 1
  14. Init 14 . 1

  15. Init First process started by the kernel Runs boot sequence

    Mounts lesystems Starts the network Starts essential subsystems: syslog, ntpd, cron Starts background daemons: ssh, smtpd Root of process tree 15 . 1
  16. Multi-User mode 16 . 1

  17. Multi-User mode 17 . 1

  18. Multi-User mode All facilities up and running Ready to run

    user programs 18 . 1
  19. Our focus today 19 . 1

  20. A small detour: the process tree There is a process

    hierarchy in Unix The kernel runs only one user process 20 . 1
  21. A small detour: the process tree 21 . 1

  22. A small detour: the process tree 22 . 1

  23. Init duties Mount lesystems, start everything Stay around to watch

    over hierarchy 23 . 1
  24. Init resiliency What happens when init crashes? 24 . 1

  25. Init resiliency 25 . 1

  26. Init simpli ed How would you? Mount lesystems Start the

    network Start essential subsystems: syslog, ntpd, cron Start background daemons: ssh, smtpd 26 . 1
  27. Standard Init swapon -a umount -a >/dev/null 2>&1 mount -a

    -t nonfs . /etc/rc.conf sh /etc/netstart if [ X${rwhod} = X"YES" ]; then echo -n ' rwhod'; rwhod fi if [ X${lpd} = X"YES" ]; then echo -n ' printer'; lpd fi . /etc/rc.local 27 . 1
  28. Standard Init #define _PATH_BSHELL "/bin/sh" #define _PATH_RUNCOM "/etc/rc" /* ...

    */ execv(_PATH_BSHELL, argv); /* ... */ 28 . 1
  29. Init ow 29 . 1

  30. Are we there? How does the system stop? How do

    I restart or stop services? 30 . 1
  31. Stopping the system /etc/rc.shutdown 31 . 1

  32. Stopping or restarting services Stop: pkill <service> Restart: pkill -HUP

    <service> 32 . 1
  33. This isn't ideal It's hard to keep track of startup

    order Plenty of services didn't react well to standard signals No way to easily gather service status No way to ensure a critical service stays up This puts a lot on application packagers Especially on non-standardized systems Hello ! 33 . 1
  34. Case in point: daemons int daemon(void) { switch (fork()) {

    case -1: return (-1); case 0: break; default: _exit(0); } if (setsid() == -1) return (-1); (void)chdir("/"); (void)close(STDIN_FILENO); (void)close(STDOUT_FILENO); (void)close(STDERR_FILENO); return (0); } 34 . 1
  35. Case in point: daemons How do I know the child's

    PID? How do I keep track of service availability 35 . 1
  36. Common things applications do 36 . 1

  37. More things applications must do 37 . 1

  38. A better world? 38 . 1

  39. The init landscape System V Upstart SystemD 39 . 1

  40. System V You all know it /etc/init.d/rc5.d/S99blargh Introduces runlevels Enforces

    the concept of service startup, status, and shutdown. inittab(5) to de ne runlevels as state transitions Still leaves much to be desired 40 . 1
  41. Ubuntu Upstart Simple DSL Event based start on (net-device-up and

    local- filesystems) Support for environment, logging, and PID tracking 41 . 1
  42. Ubuntu Upstart description "Warp agent" start on runlevel [2345] stop

    on runlevel [!2345] respawn respawn limit 5 60 limit nofile 8192 8192 pre-start script [ -x "/usr/sbin/warp-agent" ] || exit 0 [ -r "/etc/warp/agent.json" ] || exit 0 end script exec /usr/sbin/warp-agent /etc/warp/agent.json 42 . 1
  43. Ubuntu Upstart 43 . 1

  44. Ubuntu Upstart Drawbacks Very brittle job supervision Left the process

    in un xable states on a regular basis 44 . 1
  45. SystemD DSL based Builds a dependency graph Support for environment,

    logging, PID tracking Opt-in support for resource constraints, rewalling 45 . 1
  46. SystemD [Unit] Description="Warp agent" ConditionPathExists=/usr/sbin/warp-agent ConditionPathExists=/etc/warp/agent.json [Service] ExecStart=/usr/sbin/warp-agent /etc/warp/agent.json LimitNOFILE=8192

    [Install] WantedBy=multi-user.target 46 . 1
  47. SystemD 47 . 1

  48. SystemD drawbacks Hard to avoid heated discussions Wide scope means

    more room for error Mixed focus on desktop & servers can be off-puting for admins Heavily coupled to DBUS That giant, little-known attack vector on your system 48 . 1
  49. Shepperd (define nginx (make <service> #:provides '(nginx web-server) #:start (make-forkexec-constructor

    (list "nginx")) #:stop (make-kill-destructor))) (register-services nginx) 49 . 1
  50. Plenty more Daemontools SMF OpenRC BSD rc.d runit 50 .

    1
  51. Plenty more http://blog.darknedgy.net/technology/2015/09/05/0/ 51 . 1

  52. Thanks! Questions? We're hiring! 52 . 1