Init: then and now - Speaker Deck

Slide 1

Slide 1 text

Init: then and now A story of starting systems up 1

Slide 2

Slide 2 text

Grüß Gott Pierre-Yves Ritschard CTO & Co-founder at Unix since 96 Open-Source developer at OpenBSD, Riemann, Collectd, and more @pyr Exoscale 2 . 1

Slide 3

Slide 3 text

Exoscale Infrastructure as a service Part of A1 Digital Zones in Frankfurt, Vienna, Zürich, Geneva 3 . 1

Slide 4

Slide 4 text

How do systems start? 4 . 1

Slide 5

Slide 5 text

The rmware 5 . 1

Slide 6

Slide 6 text

The rmware 6 . 1

Slide 7

Slide 7 text

The rmware Exposes BUS access Rudimentary way to communicate list devices Looks for a bootloader in on devices Usually very basic environment (sometimes 16- bit) 7 . 1

Slide 8

Slide 8 text

The bootloader 8 . 1

Slide 9

Slide 9 text

The bootloader 9 . 1

Slide 10

Slide 10 text

The bootloader An intermediate mini-OS Rudimentary le-system support Prepares environment to start kernel 10 . 1

Slide 11

Slide 11 text

The kernel 11 . 1

Slide 12

Slide 12 text

The kernel 12 . 1

Slide 13

Slide 13 text

The kernel Discovers devices on Buses Prepares environment to run process(es) using a common format Exposes standard facilities for programs: Abstracted I/O: open(2) Memory management: mmap(2), malloc(3) Process handling: fork(2), signal(2) Network abstractions: socket(2), bind(2), connect(2) 13 . 1

Slide 14

Slide 14 text

Init 14 . 1

Slide 15

Slide 15 text

Init First process started by the kernel Runs boot sequence Mounts lesystems Starts the network Starts essential subsystems: syslog, ntpd, cron Starts background daemons: ssh, smtpd Root of process tree 15 . 1

Slide 16

Slide 16 text

Multi-User mode 16 . 1

Slide 17

Slide 17 text

Multi-User mode 17 . 1

Slide 18

Slide 18 text

Multi-User mode All facilities up and running Ready to run user programs 18 . 1

Slide 19

Slide 19 text

Our focus today 19 . 1

Slide 20

Slide 20 text

A small detour: the process tree There is a process hierarchy in Unix The kernel runs only one user process 20 . 1

Slide 21

Slide 21 text

A small detour: the process tree 21 . 1

Slide 22

Slide 22 text

A small detour: the process tree 22 . 1

Slide 23

Slide 23 text

Init duties Mount lesystems, start everything Stay around to watch over hierarchy 23 . 1

Slide 24

Slide 24 text

Init resiliency What happens when init crashes? 24 . 1

Slide 25

Slide 25 text

Init resiliency 25 . 1

Slide 26

Slide 26 text

Init simpli ed How would you? Mount lesystems Start the network Start essential subsystems: syslog, ntpd, cron Start background daemons: ssh, smtpd 26 . 1

Slide 27

Slide 27 text

Standard Init swapon -a umount -a >/dev/null 2>&1 mount -a -t nonfs . /etc/rc.conf sh /etc/netstart if [ X${rwhod} = X"YES" ]; then echo -n ' rwhod'; rwhod fi if [ X${lpd} = X"YES" ]; then echo -n ' printer'; lpd fi . /etc/rc.local 27 . 1

Slide 28

Slide 28 text

Standard Init #define _PATH_BSHELL "/bin/sh" #define _PATH_RUNCOM "/etc/rc" /* ... */ execv(_PATH_BSHELL, argv); /* ... */ 28 . 1

Slide 29

Slide 29 text

Init ow 29 . 1

Slide 30

Slide 30 text

Are we there? How does the system stop? How do I restart or stop services? 30 . 1

Slide 31

Slide 31 text

Stopping the system /etc/rc.shutdown 31 . 1

Slide 32

Slide 32 text

Stopping or restarting services Stop: pkill Restart: pkill -HUP 32 . 1

Slide 33

Slide 33 text

This isn't ideal It's hard to keep track of startup order Plenty of services didn't react well to standard signals No way to easily gather service status No way to ensure a critical service stays up This puts a lot on application packagers Especially on non-standardized systems Hello ! 33 . 1

Slide 34

Slide 34 text

Case in point: daemons int daemon(void) { switch (fork()) { case -1: return (-1); case 0: break; default: _exit(0); } if (setsid() == -1) return (-1); (void)chdir("/"); (void)close(STDIN_FILENO); (void)close(STDOUT_FILENO); (void)close(STDERR_FILENO); return (0); } 34 . 1

Slide 35

Slide 35 text

Case in point: daemons How do I know the child's PID? How do I keep track of service availability 35 . 1

Slide 36

Slide 36 text

Common things applications do 36 . 1

Slide 37

Slide 37 text

More things applications must do 37 . 1

Slide 38

Slide 38 text

A better world? 38 . 1

Slide 39

Slide 39 text

The init landscape System V Upstart SystemD 39 . 1

Slide 40

Slide 40 text

System V You all know it /etc/init.d/rc5.d/S99blargh Introduces runlevels Enforces the concept of service startup, status, and shutdown. inittab(5) to de ne runlevels as state transitions Still leaves much to be desired 40 . 1

Slide 41

Slide 41 text

Ubuntu Upstart Simple DSL Event based start on (net-device-up and local- filesystems) Support for environment, logging, and PID tracking 41 . 1

Slide 42

Slide 42 text

Ubuntu Upstart description "Warp agent" start on runlevel [2345] stop on runlevel [!2345] respawn respawn limit 5 60 limit nofile 8192 8192 pre-start script [ -x "/usr/sbin/warp-agent" ] || exit 0 [ -r "/etc/warp/agent.json" ] || exit 0 end script exec /usr/sbin/warp-agent /etc/warp/agent.json 42 . 1

Slide 43

Slide 43 text

Ubuntu Upstart 43 . 1

Slide 44

Slide 44 text

Ubuntu Upstart Drawbacks Very brittle job supervision Left the process in un xable states on a regular basis 44 . 1

Slide 45

Slide 45 text

SystemD DSL based Builds a dependency graph Support for environment, logging, PID tracking Opt-in support for resource constraints, rewalling 45 . 1

Slide 46

Slide 46 text

SystemD [Unit] Description="Warp agent" ConditionPathExists=/usr/sbin/warp-agent ConditionPathExists=/etc/warp/agent.json [Service] ExecStart=/usr/sbin/warp-agent /etc/warp/agent.json LimitNOFILE=8192 [Install] WantedBy=multi-user.target 46 . 1

Slide 47

Slide 47 text

SystemD 47 . 1

Slide 48

Slide 48 text

SystemD drawbacks Hard to avoid heated discussions Wide scope means more room for error Mixed focus on desktop & servers can be off-puting for admins Heavily coupled to DBUS That giant, little-known attack vector on your system 48 . 1

Slide 49

Slide 49 text

Shepperd (define nginx (make #:provides '(nginx web-server) #:start (make-forkexec-constructor (list "nginx")) #:stop (make-kill-destructor))) (register-services nginx) 49 . 1