Papers We Love: Jails and Zones

Slide 1

Slide 1 text

Papers We Love: Jails and Zones CTO [email protected] Bryan Cantrill @bcantrill

Slide 2

Slide 2 text

Papers we love: Jails and Zones • Discussing two important papers that form the foundation of thinking about OS-based virtualization and containers: • Jails: Conﬁning the Omnipotent Root by Poul-Henning Kamp and Robert Watson, presented at SANE 2000 • Solaris Zones: Operating System Support for Consolidating Commercial Workloads by Dan Price and Andy Tucker, presented at LISA 2004 • As much as possible, want to let these papers speak for themselves — and provoke discussion!

Slide 3

Slide 3 text

Jails: Problem statement

Slide 4

Slide 4 text

Jails: Prior work

Slide 5

Slide 5 text

Jails aside: chroot(2)

Slide 6

Slide 6 text

Jails: Proposed solution

Slide 7

Slide 7 text

Jails: Advantages

Slide 8

Slide 8 text

Jails: jail(2)

Slide 9

Slide 9 text

Jails: Conﬁning the ﬁlesystem

Slide 10

Slide 10 text

Jails: Conﬁning the network

Slide 11

Slide 11 text

Jails: Implementation

Slide 12

Slide 12 text

Jails: Network management complexities

Slide 13

Slide 13 text

Jails: Filesystem management complexities

Slide 14

Slide 14 text

Jails: User management complexities

Slide 15

Slide 15 text

Jails: Unintended consequences

Slide 16

Slide 16 text

Jails: Networking limitations

Slide 17

Slide 17 text

Jails: Resource management limitations

Slide 18

Slide 18 text

Jails: Management limitations

Slide 19

Slide 19 text

Jails: Epilogue • Jails became easier to manage with jls/jps/ezjail/iocage • Jails were allowed to have multiple IPv4 addresses • Some jail-based resource management was added, including CPU binding and • System V IPC was virtualized, but remains out-of-tree • VIMAGE added exclusive IP stacks to jails, but it remains a build- time option and “is considered experimental”

Slide 20

Slide 20 text

Zones: Problem statement

Slide 21

Slide 21 text

Zones: Problem statement detail

Slide 22

Slide 22 text

Zones: Proposed solution

Slide 23

Slide 23 text

Zones: Block diagram

Slide 24

Slide 24 text

Zones: Design principles

Slide 25

Slide 25 text

Zones: Design principles, cont.

Slide 26

Slide 26 text

Zones: State model

Slide 27

Slide 27 text

Zones: Conﬁguration

Slide 28

Slide 28 text

Zones: Installation

Slide 29

Slide 29 text

Zones: Application environment

Slide 30

Slide 30 text

Zones: Virtual platform

Slide 31

Slide 31 text

Zones: Console

Slide 32

Slide 32 text

Zones: Process model

Slide 33

Slide 33 text

Zones: Process model, cont.

Slide 34

Slide 34 text

Zones: IPC

Slide 35

Slide 35 text

Zones: System V IPC

Slide 36

Slide 36 text

Zones: Networking

Slide 37

Slide 37 text

Zones: Filesystem

Slide 38

Slide 38 text

Zones: Resource management

Slide 39

Slide 39 text

Zones: Observability and debugging

Slide 40

Slide 40 text

Zones: Security experience

Slide 41

Slide 41 text

Zones: Workloads

Slide 42

Slide 42 text

Zones: Epilogue • Crossbow added virtual NICs and exclusive IP stacks — and anti- spoof allowed exclusive IP stacks to be deployed safely • Resource management became much more complete, adding memory capping, CPU capping, I/O throttling • ZFS revolutionized zone installation/conﬁguration • With introduction of IPS packaging, Solaris got rid of so-called “sparse root” zones... • ...and Joyent added sparse root zones back to SmartOS (thanks to no IPS and no global zone package management)

Slide 43

Slide 43 text

Zones: Epilogue, cont. • Sun added notion of branded zones in 2006, including a nascent Linux brand (LX) — and then ripped LX out in 2010 • LX brand revived by Joyent in 2014 in SmartOS and completed (ﬁrst deployed into production in early 2015) • Overlay network support added to SmartOS by Joyent, allowing software-deﬁned VXLAN-based networks in non-global zones

Slide 44

Slide 44 text

Jails and Zones: Conclusions • Each of these technologies has served to inspire the other: zones was explicitly inspired by jails — and the jails networking work has been explicitly inspired by Crossbow • These two papers are important because they capture not just the what, but the why of their respective works • These technologies were both ahead of their time; it’s invaluable now to be able to understand their motivations! • In the words of the late, great Jim Gray: You need to write more!