Bryan Cantrill on Jails & Solaris Zones

Jails: Confining the omnipotent root.
===========================
The traditional UNIX security model is simple but inexpressive. Adding fine-grained access control improves the expressiveness, but often dramatically increases both the cost of system management and implementation complexity. In environments with a more complex management model, with delegation of some management functions to parties under varying degrees of trust, the base UNIX model and most natural extensions are inappropriate at best. Where multiple mutually untrusting parties are introduced, ‘‘inappropriate’’ rapidly transitions to‘‘nightmarish’’, especially with regards to data integrity and privacy protection. The FreeBSD ‘‘Jail’’ facility provides the ability to partition the operating system environment, while maintaining the simplicity of the UNIX‘‘root’’ model. In Jail, users with privilege find that the scope of their requests is limited to the jail, allowing system administrators to delegate management capabilities for each virtual machine environment. Creating virtual machines in this manner has many potential uses; the most popular thus far has been for providing virtual machine services in Internet Service Provider environments.

Solaris Zones: Operating System Support for Consolidating Commercial Workloads
===========================================
Server consolidation, which allows multiple workloads to run on the same system, has become increasingly important as a way to improve the utilization of computing resources and reduce costs. Consolidation is common in mainframe environments, where technology to support running multiple workloads and even multiple operating systems on the same hardware has been evolving since the late 1960’s. This technology is now becoming an important differentiator in the UNIX and Linux server market as well, both at the low end (virtual web hosting) and high end(traditional data center server consolidation).This paper introduces Solaris Zones (zones), a fully realized solution for server consolidation projects in a commercial UNIX operating system. By creating virtualized application execution environments within a single instance of the operating system, the facility strikes a unique balance between competing requirements. On the one hand, a system with multiple workloads needs to run those workloads in isolation, to ensure that applications can neither observe data from other applications nor affect their operation. It must also prevent applications from over-consuming system resources. On the other hand, the system as a whole has to be flexible, manageable, and observable, in order to reduce administrative costs and increase efficiency. By focusing on the support of multiple application environments rather than multiple operating system instances, zones meets isolation requirements without sacrificing manageability.

Transcript

1. Papers We Love: Jails and Zones CTO bryan@joyent.com Bryan Cantrill @bcantrill
2. Papers we love: Jails and Zones • Discussing two important papers that form the foundation of thinking about OS-based virtualization and containers: • Jails: Confining the Omnipotent Root by Poul-Henning Kamp and Robert Watson, presented at SANE 2000 • Solaris Zones: Operating System Support for Consolidating Commercial Workloads by Dan Price and Andy Tucker, presented at LISA 2004 • As much as possible, want to let these papers speak for themselves — and provoke discussion!
3. Jails: Problem statement
4. Jails: Prior work
5. Jails aside: chroot(2)
6. Jails: Proposed solution
7. Jails: Advantages
8. Jails: jail(2)
9. Jails: Confining the filesystem
10. Jails: Confining the network
11. Jails: Implementation
12. Jails: Network management complexities
13. Jails: Filesystem management complexities
14. Jails: User management complexities
15. Jails: Unintended consequences
16. Jails: Networking limitations
17. Jails: Resource management limitations
18. Jails: Management limitations
19. Jails: Epilogue • Jails became easier to manage with jls/jps/ezjail/iocage • Jails were allowed to have multiple IPv4 addresses • Some jail-based resource management was added, including CPU binding and • System V IPC was virtualized, but remains out-of-tree • VIMAGE added exclusive IP stacks to jails, but it remains a build- time option and “is considered experimental”
20. Zones: Problem statement
21. Zones: Problem statement detail
22. Zones: Proposed solution
23. Zones: Block diagram
24. Zones: Design principles
25. Zones: Design principles, cont.
26. Zones: State model
27. Zones: Configuration
28. Zones: Installation
29. Zones: Application environment
30. Zones: Virtual platform
31. Zones: Console
32. Zones: Process model
33. Zones: Process model, cont.
34. Zones: IPC
35. Zones: System V IPC
36. Zones: Networking
37. Zones: Filesystem
38. Zones: Resource management
39. Zones: Observability and debugging
40. Zones: Security experience
41. Zones: Workloads
42. Zones: Epilogue • Crossbow added virtual NICs and exclusive IP stacks — and anti- spoof allowed exclusive IP stacks to be deployed safely • Resource management became much more complete, adding memory capping, CPU capping, I/O throttling • ZFS revolutionized zone installation/configuration • With introduction of IPS packaging, Solaris got rid of so-called “sparse root” zones... • ...and Joyent added sparse root zones back to SmartOS (thanks to no IPS and no global zone package management)
43. Zones: Epilogue, cont. • Sun added notion of branded zones in 2006, including a nascent Linux brand (LX) — and then ripped LX out in 2010 • LX brand revived by Joyent in 2014 in SmartOS and completed (first deployed into production in early 2015) • Overlay network support added to SmartOS by Joyent, allowing software-defined VXLAN-based networks in non-global zones
44. Jails and Zones: Conclusions • Each of these technologies has served to inspire the other: zones was explicitly inspired by jails — and the jails networking work has been explicitly inspired by Crossbow • These two papers are important because they capture not just the what, but the why of their respective works • These technologies were both ahead of their time; it’s invaluable now to be able to understand their motivations! • In the words of the late, great Jim Gray: You need to write more!