a new idea, having originated via filesystem containers with chroot in Seventh Edition Unix • chroot originated with Bill Joy, but specifics are blurry; according to Kirk McKusick, via Poul-Henning Kamp and Robert Watson:
for operating legacy stacks unmodified has been to virtualize the hardware • A virtual machine is presented upon which each tenant runs an operating system that they choose (but must also manage) • Effective for running legacy stacks, but with a clear inefficiency: there are as many operating systems on a machine as tenants: • Operating systems are heavy and don’t play well with others with respect to resources like DRAM, CPU, I/O devices, etc.! • Still, hardware-level virtualization became de facto in the cloud
run containers in multi-tenant production since ~2006... • Adding support for hardware-based virtualization circa 2011 strengthened our resolve with respect to OS-based virtualization! • We saw that the operational characteristics of OS containers — performance, elasticity, tenancy — opened up new possibilities...
we built Manta, a ZFS- and container- based internet-facing object store offering in situ compute • Containers allow the compute to be moved to the data instead of having to backhaul data to transient compute • And as a reminder, the OS — Unix! — was built around the notion of ad hoc computation
it was not merely a new system, but a new way of thinking about systems • The Unix philosophy, as articulated by Doug McIlroy: • Write programs that do one thing and do it well • Write programs to work together • Write programs that handle text streams, because that is a universal interface • Four decades later, this philosophy remains as relevant as ever!
Bentley posed the challenge that became the Epic Rap Battle of computer science history: Read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies. • Don Knuth’s solution: a purpose-built algorithm in WEB, a Pascal- like literate programming system of his own invention • Doug McIlroy’s solution shows the power of the Unix philosophy: tr -cs A-Za-z '\n' | tr A-Z a-z | \ sort | uniq -c | sort -rn | sed ${1}q
scalable variant of McIlroy’s solution to Bentley’s challenge: mfind -t o /bcantrill/public/v7/usr/man | \ mjob create -o -m "tr -cs A-Za-z '\n' | \ tr A-Z a-z | sort | uniq -c" -r \ "awk '{ x[\$2] += \$1 } END { for (w in x) { print x[w] \" \" w } }' | \ sort -rn | sed ${1}q" • As with McIlroy’s solution, the container allows for not just for terser expression but faster execution!
in the power of containers as not merely better infrastructure, but as transformative • Our biggest challenge with Manta was that containers were not well understood: it felt “easy” to us, but was confusing to many • Would the world ever really figure out containers?!
rapid provisioning + shared underlying filesystem of containers to allow developers to think operationally • Operational dependencies are encoded via an image • Images can be reliably and reproducibly deployed as a container • Images can be quickly deployed — and re-deployed • Coupled with the mainstream adoption of open source and the leading adoption of microservices... • Containers accelerate software development
cloud management software with a Docker Remote API endpoint • With Triton, the notion of a Docker host is virtualized: to the Docker client, the datacenter is a large Docker host • One never allocates VMs with Triton; one only allocates containers — and all Triton containers run directly on-the-metal • All of Triton is open source — you can download it and run it for yourself (or run it on our public cloud)
ecosystem can be viewed broadly as two different approaches: • A framework approach in which the framework is in control, but alternatives are “pluggable” • A library approach in which the developer/operator is in control, composing functionality out of well-defined modules • The “all in” nature of frameworks may make initial adoption easier, but they ultimately sacrifice flexibility!
framework approach to orchestration may not accommodate applications that demand local persistence • We opted for orchestration that lives not in an all-knowing framework, but rather with the application, in the container • In-container logic allows for service registration/discovery, health checks, etc. — but does not handle restart • This is application-centric orchestration, and we have dubbed this the autopilot pattern: https://autopilotpattern.io
ContainerPilot, an embodiment of the autopilot pattern for Docker containers • All open source (https://github.com/joyent/containerpilot) — and not Triton or Joyent specific; can be used anywhere
microservices, there is an unstated truth: you are developing a distributed system • While more resilient to certain classes of force majeure failure, distributed systems remain vulnerable to software defects • We must be able to debug such systems; hope is not a strategy! • Distributed systems are hard to debug — and are more likely to exhibit behavior non-reproducible in development • With container-based systems, we must think in terms of the system, not merely the program!
seems very likely to hold for containers: greater efficiency will result in a net increase in consumption! • Efficiency gains from containers are in terms of developer time... • ...but requiring containers to be scheduled in VMs induces operational inefficiencies: every operator must now think like a cloud operator — maximizing density within fixed-cost VMs • Greater consumption + operational inefficiencies threaten to slow the container revolution — or make it explosive in terms of cost • To realize the full economic promise of the container revolution, we need container-native infrastructure!
computing is greatly exaggerated! • There are three key determinants for public v. on-premises: • Economics: Rent vs. buy; OPEX vs. CAPEX • Risk Management: Security/compliance — and also risk factors associated with operator-as-threat • Latency: The speed of light is a constant! • Economics dominates: “private cloud” efforts that do not deliver public cloud economics are doomed to (continue to) fail!
term is clearly meaningless… • ...but on the other, it highlights that containers have liberated us to think in terms of higher-level functionality • Thinking in functions is great (viz. Manta), but confining functions to containers to single-tenant VMs undermines their economics • The virtual machine is a vestigial abstraction; we must reject container-based infrastructure that implicitly assumes it! • So don’t say “#serverless” when you mean...
bcantrill
ammeep
ufuk
afnizarnur
bermonpainter
jlugia
morganepeng
swwweet
trishagee
lemiorhan
matthewcrist
iamctodd
3n
![The Container Revolution Reflections after the first decade CTO [email protected]](https://files.speakerdeck.com/presentations/50678dc8ad2549c68af31c600c807712/slide_0.jpg){kind=link}
