Joyent's Triton: containers, SmartOS, private cloud

Buzzwords aside, what is it? This talk is about where Triton came from, what it does and why it's a good fit for modern application architectures.

Transcript

2. SmartOS microservices on Triton — Jon Stuart — jon@fastmailteam.com — @zomoco
3. SmartOS microservices on Triton
4. SmartOS microservices on Triton !
6. SmartOS
7. SmartOS — Solaris
8. SmartOS — Solaris → OpenSolaris
9. SmartOS — Solaris → OpenSolaris → illumos
10. SmartOS — Solaris → OpenSolaris → illumos → SmartOS
11. SmartOS — Solaris → OpenSolaris → illumos → SmartOS and OmniOS, ...
12. SmartOS — Solaris → OpenSolaris → illumos → SmartOS and OmniOS, ... — Fork Yeah!
13. SmartOS — Solaris → OpenSolaris → illumos → SmartOS and OmniOS, ... — Fork Yeah! — Solaris RIP September 2017
14. SmartOS — Zones — ZFS — SMF — DTrace — pkgsrc — Crufty UNIX history
15. Zones?
16. SmartOS > Zones
17. SmartOS > Zones — chroot → jails → zones
18. SmartOS > Zones — chroot → jails → zones — Bryan Cantrill: Jails and Solaris zones
19. SmartOS > Zones — chroot → jails → zones — Bryan Cantrill: Jails and Solaris zones — "thin" virtualization
20. SmartOS > Zones — chroot → jails → zones — Bryan Cantrill: Jails and Solaris zones — "thin" virtualization — network, cpu, memory, io isolation
21. SmartOS > Zones — chroot → jails → zones — Bryan Cantrill: Jails and Solaris zones — "thin" virtualization — network, cpu, memory, io isolation — and, of course, a chrooted fs
22. SmartOS > Zones
23. SmartOS > Zones — run SmartOS itself with readonly /usr shared from "global zone"
24. SmartOS > Zones — run SmartOS itself with readonly /usr shared from "global zone" — run KVM and so anything
25. SmartOS > Zones — run SmartOS itself with readonly /usr shared from "global zone" — run KVM and so anything — also, Linux userland on illumos syscall magic
26. SmartOS > Zones — run SmartOS itself with readonly /usr shared from "global zone" — run KVM and so anything — also, Linux userland on illumos syscall magic — Docker
27. ZFS?
28. SmartOS > ZFS
29. SmartOS > ZFS — you know, ZFS
30. SmartOS > ZFS — you know, ZFS — each zone gets a z-thing
31. SmartOS > ZFS — you know, ZFS — each zone gets a z-thing — which is a filesystem for regular or LX zones
32. SmartOS > ZFS — you know, ZFS — each zone gets a z-thing — which is a filesystem for regular or LX zones — or a zvol for KVM zones
33. SmartOS > ZFS — you know, ZFS — each zone gets a z-thing — which is a filesystem for regular or LX zones — or a zvol for KVM zones — we can delegate zfs into the zone so it can create further datasets
34. SMF?
35. SmartOS > SMF
36. SmartOS > SMF — "I know, let's replace init"
37. SmartOS > SMF — "I know, let's replace init" — With something that uses XML!
38. SmartOS > SMF — "I know, let's replace init" — With something that uses XML! — But it's not terrible merely super engineered
39. DTrace?
40. SmartOS > DTrace
41. SmartOS > DTrace — all the observability tooling I do not know how to use properly
42. SmartOS > DTrace — all the observability tooling I do not know how to use properly — dynamic tracing of kernel and userland that's cheap and safe
43. pkgsrc?
44. SmartOS > pkgsrc
45. SmartOS > pkgsrc — Originally from NetBSD
46. SmartOS > pkgsrc — Originally from NetBSD — Binary distribution
47. SmartOS > pkgsrc — Originally from NetBSD — Binary distribution — Huge source tree
48. SmartOS > pkgsrc — Originally from NetBSD — Binary distribution — Huge source tree — It's OK!
49. Cru!y UNIX history?
50. SmartOS > Cru!y UNIX history
51. SmartOS > Cru!y UNIX history — a pleasing mix of SysV, BSD and GNU deliciously blended to confuse
52. SmartOS > Cru!y UNIX history — a pleasing mix of SysV, BSD and GNU deliciously blended to confuse — rando synthetic filesystems, doors, $0, LWP / threading, killall
53. Triton
54. Triton
55. Triton — Formerly "Smart Data Center"
56. Triton — Formerly "Smart Data Center" — What Joyent built to sell hosted SmartOS: "Public cloud"
57. Triton — Formerly "Smart Data Center" — What Joyent built to sell hosted SmartOS: "Public cloud" — Think of it like running your own AWS EC2
58. Triton — Formerly "Smart Data Center" — What Joyent built to sell hosted SmartOS: "Public cloud" — Think of it like running your own AWS EC2 — Almost all open source
59. Triton — Formerly "Smart Data Center" — What Joyent built to sell hosted SmartOS: "Public cloud" — Think of it like running your own AWS EC2 — Almost all open source — Manages the bare metal, provisioning and maintenance of the cluster and zone lifecycle
60. Strange!
61. Triton > Strange
62. Triton > Strange — All but one server boots over the network from the head node
63. Triton > Strange — All but one server boots over the network from the head node — The head node itself boots off a USB stick
65. Triton > Strange
66. Triton > Strange — Separate network for Triton comms between nodes
67. Triton > Strange — Separate network for Triton comms between nodes — Runs a bazillion tiny management services and everything has a JSON API cnapi, vmapi, napi, ...
68. How do I even?
69. Triton > How do I even?
70. Triton > How do I even? — Admin UI web interface
71. Triton > How do I even? — Admin UI web interface — triton and sdc-* CLI
72. Triton > How do I even? — Admin UI web interface — triton and sdc-* CLI — "Users" own zones, networks
73. Triton > How do I even? — Admin UI web interface — triton and sdc-* CLI — "Users" own zones, networks — User portal
74. Microservices
75. Microservices
76. Microservices — Lightweight
77. Microservices — Lightweight — A proper pid 1
78. Microservices — Lightweight — A proper pid 1 — Managed software packaging
79. Microservices — Lightweight — A proper pid 1 — Managed software packaging — Configuration Management
80. Microservices — Lightweight — A proper pid 1 — Managed software packaging — Configuration Management — Regular network
81. Microservices — Lightweight — A proper pid 1 — Managed software packaging — Configuration Management — Regular network — Density and scale
82. Triton at FastMail
83. Triton at FastMail — 15 physical servers — 180 zones mostly SmartOS, a few LX — 40 service roles — Three customer products
84. Triton at FastMail — 15 physical servers — 180 zones mostly SmartOS, a few LX — 40 service roles — Three customer products including Topicbox
85. Caveats
86. Caveats
87. Caveats — ZFS
88. Caveats — ZFS — Network infrastucture
89. Caveats — ZFS — Network infrastucture — Community and ownership