Paris container Day 2017 - Production FS: Adapt or Die - Claudia Beresford & Tiago Scolari, Pivotal

Transcript

1. Production FS: Adapt or Die
2. hello! Claudia & Tiago callisto13 tscolari
3. Summary ✘ Glossary - What is a Root Filesystem? ✘ What is CloudFoundry? ✘ Warden ✘ Garden Linux ✘ Garden runC ✘ GrootFS + Garden runC ✘ The Future ✘ Questions?
4. 1) Directory hierarchy and structure 1) Type: How the data is organised What is a FileSystem?
5. ✘ Top of the dir structure ✘ Mount point for other FSes at boot ✘ Contains all critical startup files ✘ Sets state of system ✘ Has tools for recovery of broken system and data What is a Root FileSystem?
6. What is a RootFS? Host
7. What is a RootFS? Container Host
8. ✘ Open Source Platform as a Service ✘ Development started in 2009 (VMWare) ✘ First released in 2011 ✘ Run application inside containers ✘ Supports buildpacks and Docker images What is CloudFoundry?
9. Provide framework and runtime support for applications Officially supported: ◦ Binary ◦ Go ◦ Java ◦ .Net Core ◦ Node.js Buildpacks? ◦ PHP ◦ Python ◦ Ruby ◦ Static File
10. Buildpack Buildpack RootFS Application Code RootFS Droplet Compilation
11. Droplet
13. ✘ Multitenancy ✘ Application Quotas Isolation ✘ Independent of Host FS What did CF need?
14. 2011 ✘ Kernel 2.6 ✘ Containers? ✘ No user namespace available
15. 1. AUFS + WARDEN 2011
16. Warden ✘ Developed in Ruby and C ✘ Initially with LXC ✘ Coupled to Linux Namespaces (exc User) & cgroups
17. WHY AUFS? ✘ Mounting the rootfs was faster than copying it ✘ No duplicated files But... ✘ No support for quotas ✘ Not in Mainline Kernel
18. RootFS Mnt: RootFS RW Layer Droplet Mnt: RootFS RW Layer Droplet Mnt: RootFS RW Layer Droplet ... root root root root unique uid APP1 APP2 APP3
19. 2013-2014 ✘ User namespaces ✘ Security ✘ Scalability ✘ Containers
20. 2. BTRFS + GARDEN LINUX 2014/2015
21. Garden & Diego ✘ Replacement for Warden ✘ Go (w)arden - Garden ✘ Platform Agnostic API - future support for windows ✘ New scheduler ✘ DEA(Go) - Diego
22. + Docker Images ✘ No control over RootFS anymore ✘ Security Risks ✘ User namespaces
23. Why BTRFS? ✘ Dependence on Docker graph driver ✘ Built in support for quotas ✘ The other options were: Overlay - not mature DeviceMapper - required LVM ZFS - proprietary
25. Everything was changing ✘ New scheduler ✘ New container runtime ✘ New container Filesystem ✘ New IAAS
26. And... ✘ Huge Performance Hit: BTRFS blamed (eventually) ✘ Theory was BTRFS garbage collection was consuming all IOPS from the cells ✘ BTRFS new and didn’t have enough support at the time
27. 3. AUFS + GARDEN LINUX October 2015
28. Why AUFS, again? ✘ Familiarity ✘ But… Quotas?
29. RootFS Mnt: RootFS mnt:/dev/loop1 Droplet ... root root APP1 Mnt: RootFS mnt:/dev/loop2 Droplet root APP2 Mnt: RootFS mnt:/dev/loop3 Droplet root APP3 sparse
30. 2015 ✘ runC
31. 4. AUFS + GARDEN RunC May 2016
32. Open Containers Initiative / RunC ✘ Open Standard for containers specification ✘ Implementation of OCI container specs
33. Garden RUNC ✘ Rewritten to use RunC ✘ More GO, Less C ✘ More Security
34. AUFS... ✘ More bugs ✘ Distraction
35. 2015/2016 ✘ OCI: Image-Spec ✘ GrootFS - new project to replace Garden Linux backend Dedicated team ✘ Security: Garden runC Rootless
36. 5. BTRFS + GROOTFS + GARDEN runC June 2016
37. Why BTRFS, again? ✘ Snapshotting: plays well with container images ✘ Could be (almost) rootless ✘ Quotas ✘ Previous issues fixed in kernel 4.4 ✘ Big companies investing ✘ Support from Canonical
38. Container Image
39. Snapshot: rootfs Droplet 1001 Buildpack App Snapshot: layer1 1001 Docker Img App Snapshot: layer2 Snapshot: layer3
40. But... ✘ New performance issue: `btrfs enable quota`
41. OVERLAY/XFS + GROOTFS + GARDEN runC 6. February 2017
42. Why Overlay+XFS? ✘ Maturity ✘ Overlay for layering ✘ XFS for quotas
43. XFS folder app1/ Overlay mount upperdir: app1/diff lowerdirs: layer1:layer2:layer3 App 1 XFS folder app2/ Overlay mount upperdir: app2/diff lowerdirs: layer1:layer2:layer3 App 2
45. The Future
46. What’s Next? ✘ EXT4 Kernel 4.5 in Stemcell Match host FS ✘ ShiftFS On the fly user mappings No translation layer
47. Conclusion ✘ Nothing is forever ✘ There are always risks Agility is key ✘ Focus!
48. thanks! Any questions? callisto13 / cberesford@pivotal.io tscolari / tscolari@pivotal.io Slide template by SlidesCarnival