Upgrade to Pro — share decks privately, control downloads, hide ads and more …

firecracker-containerd and SOCI Snapshotter

Kazuyoshi Kato
August 20, 2022
210

firecracker-containerd and SOCI Snapshotter

Kazuyoshi Kato

August 20, 2022
Tweet

Transcript

  1. © 2022, Amazon Web Services, Inc. or its affiliates. ©

    2022, Amazon Web Services, Inc. or its affiliates. firecracker-containerd and SOCI Snapshotter Kazuyoshi Kato (he/him) Sr. Software Development Engineer Amazon Web Services
  2. © 2022, Amazon Web Services, Inc. or its affiliates. Linux

    container primitives • Namespaces – Visibility restrictions • Control groups (cgroups) – Resource limits • Capabilities – Permission rules • Seccomp – Syscall allow/deny lists 2
  3. © 2022, Amazon Web Services, Inc. or its affiliates. Is

    this secure enough? 3 Your app Your sidecar Malicious app Linux kernel Container Container Container
  4. © 2022, Amazon Web Services, Inc. or its affiliates. runc

    CVE-2019-5736 • A malicious actor could overwrite the host runc binary through /proc/self/exe • https://aws.amazon.com/blogs/compute/anatomy-of-cve-2019- 5736-a-runc-container-escape/ 4
  5. © 2022, Amazon Web Services, Inc. or its affiliates. Firecracker

    • “Firecracker is an open source virtualization technology that is purpose-built for creating and managing secure, multi-tenant container and function-based services” • Open-source virtual machine monitor written in Rust • Utilizes hardware-assisted virtualization through Linux’s KVM • Minimalistic design to support only ”serverless” workloads • Not aware about Linux containers 5
  6. © 2022, Amazon Web Services, Inc. or its affiliates. Running

    containers with Firecracker 6 Your app Your sidecar Malicious app Linux kernel Linux kernel Firecracker Firecracker Linux kernel Container Container Container
  7. © 2022, Amazon Web Services, Inc. or its affiliates. Firecracker

    + containerd = firecracker-containerd • Secure isolation through Firecracker’s virtualization • Convenience and familiarity of containers from containerd • https://github.com/firecracker-microvm/firecracker-containerd 7
  8. © 2022, Amazon Web Services, Inc. or its affiliates. firecracker-containerd

    implementation 8 fc-control (plugin) containerd-shim- aws-firecracker (runtime/shim) agent runc container client (e.g. ctr) firecracker-containerd (daemon) containerd Firecracker microVM
  9. © 2022, Amazon Web Services, Inc. or its affiliates. Working

    with containerd community • Two maintainers from Amazon Web Services • OpenTelemetry tracing support • Device Mapper Snapshotter 9
  10. © 2022, Amazon Web Services, Inc. or its affiliates. What

    are snapshotters? • Snapshotter converts container images to filesystems • “Graph Driver” in Docker Engine • overlay (default) • devmapper (used by firecracker-containerd) • btrfs, aufs, zfs, … 10
  11. © 2022, Amazon Web Services, Inc. or its affiliates. Lazy-loading

    snapshotters • Stargz Snapshotter • Nydus Snapshotter • Downloading a container image and assembling the filesystem from the image is time-consuming • Containers don’t need the all files on the images to start doing useful work 11
  12. © 2022, Amazon Web Services, Inc. or its affiliates. Lazy-loading

    without conversion • Explicitly converting images and managing them is cumbersome • Implicitly converting images have negative security implications • For example, image signing wouldn’t work if AWS implicitly converts images 12
  13. © 2022, Amazon Web Services, Inc. or its affiliates. SOCI

    Snapshotter • SOCI Snapshotter is a new lazy-loading snapshotter, based on Stargz Snapshotter • Utilizes FUSE and HTTP’s ranged GET • No image conversion • Workload-specific load order optimization • https://github.com/awslabs/soci-snapshotter 13
  14. © 2022, Amazon Web Services, Inc. or its affiliates. SOCI:

    Seekable OCI 14 OCI Image Layer 1 Layer 2 Layer 3 SOCI Index zTOC 1 zTOC 2 zTOC 3
  15. © 2022, Amazon Web Services, Inc. or its affiliates. zTOC

    15 Checkpoint M /bin/ls TOC entry Compressed Span M zTOC N Compressed Layer N Uncompressed Span M /bin/ls data
  16. © 2022, Amazon Web Services, Inc. or its affiliates. Workload-specific

    load order optimization • The list of to-be-prefetched files wouldn’t be 1:1 to container images • Base images (e.g. Python 3) would have multiple possible prefetch lists, depending on upper application layers • The lists could be more dynamic than container images themselves. 16
  17. © 2022, Amazon Web Services, Inc. or its affiliates. Workload-specific

    load order optimization 17 Python 3 ML training application Web application Load order document Load order document Python 3 Debian Bullseye Debian Bullseye
  18. © 2022, Amazon Web Services, Inc. or its affiliates. What’s

    next? • Better code sharing with Stargz Snapshotter • Support OCI Reference Types instead of ORAS • Finalize SOCI Index and zTOC format (e.g. getting rid of encoding/gob) • Load order optimization • https://github.com/awslabs/soci-snapshotter 18
  19. © 2022, Amazon Web Services, Inc. or its affiliates. Thank

    you! © 2022, Amazon Web Services, Inc. or its affiliates. Kazuyoshi Kato @kzys [email protected]