The Latest Trends in Lightweight Virtualization Technologies from Aspects of Unikernel Linux and Lupine Linux

Transcript

1. Manabu Sugimoto R&D Center, Sony Corporation Copyright 2020 Sony Corporation The Latest Trends in Lightweight Virtualization Technologies from Aspects of Unikernel Linux and Lupine Linux
2. R&D Center 2 About me  Manabu Sugimoto  System software engineer, Sony  New employee in 2020  Interests  Operating systems, Unikernels, Container technologies, Software analysis  Research topic when I was a university student  Improving operating systems dependability
3. R&D Center 3 Outline of the Talk  Lightweight Virtualization Technologies  About Unikernels  Unikernel Linux (UKL)  Lupine Linux  Intra-Unikernel Isolation  Key Takeaways
4. R&D Center 4 Lightweight Virtualization Technologies
5. R&D Center 5 New Approaches Today’s Lightweight Virtualization Hardware Container Process Process Container Process Process LXC [1] (2008) Container Process gVisor Container Process gVisor gVisor [2] (2018) VM Process Linux VM Process Linux LightVM [4] (2017) Unikernel Process LibOS Unikernel Process LibOS Unikernel [5] (2013) X-Container Process X-LibOS X-Container Process X-LibOS X-Containers [6] (2019) UKL Process Linux UKL Process Linux Unikernel Linux [7] (2019) Lupine Process Lupine Linux [8] (2020) Lupine Process Lupine Linux Hardware Linux OS Hardware Xen-PV VM Process Clear Linux VM Process Clear Linux Clear Container [3] (2015) Hardware KVM/QEMU Hardware X-Kernel Hardware Hypervisor Hardware KVM/QEMU Hardware KVM/Firecracker Containers on Linux OS Process on Library OS (LibOS) using Hypervisor Containers in VM using Hypervisor Linux OS Lupine Linux
6. R&D Center 6 What is Unikernel?  Specialized machine images constructed by Library OS (LibOS) such as Exokernel [9]  LibOS builds the kernel components as a shared library and runs in userspace  One application and LibOS into a binary can be executed as a VM on a hypervisor  All the code executes with the highest privilege level  There is no need for memory protection within unikernel, done with hypervisor  Unikernels can be grouped into two categories  Language-based  MirageOS [5], HalVM [10], runtime.js [11], Ling [12] ClickOS [13] , IncludeOS [14]  POSIX-like  OSv [15], HermiTux [16], Rumprun [17], RustyHermit [18] Filesystem Network Stack Virtual Memory Kernel components as a shared library etc... Hardware Unikernel Unikernel LibOS LibOS App App Hypervisor Unikernel
7. R&D Center 7 Problems of Unikernels  Very hard to deploy on existing systems due to lack of compatibility  In case of Language-based unikernels  Applications need to be written from scratch (clean-slate approach)  Developers may use unfamiliar languages such as OCaml  In case of POSIX-like unikernels  Developers must reimplement kernel functionality from scratch (strip-down approach)  Lose the opportunity to benefit from the community Unikernel LibOS App Unikernel LibOS App Language-based unikernels POSIX-like unikernels clean slate strip down
8. R&D Center 8 Solutions to the Problems of Unikernels - Unikernel Linux and Lupine Linux
9. R&D Center 9 Unikernel Linux (UKL)  UKL [7] is a unikernel based on Linux kernel  UKL uses many of the advantage of Linux (e.g., performance, security)  Research and development by Boston University and RedHat  Key Goals  Applications and user libraries should be integrated into a unikernel without modification  Avoid overheads of context switches by privilege transitions  Allow developers to co-optimize the application and kernel code  Changes in Linux source code should be minimal to be accepted upstream
10. R&D Center 10 Architecture Overview  Minimal changes to Linux code  Add a new kernel configuration to compile the Linux kernel as UKL  Add a call to an undefined symbol to invoke applications code before the first process  Create a small UKL library as stubs for syscalls  Change glibc to make syscalls into the kernel  Modify the kernel linking stage to include the application code, glibc, and UKL library KVM/QEMU UKL Linux Kernel App glibc UKL Library Single binary func calls func calls func calls 1. Compile Linux as UKL by configuration 2. Directly invoke app without fork 3. Use UKL func instead of syscalls run app
11. R&D Center 11 Build Process  UKL build process is straightforward  Compile glibc into an archive of object files without any linking  Compile the application code into object files without any linking  Compile the UKL library into an object file  Build the Linux kernel with UKL config option turned on KVM/QEMU UKL Linux Kernel App glibc UKL Library built-in.a + lib.a + ... ukl.o libc.a app.o vmlinux
12. R&D Center 12 Lupine Linux  Lupine [8] makes Linux behave like a unikernel and achieves high performance  Apply kernel configuration specialization and system call overhead elimination  Run an application as a kernel-mode process by using Kernel Mode Linux (KML) [19]  Research and development by Illinois University and IBM Watson Research Center  Key Goals  Eliminate unnecessary functionality from the kernel (e.g., multiprocessing)  Tailor the kernel as much as possible to the particular application  Leverage the robustness and performance of Linux kernel
13. R&D Center 13 Firecracker’s microVM Architecture Overview  Generate application-specific Lupine kernel binary and root filesystem Syscalls overhead elimination Specialization for unikernel Application-specific requirements Lupin config Linux Kernel includes KML patch startup script Container image Unmodified app binary Metadata entrypoint env variables Libraries libc.so root filesystem (rootfs) KML-enabled musl libc Lupine kernel binary
14. R&D Center 14 Specialization for Unikernel  Remove unnecessary kernel configurations for unikernel domain  Application-specific options  An application-centric approach can be applied towards Linux kernel configuration  Unnecessary options in microVM’s configurations  Options that stem from the single-process nature of unikernels (e.g., cgroups, namespaces)  Options that stem from the expected virtual hardware environment (e.g., power management) All 15,953 configurations of Linux kernel v4.0 833 microvm configurations 283 (34%) configurations of Lupine-base 550 (66%) application-specific and unnecessary options Specialization
15. R&D Center 15 Eliminating System Call Overhead  Lupine Linux uses Kernel Mode Linux (KML) [19]  Enable normal user processes to run in kernel mode  Call kernel routines directly without privilege transitions or context switches  For the implementation of KML in Lupine  Apply the modified KML patch to the Linux kernel  Before loading the program inside kernel, type-checker ensures memory and control flow safety Linux Kernel User Program User Program Type-checker 1. Create a program in TALx86 2. Type-check the program 4. Load the program inside Kernel 3. Ensure memory and control flow safety
16. R&D Center 16 Memory Isolation inside Unikernel - Intra-Unikernel Isolation
17. R&D Center 17 Intra-Unikernel Isolation  An isolation scheme for components within a unikernel instance [20]  Provide memory space isolation (e.g., kernel isolation from user)  untrusted application code, untrusted/trusted kernel components  Leverage the Intel Memory Protection Keys (MPK) technology [21]  Build on top of RustyHermit unikernel [18] written in Rust User Unsafe Kernel Memory MPK fault MPK fault Kernel Safe Kernel Memory System calls
18. R&D Center 18 Key Takeaways  Various lightweight virtualization technologies have been proposed  Unikernels, Clear Container, LightVM, gVisor, X-Containers, etc.  Unikernels are lightweight alternative to traditional virtual machines  However, existing approaches still fall short of true compatibility  New unikernels designs can be the next standard evolution of Linux  Unikernel Linux and Lupine Linux  Unikernels model can be accepted to the Linux community  The era when the unikernels are a research system is coming to an end
19. R&D Center 19 References [1] LXC - Introduction. https://linuxcontainers.org/lxc/introduction/. [2] gVisor Documentation. https://gvisor.dev/docs/. [3] Intel Clear Containers. https://clearlinux.org/containers. [4] Filipe Manco, Costin Lupu, Florian Schmidt, Jose Mendes, Simon Kuenzer, Sumit Sati, Kenichi Yasukata, Costin Raiciu, and Felipe Huici. 2017. My VM is Lighter (and Safer) than your Container. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP ’17). [5] Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: library operating systems for the cloud. In Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems (ASPLOS ’13). [6] Zhiming Shen, Zhen Sun, Gur-Eyal Sela, Eugene Bagdasaryan, Christina Delimitrou, Robbert Van Renesse, and Hakim Weatherspoon. 2019. X-Containers: Breaking Down Barriers to Improve Performance and Isolation of Cloud-Native Containers. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’19). [7] Ali Raza, Parul Sohal, James Cadden, Jonathan Appavoo, Ulrich Drepper, Richard Jones, Orran Krieger, Renato Mancuso, and Larry Woodman. 2019. Unikernels: The Next Stage of Linux’s Dominance. In Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS ’19). [8] Hsuan-Chi Kuo, Dan Williams, Ricardo Koller, and Sibin Mohan. 2020. A Linux in unikernel clothing. In Proceedings of the Fifteenth European Conference on Computer Systems (EuroSys ’20). [9] D. R. Engler, M. F. Kaashoek, and J. O’Toole. 1995. Exokernel: an operating system architecture for application-level resource management. In Proceedings of the fifteenth ACM symposium on Operating systems principles (SOSP ’95). [10] GALOISINC. The Haskell Lightweight VirtualMachine (HALVM) source archive. https://github.com/GaloisInc/HaLVM. [11] Javascript library operating system for the cloud. http://runtimejs.org/. [12] LING. http://erlangonxen.org. [13] Joao Martins, Mohamed Ahmed, Costin Raiciu, Vladimir Olteanu, Michio Honda, Roberto Bifulco, and Felipe Huici. 2014. ClickOS and the art of network function virtualization. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI ’14). [14] A. Bratterud, A. Walla, H. Haugerud, P. E. Engelstad and K. Begnum, "IncludeOS: A Minimal, Resource Efficient Unikernel for Cloud Services," 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom ’15) [15] Avi Kivity, Dor Laor, Glauber Costa, Pekka Enberg, Nadav Har’El, Don Marti, and Vlad Zolotarov. 2014. OSv: optimizing the operating system for virtual machines. In Proceedings of the 2014 USENIX conference on USENIX Annual Technical Conference (USENIX ATC ’14). [16] Pierre Olivier, Daniel Chiba, Stefan Lankes, Changwoo Min, and Binoy Ravindran. 2019. A binary-compatible unikernel. In Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ‘19). [17] The rumprun unikernel and toolchain for various platforms. https://github.com/rumpkernel/Rumprun. [18] Stefan Lankes, Jens Breitbart, and Simon Pickartz. 2019. Exploring Rust for Unikernel Development. In Proceedings of the 10th Workshop on Programming Languages and Operating Systems (PLOS ’19). [19] T. Maeda and A. Yonezawa. Kernel mode linux: Toward an operatingsystem protected by a type theory. In V. A. Saraswat, editor,Ad-vances in Computing Science – ASIAN 2003. [20] Mincheol Sung, Pierre Olivier, Stefan Lankes, and Binoy Ravindran. 2020. Intra-unikernel isolation with Intel memory protection keys. In Proceedings of the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE ’20). [21] Jonathan Corbet. 2015. Memory protection keys.Linux Weekly News(2015). https://lwn.net/Articles/643797/.
20. SONY is a registered trademark of Sony Corporation. Names of Sony products and services are the registered trademarks and/or trademarks of Sony Corporation or its Group companies. Other company names and product names are registered trademarks and/or trademarks of the respective companies.