OSv Designed for the cloud Pekka Enberg @penberg Tzach Livyatan @TzachL Cloudius Systems FOSDEM 2014 

About Cloudius ● Founded in December 2012 ● ~14 people ● 7 countries ● Open source experience ● Linux Foundation member 

Mission ● Be the best OS powering VMs in the cloud. 

What is OSv? 

OSv ● “Cloud OS” written from scratch ● C++11 ● BSD licensed ● Networking stack and ZFS imported from FreeBSD ○ We are modifying the networking stack ○ Switching to OpenZFS in the future 

Why a new operating system? 

Typical Cloud Stack Application Application Server JVM Operating System Hypervisor Hardware Protection and abstraction 

● Single application per VM ● No kernel and userspace separation ● No fork() ● MMU access via OSv APIs ● POSIX APIs for compatibility “Library OS” 

“Library OS” Application Application Server JVM + OSv Hypervisor Hardware 

Features 

OS Services ● Scheduler ● Memory management ○ mmap ○ demand paging ● Networking ● File system ○ ZFS and ramfs 

● Linux system calls ○ uname() reports Linux for compatibility ● libc ● OSv APIs APIs 

● Pure 64-bit OS ● 64-bit x86 ○ KVM ○ Xen HVM ○ Xen PV - incomplete ○ VMware and VirtualBox - in progress ● 64-bit ARM - planned ● Others - help wanted! Architectures 

Status ● JVM languages ○ Java, JRuby, Scala, Groovy, Clojure, JavaScript ● JVM applications ○ Cassandra ○ Tomcat ● Other runtimes ○ mruby ○ I would love to see someone port Node.js... ● Native applications ○ memcached ○ haproxy 

Demo 

Performance 

Performance ● Outperforms Linux ○ SpecJVM, memcached, netperf ● Same as Linux ○ Tomcat ○ Cassandra ● < 1 second boot time ● 4x faster context switch in scheduler microbenchmarks than Linux 

Image Size ● 17 MB minimal image ○ kernel + libc ● 29 MB mruby image ○ kernel + libc + mruby ● 465 MB default image (!) ○ kernel + libc + openjdk ○ openjdk is 127 MB ○ ZFS intent log related, working on a fix 

How is OSv different? 

● Linux network channels by Van Jacobson ○ Proposed in 2006, never merged ○ Packet processing overhead reduced by 25% for 1 CPU and 20% for 2 CPUs (source: lwn.net) ● Network channels in OSv ○ Channel connected to the application ● Netperf throughput increases 30% from ~36 Gbps to ~47 Gbps ○ Linux on host, OSv in guest Network channels 

JVM Balloon ● GC heap autotuning ● Steal memory from JVM on OS memory pressure ● Running on unmodified JVM ○ Compacting GC causes interesting problems ● Nice presentation by Glauber Costa on Youtube 

● Replace GC card tables with MMU remapping ● Modified OpenJDK ● Azul C4’s GC uses similar MMU tricks on x86 ○ Out-of-tree Linux patches JVM and MMU 

Admin 

OS Admin Traditional OS ● Built for a standalone desktop ● CLI oriented ● Human oriented ● File base configuration ● Hard to automate Cloud OS ● Built for mass deployment ● API oriented ● Machine oriented ● API base configuration ● Easy to automate 

Admin as collection of services OSv OSv OSv OSv Operation Configuration Packaging OSv Trace / Log Binary REST Metering Centralize Administration 

Operation Service Base on REST API (GET, POST, …) Successful Sync operations - 200 Successful A-sync operation - 202 A-sync operation result available in log with correlation id REST API definition use Swagger OSv OSv OSv Operations OSv 

Configuration Service Level of configuration: 1. Image configuration: capture as part of the image. 2. Init configuration: update at init phase (as in cloud-init) 3. Runtime configuration: instance lifetime No configuration overriding Same API for all configuration levels API is the only way to access configuration Configuration is ordered in a tree structure OSv OSv OSv Configuration OSv 

Packaging Services pull Base App repository run search new App ops commit push run util OSv OSv OSv Packaging OSv 

Packaging Services Repository operations search pull user/app - download pre built image push user/app - upload a new image to the repository VM operations run app - start OSv instance kill app - kill a running OSv node VM interactions app deploy bar.jar - deploy a jar to an instance app deploy foo.rpm - add a RPM to an instance run app cli - open an interactive shell OSv OSv OSv Packaging OSv 

Log / Trap Service OSv OSv OSv Log Traps Requirements Log Traps Direction Outbound Outbound Bandwidth High Low Reliable No Yes Format Binary/Text Binary/Text Alternatives RabbitMQ ZeroMQ Gelf ... See Log SNMP MQTT JMX 

Alpha Release 3/2014 Optimized ZFS cache Dirty page writeback JVM page table HV support: GCE, VMW, VBox Git init osv, 12/201 2 Java hello world, 01/2013 support for 64 vcpu 02/2013 UDP, 03/2013 ZFS support > 1Gbps netperf, 6/2013 Cassandra outperform s Linux, 8/2013 Native REST API Memcached gain > 70% 1/2014 Cli and web interface 9/2014 Tomcat, HAProxy modules 10/2013 Net channels JVM ballooning >47Gbps netperf 2/2014 Image Repository JVM Read barrier elimination DPDK support More ... Roadmap 

So what about containers? 

Containers ● Docker is awesome ○ Linux control groups not so awesome... ● Fast boot ● Fast provisioning ● Performance 

Containers under the hood ● Shared kernel image ○ Kernel upgrades ○ Hardware maintenance ● Complexity in the kernel ● Copy-on-write userspace 

OSv - best of both worlds? ● Fast boot, fast provisioning, performance ● Virtualized ○ Live migration ○ Elasticity ○ MMU access ● Great blog post on Google+ “OSv vs containers” by Glauber Costa 

That’s it! 

Thank you! ● Web ○ http://osv.io ● Github ○ http://github.com/cloudius-systems/osv ● Mailing List ○ [email protected] ● You should follow us on Twitter: ○ @CloudiusSystems ○ @penberg ○ @TzachL