Unikernels - Dutch Devops Engineers workshop

Material for the "Writing a Unikernel Microservice with Mirage" workshop given at the Dutch Devops Engineers meetup. http://www.meetup.com/devops-engineers/events/216306632/

Transcript

1. Unikernels Compile your OS for your application
2. Deployment trend general-purpose -> single-purpose
3. Deployment trend general-purpose -> single-purpose i.e. microservices
4. Deployment trend general-purpose -> single-purpose but: still deployed on general-purpose foundation
5. Smashing the Stack Typical layering application jvm unix tooling (shell scripts, daemons, ...) container (docker) os (linux) hypervisor (xen) hardware
6. Smashing the Stack Typical layering Unikernel layering application application jvm unikernel runtime unix tooling (shell scripts, daemons, ...) container (docker) os (linux) hypervisor (xen) hypervisor (xen) hardware hardware
7. Smashing the Stack Typical layering Unikernel layering application application jvm unikernel runtime ‘library os’ specifically compiled for this application unix tooling (shell scripts, daemons, ...) container (docker) os (linux) hypervisor (xen) hypervisor (xen) hardware hardware
8. Single ecosystem • Unikernels for: ◦ Haskell (HalVM) ◦ Go (Clive) ◦ Java (OSv) ◦ C++ (ClickOS) • Today: ◦ OCaml (MirageOS)
9. OCaml overview • High-level functional language • Compiles to efficient native code • Statically typed (with inference)
10. First taste of OCaml print_string “Hello, world"
11. First taste of OCaml let () = print_string "Hello, world"
12. First taste of OCaml let greet = print_string “Hello, world” let () = greet
13. First taste of OCaml let greet name = print_string ("Hello, " ^ name) let () = greet "John"
14. First taste of OCaml module Greeter = struct let greet name = print_string ("Hello, " ^ name) end let () = Greeter.greet "John"
15. First taste of OCaml module Greeter = struct let greet name = let greeting = "Hello, " ^ name in print_string greeting end let () = Greeter.greet "John"
16. Mirage overview • config.ml: mirage configuration ◦ select OS components to pass to your app ◦ high-level configuration (e.g. DHCP vs static IP) • ‘mirage config --unix’ or ‘mirage config --xen’ ◦ generates build system (Makefile) ◦ generates bootstrap code (main.ml)
17. Workshop today • Given: ◦ VirtualBox Linux image ◦ ocaml, opam and mirage tools installed ◦ bare-bones application • Goals ◦ Implement a HTTP service ◦ Run on Linux ◦ Run on Xen
18. Workshop today First steps Unikernel application application unikernel runtime xen linux host os + tools linux host os + tools virtualization (vagrant+virtualbox) virtualization (vagrant+virtualbox) your os (OSX/linux) your os (OSX/linux) your laptop your laptop
19. Workshop today • download http://we.tl/A5mQpDnEbi • vagrant box add virtualbox-xen.box --name virtualbox-xen • vagrant init virtualbox-xen • vagrant up • vagrant ssh • opam install utop
20. OCaml REPL: utop utop # let nine = 3 * 3;; val nine : int = 9 utop # let exp x = x * x;; val exp : int -> int = <fun> utop # let foo = let bar = 8 in bar + 2
21. OCaml REPL: utop utop # #require “cohttp” utop # #require “mirage” utop # open Mirage;; utop # Header.init_with;; - : string -> string -> Header.t = <fun>
22. OCaml LightWeight Threads (Lwt) utop # return;; - ‘a -> ‘a Lwt.t = <fun> utop # Lwt_main.run (return “Hi!”);; - : string = “Hi!” utop # Lwt_main.run (return ());; - : unit = ()
23. OCaml LightWeight Threads (Lwt) utop # (>>=);; - : ‘a Lwt.t -> (‘a -> ‘b Lwt.t) -> ‘b Lwt.t = <fun> utop # return "hallo" >>= fun p -> return ();; utop # (>|=);; - : 'a Lwt.t -> ('a -> 'b) -> 'b Lwt.t = <fun> utop # return “hallo” >|= fun p -> ();; (http://ocsigen.org/lwt/api/Lwt)
24. OCaml Module types (* https://github.com/mirage/mirage/blob/master/types/V1.mli#L157 *) open V1_LWT module Main (C: CONSOLE) = struct let start c = C.log c “Hello world”; return () end;;
25. OCaml Module types (* https://github.com/mirage/mirage/blob/master/types/V1.mli#L157 *) open V1_LWT module Main (C: CONSOLE) = struct let start c = C.log c “Hello world”; OS.Time.sleep 3.0 end;;
26. Mirage config.ml with console open Mirage let main = foreign "MyApp.Main" (console @-> job) let () = register "myapp" [ main $ default_console ]
27. Xen • mirage configure --xen • sudo xl create -c myapp.xl • sudo xl list • sudo xl destroy 6 • sudo xl destroy myapp
28. Mirage config with networking open Mirage let httpsrv = let stackv4 = direct_stackv4_with_default_ipv4 default_console tap0 in let conduit = conduit_direct (stackv4) in http_server conduit let main = let libraries = [ "cohttp" ] in foreign ~libraries "MyApp.Main" (http @-> console @-> job) let () = register "myapp" [ main $ httpsrv $ default_console ]
29. Mirage app with networking open Lwt open V1_LWT open Cohttp module Main (S: Cohttp_lwt.Server) (C: CONSOLE) = struct let start srv console = let callback conn_id request body = …. in srv (`TCP 8080) (S.make ~callback ()) end
30. Xen networking configuration • Add to myapp.xl ◦ vif = [ ‘bridge=br0’ ] • Linux plumbing ◦ sudo ifup br0 ◦ sudo ifconfig br0 add 10.0.0.1 ▪ this should also set up the routes • Test it ◦ telnet 10.0.0.1 8080
31. Hands-on time! Resources: • These sheets: http://bit.ly/1IWkhyd • Vagrant image: ◦ http://we.tl/A5mQpDnEbi ◦ Based on https://github.com/mattgray/mirage-xen-virtualbox • Example projects: ◦ https://github.com/mirage/mirage-skeleton ◦ especially ‘console’, ‘stackv4’, perhaps ‘conduit_server’ • And docs to go with them: ◦ https://mirage.io/wiki/hello-world • HTTP ◦ http://roscidus.com/blog/blog/2014/07/28/my-first-unikernel/#the-http- server