Native Clojure

GraalVM challenges the status quo on the JVM. This newly-released just-in-time compiler brings substantial speed improvements and support for polyglot applications. It also allows us to translate our JVM bytecode into small self-contained native binaries. In this session we’ll explore Graal’s impact on Clojure.

We’ll use GraalVM to build native binaries with simple command-line tools. We’ll discuss the method’s limitations and their impact. Finally, we’ll build complete Clojure web applications weighing a fraction of their traditional JVM incarnations.

As seen at ClojuTRE 2018 in Helsinki.

Transcript

1. Native Clojure
 with GraalVM Jan Stępień @janstepien 14.09.2018 // ClojuTRE
2. graalvm.org
3. Native Clojure
5. (ns pprint.main (:require [clojure.pprint :refer [pprint]]) (:gen-class)) (defn -main [& path] (-> (read *in*) (get-in (mapv read-string path)) pprint))
6. $ echo {:a [5 3]} \ | time java -jar pprint.jar :a 1 3 1.51 real 112MB maximum resident set size
7. $ echo {:a [5 3]} \ | java -XX:TieredStopAtLevel=1 \ -jar pprint.jar :a 1 3 1.06 real 96MB maximum resident set size
8. $ echo {:a [5 3]} \ | time lumo pprint.cljs :a 1 3 0.56 real 128MB maximum resident set size
9. $ native-image --no-server \ --static -jar pprint.jar classlist: 3,596.11 ms (...) image: 4,671.96 ms write: 14,763.97 ms [total]: 122,614.21 ms
10. $ echo {:a [5 3]} \ | time ./pprint :a 1 3 0.01 real 12MB maximum resident set size
11. Time Memory JVM 1.10 s 100 MB Lumo 0.60 s 130 MB Native 0.01 s 12 MB
12. Command Line Tools
13. (ns json2edn.core (:require [clojure.data.json :as json])) (defn -main [] (->> (repeatedly #(json/read *in* :key-fn keyword :eof-error? false :eof-value ::eof)) (take-while #(not= ::eof %)) (run! prn))) ;; See also Taylor Wood’s piece at ;; blog.taylorwood.io/2018/05/02/graal-clojure
14. $ cat package.json | time json2edn {:description "Check if a value is an object", :engines {:node ">=0.10.0"}, :license "MIT", :repository "sindresorhus/is-obj", :name "is-obj", :scripts {:test "xo && ava"}, :keywords ["obj" "object" "is" "check" "test" "type"], :author {:name "Sindre Sorhus", :email "sindresorhus@gmail.com", :url "sindresorhus.com"}, :files ["index.js"], :version “1.0.1"} 0.01 real 10MB maximum resident set size
16. user=> (require '[spec-provider.provider :as sp]) nil user=> (def inferred-specs #_=> (sp/infer-specs #_=> [{:a 8 :b "foo" :c :k} #_=> {:a 10 :b "bar" :c "k"} #_=> {:a 1 :b "baz" :c "k"}] #_=> :toy/small-map)) #'user/inferred-specs user=>
17. user=> (require '[spec-provider.provider :as sp]) nil user=> (def inferred-specs #_=> (sp/infer-specs #_=> [{:a 8 :b "foo" :c :k} #_=> {:a 10 :b "bar" :c "k"} #_=> {:a 1 :b "baz" :c "k"}] #_=> :toy/small-map)) #'user/inferred-specs user=> (sp/pprint-specs inferred-specs 'toy 's) (s/def ::c (s/or :keyword keyword? :string string?)) (s/def ::b string?) (s/def ::a integer?) (s/def ::small-map (s/keys :req-un [::a ::b ::c]))
18. (ns spec-provider.main (:require [spec-provider.provider :as sp] [clojure.string :as str] [clojure.edn :as edn]) (:gen-class)) (defn -main [spec-name] (let [kw (edn/read-string spec-name) read-one #(edn/read {:eof ::eof} *in*) samples (sequence (take-while (complement #{::eof})) (repeatedly read-one)) specs (sp/infer-specs samples kw)] (sp/pprint-specs specs (namespace kw) 's))))
20. $ cat node_modules/*/package.json \ | json2edn | spec-provider :node/package | zprint
21. $ cat node_modules/*/package.json \ | json2edn | spec-provider :node/package | zprint ;; ... (s/def ::repository (s/or :map (s/keys :req-un [::type ::url]) :simple string?)) (s/def ::engines (s/keys :req-un [::node] :opt-un [::iojs ::npm])) (s/def ::description string?) (s/def ::package (s/keys :req-un [::description ::name ::repository ::version] :opt-un [::author ::bin ::browser ::bugs ::contributors ::dependencies ::devDependencies ::engines ::files ::homepage ::icon ::keywords ::license ::main ::maintainers ::optionalDependencies ::peerDependencies ::scripts ::verb ::xo])) 0.29 real 103MB maximum resident set size
22. Web Applications
23. $ curl localhost:8080/kv/city -i -X PUT -d val=Helsinki HTTP/1.1 201 Created Content-Length: 9 Date: Fri, 14 Sep 2018 10:04:22 GMT Helsinki $ curl localhost:8080/kv/city -i HTTP/1.1 200 OK Content-Length: 9 Date: Fri, 14 Sep 2018 10:04:27 GMT Helsinki
24. (ns webkv.main (:require [org.httpkit.server :as http] [ring.middleware.defaults :refer [wrap-defaults api-defaults]] [bidi.ring :refer [make-handler]]) (:gen-class)) ;; We want to be sure none of our calls relies ;; on reflection. (set! *warn-on-reflection* 1) ;; This is where we store our data. (def ^String tmpdir (System/getProperty "java.io.tmpdir")) ;; That's how we find a file given a key. ;; Keys must match the given pattern. (defn file [^String key] {:pre [(re-matches #"^[A-Za-z-]+$" key)]} (java.io.File. tmpdir key)) ;; Here we handle GET requests. We just ;; read from a file. (defn get-handler [{:keys [params]}] {:body (str (slurp (file (params :key))) "\n")}) ;; This is our PUT request handler. Given ;; a key and a value we write to a file. (defn put-handler [{:keys [params]}] (let [val (params :val)] (spit (file (params :key)) val) {:body (str val "\n"), :status 201})) ;; Here's the routing tree of our application. ;; We pick the handler depending on the HTTP ;; verb. On top of that we add an extra middle- ;; ware to parse data sent in requests. (def handler (-> ["/kv/" {[:key] {:get #'get-handler :put #'put-handler}}] (make-handler) (wrap-defaults api-defaults))) ;; Finally, we've got all we need to expose ;; our handler over HTTP. (defn -main [] (http/run-server handler {:port 8080}) (println " http://localhost:8080")) 7 M B
25. FROM ubuntu AS BASE RUN apt-get update RUN apt-get install -yy curl leiningen build-essential zlib1g-dev RUN cd /opt && curl -sL https://github.com/.../graalvm.tar.gz \ | tar -xzvf - ADD project.clj . RUN lein deps ADD src src RUN lein uberjar RUN /opt/graalvm-ce-1.0.0-rc6/bin/native-image \ -H:EnableURLProtocols=http --static --no-server \ -cp target/webkv-0.0.0-standalone.jar webkv.main FROM scratch COPY --from=BASE /webkv.main / CMD ["/webkv.main"] 13 M B
26. How Is That Even Possible
27. Time Memory JVM 1.10 s 100 MB Lumo 0.60 s 130 MB Native 0.01 s 12 MB
29. static { ... } initialiser Static class
30. https:/ /blog.ndk.io/clojure-compilation
32. Limitations
34. ;; Reflection (set! *warn-on-reflection* true)
35. ;; Dynamic class loading (require 'example.core :reload)
36. github.com/taylorwood/lein-native-image github.com/taylorwood/clj.native-image
37. Jan Stępień @janstepien janstepien.com Native Clojure
 with GraalVM 14.09.2018 // ClojuTRE
38. JEP 243 JVM Compiler Interface JD K 9
39. Truffle TruffleRuby GraalVM JVMCI Graal JVM