Finagle & Clojure - Speaker Deck

Slide 1

Slide 1 text

Finagle & Clojure by Alexey Kachayev for #FinagleCon 2015

Slide 2

Slide 2 text

About Me ‣ Alexey Kachayev, @kachayev ‣ CTO at Attendify.com • Almost-all-day-coding-kind-of-CTO ‣ Active open source contributor

Slide 3

Slide 3 text

Attendify Use Case Detailed Event information. Private event community and social network. A fully featured event app with essential content.

Slide 4

Slide 4 text

Attendify Use Case ‣ Private social networks for events (thousands) ‣ A lot of microservices both in Scala & Clojure ‣ Finagle to run services written in Scala ‣ Finagle to run services written in Clojure (???)

Slide 5

Slide 5 text

finagle-clojure ‣ https://github.com/finagle/finagle-clojure ‣ “A thin Clojure wrapper around Finagle” ‣ Scala interop ‣ Bridge to Thrift/ThriftMux and more ‣ lein template (quick start)

Slide 6

Slide 6 text

lein template $ lein new finagle-‐clojure schedule $ tree -‐L 2 . !"" schedule #"" README.md #"" project.clj #"" schedule-‐client #"" schedule-‐core !"" schedule-‐service

Slide 7

Slide 7 text

Thrift deﬁnition namespace java schedule.thrift struct SessionRequest { 1: string id } struct Session { 1: string id 2: string title 3: string speaker } service Schedule { Session fetchSession(1: SessionRequest req) }

Slide 8

Slide 8 text

Service implementation (ns schedule.service (:import [schedule.thrift Schedule Session]) (:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift]) (:gen-‐class)) (defn make-‐service [] (thrift/service Schedule (fetchSession [req] (let [id (.id req)] (f/value (Session. id "Clojure" "Alexey")))))) (defn -‐main [& args] (f/await (thrift/serve ":9999" (make-‐service))))

Slide 9

Slide 9 text

Client implementation (ns schedule.client (:import [schedule.thrift Schedule]) (:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift])) (defn make-‐client [address] (thrift/client address Schedule)) (defn session-‐title [client id] (-‐> (.fetchSession client (SessionRequest. id)) (f/map [v] (.title v))))

Slide 10

Slide 10 text

Client implementation (defn speaker-‐name [client session-‐id] (-‐> (.fetchSession client (SessionRequest. session-‐id)) (f/flatmap [v] (let [speaker-‐id (.speaker v) req (SpeakerRequest. speaker-‐id)] (-‐> (.fetchSpeaker client req) (f/map [v] (.firstName v))))))) (defn -‐main [& args] (let [c (make-‐client "localhost:9999")] (println (f/await (session-‐title c "42"))) (println (f/await (speaker-‐name c "42")))))

Slide 11

Slide 11 text

ﬁnagle-clojure? ‣ Scala API is not idiomatic for Clojure • easy to translate examples from the Internet • hard to play with the rest of the code ‣ Should we write Scala code using Clojure syntax?

Slide 12

Slide 12 text

Everything is a Data ‣ Builder API for servers & clients ‣ Hash-maps instead chained mutators ‣ Hash-maps instead of HTTP Request/Response builders

Slide 13

Slide 13 text

Not A Clojure (-‐> (builder-‐server/builder) (builder-‐server/codec http-‐codec/http) (builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world)) (new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})

Slide 14

Slide 14 text

Clojure (-‐> (builder-‐server/builder) (builder-‐server/codec http-‐codec/http) (builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world)) (new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})

Slide 15

Slide 15 text

Scala Futures in Clojure ‣ Two branches of execution (success & errors) ‣ Exception-driven errors handling ‣ Hard to manage execution context ‣ There is a better way to work with chaining

Slide 16

Slide 16 text

Clojure Futures ‣ A lot of “native” primitives: • futures, promises • agents • pmap, parallel reducers ‣ Clojure Futures are not that great for doing I/O

Slide 17

Slide 17 text

Manifold ‣ https://github.com/ztellman/manifold ‣ Abstraction for event-based async programming ‣ Streams & deferreds ‣ Deferreds: @, zip, chain, success, catch ‣ Pluggable concurrent executors

Slide 18

Slide 18 text

Finagle & Deferred (def req (SessionRequest. "42")) (def f1 (.fetchSession client req)) (def d4 (d/deferred)) (f/on-‐success f1 [v] (d/success! d4 v)) (f/on-‐failure f1 [v] (d/error! d4 v)) user> @d4 user> #object[Session[…]] user> @(d/chain d4 #(.title %)) user> "Finagle & Clojure"

Slide 19

Slide 19 text

Finagle & Manifold (defn future-‐>deferred [sf] (let [d1 (d/deferred)] (f/on-‐success sf [v] (d/success! d1 v)) (f/on-‐failure sf [v] (d/error! d1 v)) d1)) (defn fetch-‐session [c id] (f-‐>d (.fetchSession c (SessionRequest. id)))) (defn fetch-‐speaker [c id] (f-‐>d (.fetchSpeaker c (SpeakerRequest. id))))

Slide 20

Slide 20 text

Let-Flow (defn speaker-‐name [c session] (let-‐flow [session (fetch-‐session c session) sid (.speaker session) speaker (fetch-‐speaker c sid)] (str (.firstName speaker) " " (.lastName speaker)))) user> (speaker-‐name client "42") user> #object[Deferred] user> @(speaker-‐name client "42") user> "Alexey Kachayev"

Slide 21

Slide 21 text

Slide 22

Slide 22 text

core.async ‣ https://github.com/clojure/core.async ‣ “Library designed to provide facilities for async programming and communication” ‣ CSP as a library ‣ go macro to turn async code into a state machine ‣ Widely adopted by Clojure community

Slide 23

Slide 23 text

Finagle & core.async (defn future-‐>chan [sf] (let [c1 (a/chan 1)] (f/on-‐success sf [v] (a/put! c1 v)) (f/on-‐failure sf [v] (a/put! c1 (e/left v))) c1)) (defn fetch-‐session [c id] (f-‐>c (.fetchSession c (SessionRequest. id)))) (defn fetch-‐speaker [c id] (f-‐>c (.fetchSpeaker c (SpeakerRequest. id))))

Slide 24

Slide 24 text

Finagle & core.async (defn speaker-‐name [c session-‐id] (go (let [session ( (speaker-‐name client "42") user> #object[ManyToManyChannel] user> ( "Alexey Kachayev"

Slide 25

Slide 25 text

Finagle & core.async (defn speaker-‐name [c session-‐id] (go (let [session (

Slide 26

Slide 26 text

What About Server (:import [com.twitter.util Promise]) (defn propagate-‐to [promise value] (if (e/left? value) (.setException promise (e/left-‐value value)) (.setValue promise (e/right-‐value value)))) (defn chan-‐>future [c] (let [promise (Promise.)] (a/take! c (partial propagate-‐to promise)) promise))

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Finagle & core.async ‣ Clean & concise async code ‣ A lot of built-in primitives (timeout, pub/sub etc) ‣ Compatibility with tons of Clojure libraries based on core.async ‣ Limited usage in request-reply world though

Slide 29

Slide 29 text

Stitch → Muse ‣ Stitch is a Scala library for composing RPC services ‣ Created in Twitter, introduces by Jake Donham ‣ As Stitch is not open sourced… ‣ github.com/kachayev/muse ‣ EuroClojure talk about it

Slide 30

Slide 30 text

Stitch → Muse ‣ Runs independent data fetches concurrently • Uses BFS to group fetches level-by-level ‣ Caches previously made fetches during execution ‣ Batches requests when applicable ‣ Uses the idea of building and interpreting AST ‣ Uses core.async to deal with concurrency

Slide 31

Slide 31 text

More Clojure Codec ‣ Scala has Scodec & ﬁnagle-serial ‣ Clojure has Fressian • Designed with EDN in mind • Rich set of core types & extensions • Middleware-friendly with tagged objects ‣ Working on Fressian codec right now

Slide 32

Slide 32 text

EDN & Fressian {:id 42 :title "Finagle & Clojure" :tracks ["Libraries" "Practice"] :tags #{"Finagle" "Clojure"} :speaker {:name "Alexey Kachayev"} :at #inst "2015-‐08-‐13T14:30:00" :duration #duration "10s"}

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Conclusion ‣ Finagle is great ‣ Clojure is amazing ‣ Finagle can solve a lot of problems for Clojure ecosystem ‣ Finagle & Clojure integration still requires a lot of work

Slide 35

Slide 35 text

Thank You! Questions?