Slide 1

Slide 1 text

Finagle & Clojure by Alexey Kachayev for #FinagleCon 2015

Slide 2

Slide 2 text

About Me ‣ Alexey Kachayev, @kachayev ‣ CTO at Attendify.com • Almost-all-day-coding-kind-of-CTO ‣ Active open source contributor

Slide 3

Slide 3 text

Attendify Use Case Detailed Event information. Private event community and social network. A fully featured event app with essential content.

Slide 4

Slide 4 text

Attendify Use Case ‣ Private social networks for events (thousands) ‣ A lot of microservices both in Scala & Clojure ‣ Finagle to run services written in Scala ‣ Finagle to run services written in Clojure (???)

Slide 5

Slide 5 text

finagle-clojure ‣ https://github.com/finagle/finagle-clojure ‣ “A thin Clojure wrapper around Finagle” ‣ Scala interop ‣ Bridge to Thrift/ThriftMux and more ‣ lein template (quick start)

Slide 6

Slide 6 text

lein template $  lein  new  finagle-­‐clojure  schedule   $  tree  -­‐L  2   .   !""  schedule          #""  README.md          #""  project.clj          #""  schedule-­‐client          #""  schedule-­‐core          !""  schedule-­‐service

Slide 7

Slide 7 text

Thrift definition namespace  java  schedule.thrift   struct  SessionRequest  {      1:  string  id   }   struct  Session  {      1:  string  id      2:  string  title      3:  string  speaker   }   service  Schedule  {          Session  fetchSession(1:  SessionRequest  req)   }

Slide 8

Slide 8 text

Service implementation (ns  schedule.service      (:import  [schedule.thrift  Schedule  Session])      (:require  [finagle-­‐clojure.futures  :as  f]                          [finagle-­‐clojure.thrift  :as  thrift])      (:gen-­‐class))   (defn  make-­‐service      []      (thrift/service  Schedule          (fetchSession  [req]              (let  [id  (.id  req)]                  (f/value  (Session.  id  "Clojure"  "Alexey"))))))   (defn  -­‐main      [&  args]      (f/await  (thrift/serve  ":9999"  (make-­‐service))))

Slide 9

Slide 9 text

Client implementation (ns  schedule.client      (:import  [schedule.thrift  Schedule])      (:require  [finagle-­‐clojure.futures  :as  f]                          [finagle-­‐clojure.thrift  :as  thrift]))   (defn  make-­‐client      [address]      (thrift/client  address  Schedule))   (defn  session-­‐title  [client  id]      (-­‐>  (.fetchSession  client  (SessionRequest.  id))              (f/map  [v]  (.title  v))))

Slide 10

Slide 10 text

Client implementation (defn  speaker-­‐name  [client  session-­‐id]      (-­‐>  (.fetchSession  client  (SessionRequest.  session-­‐id))              (f/flatmap  [v]                  (let  [speaker-­‐id  (.speaker  v)                              req  (SpeakerRequest.  speaker-­‐id)]                      (-­‐>  (.fetchSpeaker  client  req)                              (f/map  [v]  (.firstName  v)))))))   (defn  -­‐main  [&  args]      (let  [c  (make-­‐client  "localhost:9999")]          (println  (f/await  (session-­‐title  c  "42")))          (println  (f/await  (speaker-­‐name  c  "42")))))

Slide 11

Slide 11 text

finagle-clojure? ‣ Scala API is not idiomatic for Clojure • easy to translate examples from the Internet • hard to play with the rest of the code ‣ Should we write Scala code using Clojure syntax?

Slide 12

Slide 12 text

Everything is a Data ‣ Builder API for servers & clients ‣ Hash-maps instead chained mutators ‣ Hash-maps instead of HTTP Request/Response builders

Slide 13

Slide 13 text

Not A Clojure (-­‐>  (builder-­‐server/builder)          (builder-­‐server/codec  http-­‐codec/http)          (builder-­‐server/bind-­‐to  3000)          (builder-­‐server/named  “test”)          (builder-­‐server/build  hello-­‐world)) (new-­‐server  hello-­‐world  {:codec  http-­‐codec/http                                                    :tls  nil                                                    :max-­‐request-­‐size  200                                                    :max-­‐response-­‐size  400                                                    :bind-­‐to  3000                                                    :name  :test})

Slide 14

Slide 14 text

Clojure (-­‐>  (builder-­‐server/builder)          (builder-­‐server/codec  http-­‐codec/http)          (builder-­‐server/bind-­‐to  3000)          (builder-­‐server/named  “test”)          (builder-­‐server/build  hello-­‐world)) (new-­‐server  hello-­‐world  {:codec  http-­‐codec/http                                                    :tls  nil                                                    :max-­‐request-­‐size  200                                                    :max-­‐response-­‐size  400                                                    :bind-­‐to  3000                                                    :name  :test})

Slide 15

Slide 15 text

Scala Futures in Clojure ‣ Two branches of execution (success & errors) ‣ Exception-driven errors handling ‣ Hard to manage execution context ‣ There is a better way to work with chaining

Slide 16

Slide 16 text

Clojure Futures ‣ A lot of “native” primitives: • futures, promises • agents • pmap, parallel reducers ‣ Clojure Futures are not that great for doing I/O

Slide 17

Slide 17 text

Manifold ‣ https://github.com/ztellman/manifold ‣ Abstraction for event-based async programming ‣ Streams & deferreds ‣ Deferreds: @, zip, chain, success, catch ‣ Pluggable concurrent executors

Slide 18

Slide 18 text

Finagle & Deferred (def  req  (SessionRequest.  "42"))   (def  f1  (.fetchSession  client  req))   (def  d4  (d/deferred))   (f/on-­‐success  f1  [v]  (d/success!  d4  v))   (f/on-­‐failure  f1  [v]  (d/error!  d4  v))   user>  @d4   user>  #object[Session[…]]   user>  @(d/chain  d4  #(.title  %))   user>  "Finagle  &  Clojure"

Slide 19

Slide 19 text

Finagle & Manifold (defn  future-­‐>deferred  [sf]      (let  [d1  (d/deferred)]          (f/on-­‐success  sf  [v]  (d/success!  d1  v))          (f/on-­‐failure  sf  [v]  (d/error!  d1  v))          d1))   (defn  fetch-­‐session  [c  id]      (f-­‐>d  (.fetchSession  c  (SessionRequest.  id))))   (defn  fetch-­‐speaker  [c  id]      (f-­‐>d  (.fetchSpeaker  c  (SpeakerRequest.  id))))

Slide 20

Slide 20 text

Let-Flow (defn  speaker-­‐name  [c  session]      (let-­‐flow  [session  (fetch-­‐session  c  session)                            sid  (.speaker  session)                            speaker  (fetch-­‐speaker  c  sid)]          (str  (.firstName  speaker)  "  "                      (.lastName  speaker))))   user>  (speaker-­‐name  client  "42")   user>  #object[Deferred]   user>  @(speaker-­‐name  client  "42")   user>  "Alexey  Kachayev"

Slide 21

Slide 21 text

Let-Flow (defn  speaker-­‐name  [c  session]      (let-­‐flow  [session  (fetch-­‐session  c  session)                            sid  (.speaker  session)                            speaker  (fetch-­‐speaker  c  sid)]          (str  (.firstName  speaker)  "  "                      (.lastName  speaker)))) values deferreds

Slide 22

Slide 22 text

core.async ‣ https://github.com/clojure/core.async ‣ “Library designed to provide facilities for async programming and communication” ‣ CSP as a library ‣ go macro to turn async code into a state machine ‣ Widely adopted by Clojure community

Slide 23

Slide 23 text

Finagle & core.async (defn  future-­‐>chan  [sf]      (let  [c1  (a/chan  1)]          (f/on-­‐success  sf  [v]  (a/put!  c1  v))          (f/on-­‐failure  sf  [v]  (a/put!  c1  (e/left  v)))          c1))   (defn  fetch-­‐session  [c  id]      (f-­‐>c  (.fetchSession  c  (SessionRequest.  id))))   (defn  fetch-­‐speaker  [c  id]      (f-­‐>c  (.fetchSpeaker  c  (SpeakerRequest.  id))))

Slide 24

Slide 24 text

Finagle & core.async (defn  speaker-­‐name  [c  session-­‐id]      (go          (let  [session  (  (speaker-­‐name  client  "42")   user>  #object[ManyToManyChannel]   user>  (  "Alexey  Kachayev"

Slide 25

Slide 25 text

Finagle & core.async (defn  speaker-­‐name  [c  session-­‐id]      (go          (let  [session  (

Slide 26

Slide 26 text

What About Server (:import  [com.twitter.util  Promise])   (defn  propagate-­‐to  [promise  value]      (if  (e/left?  value)          (.setException  promise  (e/left-­‐value  value))          (.setValue  promise  (e/right-­‐value  value))))   (defn  chan-­‐>future  [c]      (let  [promise  (Promise.)]          (a/take!  c  (partial  propagate-­‐to  promise))          promise))

Slide 27

Slide 27 text

What About Server (:import  [com.twitter.util  Promise])   (defn  propagate-­‐to  [promise  value]      (if  (e/left?  value)          (.setException  promise  (e/left-­‐value  value))          (.setValue  promise  (e/right-­‐value  value))))   (defn  chan-­‐>future  [c]      (let  [promise  (Promise.)]          (a/take!  c  (partial  propagate-­‐to  promise))          promise))

Slide 28

Slide 28 text

Finagle & core.async ‣ Clean & concise async code ‣ A lot of built-in primitives (timeout, pub/sub etc) ‣ Compatibility with tons of Clojure libraries based on core.async ‣ Limited usage in request-reply world though

Slide 29

Slide 29 text

Stitch → Muse ‣ Stitch is a Scala library for composing RPC services ‣ Created in Twitter, introduces by Jake Donham ‣ As Stitch is not open sourced… ‣ github.com/kachayev/muse ‣ EuroClojure talk about it

Slide 30

Slide 30 text

Stitch → Muse ‣ Runs independent data fetches concurrently • Uses BFS to group fetches level-by-level ‣ Caches previously made fetches during execution ‣ Batches requests when applicable ‣ Uses the idea of building and interpreting AST ‣ Uses core.async to deal with concurrency

Slide 31

Slide 31 text

More Clojure Codec ‣ Scala has Scodec & finagle-serial ‣ Clojure has Fressian • Designed with EDN in mind • Rich set of core types & extensions • Middleware-friendly with tagged objects ‣ Working on Fressian codec right now

Slide 32

Slide 32 text

EDN & Fressian {:id  42    :title  "Finagle  &  Clojure"    :tracks  ["Libraries"  "Practice"]    :tags  #{"Finagle"  "Clojure"}    :speaker  {:name  "Alexey  Kachayev"}    :at  #inst  "2015-­‐08-­‐13T14:30:00"    :duration  #duration  "10s"}

Slide 33

Slide 33 text

EDN & Fressian {:id  42    :title  "Finagle  &  Clojure"    :tracks  ["Libraries"  "Practice"]    :tags  #{"Finagle"  "Clojure"}    :speaker  {:name  "Alexey  Kachayev"}    :at  #inst  "2015-­‐08-­‐13T14:30:00"    :duration  #duration  "10s"} tags

Slide 34

Slide 34 text

Conclusion ‣ Finagle is great ‣ Clojure is amazing ‣ Finagle can solve a lot of problems for Clojure ecosystem ‣ Finagle & Clojure integration still requires a lot of work

Slide 35

Slide 35 text

Thank You! Questions?