Finagle & Clojure

Finagle & Clojure by Alexey Kachayev for #FinagleCon 2015

About Me ‣ Alexey Kachayev, @kachayev ‣ CTO at Attendify.com
• Almost-all-day-coding-kind-of-CTO ‣ Active open source contributor

Attendify Use Case Detailed Event information. Private event community and
social network. A fully featured event app with essential content.

Attendify Use Case ‣ Private social networks for events (thousands)
‣ A lot of microservices both in Scala & Clojure ‣ Finagle to run services written in Scala ‣ Finagle to run services written in Clojure (???)

finagle-clojure ‣ https://github.com/finagle/finagle-clojure ‣ “A thin Clojure wrapper around Finagle”
‣ Scala interop ‣ Bridge to Thrift/ThriftMux and more ‣ lein template (quick start)

lein template $ lein new finagle-‐clojure schedule $ tree
-‐L 2 . !"" schedule #"" README.md #"" project.clj #"" schedule-‐client #"" schedule-‐core !"" schedule-‐service

Thrift deﬁnition namespace java schedule.thrift struct SessionRequest {
1: string id } struct Session { 1: string id 2: string title 3: string speaker } service Schedule { Session fetchSession(1: SessionRequest req) }

Service implementation (ns schedule.service (:import [schedule.thrift Schedule Session])
(:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift]) (:gen-‐class)) (defn make-‐service [] (thrift/service Schedule (fetchSession [req] (let [id (.id req)] (f/value (Session. id "Clojure" "Alexey")))))) (defn -‐main [& args] (f/await (thrift/serve ":9999" (make-‐service))))

Client implementation (ns schedule.client (:import [schedule.thrift Schedule])
(:require [finagle-‐clojure.futures :as f] [finagle-‐clojure.thrift :as thrift])) (defn make-‐client [address] (thrift/client address Schedule)) (defn session-‐title [client id] (-‐> (.fetchSession client (SessionRequest. id)) (f/map [v] (.title v))))

Client implementation (defn speaker-‐name [client session-‐id] (-‐> (.fetchSession
client (SessionRequest. session-‐id)) (f/flatmap [v] (let [speaker-‐id (.speaker v) req (SpeakerRequest. speaker-‐id)] (-‐> (.fetchSpeaker client req) (f/map [v] (.firstName v))))))) (defn -‐main [& args] (let [c (make-‐client "localhost:9999")] (println (f/await (session-‐title c "42"))) (println (f/await (speaker-‐name c "42")))))

ﬁnagle-clojure? ‣ Scala API is not idiomatic for Clojure •
easy to translate examples from the Internet • hard to play with the rest of the code ‣ Should we write Scala code using Clojure syntax?

Everything is a Data ‣ Builder API for servers &
clients ‣ Hash-maps instead chained mutators ‣ Hash-maps instead of HTTP Request/Response builders

Not A Clojure (-‐> (builder-‐server/builder) (builder-‐server/codec
http-‐codec/http) (builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world)) (new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})

Clojure (-‐> (builder-‐server/builder) (builder-‐server/codec http-‐codec/http)
(builder-‐server/bind-‐to 3000) (builder-‐server/named “test”) (builder-‐server/build hello-‐world)) (new-‐server hello-‐world {:codec http-‐codec/http :tls nil :max-‐request-‐size 200 :max-‐response-‐size 400 :bind-‐to 3000 :name :test})

Scala Futures in Clojure ‣ Two branches of execution (success
& errors) ‣ Exception-driven errors handling ‣ Hard to manage execution context ‣ There is a better way to work with chaining

Clojure Futures ‣ A lot of “native” primitives: • futures,
promises • agents • pmap, parallel reducers ‣ Clojure Futures are not that great for doing I/O

Manifold ‣ https://github.com/ztellman/manifold ‣ Abstraction for event-based async programming ‣
Streams & deferreds ‣ Deferreds: @, zip, chain, success, catch ‣ Pluggable concurrent executors

Finagle & Deferred (def req (SessionRequest. "42")) (def f1
(.fetchSession client req)) (def d4 (d/deferred)) (f/on-‐success f1 [v] (d/success! d4 v)) (f/on-‐failure f1 [v] (d/error! d4 v)) user> @d4 user> #object[Session[…]] user> @(d/chain d4 #(.title %)) user> "Finagle & Clojure"

Finagle & Manifold (defn future-‐>deferred [sf] (let [d1
(d/deferred)] (f/on-‐success sf [v] (d/success! d1 v)) (f/on-‐failure sf [v] (d/error! d1 v)) d1)) (defn fetch-‐session [c id] (f-‐>d (.fetchSession c (SessionRequest. id)))) (defn fetch-‐speaker [c id] (f-‐>d (.fetchSpeaker c (SpeakerRequest. id))))

Let-Flow (defn speaker-‐name [c session] (let-‐flow [session (fetch-‐session
c session) sid (.speaker session) speaker (fetch-‐speaker c sid)] (str (.firstName speaker) " " (.lastName speaker)))) user> (speaker-‐name client "42") user> #object[Deferred] user> @(speaker-‐name client "42") user> "Alexey Kachayev"

Let-Flow (defn speaker-‐name [c session] (let-‐flow [session (fetch-‐session
c session) sid (.speaker session) speaker (fetch-‐speaker c sid)] (str (.firstName speaker) " " (.lastName speaker)))) values deferreds

core.async ‣ https://github.com/clojure/core.async ‣ “Library designed to provide facilities for
async programming and communication” ‣ CSP as a library ‣ go macro to turn async code into a state machine ‣ Widely adopted by Clojure community

Finagle & core.async (defn future-‐>chan [sf] (let [c1
(a/chan 1)] (f/on-‐success sf [v] (a/put! c1 v)) (f/on-‐failure sf [v] (a/put! c1 (e/left v))) c1)) (defn fetch-‐session [c id] (f-‐>c (.fetchSession c (SessionRequest. id)))) (defn fetch-‐speaker [c id] (f-‐>c (.fetchSpeaker c (SpeakerRequest. id))))

Finagle & core.async (defn speaker-‐name [c session-‐id] (go
(let [session (<! (fetch-‐session c session-‐id)) speaker-‐id (.speaker session) speaker (<! (fetch-‐speaker c speaker-‐id))] (str (.firstName speaker) " " (.lastName speaker))))) user> (speaker-‐name client "42") user> #object[ManyToManyChannel] user> (<!! (speaker-‐name client "42")) user> "Alexey Kachayev"

Finagle & core.async (defn speaker-‐name [c session-‐id] (go
(let [session (<! (fetch-‐session c session-‐id)) speaker-‐id (.speaker session) speaker (<! (fetch-‐speaker c speaker-‐id))] (str (.firstName speaker) " " (.lastName speaker))))) state machine

What About Server (:import [com.twitter.util Promise]) (defn propagate-‐to [promise
value] (if (e/left? value) (.setException promise (e/left-‐value value)) (.setValue promise (e/right-‐value value)))) (defn chan-‐>future [c] (let [promise (Promise.)] (a/take! c (partial propagate-‐to promise)) promise))

Finagle & core.async ‣ Clean & concise async code ‣
A lot of built-in primitives (timeout, pub/sub etc) ‣ Compatibility with tons of Clojure libraries based on core.async ‣ Limited usage in request-reply world though

Stitch → Muse ‣ Stitch is a Scala library for
composing RPC services ‣ Created in Twitter, introduces by Jake Donham ‣ As Stitch is not open sourced… ‣ github.com/kachayev/muse ‣ EuroClojure talk about it

Stitch → Muse ‣ Runs independent data fetches concurrently •
Uses BFS to group fetches level-by-level ‣ Caches previously made fetches during execution ‣ Batches requests when applicable ‣ Uses the idea of building and interpreting AST ‣ Uses core.async to deal with concurrency

More Clojure Codec ‣ Scala has Scodec & ﬁnagle-serial ‣
Clojure has Fressian • Designed with EDN in mind • Rich set of core types & extensions • Middleware-friendly with tagged objects ‣ Working on Fressian codec right now

EDN & Fressian {:id 42 :title "Finagle & Clojure"
:tracks ["Libraries" "Practice"] :tags #{"Finagle" "Clojure"} :speaker {:name "Alexey Kachayev"} :at #inst "2015-‐08-‐13T14:30:00" :duration #duration "10s"}

EDN & Fressian {:id 42 :title "Finagle & Clojure"
:tracks ["Libraries" "Practice"] :tags #{"Finagle" "Clojure"} :speaker {:name "Alexey Kachayev"} :at #inst "2015-‐08-‐13T14:30:00" :duration #duration "10s"} tags

Conclusion ‣ Finagle is great ‣ Clojure is amazing ‣
Finagle can solve a lot of problems for Clojure ecosystem ‣ Finagle & Clojure integration still requires a lot of work

Thank You! Questions?

Finagle & Clojure

Finagle & Clojure

More Decks by Oleksii Kachaiev

Other Decks in Programming

Featured

Transcript