Production Clojure: An Experience Report

by Shantanu Kumar

Slide 1

Slide 1 text

Production Clojure: An Experience Report Shantanu Kumar, Concur @kumarshantanu 1

Slide 2

Slide 2 text

Clojure @ Concur Rebuilding for scale, resilience, harmony 2

Slide 3

Slide 3 text

Scope • Rebuilding the core platform • Pervasive decoupling and fault tolerance • Common services across products • Authentication, Authorization, Proﬁle, Identity… 3

Slide 4

Slide 4 text

Approach • Web services (microservices) • Shared nothing architecture • N, N+1 versions • Open Standards • REST • JSON • JSON Web Token (JWT) • X509, PKI • OAuth2, SAML 4

Slide 5

Slide 5 text

Building microservices:  First cut — The prototype • Leiningen (for builds) • Conﬁguration properties ﬁle • Ring + Compojure • Jetty server (via uberjar) • clojure/java.jdbc (SQL Server) • couchbase-clj • 150-200ms response time 5

Slide 6

Slide 6 text

–Steve Brown “Anything worth doing is worth doing poorly until you learn to do it well.” 6

Slide 7

Slide 7 text

Initial challenges • Application conﬁguration/initialization • Logging • Policy based resiliency • Performance metrics/tuning 7

Slide 8

Slide 8 text

Challenge mitigation 8 (polite term for a framework) Developed a web service “Shell”

Slide 9

Slide 9 text

“Shell” constituents • Logging: SLF4j + Logback + tools.logging + JSON • Mapped Diagnostic Context (MDC) wrapper • Web server: http-kit • JSON serialization: Cheshire • c3p0 (Instrumented JDBC connection pool) • Clojure equivalent of Hystrix core 9

Slide 10

Slide 10 text

“Shell” App initialization 10

Slide 11

Slide 11 text

“Shell” Resiliency • In-house Clojure equivalent of Netﬂix Hystrix-core • Circuit breaker, Semaphore, Thread-pool, Metrics • Integrates with Hystrix-dashboard via SSE 11

Slide 12

Slide 12 text

Performance (after tuning) • Sub milli-second response time (within datacenter) • Because in microservices, every service adds up • Measure in ns, optimize microseconds • Simulate load, stress for long duration 12

Slide 13

Slide 13 text

External components • ElasticSearch, Logstash, Kibana (ELK) • Appdynamics • Yourkit proﬁler • Health checks (HTTP) • SQL Server • Couchbase 13

Slide 14

Slide 14 text

Clojure impressions • First class values • Immutability, isolation (safety) • Functional programming (HoF, Transducers) • REPL • Easy side effects (I/O) • Java interop • Small language, easy to learn & reason about • Homoiconicity and macros 14

Slide 15

Slide 15 text

## First class values 15

Slide 16

Slide 16 text

## Immutability, Isolation (Safety) • Epochal time model • Pro concurrency • Persistent data structures • Good performance 16

Slide 17

Slide 17 text

## Functional Programming 17

Slide 18

Slide 18 text

## Easy side effects 18

Slide 19

Slide 19 text

## Java Interop 19

Slide 20

Slide 20 text

## Homoiconicity & Macros 20

Slide 21

Slide 21 text

Practices • IDE/editor — To each his own (Emacs, Eclipse, vi) • Most things are conﬁgurable (properties ﬁle) • Tests and launch must work from command line • Unit and integration tests are a must • Docker in development, QA and production • Internal Maven repository for JAR artifacts 21

Slide 22

Slide 22 text

Libraries (Some factored out from web service “shell”) 22

Slide 23

Slide 23 text

Conﬁguration lookup — Keypin • https://github.com/kumarshantanu/keypin • Fail fast • Value parsing • Validation • Optional default 23

Slide 24

Slide 24 text

Keypin example 24

Slide 25

Slide 25 text

String concatenation — Stringer • https://github.com/kumarshantanu/stringer • Fast string concatenation • Pre-concatenation of literals at compile time • Caveat — not function 25

Slide 26

Slide 26 text

Stringer latency 26

Slide 27

Slide 27 text

Ring request matching — CalfPath • https://github.com/kumarshantanu/calfpath • À la carte • Low overhead • Composable 27

Slide 28

Slide 28 text

CalfPath latency 28

Slide 29

Slide 29 text

JDBC access — Asphalt • https://github.com/kumarshantanu/asphalt • Functional design • Named parameters • Extensible (works with SQL generation libraries) • Precise control via type hints 29

Slide 30

Slide 30 text

Asphalt example 30

Slide 31

Slide 31 text

Comparative micro- benchmarking — Citius • https://github.com/kumarshantanu/citius • Comparative benchmarks using Criterium • Simulate load with concurrency • Setup can be tweaked • Built-in comparative latency bar charts 31

Slide 32

Slide 32 text

Citius example 32

Slide 33

Slide 33 text

Inter-layer latency ﬁnder — Espejito • https://github.com/kumarshantanu/espejito • Finds latency across layers • Summary reporting (customizable) • Low overhead • Programmable for conditional trigger 33

Slide 34

Slide 34 text

Espejito examples 34

Slide 35

Slide 35 text

http://j.mp/packt-chpp2 35

Slide 36

Slide 36 text

Learning • Clojure (the language) is very well designed • Library eco-system is great, but still maturing • Java-interop is not the panacea • Adoption: Status-quo > other FP languages • Parentheses may cause knee-jerk reaction • Being on the JVM is a huge win 36

Slide 37

Slide 37 text

Thank you! @kumarshantanu 37