Save 37% off PRO during our Black Friday Sale! »

Clojure Redeployed

Clojure Redeployed

Slides from a talk I delivered on 28.03.2015 at LambdaCon in Bologna, Italy.

At stylefruits we've been using Clojure on production since late 2012. We've made a lot of mistakes and we've learned a lot. This talk is dedicated to the newest addition to our infrastructure: a Clojure REST-inspired API. Taught by our previous mistakes in the realm of deployment and scalability we've set our requirements high: we're aiming for continuous delivery, high availability, dynamic scaling and pauseless deployments under load. Aside from deployment matters, Clojure proves to be an excellent choice to build a modular and stateless service. In this talk we're going to share our experiences, both good and bad, gained while prototyping, deploying and maintaining our newest project.



Jan Stępień

March 28, 2015


  1. Clojure Redeployed Jan Stępień @janstepien

  2. © iwillbehomesoon 2012,

  3. I work at

  4. I work at

  5. None
  6. None
  7. Elasticsearch MySQL

  8. Elasticsearch MySQL So how about...

  9. Elasticsearch MySQL So how about... Clojure?

  10. Expectations set high ▶ short feedback loop ▶ continuous delivery

    ▶ automated and pauseless deployment ▶ ...under load ▶ high availability, dynamic scaling
  11. we’ve called it Ogrom* * plethora, immensity, enormity

  12. Outline 1. Architecture 2. Deployment and Operations 3. Lessons Learned

  13. Architecture to use a bold word

  14. sources protocols run core

  15. protocols define interfaces (defprotocol CollageSource (collage-by-id [ctx collage-id]))

  16. sources wrap data sources (reify CollageSource (collage-by-id [ctx collage-id] (fetch-from-mysql

    ctx collage-id)))
  17. sources protocols run core

  18. core defines HTTP endpoints each core endpoint ▶ is independent

    ▶ has all dependencies injected ▶ is absolute-path-agnostic ▶ is a value
  19. core uses Liberator (defresource product :exists? find-product-or-return-nil :handle-ok render-product :handle-not-found

  20. :exists? :handle-ok 200 true :handle-not-found 404 false

  21. core uses bidi (def routes ["/" {"product" product "ratings" ratings}])

  22. sources protocols run core

  23. run wires everything together products GET /products POST /comments comments

    GET /comments users
  24. /v1 /de-DE GET /products GET /comments POST /comments

  25. /v1 /de-DE GET /products GET /comments POST /comments GET /v1/de-DE/comments

  26. routing GET /comments POST /comments GET /products users third-party API

    MySQL products Elasticsearch comments
  27. run instantiates a self-contained service MySQL Elasticsearch logstash 3rd-party API

  28. MySQL Elasticsearch logstash 3rd-party API ogrom MySQL Elasticsearch logstash 3rd-party

    API ogrom MySQL Elasticsearch logstash 3rd-party API ogrom MySQL Elasticsearch logstash 3rd-party API ogrom
  29. /v1 /fr-FR /nl-NL /de-DE /en-GB GET /products GET /comments POST

  30. MySQL Elasticsearch logstash 3rd-party API ogrom

  31. Deployment and Operations

  32. Deployment used to be cumbersome 1. Create an überjar 2.

    Upload to n boxes 3. Restart processes 4. Juggle load balancers
  33. sources protocols run core ▶ sources, protocols, core are sem-versioned

    ▶ run is deployed straight from master
  34. sources run protocols core CI on every push to master

    S3 depl. tool überjar commit id
  35. Elastic Beanstalk and its environments ▶ EB wraps EC2 and

    ELB ▶ environments with instances ▶ runs Docker containers
  36. deployment tool AWS API CI commit id box 1 box

    2 S3 (überjar)
  37. Beanstalk 1 Beanstalk 2

  38. CloudFront Beanstalk 1 Beanstalk 2

  39. Lessons Learned and Future Work

  40. Feedback loops are shorter ▶ Productive development environment ▶ Bug

    fixes and features can be deploy the same morning they were asked for
  41. Splitting wasn’t strictly necessary sources protocols run core

  42. Docker and empowered developers ▶ Developers are in control over

    deployment ▶ Reduced dependency on the ops team ▶ Production env ≈ development env
  43. There are some trade-o s ▶ Beanstalk allows opening only

    a single port when running Docker containers ▶ Deployment under load vs. JIT compilation ▶ The deployment takes a while: Clojure compile times, JVM start-up time, etc. ▶ There’s no staging environment
  44. Ogrom is an improvement ▶ Client teams are happy with

    it ▶ It performs well under load and is durable ▶ Our main web app uses the Ogrom too now
  45. Plans for the future ▶ Ring Swagger ▶ Clojure 1.7

    and its reduced compilation time ▶ Clojure(Script) client API
  46. None
  47. Clojure Redeployed Jan Stępień @janstepien