Save 37% off PRO during our Black Friday Sale! »

Moving People with Clojure

Moving People with Clojure

At Go-Jek, one of Indonesia’s fastest growing startups, we use Clojure in production to efficiently provide courier, transport, and shopping services all over the country. In this talk we will cover our journey of rewriting and maintaining Allocation Service, core of Go-Jek business, in Clojure. This service matches the best driver from a fleet of more than 100k drivers at the rate of more than 100k bookings per hour efficiently, with only 8 machines, and the rate doubling every six months. Switching to Clojure has enabled us to ship new features at 5x speed compared to the previous (Golang) service. Through this talk we intend to share the learnings, difficulties, and motivations for the same.

4d29918c109bc75d2a1fd8420660d72b?s=128

Abhinav Sarkar

July 20, 2017
Tweet

Transcript

  1. Moving People with Clojure EuroClojure 2017 / Berlin Kartik Gupta

    & Abhinav Sarkar
  2. None
  3. • Indonesian transport, logistics and payments startup • First billion

    dollar startup in Indonesia • Operating in more than 25 cities with more than 300.000 motorbike and car drivers • More than 25 products, using the same fleet • Processing more than 1000 orders per second
  4. None
  5. Prologue

  6. Prologue Jan 2015 App launched 800 drivers New verticals 30.000

    drivers Java monolith Go service Aug 2015
  7. Prologue Feb 2016 Business grew 200.000 drivers Rewrite starts Go

    service Clojure service Aug 2016 Rewrite ends Oct 2016 Rewrite in prod Nov 2016
  8. The Rewrite

  9. Architecture Customers Order Management Systems Allocation Service Drivers Call Callback

  10. Why Rewrite? • Go is imperative and mutation heavy •

    Domain heavy code is hard to reason about • Moving fast is more important than high performance • Immutability, laziness and dynamic typing in Clojure • leads to simpler domain specific code
  11. None
  12. None
  13. Testing the Rewrite • Two weeks in testing • Duplicating

    live traffic • Feature toggles • Mock downstream services • Match data and external effects
  14. Duplicate Traffic Live Traffic Shared Services Actual Services Mock Services

    Toggled-off Services Go Allocation Service Clojure Allocation Service OMSes
  15. Releasing the Rewrite • AOT compiled uberjar • saves startup

    time • Monitoring: application and business metrics, errors • Incremental rollout over two weeks
  16. 1% Live Traffic 99% Live Traffic Go Allocation Service Clojure

    Allocation Service OMSes
  17. Lessons Learned

  18. Generative Testing • At all levels (unit, functional and integration)

    • Using test.check library • Handwritten generators
  19. Generative Testing

  20. Generative Testing

  21. prismatic/schema

  22. swagger

  23. • The protected function call is wrapped in a circuit

    breaker object, which monitors for failures. • Once the failures reach a certain threshold, the circuit breaker trips. • Once tripped all further calls to the circuit breaker return with an error, without the protected call being made at all. Circuit Breaker
  24. Netflix Hystrix

  25. Hystrix Dashboard

  26. etcd service nodes Config change request Config change callbacks load

    balancer etcd Config change call
  27. tolitius/mount

  28. tolitius/mount

  29. Either Monad

  30. Either Monad

  31. Epilogue

  32. Epilogue • Around 1 new release per week • Experimentation

    • Clojure Sessions • More services in Clojure
  33. Thank You Abhinav Sarkar @abhin4v nilenso.com @nilenso Kartik Gupta @kartik7153

    gojekengineering.com @gojektech