Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Clojure Ireland Talk June 2015

Clojure Ireland Talk June 2015

Brief rundown of some Netflix libraries we've found useful when rolling out new backend services at Logentries.

Chris Mowforth

June 03, 2015

More Decks by Chris Mowforth

Other Decks in Programming



  2. ME Chris Mowforth chris[at]mowforth.com @m0wfo

  3. CAVEATS 1. No affiliation with Netflix! 2. Just because it

    works for us…
  4. LOGENTRIES • Founded in ’10 • Massive growth: • Customer

    base • Volume sent • Volume searched Now we have to make it scale!
  5. IN THE BEGINNING Log processing infrastructure Web Tier (Java) (Django/JS)

  6. LATER ON Log processing infrastructure Web Tier (Java) (Django/JS) Browser

    Service x (JVM-based) Service y (JVM-based) Service ? (JVM-based) We went hiring :)
  7. LATER STILL… We got some dev-ops guys Log processing infrastructure

    Web Tier (Java) (Django/JS) Browser Service x (JVM-based) Service y (JVM-based) Service ? (JVM-based) Magic continuous-delivery system
  8. CONSTRAINTS 1. Avoid new services incurring
 excessive ops burden 2.

    Minimise engineering time spent
 developing non-essential* IP
  9. NON-ESSENTIAL? People value us for: • Reliable log ingestion •

    Real-time alerts • Simple, expressive query language • Extending our analytics capabilities
  10. NON-ESSENTIAL? This is critical, but not what we’re
 valued for:

    • Configuration • Load-balancing • Fault-tolerance (bulkheading) • Metrics & monitoring • Routing, production debugging, TiP
  11. NON-ESSENTIAL? Unless you have a good reason, you should allocate

    engineering time to the former.
  12. NON-ESSENTIAL? A common reason is, “there’s no off-the-shelf FOSS solution”

    Enter Netflix :)
  13. [email protected] Problem Solution Configuration Archaius Async / Reactive APIs Rx{Java/Scala/Clojure/…}

    Fault-tolerance / Monitoring Hystrix / Turbine

  15. ARCHAIUS Make config simple

  16. ARCHAIUS Take config from anywhere: • Environment vars • Properties

    file on disk • File at URL • ZooKeeper • JMX • etcd • …
  17. ARCHAIUS “Can we schedule some downtime to change value x?”

    private static final int TIMEOUT = 1000;
  18. ARCHAIUS “Nah, we can just change it.” private static final

    DynamicIntProperty TIMEOUT = DynamicPropertyFactory
 .getIntProperty("service.timeout", 1000); TIMEOUT.addCallback(() ->
 System.out.println("value changed!"));
  19. #2:

  20. RxJava "ReactiveX is a combination of the best ideas from

    the Observer pattern, the Iterator pattern, and functional programming" http://reactivex.io/
  21. RxJava Async by default. Even if it’s not :) public

    interface TagService { List<Tag> getTags(UUID accountId); }
  22. RxJava Async by default. public interface TagService { Observable<Tag> getTags(UUID

    accountId); }
  23. RxJava Async by default. List<Tag> tags = tagService.getTags("5bd8bda8-b2ae-4d83- a20f-50a5327a4a13"); List<Tag>

    publicTags = new ArrayList<>(); for (Tag tag : tags) { if (!tag.isPrivate()) { publicTags.add(tag); } }
  24. RxJava Async by default. tagService.getTags("5bd8bda8-b2ae-4d83-a20f-50a5327a4a13") .filter(tag -> !tag.isPrivate()) .subscribe(tag ->

  25. RxJava Async by default. (defn rx-fn [fn] (reify Func1 (call

    [this obj] (fn obj)))) (defn rx-action [fn] (reify Action1 (call [this e] (fn e)))) (-> (.getTags tagService "5bd8bda8-b2ae-4d83-a20f-50a5327a4a13") (.filter (rx-fn #(not (.isPrivate %)))) (.subscribe (rx-action #(println (.getId %)))))
  26. #3:

  27. HYSTRIX Expect failure Make it easy to react

  28. HYSTRIX Bulkheading: • Mock responses- sometimes it's ok • Just

    die- no point burning a thread if 
 it'll fail anyway
  29. HYSTRIX class SomeServiceRequest extends HystrixRequest<String> { @Override public String run()

    throws Exception { // some fallible API call } @Override public String getFallback() { // return some stale data, if viable } }

  31. END

  32. LINKS ★ http://netflix.github.io/#repo ★ http://techblog.netflix.com/2013/02/rxjava-netflix-api.html ★ http://techblog.netflix.com/2012/11/hystrix.html ★ http://techblog.netflix.com/2013/06/announcing-zuul-edge- service-in-cloud.html