Save 37% off PRO during our Black Friday Sale! »

How to make our concurrency tools suck less @ Servers.com

How to make our concurrency tools suck less @ Servers.com

Friday internal meetup

#GIL #Eventmachine #Fiber #Ruby #Benchmarks

Cb09696b034cce3cc79cab80a4bba4a3?s=128

exAspArk

June 24, 2016
Tweet

Transcript

  1. How to make our concurrency tools suck less exAspArk Evgeny

    Li
  2. • Multiple parallel requests into the app from 1 user,

    e.g. to get credentials to each OpenStack computing region • Concurrent HTTP requests from the app, e.g. to combine information from different microservices Challenges
  3. GIL https://twitter.com/reubenbond/status/662061791497744384

  4. • rainbows • batch_api • rack-fiber_pool • eventmachine • sinatra-synchrony

    • em-synchrony • em-http-request • em-socksify • em-resolv-replace • delayed_job What we currently use Gemfile.lock
  5. Rainbows! is an HTTP server for sleepy Rack applications. It

    is based on unicorn, but designed to handle applications that expect long request/response times and/or slow clients. For network concurrency, models we currently support are: • EventMachine • FiberPool • ThreadPool • etc. rainbows http://rainbows.bogomips.org/
  6. rainbows http://bogomips.org/rainbows.git/log/

  7. The Batch API is implemented as a Rack middleware. Useful

    for reducing HTTP overhead by combining requests. sequential – option to execute all operations sequentially, rather than in parallel. This parameter is currently REQUIRED and must be set to true. (In the future the Batch API will offer parallel processing for thread-safe apps, and hence this parameter must be supplied in order to explicitly preserve expected behavior.) batch_api https://github.com/arsduo/batch_api
  8. An event-driven I/O and lightweight concurrency library for Ruby. It

    provides event-driven I/O using the Reactor pattern, much like Node.js. Pros: • Good performance for networked apps (web server, proxy) • No memory overhead per connection • No threaded programming eventmachine https://github.com/eventmachine/eventmachine
  9. https://www.youtube.com/watch?v=5AxtY4dfuwc RailsConf 2016 - Introduction to Concurrency in Ruby

  10. Cons: • One CPU • Quite a huge lib: 17k

    + 10k(C++) vs 6k LOC in Celluloid • Hard to debug complex systems: errors, callbacks • Work done per tick should be small • Lack of good and supported libraries around EM eventmachine
  11. Migration trend: • Concurrent-Ruby vs EM, Websocket-Driver vs Faye- Websocket

    in ActionCable • Bunny vs AMQP • Angelo vs Sinatra-Synchrony eventmachine
  12. How it works: • Loads EventMachine and EM-Synchrony • Inserts

    the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
  13. How it works: • Loads EventMachine and EM-Synchrony • Inserts

    the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
  14. I released sinatra-synchrony so I could use EventMachine without callbacks.

    Since then, I’ve changed my mind. Ruby developers need to stop using EventMachine. It’s the wrong direction. sinatra-synchrony http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns
  15. If you are looking for a similar solution, check out

    Angelo, which is similar to Sinatra but uses Celluloid, Celluloid::IO, Reel, and doesn't compromise real threads or require stupid EM monkey patches. This is a way, way, way better solution than sinatra-synchrony, rack, or EM will ever be. I will not be maintaining this gem anymore. If anyone is interested in maintaining it, feel free to inquire, but I recommend not using EventMachine or sinatra- synchrony anymore. sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
  16. Collection of convenience classes and primitives to help untangle evented

    code, plus a number of patched EM clients to make them Fiber aware. https://github.com/igrigorik/em-synchrony/ em-synchrony
  17. !OVOLEHPV\QFKURQ\ DFWLYHUHFRUGUE DPTSUE FRQQHFWLRQBSRROUE FRUHBH[WUE HPKLUHGLVUE HPKWWSUE HPMDFNUE HPPHPFDFKHUE HPPRQJRUE

    HPPXOWLUE HPUHGLVUE HPUHPFDFKHGUE ƉEHUBLWHUDWRUUE LWHUDWRUUE NH\ERDUGUE PHFKDQL]HUE PRQJRUE PRQJRLGUE P\VTOUE WFSVRFNHWUE WKUHDGUE em-synchrony https://github.com/igrigorik/em-synchrony/tree/master/lib/em-synchrony
  18. Database based asynchronous priority queue system. Pros: • Easy to

    use • Processes Cons: • No master process and forks • No dashboards • No plugins • Uses Timeout delayed_job https://github.com/collectiveidea/delayed_job
  19. Nobody writes code to defend against an exception being raised

    on literally any line. That's not even possible. So Thread.raise is basically like a sneak attack on your code that could result in almost anything. It would probably be okay if it were pure-functional code that did not modify any state. But this is Ruby, so that's unlikely :) Why Ruby’s Timeout is dangerous http://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/
  20. No Timeout in Sidekiq https://github.com/mperham/sidekiq/issues/862#issuecomment-76982399

  21. How do we live with that FODVV2XU'HOD\HG-RE GHIVWDUW LQBHPBV\QFKURQ\^RXUBKDUGBZRUNHUVWDUW` HQG

    GHILQBHPBV\QFKURQ\ UHWXUQXQOHVVEORFNBJLYHQ" (0V\QFKURQ\GR '-VKRXOGKDQGOHHUURUVEXWVRPHWLPHVLQ(0)LEHU,WHUDWRUWKH\DUHQRWUHVFXHG (0HUURUBKDQGOHU^_HUURU_#HUURU HUURU(0VWRS`VRZHQHHGWRKDQGOHWKHPPDQXDOO\ODWHU  \LHOG (0VWRS HQG HQVXUH HQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ZRUNVZLWK$5 RXUBKDUGBZRUNHUKDQGOHBHUURUBKDUGHU #HUURU LI#HUURUKDQGOHHUURUVRXWRI(0 HQG GHIHQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ WUDQVDFWLRQBPDQDJHU 'HOD\HG-REFRQQHFWLRQWUDQVDFWLRQBPDQDJHU UHWXUQLIWUDQVDFWLRQBPDQDJHUFXUUHQWBWUDQVDFWLRQ WUDQVDFWLRQBPDQDJHUFODVV18//B75$16$&7,21 WUDQVDFWLRQBPDQDJHUUROOEDFNBWUDQVDFWLRQ HQG HQG
  22. • Let’s discuss the problems • Replace EM and all

    other EMish dependencies • Use Puma to survive under high load • Use Curb / Typhoeus to make concurrent HTTP requests • Contribute to open source ;) Solutions
  23. puma https://github.com/puma/puma

  24. It automatically yields control back to the process when an

    application thread waits on I/O. If, for example, your application is waiting for an HTTP response from a payments provider, Puma can still accept requests in the Reactor thread or even complete other requests in different application threads. Puma has a "clustered" mode, where it combines its multi- threaded model with Unicorn's multi-process model. In clustered mode, then, Puma can deal with slow requests (thanks to a separate master process whose responsibility it is to download requests and pass them on) and slow application responses (thanks to spawning multiple workers). puma https://www.nateberkopec.com/2015/07/29/scaling-ruby-apps-to-1000-rpm.html
  25. Concurrent HTTP requests https://github.com/exAspArk/concurrent_http_requests

  26. • Less monkey patches and dependencies • Modern supported and

    reliable libraries • The same performance • Easier to start for new developers Profit
  27. • https://bearmetal.eu/theden/how-do-i-know-whether- my-rails-app-is-thread-safe-or-not/ • https://www.toptal.com/ruby/ruby-concurrency-and- parallelism-a-practical-primer • https://github.com/celluloid • https://github.com/ruby-concurrency/concurrent-ruby

    • Parallel vs Concurrent • Software Transactional Memory vs locks • Communicating Sequential Processes vs actors • etc. Further reading
  28. Thank you! https://giphy.com/gifs/redbull-26vUsARImRG7qT6ww

  29. Update: Proof of Concept • puma • sinatra • typhoeus

    + faraday https://gist.github.com/exAspArk/7213b8b12a2340f0b74811d72f9be8ac
  30. Update: Proof of Concept Workers Threads Concurrent requests from server

    Concurrent requests to server Time 2 2 1 1 1.6 2 2 1 4 1.6 2 2 1 8 4.5 2 2 1 16 6.4 2 2 1 32 13.5 2 2 5 1 1.6 2 2 5 4 1.6 2 2 5 8 4.5 2 2 5 16 6.9 2 2 5 32 14.1
  31. Update: Proof of Concept • Number of concurrent requests from

    server doesn’t really affect performance • We need to use as many puma workers × threads as number of concurrent requests to server