How to make our concurrency tools suck less @ Servers.com

Friday internal meetup

#GIL #Eventmachine #Fiber #Ruby #Benchmarks

Transcript

1. How to make our concurrency tools suck less exAspArk Evgeny Li
2. • Multiple parallel requests into the app from 1 user, e.g. to get credentials to each OpenStack computing region • Concurrent HTTP requests from the app, e.g. to combine information from different microservices Challenges
3. GIL https://twitter.com/reubenbond/status/662061791497744384
4. • rainbows • batch_api • rack-fiber_pool • eventmachine • sinatra-synchrony • em-synchrony • em-http-request • em-socksify • em-resolv-replace • delayed_job What we currently use Gemfile.lock
5. Rainbows! is an HTTP server for sleepy Rack applications. It is based on unicorn, but designed to handle applications that expect long request/response times and/or slow clients. For network concurrency, models we currently support are: • EventMachine • FiberPool • ThreadPool • etc. rainbows http://rainbows.bogomips.org/
6. rainbows http://bogomips.org/rainbows.git/log/
7. The Batch API is implemented as a Rack middleware. Useful for reducing HTTP overhead by combining requests. sequential – option to execute all operations sequentially, rather than in parallel. This parameter is currently REQUIRED and must be set to true. (In the future the Batch API will offer parallel processing for thread-safe apps, and hence this parameter must be supplied in order to explicitly preserve expected behavior.) batch_api https://github.com/arsduo/batch_api
8. An event-driven I/O and lightweight concurrency library for Ruby. It provides event-driven I/O using the Reactor pattern, much like Node.js. Pros: • Good performance for networked apps (web server, proxy) • No memory overhead per connection • No threaded programming eventmachine https://github.com/eventmachine/eventmachine
9. https://www.youtube.com/watch?v=5AxtY4dfuwc RailsConf 2016 - Introduction to Concurrency in Ruby
10. Cons: • One CPU • Quite a huge lib: 17k + 10k(C++) vs 6k LOC in Celluloid • Hard to debug complex systems: errors, callbacks • Work done per tick should be small • Lack of good and supported libraries around EM eventmachine
11. Migration trend: • Concurrent-Ruby vs EM, Websocket-Driver vs Faye- Websocket in ActionCable • Bunny vs AMQP • Angelo vs Sinatra-Synchrony eventmachine
12. How it works: • Loads EventMachine and EM-Synchrony • Inserts the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
13. How it works: • Loads EventMachine and EM-Synchrony • Inserts the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
14. I released sinatra-synchrony so I could use EventMachine without callbacks. Since then, I’ve changed my mind. Ruby developers need to stop using EventMachine. It’s the wrong direction. sinatra-synchrony http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns
15. If you are looking for a similar solution, check out Angelo, which is similar to Sinatra but uses Celluloid, Celluloid::IO, Reel, and doesn't compromise real threads or require stupid EM monkey patches. This is a way, way, way better solution than sinatra-synchrony, rack, or EM will ever be. I will not be maintaining this gem anymore. If anyone is interested in maintaining it, feel free to inquire, but I recommend not using EventMachine or sinatra- synchrony anymore. sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony
16. Collection of convenience classes and primitives to help untangle evented code, plus a number of patched EM clients to make them Fiber aware. https://github.com/igrigorik/em-synchrony/ em-synchrony
17. !OVOLEHPV\QFKURQ\ DFWLYHUHFRUGUE DPTSUE FRQQHFWLRQBSRROUE FRUHBH[WUE HPKLUHGLVUE HPKWWSUE HPMDFNUE HPPHPFDFKHUE HPPRQJRUE HPPXOWLUE HPUHGLVUE HPUHPFDFKHGUE ƉEHUBLWHUDWRUUE LWHUDWRUUE NH\ERDUGUE PHFKDQL]HUE PRQJRUE PRQJRLGUE P\VTOUE WFSVRFNHWUE WKUHDGUE em-synchrony https://github.com/igrigorik/em-synchrony/tree/master/lib/em-synchrony
18. Database based asynchronous priority queue system. Pros: • Easy to use • Processes Cons: • No master process and forks • No dashboards • No plugins • Uses Timeout delayed_job https://github.com/collectiveidea/delayed_job
19. Nobody writes code to defend against an exception being raised on literally any line. That's not even possible. So Thread.raise is basically like a sneak attack on your code that could result in almost anything. It would probably be okay if it were pure-functional code that did not modify any state. But this is Ruby, so that's unlikely :) Why Ruby’s Timeout is dangerous http://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/
20. No Timeout in Sidekiq https://github.com/mperham/sidekiq/issues/862#issuecomment-76982399
21. How do we live with that FODVV2XU'HOD\HG-RE GHIVWDUW LQBHPBV\QFKURQ\^RXUBKDUGBZRUNHUVWDUW` HQG GHILQBHPBV\QFKURQ\ UHWXUQXQOHVVEORFNBJLYHQ" (0V\QFKURQ\GR '-VKRXOGKDQGOHHUURUVEXWVRPHWLPHVLQ(0)LEHU,WHUDWRUWKH\DUHQRWUHVFXHG (0HUURUBKDQGOHU^_HUURU_#HUURU HUURU(0VWRS`VRZHQHHGWRKDQGOHWKHPPDQXDOO\ODWHU  \LHOG (0VWRS HQG HQVXUH HQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ZRUNVZLWK$5 RXUBKDUGBZRUNHUKDQGOHBHUURUBKDUGHU #HUURU LI#HUURUKDQGOHHUURUVRXWRI(0 HQG GHIHQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ WUDQVDFWLRQBPDQDJHU 'HOD\HG-REFRQQHFWLRQWUDQVDFWLRQBPDQDJHU UHWXUQLIWUDQVDFWLRQBPDQDJHUFXUUHQWBWUDQVDFWLRQ WUDQVDFWLRQBPDQDJHUFODVV18//B75$16$&7,21 WUDQVDFWLRQBPDQDJHUUROOEDFNBWUDQVDFWLRQ HQG HQG
22. • Let’s discuss the problems • Replace EM and all other EMish dependencies • Use Puma to survive under high load • Use Curb / Typhoeus to make concurrent HTTP requests • Contribute to open source ;) Solutions
23. puma https://github.com/puma/puma
24. It automatically yields control back to the process when an application thread waits on I/O. If, for example, your application is waiting for an HTTP response from a payments provider, Puma can still accept requests in the Reactor thread or even complete other requests in different application threads. Puma has a "clustered" mode, where it combines its multi- threaded model with Unicorn's multi-process model. In clustered mode, then, Puma can deal with slow requests (thanks to a separate master process whose responsibility it is to download requests and pass them on) and slow application responses (thanks to spawning multiple workers). puma https://www.nateberkopec.com/2015/07/29/scaling-ruby-apps-to-1000-rpm.html
25. Concurrent HTTP requests https://github.com/exAspArk/concurrent_http_requests
26. • Less monkey patches and dependencies • Modern supported and reliable libraries • The same performance • Easier to start for new developers Profit
27. • https://bearmetal.eu/theden/how-do-i-know-whether- my-rails-app-is-thread-safe-or-not/ • https://www.toptal.com/ruby/ruby-concurrency-and- parallelism-a-practical-primer • https://github.com/celluloid • https://github.com/ruby-concurrency/concurrent-ruby • Parallel vs Concurrent • Software Transactional Memory vs locks • Communicating Sequential Processes vs actors • etc. Further reading
28. Thank you! https://giphy.com/gifs/redbull-26vUsARImRG7qT6ww
29. Update: Proof of Concept • puma • sinatra • typhoeus + faraday https://gist.github.com/exAspArk/7213b8b12a2340f0b74811d72f9be8ac
30. Update: Proof of Concept Workers Threads Concurrent requests from server Concurrent requests to server Time 2 2 1 1 1.6 2 2 1 4 1.6 2 2 1 8 4.5 2 2 1 16 6.4 2 2 1 32 13.5 2 2 5 1 1.6 2 2 5 4 1.6 2 2 5 8 4.5 2 2 5 16 6.9 2 2 5 32 14.1
31. Update: Proof of Concept • Number of concurrent requests from server doesn’t really affect performance • We need to use as many puma workers × threads as number of concurrent requests to server