Share
Embed
How to make our concurrency tools suck less @ Servers.com
Friday internal meetup
#GIL #Eventmachine #Fiber #Ruby #Benchmarks
exAspArk
June 24, 2016
Transcript
31 slides
-
• Multiple parallel requests into the app from 1 user,
e.g. to get credentials to each OpenStack computing region • Concurrent HTTP requests from the app, e.g. to combine information from different microservices Challenges -
• rainbows • batch_api • rack-fiber_pool • eventmachine • sinatra-synchrony
• em-synchrony • em-http-request • em-socksify • em-resolv-replace • delayed_job What we currently use Gemfile.lock -
Rainbows! is an HTTP server for sleepy Rack applications. It
is based on unicorn, but designed to handle applications that expect long request/response times and/or slow clients. For network concurrency, models we currently support are: • EventMachine • FiberPool • ThreadPool • etc. rainbows http://rainbows.bogomips.org/ -
The Batch API is implemented as a Rack middleware. Useful
for reducing HTTP overhead by combining requests. sequential – option to execute all operations sequentially, rather than in parallel. This parameter is currently REQUIRED and must be set to true. (In the future the Batch API will offer parallel processing for thread-safe apps, and hence this parameter must be supplied in order to explicitly preserve expected behavior.) batch_api https://github.com/arsduo/batch_api -
An event-driven I/O and lightweight concurrency library for Ruby. It
provides event-driven I/O using the Reactor pattern, much like Node.js. Pros: • Good performance for networked apps (web server, proxy) • No memory overhead per connection • No threaded programming eventmachine https://github.com/eventmachine/eventmachine -
Cons: • One CPU • Quite a huge lib: 17k
+ 10k(C++) vs 6k LOC in Celluloid • Hard to debug complex systems: errors, callbacks • Work done per tick should be small • Lack of good and supported libraries around EM eventmachine -
Migration trend: • Concurrent-Ruby vs EM, Websocket-Driver vs Faye- Websocket
in ActionCable • Bunny vs AMQP • Angelo vs Sinatra-Synchrony eventmachine -
How it works: • Loads EventMachine and EM-Synchrony • Inserts
the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony -
How it works: • Loads EventMachine and EM-Synchrony • Inserts
the Rack::FiberPool middleware • Adds em-http-request to do concurrent HTTP calls • Patches TCPSocket via EM-Synchrony • Patches Rack::Test to run tests within an EventMachine • Patches Resolv via em-resolv-replace sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony -
I released sinatra-synchrony so I could use EventMachine without callbacks.
Since then, I’ve changed my mind. Ruby developers need to stop using EventMachine. It’s the wrong direction. sinatra-synchrony http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns -
If you are looking for a similar solution, check out
Angelo, which is similar to Sinatra but uses Celluloid, Celluloid::IO, Reel, and doesn't compromise real threads or require stupid EM monkey patches. This is a way, way, way better solution than sinatra-synchrony, rack, or EM will ever be. I will not be maintaining this gem anymore. If anyone is interested in maintaining it, feel free to inquire, but I recommend not using EventMachine or sinatra- synchrony anymore. sinatra-synchrony https://github.com/kyledrake/sinatra-synchrony -
Collection of convenience classes and primitives to help untangle evented
code, plus a number of patched EM clients to make them Fiber aware. https://github.com/igrigorik/em-synchrony/ em-synchrony -
!OVOLEHPV\QFKURQ\ DFWLYHUHFRUGUE DPTSUE FRQQHFWLRQBSRROUE FRUHBH[WUE HPKLUHGLVUE HPKWWSUE HPMDFNUE HPPHPFDFKHUE HPPRQJRUE
HPPXOWLUE HPUHGLVUE HPUHPFDFKHGUE ƉEHUBLWHUDWRUUE LWHUDWRUUE NH\ERDUGUE PHFKDQL]HUE PRQJRUE PRQJRLGUE P\VTOUE WFSVRFNHWUE WKUHDGUE em-synchrony https://github.com/igrigorik/em-synchrony/tree/master/lib/em-synchrony -
Database based asynchronous priority queue system. Pros: • Easy to
use • Processes Cons: • No master process and forks • No dashboards • No plugins • Uses Timeout delayed_job https://github.com/collectiveidea/delayed_job -
Nobody writes code to defend against an exception being raised
on literally any line. That's not even possible. So Thread.raise is basically like a sneak attack on your code that could result in almost anything. It would probably be okay if it were pure-functional code that did not modify any state. But this is Ruby, so that's unlikely :) Why Ruby’s Timeout is dangerous http://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/ -
How do we live with that FODVV2XU'HOD\HG-RE GHIVWDUW LQBHPBV\QFKURQ\^RXUBKDUGBZRUNHUVWDUW` HQG
GHILQBHPBV\QFKURQ\ UHWXUQXQOHVVEORFNBJLYHQ" (0V\QFKURQ\GR '-VKRXOGKDQGOHHUURUVEXWVRPHWLPHVLQ(0)LEHU,WHUDWRUWKH\DUHQRWUHVFXHG (0HUURUBKDQGOHU^_HUURU_#HUURU HUURU(0VWRS`VRZHQHHGWRKDQGOHWKHPPDQXDOO\ODWHU \LHOG (0VWRS HQG HQVXUH HQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ZRUNVZLWK$5 RXUBKDUGBZRUNHUKDQGOHBHUURUBKDUGHU #HUURU LI#HUURUKDQGOHHUURUVRXWRI(0 HQG GHIHQVXUHBWUDQVDFWLRQBƉQLVKHGBFRUUHFWO\ WUDQVDFWLRQBPDQDJHU 'HOD\HG-REFRQQHFWLRQWUDQVDFWLRQBPDQDJHU UHWXUQLIWUDQVDFWLRQBPDQDJHUFXUUHQWBWUDQVDFWLRQ WUDQVDFWLRQBPDQDJHUFODVV18//B75$16$&7,21 WUDQVDFWLRQBPDQDJHUUROOEDFNBWUDQVDFWLRQ HQG HQG -
• Let’s discuss the problems • Replace EM and all
other EMish dependencies • Use Puma to survive under high load • Use Curb / Typhoeus to make concurrent HTTP requests • Contribute to open source ;) Solutions -
It automatically yields control back to the process when an
application thread waits on I/O. If, for example, your application is waiting for an HTTP response from a payments provider, Puma can still accept requests in the Reactor thread or even complete other requests in different application threads. Puma has a "clustered" mode, where it combines its multi- threaded model with Unicorn's multi-process model. In clustered mode, then, Puma can deal with slow requests (thanks to a separate master process whose responsibility it is to download requests and pass them on) and slow application responses (thanks to spawning multiple workers). puma https://www.nateberkopec.com/2015/07/29/scaling-ruby-apps-to-1000-rpm.html -
• Less monkey patches and dependencies • Modern supported and
reliable libraries • The same performance • Easier to start for new developers Profit -
• https://bearmetal.eu/theden/how-do-i-know-whether- my-rails-app-is-thread-safe-or-not/ • https://www.toptal.com/ruby/ruby-concurrency-and- parallelism-a-practical-primer • https://github.com/celluloid • https://github.com/ruby-concurrency/concurrent-ruby
• Parallel vs Concurrent • Software Transactional Memory vs locks • Communicating Sequential Processes vs actors • etc. Further reading -
Update: Proof of Concept • puma • sinatra • typhoeus
+ faraday https://gist.github.com/exAspArk/7213b8b12a2340f0b74811d72f9be8ac -
Update: Proof of Concept Workers Threads Concurrent requests from server
Concurrent requests to server Time 2 2 1 1 1.6 2 2 1 4 1.6 2 2 1 8 4.5 2 2 1 16 6.4 2 2 1 32 13.5 2 2 5 1 1.6 2 2 5 4 1.6 2 2 5 8 4.5 2 2 5 16 6.9 2 2 5 32 14.1 -
Update: Proof of Concept • Number of concurrent requests from
server doesn’t really affect performance • We need to use as many puma workers × threads as number of concurrent requests to server