High-Performance Asynchronous Applications with (or without) Ruby - By Scott Tadman

Scott Tadman's presentation at Burlington Ruby Conf. 2012

Transcript

1. High-Performance Asynchronous Applications With (Or Without!) Ruby 1 Monday, July 30, 12
2. PostageApp • Simply: “Email as a Service” • Created in 2009 as an experiment. • Boils down to: Laziness, Impatience and Hubris. 2 Monday, July 30, 12
3. Why PostageApp? • Sending email is generally not fun. • Sending email correctly is even less fun. 3 Monday, July 30, 12
4. Why PostageApp? • Sending email is generally not fun: • Email support in most frameworks is usually primitive. • SMTP is not a reliable transport. • Creating a background job is not always possible. 4 Monday, July 30, 12
5. Why PostageApp? • Sending email correctly is even less fun: • Standards for composing HTML email are tricky. • Changes to templates usually require re-deploying. • Previews not always easy to simulate. 5 Monday, July 30, 12
6. Mission Accomplished • Prototype is a Ruby on Rails application • Sending engine is a Rails-based process. • 100% ActiveRecord and MySQL 6 Monday, July 30, 12
7. So You’ve Built A SaaS App? 7 Monday, July 30, 12
8. Now what? 8 Monday, July 30, 12
9. Scaling! 9 Monday, July 30, 12
10. How does Rails Scale? • Front-end performance is great! • Rails background workers...not so great. 10 Monday, July 30, 12
11. Rails-Based Workers • Each worker process was limited: • Could only send one email at a time. • Every email required a new connection. • Process footprint was large (~100MB) 11 Monday, July 30, 12
12. Why Use Rails for Workers? • Easy. • Literally. • Synchronous, blocking code is: • Easy to write. • Not hard to debug. 12 Monday, July 30, 12
13. Why is Rails Synchronous? • Long tradition in PHP, Python, Perl, etc. • Database calls are supposed to be fast. • Not fast enough? • Optimize. • Cache. • Optimize some more! Get creative. 13 Monday, July 30, 12
14. Limits of Synchronous • Some things can never be cached or optimized: • Third-party external servers. • Third-party external networks. • Basically stuff you have no control over. 14 Monday, July 30, 12
15. External systems... 15 Monday, July 30, 12
16. ...on external networks? 16 Monday, July 30, 12
17. Uh oh... 17 Monday, July 30, 12
18. Problems with Blocking 18 • Entire process grinds to a halt while waiting. • Memory is still in use. • Lights on, nobody home. • More workers requires more memory. Monday, July 30, 12
19. If only... • Your processing application could: • Do other things while waiting for a response? • Juggle multiple jobs at the same time? • Make use of multiple CPU cores? 19 Monday, July 30, 12
20. Threads! 20 Monday, July 30, 12
21. Multi-Threaded Code 21 • So you make your application multi-threaded. • Now yohaveu protwoblems: • Partition resources carefully? • Lock shared resources aggressively? • No magic bullet here. • Oh no. Deadlocks. Monday, July 30, 12
22. Hmm... 22 Monday, July 30, 12
23. Let’s Go Async! • All the cool kids are doing it. • JavaScript in the browser: • jQuery $.ajax • JavaScript on the server: • Node.js 23 Monday, July 30, 12
24. Asynchronous JavaScript • In the browser: • Blocking calls can lock the JavaScript VM. • Your computer never needs more beach-ball. • The “A” in AJAX actually stands for “Awesome” • The “X” stands for “JSON” 24 Monday, July 30, 12
25. Asynchronous Server? • Services implemented this way: • IRC • DNS • SNMP • SMTP! 25 Monday, July 30, 12
26. select() • Workhorse of traditional UNIX networking. • Improved upon with epoll and kqueue • Want to use those in Ruby? • EventMachine 26 Monday, July 30, 12
27. EventMachine • Ruby’s answer to Node.js • Implements the “Event Machine” pattern. • Built on similar libraries. • Recognizes slow networking is the problem to solve. 27 Monday, July 30, 12
28. Network Calls • In your application is heavily networked: • Database queries. • HTTP requests. • External APIs. • Shared cache systems. 28 Monday, July 30, 12
29. Event Driven • An “event” is something that happens. • Client example: Clicking on a page. • Server example: Receiving DNS request. 29 Monday, July 30, 12
30. Event Loop • In a nutshell: • Wait for stuff to happen. • When stuff happens, deal with it... • ...quickly. • Seriously, are you done yet? 30 Monday, July 30, 12
31. Event Responses • Break up longer operations into small units of work. • Keep the event stream flowing. • For best results: • Use external resources to do heavy lifting (DB, etc.) • Enable long-running tasks to pause and resume. • “Break time” 31 Monday, July 30, 12
32. Ruby’s Async Toolbox • Language features: • Blocks. • Threads. • Fibers...? 32 Monday, July 30, 12
33. Blocks • A block of code that’s passed in to a function. • Many ways to create: • lambda • Proc.new • do ... end • Blocks make Ruby very flexible. 33 Monday, July 30, 12
34. Definition of a Block • A small unit of code passed to a method. • May be executed zero or more times. • May be executed immediately... • ...or at some unspecified time in the future. 34 Monday, July 30, 12
35. lambda vs. function() • Ruby has lambda { } • JavaScript has function () • Superficially very similar. • Ruby’s syntax advantage: • Append do...end to any method call. • Makes passing methods almost too easy. 35 Monday, July 30, 12
36. Async JavaScript • Example: • async_method(function(r) { ... }); • async_method({ callback: function(r) { ... } }); • $.ajax(...).done(function (data) { ... }); 36 Monday, July 30, 12
37. Async JavaScript • Example from jQuery’s documentation: var jqxhr = $.ajax( "example.php" ) .done(function() { alert("success"); }) .fail(function() { alert("error"); }) .always(function() { alert("complete"); }); • That looks easy enough. • What about chaining operations? • Uh oh... 37 Monday, July 30, 12
38. Asynchronous Ruby • Give a method a block to call: • ...when the operation is complete. • ...when something went wrong. • ...when it timed out. 38 Monday, July 30, 12
39. Asynchronous Ruby • Chaining operations: • Often the result of one action informs the next: • Fetching additional records. • Error recovery. • May skip steps if data already cached. 39 Monday, July 30, 12
40. Nested Calls • Typical Example: • User... • ...belongs to an Account... • ...which has Notices. • ...has many Messages. 40 Monday, July 30, 12
41. Nested Calls • Synchronous Implementation: • user = User.find(...) • account = user.account • notice = Notice.find(account.last_notice_id) • m_count = Message.where(:user_id => user.id).count 41 Monday, July 30, 12
42. Nested Calls • Asynchronous Implementation: • User.async_find(...) do |user| • Account.async_find(user.account_id) do |account| • Notice.async_find(account.last_notice_id) do | notice| • Message.where(:user_id => user.id).async_count ... 42 Monday, July 30, 12
43. Asynchronous Calls • Without convention you have anarchy: • Multiple callback styles. • Differing return types and result codes. • Predictable, dependable behavior is essential. • Limitations imposed by Ruby need to be respected: • Don’t try to make it something it isn’t. 43 Monday, July 30, 12
44. Asynchronous House Rules • A well-behaving asynchronous method will... • Call the supplied block with a well-defined response: • Once and once only. • Always. • Even if something unexpected happens. • Never trigger any exceptions it can’t handle. 44 Monday, July 30, 12
45. Asynchronous Exceptions def async_example(&block) other_async_method do |result| yield(result) end rescue SomeException ... end 45 Monday, July 30, 12
46. Asynchronous Exceptions def async_example(&block) other_async_method do |result| yield(result) end rescue SomeException ... end 46 Monday, July 30, 12
47. Asynchronous Exceptions def async_example(&block) other_async_method do |result| yield(result) end rescue SomeException ... end 47 Monday, July 30, 12
48. Asynchronous Exceptions • Don’t do things that might cause trouble: • Be aware of what exceptions methods can cause. • Catch and handle them where they occur. • Exceptions will not be caught by the caller. • Exceptions will crash your entire application. 48 Monday, July 30, 12
49. Asynchronous Conditions conditional_async_method do |result_id| if (cached = @cache[result_id]) yield(cached) else fetch_and_cache(result_id) do |result| yield(result) end end end 49 Monday, July 30, 12
50. Leaving the Nest • Multiply nested asynchronous calls: • Tend to grow more complicated. • Optional steps are hard to express. • End up very hard to debug. • Make for a very deep stack. • Maybe there’s a better way. 50 Monday, July 30, 12
51. Leaving the Nest • Reasons to find an alternative: • Do you know what your application is doing? • What callbacks are still outstanding? • Why the application is not responding? • Where that asynchronous call was initiated? 51 Monday, July 30, 12
52. Stack Trace • EventMachine’s core is an event loop. • while (true) do ... end • The ... is the magical EventMachine stuff. • Asynchronous code not reflected in stack trace... • ...unless executing at that exact moment. • ...which is unlikely. 52 Monday, July 30, 12
53. How to fix this? 53 Monday, July 30, 12
54. State Machine Pattern • A state machine is: • One or more formally defined steps: • To complete an operation. • To handle conditions. • Basically like a flow-chart. 54 Monday, July 30, 12
55. State Machine Benefits • Using a state machine to track async code: • Encapsulates a multi-stage process. • Provides insight into completion status. • Easy to hook in to for logging and benchmarking. • Makes it easy to find where things have stalled. • Easily rendered as a diagrams. 55 Monday, July 30, 12
56. Use Case: SMTP • Line-based protocol • Very simple syntax • Well documented in various RFCs 56 Monday, July 30, 12
57. Use Case: SMTP • Example command stream sent to remote server: • HELO myhostname.net • MAIL FROM:<someone@example.com> • RCPT TO:<someone@yahoo.com> • DATA • QUIT 57 Monday, July 30, 12
58. State Machine Example Example from remailer library: state :helo do enter do send_line("HELO #{hostname}") end interpret(250) do if (requires_authentication?) enter_state(:auth) else enter_state(:established) end end end 58 Monday, July 30, 12
59. State Machine Example Example from remailer library: state :auth do enter do send_line("AUTH PLAIN #{encode_authentication(username, password)}") end interpret(235) do enter_state(:established) end interpret(535) do |reply_message, continues| handle_reply_continuation(535, reply_message, continues) do |reply_code, reply_message| error_notification(reply_code, reply_message) enter_state(:quit) end end end 59 Monday, July 30, 12
60. State Machine Use Cases • Best applied to problems that: • Have a complicated, multi-step procedure. • May need to recover from crashes. • A standardized way of tracking progress is required. • Insight into what’s “going on” is necessary. 60 Monday, July 30, 12
61. Wow, okay. 61 Monday, July 30, 12
62. What else? 62 Monday, July 30, 12
63. Fibers • Facility within Ruby since 1.8.6 • Even more confusing than blocks to the uninitiated. • Not many libraries make use of them. • ...but that seems to be changing. 63 Monday, July 30, 12
64. Fibers in Ruby • Fibers are like blocks that you can pause and resume. • Asynchronous operations often involve a lot of that. • So... • Fibers + asynchronous code are best friends? 64 Monday, July 30, 12
65. High Fiber Ruby • Instead of passing callback blocks to async methods... • ...yield control to that method... • ...then resume control when a response is received. • Seems simple enough, right? • Simplicity is a compelling reason to use them. 65 Monday, July 30, 12
66. High Fiber Ruby • Blocking synchronous code: • user = User.find(...) • account = user.account • ... 66 Monday, July 30, 12
67. High Fiber Ruby • Callback-driven asynchronous code: • User.find_async(...) do |user| • Account.where(...).find_async do |account| • ... 67 Monday, July 30, 12
68. High Fiber Ruby • Fibered asynchronous code: • user = User.find(...) • account = user.account • ... 68 Monday, July 30, 12
69. How is that “asynchronous”? 69 Monday, July 30, 12
70. Fiber Wrapper Example 70 def find_async(id) fiber = Fiber.current standard_async_query(...) do |result| fiber.resume(result) end Fiber.yield end Fiber.new do user = find_async(id) end Monday, July 30, 12
71. Fiber Future • Making Rails more Fiber-friendly. • Need more Fiber-aware libraries. • Not unlike being “thread-aware” or “thread-safe” • All non-trivial applications can benefit. • ...if the cost of change is low. • Fibers can make it easy. 71 Monday, July 30, 12
72. GitHub Resources • https://github.com/eventmachine/eventmachine • eventmachine • https://github.com/igrigorik • em-synchrony, em-websocket, etc. • em-mysqlplus based on em-mysql • https://github.com/tmm1/ • em-mysql 72 Monday, July 30, 12