Scaling out a Rails app with Scala and Finagle

A lot of startups use Ruby on Rails for fast development, which makes sense -- it’s great for building CRUD apps quickly and cheaply. But building your app with Rails has it’s own drawbacks: it can get huge and messy, runtime is slow, and concurrency is an afterthought. While that shouldn’t stop you from using the framework, it’s important to think about longterm scalability during development. At Brigade, we have been developing a social network for politics using Rails. However, with time and more users, we decided to migrate more complex and modular realtime computations to Finagle micro-services. This talk will discuss the technolgies we selected (Scala, Finagle, Thrift) and how we addressed issues of debuggability, data-access, and integrating in various environments.

Transcript

1. Scaling out a Rails App With Scala and Finagle Stephanie Bian Brigade
3. Ruby on Rails • Pros ◦ Quick development ◦ ActiveRecord ◦ Community • Cons ◦ Performance doesn’t compare to JVM ◦ Lack of concurrency ◦ Ruby
6. Alignment
7. Let’s build a microservice!
8. Microservices introduce complexity • Debuggability • Complexity with data access • Integration across environments
9. Why Scala? • JVM performance • Support for asynchronous calls and computation • Static typing • Existing Scala components and experience
10. RPC Framework: Finagle • Simplicity of Futures ◦ Easy concurrent composition from Finagle documentation
11. RPC Framework: Finagle • Service (and Filters) as a Function from Finagle documentation
12. RPC Framework: Finagle • Supports numerous protocols ◦ Thrift, HTTP, MySQL, Redis from Finagle documentation
13. Protocol: Thrift • Acts as agreement between client and server • No ambiguity over request/response structure
14. Protocol: Thrift • Finagle supports Thrift with Scrooge Thrift: Generated Scala:
15. Using Java generated code: Using Scala generated code:
16. Debuggability
17. Thrift to Http Filter • Filter to convert a Thrift service to an HTTP service
18. A Readable Thrift Protocol • Thrift provides TJSONProtocol ◦ Difficult to read/write by hand
19. TJSONProtocol Thrift object: Serialized as JSON:
20. A Readable Thrift Protocol • Implemented a more readable/writeable protocol
21. Our readable protocol Serialized as JSON: Thrift object:
22. Data-Access Complexity
23. Data-access Complexity • Rails uses ActiveRecord to manage database • How do microservices get data when Rails owns the schema?
24. Alignment: Poll MySQL Rails MySQL Alignment async poll request
25. Alignment: Event-driven Rails MySQL Alignment Kafka request write event read event
26. Integration Across Environments
27. • How do engineers develop and test? • Abstract two strategies ◦ MySQL Strategy ◦ Service Strategy • AllServer which runs all microservices in a single JVM • Developed Dock to define, build, and run container with all dependencies Integration across environments
28. Summary • Finagle provides a simple RPC framework • Thrift enforces request and response structure • Debuggability with HTTP endpoints • Data-access via event-driven architecture • Integration using different strategies and easy to use tools
29. Thanks! • “10,000 times faster than MySQL?” • Check out Dock: http://github.com/brigade/dock • Check out Brigade: http://brigade.com • Follow me on Twitter: @stephbian