High Performance RPC with Finagle

@samklr
#devoxx #finagle #scala
High Performance RPC with
Sam BESSALAH
@samklr

View Slide

@samklr
RPC Redux
• Control of clients and servers
• Need for predictable performance
• Communication between different languages
• Isolated computations
• Fan out, Fan In distributed systems
• ...

View Slide

@samklr

View Slide

@samklr
Open sourced by Twitter in 2011 to solve this :

View Slide

@samklr

View Slide

@samklr
Asynchronous
Non-Blocking
Protocol Agnostic
Full Stack RPC
built on top of

View Slide

@samklr
http://monkey.org/~marius/funsrv.pdf

View Slide

@samklr
Load Balancing : Heap Balancer
Client Backoff Strategy
BackPressure
Failure Detection
Failover/Retry
Connection Pooling
Tracing (via ZipKin)
...
Production ready distributed toolbox

View Slide

@samklr
• Box
• Foursquare
• Nest
• Uber
• Pinterest
• NY Times
• Rdio
• PagerDuty
• Soundcloud
• Strava
• StumbleUpon
• ...
Known Users ...

View Slide

@samklr
@nivdul

View Slide

@samklr
Networking Layer
@nivdul

View Slide

@samklr
Building Blocks
Futures
Services
Filters

View Slide

@samklr
Futures
• Concurrency primitive in Finagle
• Different from scala.concurrent.Future and java.util.concurrent.
Future
• Less context-switching, easier cancelability
• From com.twitter.util.Future
• Composables
• Error and Exception Handling

View Slide

@samklr
Futures Combinators
• Can be composed with map, flatMap, handle and rescue
• Exception handling via:

View Slide

@samklr
Service

View Slide

@samklr
Services
class DummyService extends Service[httpx.Request, httpx.Response]
{
def apply(req: httpx.Request): Future[httpx.Response] = {
val response = req.response
response.setStatusCode(200)
response.setContentTypeJson()
response.write(requestHandler(req.getContentString()))
Future.value(response)
}

View Slide

@samklr
Filters
class Filter[-ReqIn, +RepOut, +ReqOut, -RepIn] extends
((ReqIn, Service[ReqOut, RepIn]) => Future[RepOut])
or
type Filter[Req, Rep] =
(Req, Service[Req, Rep]) => Future[Rep]

View Slide

@samklr
Filters
●
Wrap services to modify inputs or outputs (or both) …
●
Protocol and Network agnostic
●
Composable

View Slide

@samklr
Filters
val myTimeoutFilter =
new TimeoutFilter[String,Int](1.second,DefaultTimer.twitter)

View Slide

@samklr
Filters
new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val myService = new DummyService

View Slide

@samklr
Filters
new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val server = myTimeOutFilter andThen myService

View Slide

@samklr
Filters
new TimeoutFilter[String,Int](
1.second, DefaultTimer.twitter)
val service = myService andThen myTimeOutFilter
val server = Httpx.serve( “:8080”, service)
Await.ready(server)

View Slide

@samklr
Filters
Can be used for:
• Retries
• Logging
• Serialization
• Authentication
• Retries
• Timeouts
• ...

View Slide

@samklr
Clients

View Slide

@samklr
Clients
Finagle client are just full stack services
val myClient: Service[Request, Response] =
Httpx.newService("rtb-bidder.com:8081")
val bidder = Httpx.serve(":8080", myClient)

View Slide

@samklr
Finagle in a nutshell

View Slide

@samklr
Protocols
• Thrift
• Memcache
• Redis
• HTTP
• SMTP
• MySQL
• Zookeeper
• Mux
Protocol.newService( … )

View Slide

@samklr
Finagle Mux
• RPC Protocol Multiplexing
• One network connection per client-server session
• Maximization of available bandwidth without incurring the cost of
opening additional sockets
• Elimination of head-of-line blocking
• Explicit queue management
• Pure session layer (OSI )

View Slide

@samklr
Finagle Serial

View Slide

@samklr
Finagle Serial
Use binary format. Scodec, Protobuf, Thrift, Avro, etc.

View Slide

@samklr
Service discovery
●
Finagle ServerSets via ”Names”
●
Uses Zookeeper as Name server
val client = Http.newService(
“zk!myzkhost.home.com:2181!/my/zk/path” )

View Slide

@samklr
Twitter-Server
• Template for servers at Twitter built on top of Finagle
• Flags for configuration
• Logging and Metrics
• Admin HTTP UI
• Lifecycle management endpoints, handy for Docker and/or Mesos
• Tracing via ZipKin

View Slide

@samklr
Finagle Java
Service service = new Service() {
public Future apply(Request request) {
Response response = Response.apply();
response.setContent(ChannelBuffers.wrappedBuffer("yo".getBytes()));
Future future = Future.value(response);
return future;
}};
ServerBuilder.safeBuild(
service,
ServerBuilder.get()
.codec(Http.get())
.bindTo(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0))
.name("HttpServer"));
}

View Slide

@samklr
Finatra
• Library to easily build API Services on top of the Twitter Stack (Finagle,
Twitter-Server, Twitter-util, etc.)
• Supports Http and Thrift services
• Brings Sinatra like Style to Finagle for Rest Services

View Slide

@samklr
Finatra
class HelloWorldController extends Controller {
get("/hi") { request: Request =>
"Hello " + request.params.getOrElse("name", "unnamed")
}
post("/hi") { hiRequest: HiRequest =>
"Hello " + hiRequest.name + " with id " + hiRequest.id
}
}

View Slide

@samklr
Finatra
object HelloWorldServerMain extends HelloWorldServer
class HelloWorldServer extends HttpServer {
override def modules = Seq(Slf4jBridgeModule)
override def configureHttp(router: HttpRouter) {
router
.filter[LoggingMDCFilter[Request, Response]]
.filter[TraceIdMDCFilter[Request, Response]]
.filter[CommonFilters]
.add[HelloWorldController]
}
}

View Slide

@samklr
● Thin layer of purely functional basic blocks, to build
composable REST APIs.
● Built on top of Finagle
● Functional wrapper of Finagle-http
● Performance close to Bare Metal Finagle
http://github.com/finagle/finch

View Slide

@samklr
Zipkin
• Zipkin is a distributed tracing system that helps us gather timing data
for all the disparate services at Twitter. It manages both the collection
and lookup of this data through a Collector and a Query service.
• Implementation of Google’s Dapper paper
• Moved to OpenZipkin

View Slide

@samklr
ZipKin - Open ZipKin

View Slide

@samklr
Finagle : Cons
• Stuck to old versions of Netty. Currently Netty 3 and now moving to 4
• API Changes since 6.x. No more Netty API Leaking through Finagle’s
• Mismatch between scala.concurrent.Future and com.twitter.util.Future
• Slow to update
• Documentation not exhaustive. Thanks for gitter.im and the Mailing List
• Can be painful to plug to other Metrics and Monitoring systems. Hopefully
it’s changing

View Slide

@samklr
Quick Demo

View Slide

@samklr
Conclusion

View Slide

High Performance RPC with Finagle

High Performance RPC with Finagle

Sam Bessalah

More Decks by Sam Bessalah

Other Decks in Programming

Featured

Transcript