Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Switching to Asynchronous RPC Stack

Switching to Asynchronous RPC Stack

gRPC is a really nice RPC system supporting not only classical remote method execution but also many useful features from 21st century like streaming, asynchronous message processing, h2 transport. In this talk we will share pain and experience Wix gained in a journey from handcrafted blocking json-rpc method of RPC to a standard non-blocking gRPC.

Ed3896433ccf1063a9d89ef4f74f44ae?s=128

Viaceslav Pozdniakov

November 24, 2018
Tweet

Transcript

  1. Viačeslav Pozdniakov @poznia github.com/vipo Switching to Asynchronous RPC Stack

  2. Wix Engineering Locations Ukraine Israel Lithuania Vilnius Kiev Dnipro Tel-Aviv

    Be’er Sheva
  3. Numbers ▪ ~450 jvm (~Scala) (micro-)services / ~1400 instances ▪

    ~440 Node.js (micro-)services / ~1040 instances ▪ 228 back-end developers (our users) ▪ 379 front-end developers BEFORE WE EVEN START
  4. Remote Procedure Call In distributed computing, a remote procedure call

    (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction. https://en.wikipedia.org/wiki/Remote_procedure_call BEFORE WE EVEN START
  5. AGENDA Wix (micro-)services Json RPC - problematic legacy gRPC -

    challenges and solution How to implement such changes
  6. Wix (micro-)services 01

  7. Microservices Microservices are a software development technique — a variant

    of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. https://en.wikipedia.org/wiki/Microservices (MICRO-)SERVICES
  8. Good, but we need development velocity ▪ Same metrics reported

    by every service ▪ Logs in same format ▪ Support of A/B testing by every service ▪ Context propagation (MICRO-)SERVICES
  9. Context? ▪ User identity of initial requests ▪ What kind

    of user ▪ Experiment conduction data (A/B testing) ▪ User interface language ▪ ... (MICRO-)SERVICES
  10. Context propagation at Wix If service A calls service B

    and service B calls service C, service C must receive all context data sent by service A even though this data is not explicitly used by service B and all calls do not bring any parameters. (MICRO-)SERVICES A B C uid=42 uid=42 uid=42
  11. None
  12. JSON-RPC 02

  13. JSON-RPC https://en.wikipedia.org/wiki/JSON-RPC JSON-RPC

  14. Interface Definition Language An interface description language or interface definition

    language (IDL), is a specification language used to describe a software component's application programming interface (API). https://en.wikipedia.org/wiki/Interface_description_language JSON-RPC
  15. Scala as IDL trait ServiceName { def foo(bar: Bar): Unit

    def zoo(value: Int): Option[Zoo] } case class Bar(value: String) case class Zoo(val1: Long, val2: Long) JSON-RPC
  16. Typical JSON-RPC service //server class ServiceNameImpl extends ServiceName { def

    foo(bar: Bar): Unit = () def zoo(value: Int): Option[Zoo] = None } //client val aClient = rpcClientFactory.forTrait[ServiceName](url) aClient.zoo(42) // supports context propagation JSON-RPC
  17. Implementation details ▪ Some Proxy.newProxyInstance magic ▪ Jackson (https://github.com/FasterXML/jackson) JSON-RPC

  18. Jackson is too flexible @JsonDeserialize(as=classOf[BarNew]) case class Bar(@JsonProperty("name") value: String)

    case class BarNew(name: String, age: Integer = 0) @JsonTypeInfo(use=Id.CLASS, include=As.PROPERTY, property="class") trait Zoo case class ZooOne(val1: Long, val2: Long) extends Zoo case class ZooTwo(name: String) extends Zoo JSON-RPC
  19. Binary coupling/dependencies JSON-RPC Somewhy you cannot use off-the-shelf library for

    your language to talk easily over a standard protocol
  20. None
  21. gRPC 03

  22. Protocol Buffers IDL syntax = "proto3"; service ServiceName { rpc

    Echo (Message) returns (Message); }; message Message { string field = 1; repeated string fields = 2; } gRPC
  23. gRPC (gRPC Remote Procedure Call) • Over HTTP/2 ◦ multiplexed,

    instead of ordered and blocking ◦ one connection • Netty underneath • Protocol Buffers IDL compiled to almost any programming language gRPC
  24. Sounds like a plan! 1. Take ScalaPB (https://scalapb.github.io/) 2. Generate

    traits and entities 3. Use JSON-RPC right away (+ REST) 4. Eventually switch to gRPC (and expose same service impl. as in json-rpc) 5. PROFIT!!! gRPC
  25. Protocol Buffers IDL import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {

    override def echo(request: Message): Future[Message] = Future(request) } gRPC
  26. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {

    override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
  27. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {

    override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
  28. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {

    override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
  29. Stack relies on ThreadLocal 1. java.lang.ThreadLocal[T] is a global variable

    per thread 2. ~6 years old code base 3. Own and specialized json-rpc and kafka clients,.. read ThreadLocal 4. Designed for sequential request processing gRPC
  30. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName

    { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
  31. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName

    { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
  32. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName

    { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
  33. Definition of “thread”, theory A programming structure or process formed

    by linking a number of separate elements or subroutines, especially each of the tasks executed concurrently in multithreading. https://en.oxforddictionaries.com/definition/thread gRPC
  34. What practitioners do? • Finagle: server as a function Req

    => Future[Res] (https://monkey.org/~marius/funsrv.pdf) • Spray: The Magnet Pattern (http://spray.io/blog/2012-12-13-the-magnet-pattern) • Play Framework: implicit request • FP elitists: “Use Reader monad!” • Martin Odersky: Implicit Function Types (dotty) or brand new Witnesses gRPC
  35. Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends

    ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request) _ <- kafka.produce(response) } yield request } gRPC
  36. Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends

    ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request)(cs) _ <- kafka.produce(response)(cs) } yield request } gRPC
  37. ScalaPB generator plugin def printService(printer: FunctionalPrinter): FunctionalPrinter = { printer

    .add("package " + service.getFile.scalaPackageName) .newline .call(internal) .newline .add(s"object ${service.getName}WithCallScope {") .indent .newline //... gRPC
  38. CallScope • Represents a thread • Contains request context, request

    deadline • Immutable/thread safe • A builder for other CallScopes with updated values • Constructed by framework with love. Users cannot create their own one (except TestCallScope) gRPC
  39. New “CallScoped” infrastructure • Service traits are generated by our

    protoc plugin • Service traits, kafka consumers are (implicit) CallScope producers • gRPC and json-rpc clients, kafka producers are (implicit) CallScope consumers • Enables asynchronicity which is predictable gRPC
  40. Look ma, no globals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends

    ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope) = { val f1 = Future(rpcClient.boo()(cs.withUid(42)) val f2 = Future(rpcClient.boo()(cs.withUid(43)) for { _ <- f1 _ <- f2 } yield request } } gRPC
  41. Lessons learned 04

  42. Architecture we had ~2 years ago Binary coupled distributed monolith

    with thread-per-request threading model EXECUTION
  43. Architecture we are promoting Protocol Buffers IDL based distributed monolith

    (must be improved but we are OK with that) which is asynchronous and non-blocking EXECUTION
  44. Execution of such big changes • Legacy will stay here

    forever - deal with it • Create a black market first - no one will give you any resources to change status quo • Start with power-users: they are more agreeable • Be responsive, otherwise even agreeable users will leave you • Claim for legalization after a while - make the black market you created an official one EXECUTION
  45. Technical stuff • Just do not use global variables (i.e.

    ThreadLocal) • Improved coding culture ◦ Future[T] for IO ◦ implicit CallScope - compile time checks • Use FP design patterns (i.e. Reader monad). Just don’t tell your users about that. • Code generation is OK, don’t be squeamish EXECUTION
  46. The answer is there is no answer Chuck Palahniuk

  47. Thank You @poznia github.com/vipo

  48. Q&A @poznia github.com/vipo