Share
Embed
GPars vs Wild
A real life case study about GPars use in a Grails project.
Aurélien Maury
December 13, 2012
Transcript
56 slides
-
Vanity slide I’m French Aurélien Maury IT Consultant for Xebia
France @aurelienmaury Groovy world lover, evangelist -
Technical challenges No DB, lot of backends to get travel
offers Backends differ, depending on request content Different post-treatment for each backend Felix Baumgartner -
Weird legacy web services API Remain stateless to scale easily
Reliable and fast enough for large public use -
Meanwhile, serverside... For a single search, we need to solve
up to 70 backend requests. Each backend response should be visible as soon as possible to the user. The total response time should remain close to the response time of the slowest backend. (~15s) -
Hype Driven Development « Hey! What’s that Actors thing ?
Can it help ? » -
Actors Model reminder «Actor» is the primitive of concurrent computation
Has an incoming message queue Reaction loop processing messages one by one Reacts to incoming messages by: making local decisions sending more messages to other Actors replying a message to the sender -
Actors Model reminder Popularized by Akka for Scala For Groovy:
GPars! We used 1.0 beta 1 -
GPars to the rescue Groovy Parallel Systems Handle tasks concurrently
and asynchronously Concurrent collection processing Dataflow concurrency constructs Actor programming model ! -
GPars Actors Types DynamicDispatchActor Specific reaction per message type ReactiveActor
Always replies at the end of processing StaticDispatchActor Simplest message processor, advised as default choice. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Incoming HTTP requests are validated, and tagged with a correlation id. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" A POGO is stored, it contains: the correlation id, HTTP request and response -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" From now on: all message hold the correlation id. Request message is sent to the Router Actor. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Duplicate request messages are sent to specialized Business Actors. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Business Actors reply the number of messages they’ll send to Service Actors. POGO counts expected responses. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Request is split into backend compliant requests and sent to the next level. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Messages are solved against Grails services, plugged to backends (long running) -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Grails services responses are replied up. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Business post treatment is applied. Response messages are sent to Router Actor. Content is written in HttpResponse. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" When the count of expected response chunk reaches 0: HTTP response is closed and cleaned -
Let it crash... or not Nice but fragile: every uncaught
exception break an Actor Yes, ... we caught java.lang.Exception Encapsulate exceptions with Actor messages -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" An exception must propagate to the response counter. Elsewise, we have neverending requests. -
Every rose has its thorn One allocated Thread per Actor
instance at a time Long running message processing leads to contention Grails services calling backend are long running... -
The road not taken We could have gone with async
HTTP That would still have lead us to XML processing parallelization We pushed GPars further -
Round Robin Actor Inspired by Akka Balances messages on a
homogeneous Actors set Very short message processing time -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" We scale in peace Every RoundRobin has its own size. -
Checkpoint Message processing is parallelized Global ThreadPool for all Actors
GPars manages all boilerplate code for us Next moves: RoundRobin sizing Global ThreadPool sizing -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 1 1 1 1 Consider worst case ever to run in parallel for a single incoming request. Size does matter -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 16 4 1 16 4 1 21 Do the same for responses path. -
!"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-."
!"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 16 1 4 21 Sized RoundRobin for a well parallelized single incoming request. -
ThreadPool sizing GPars offers Actors grouping, so a group share
a dedicated ThreadPool For a first date, just Keep It Stupidely Simple ThreadPool must be the size of the biggest RoundRobin -Dgpars.poolsize=21 -
Checkpoint RoundRobin & ThreadPool sized... for one request! Next move:
max the number of concurrent requests -
Aiming for the top Get rid of interferences Live app
instance with iso-prod memory resources Backends replaced by custom Vert.x proxy-cache -
!"#$%&'(#)*$+,-*.,)# !&.+,$/.)0#,1 %2.,#$/.)0#,1 3'.1$/.)0#,1 4#&52#* 6#&*78 %&'89$:.)"# 6#&*78 %&'89$:.)"# 6#&*78
%&'89$:.)"# ;,(#)*'&$4)&+<* =%.&-7>+*"%''2$?$@!!%/A+21#& Throughput maxing Hype here -
Throughput maxing !"#$%&#' Breaking ? ($)' (#*'"" Unused resources ?
WIN! NO NO YES YES +,-'* '"#$%&#$,. /&$"' '"#$%&#$,. -
Throughput maxing Be patient, be brave, be picky Leads to
global performance tuning (connection pools, XML parsers) Goal reached when, under constant concurrent requests load: end-to-end response time stay constant over time with constant memory usage (and not too much GC) -
First production Running on: 1 EC2 small instance CPU capacity
of a 1.0-1.2 GHz 2007 Opteron Memory: 1.7 GB I/O Performance: Moderate JDK6 + Tomcat6 + Http11NioConnector -Xmx968m -
Under stress load 80 search by minutes 20 really concurrent,
15 sec each each search is 70 backends calls, plus XML parsing and processing, business rules applying, JSON marshalling and response writing average CPU usage: ~70% average GC activity: ~15% Limited by Memory and Backend response time -
Win Each response chunk is streamed to EmberJS UI Response
time is equivalent to the slowest backend response + post processing Lineary scalable -
Testing Unit testing is easy Integration testing is non-deterministic User
guide deserves a chapter on testing Find inspiration for your tests in Akka user guide -
Good Points User guide... guides the user! Flat learning curve
Just works! No bug has slowed us down. -
Improvements ideas Monitoring Actors activity and queue state Testing tools
and/or strategies Distributed Actors Flow specific Actors (RoundRobin, Restarter, ...) -
Eat some, it’s great Actor model is hard to catch
at first GPars is cool, it doesn’t stand in the way Thread free designing, business logic first Will definitely reuse