GPars vs Wild

A real life case study about GPars use in a Grails project.

Transcript

1. GPars vs. Wild a real life case study
2. Vanity slide I’m French Aurélien Maury IT Consultant for Xebia France @aurelienmaury Groovy world lover, evangelist
3. Agenda The Project Hype Driven Development The Wild Feedbacks
4. The Project Functional overview & technical challenges
5. Mytripset
6. Single Page App (Ember.js)
7. Powered by Grails
8. and GPars
9. Functional overview Travel planner accross Europe Plane, Train and Car routing
10. Technical challenges No DB, lot of backends to get travel offers Backends differ, depending on request content Different post-treatment for each backend Felix Baumgartner
11. Weird legacy web services API Remain stateless to scale easily Reliable and fast enough for large public use
12. Big picture !"#$%&'(#)*$+,-*.,)# / 012#&34$4%5 /.6*')'178#*# /-#.&)" HTML+JS AJAX AJAX
13. Meanwhile, serverside... For a single search, we need to solve up to 70 backend requests. Each backend response should be visible as soon as possible to the user. The total response time should remain close to the response time of the slowest backend. (~15s)
14. Hype Driven Development « Hey! What’s that Actors thing ? Can it help ? »
15. Actors Model reminder «Actor» is the primitive of concurrent computation Has an incoming message queue Reaction loop processing messages one by one Reacts to incoming messages by: making local decisions sending more messages to other Actors replying a message to the sender
16. Actors Model reminder Popularized by Akka for Scala For Groovy: GPars! We used 1.0 beta 1
17. GPars to the rescue Groovy Parallel Systems Handle tasks concurrently and asynchronously Concurrent collection processing Dataflow concurrency constructs Actor programming model !
18. GPars Actors Types DynamicDispatchActor Specific reaction per message type ReactiveActor Always replies at the end of processing StaticDispatchActor Simplest message processor, advised as default choice.
19. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Incoming HTTP requests are validated, and tagged with a correlation id.
20. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" A POGO is stored, it contains: the correlation id, HTTP request and response
21. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" From now on: all message hold the correlation id. Request message is sent to the Router Actor.
22. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Duplicate request messages are sent to specialized Business Actors.
23. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Business Actors reply the number of messages they’ll send to Service Actors. POGO counts expected responses.
24. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Request is split into backend compliant requests and sent to the next level.
25. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Messages are solved against Grails services, plugged to backends (long running)
26. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Grails services responses are replied up.
27. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" Business post treatment is applied. Response messages are sent to Router Actor. Content is written in HttpResponse.
28. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" When the count of expected response chunk reaches 0: HTTP response is closed and cleaned
29. Let it crash... or not Nice but fragile: every uncaught exception break an Actor Yes, ... we caught java.lang.Exception Encapsulate exceptions with Actor messages
30. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" An exception must propagate to the response counter. Elsewise, we have neverending requests.
31. Checkpoint Paper implementation working First implementation working Let’s add some load...
32. The Wild «On the paper, it worked, I swear.»
33. Every rose has its thorn One allocated Thread per Actor instance at a time Long running message processing leads to contention Grails services calling backend are long running...
34. The road not taken We could have gone with async HTTP That would still have lead us to XML processing parallelization We pushed GPars further
35. Round Robin Actor Inspired by Akka Balances messages on a homogeneous Actors set Very short message processing time
36. Round Robin Actor !"#$%!"&'$()*"+,-+.'$/0+1')0()*"+2 -+.'$ /0+1')03()*"+ -+.'$ /0+1')03()*"+ -+.'$ /0+1')03()*"+
37. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" We scale in peace Every RoundRobin has its own size.
38. Checkpoint Message processing is parallelized Global ThreadPool for all Actors GPars manages all boilerplate code for us Next moves: RoundRobin sizing Global ThreadPool sizing
39. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 1 1 1 1 Consider worst case ever to run in parallel for a single incoming request. Size does matter
40. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 16 4 1 16 4 1 21 Do the same for responses path.
41. !"#$%"& '()&"# *+&(# ,%-." /)01."002*+&(# 3#-1. /)01."002*+&(# '(-4 /)01."002*+&(# ,%-." !"#$1+"2*+&(# 3#-1. !"#$1+"2*+&(# '(-4 !"#$1+"2*+&(# 5&&6 '"06(.0"0 3#-1.2!"#$1+" ,%-."2!"#$1+" '(-42!"#$1+" 16 4 1 16 1 4 21 Sized RoundRobin for a well parallelized single incoming request.
42. ThreadPool sizing GPars offers Actors grouping, so a group share a dedicated ThreadPool For a first date, just Keep It Stupidely Simple ThreadPool must be the size of the biggest RoundRobin -Dgpars.poolsize=21
43. Checkpoint RoundRobin & ThreadPool sized... for one request! Next move: max the number of concurrent requests
44. Aiming for the top Get rid of interferences Live app instance with iso-prod memory resources Backends replaced by custom Vert.x proxy-cache
45. !"#$%&'(#)*$+,-*.,)# !&.+,$/.)0#,1 %2.,#$/.)0#,1 3'.1$/.)0#,1 4#&52#* 6#&*78 %&'89$:.)"# 6#&*78 %&'89$:.)"# 6#&*78 %&'89$:.)"# ;,(#)*'&$4)&+<* =%.&-7>+*"%''2$?$@!!%/A+21#& Throughput maxing Hype here
46. Throughput maxing !"#$%&#' Breaking ? ($)' (#*'"" Unused resources ? WIN! NO NO YES YES +,-'* '"#$%&#$,. /&$"' '"#$%&#$,.
47. Throughput maxing Be patient, be brave, be picky Leads to global performance tuning (connection pools, XML parsers) Goal reached when, under constant concurrent requests load: end-to-end response time stay constant over time with constant memory usage (and not too much GC)
48. First production Running on: 1 EC2 small instance CPU capacity of a 1.0-1.2 GHz 2007 Opteron Memory: 1.7 GB I/O Performance: Moderate JDK6 + Tomcat6 + Http11NioConnector -Xmx968m
49. Under stress load 80 search by minutes 20 really concurrent, 15 sec each each search is 70 backends calls, plus XML parsing and processing, business rules applying, JSON marshalling and response writing average CPU usage: ~70% average GC activity: ~15% Limited by Memory and Backend response time
50. Win Each response chunk is streamed to EmberJS UI Response time is equivalent to the slowest backend response + post processing Lineary scalable
51. Feedbacks
52. Testing Unit testing is easy Integration testing is non-deterministic User guide deserves a chapter on testing Find inspiration for your tests in Akka user guide
53. Good Points User guide... guides the user! Flat learning curve Just works! No bug has slowed us down.
54. Improvements ideas Monitoring Actors activity and queue state Testing tools and/or strategies Distributed Actors Flow specific Actors (RoundRobin, Restarter, ...)
55. Eat some, it’s great Actor model is hard to catch at first GPars is cool, it doesn’t stand in the way Thread free designing, business logic first Will definitely reuse
56. Q&A