Image by Mike Rohde
in the Cloud and at Scale
Yan Cui (@theburningmonk)
F#
Slide 2
Slide 2 text
Who is Gamesys?
• Founded in 2001
• #1 in the UK
• Handle $5 Billion in turnover annually
• First company to launch real money gaming on Facebook
• Employ 1,000 globally
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
Running in the Cloud, at Scale
• 1 Million DAU
• 250 Million requests/day
• 7 Billion rows of analytics events/month
• 2 TB of analytics data/month
Slide 7
Slide 7 text
Running in the Cloud, at Scale
• 100% cloud hosted
• AWS for Game services
• Google App Engine for back-office services
• NoSQL databases
– DynamoDB, CouchBase, Redis, Neo4j, ...
• Google BigQuery for analytics
Slide 8
Slide 8 text
What is the Cloud?
Cloud computing is an expression used to describe a
variety of computing concepts that involve a large
number of computers connected through a real-time
communication network such as the Internet....
Such virtual servers do not physically exist and can
therefore be moved around and scaled up (or down)
on the fly without affecting the end user - arguably,
rather like a cloud.
- Wikipedia
Slide 9
Slide 9 text
What is the Cloud?
IaaS
VMs, load
balancers,
storage, ...
PaaS
Runtime,
database, web
servers, ...
SaaS
Virtual desktop,
games, emails,
...
Slide 10
Slide 10 text
Why the Cloud?
• Trade capital cost with operating cost
• Lower operating cost
• Elastic scaling
• Speed and agility
• Focus on what differentiates your product
• Global infrastructure for your global audience
Slide 11
Slide 11 text
Why the Cloud - Scalability
• Scale up
• Scale out
• Scale everything!
– Your architecture is as scalable as its least scalable part
• Scalability != simple choice of language/framework
Slide 12
Slide 12 text
Travel, Collect, Craft!
Slide 13
Slide 13 text
Trap Monsters
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
Stateful Server
Elastic Load Balancer
S3
Auto scaling Group
Server A Server B
...
EC2
CloudFront
Slide 16
Slide 16 text
Stateful Server
• Need to ensure Server affinity
– All calls need to be routed to the affined server
• Need to balance load
– Session lengths vary greatly
– Some players are more active than others
– Need to avoid hot spots
• Need to avoid players hogging a server
– Need to be able to scale down!
Slide 17
Slide 17 text
Stateful Server
• Persist player state after short inactivity
• Move player to another server after persistence
Slide 18
Slide 18 text
Why Stateful Server?
• 500% efficiency increase
• 60% reduction in avg latency
• Fewer game servers
• No CouchBase cluster
• Huge saving on cost
Slide 19
Slide 19 text
The Actor Model
An actor is the fundamental unit of computation which
embodies the 3 things
• Processing
• Storage
• Communication
that are essential to computation.
-Carl Hewitt*
* http://bit.ly/HoNHbG
Slide 20
Slide 20 text
The Actor Model
• Everything is an actor
• An actor has a mailbox
• When an actor receives a message it can:
– Create new actors
– Send messages to actors it has addresses for
– Designate how to handle the next message it receives
Slide 21
Slide 21 text
Stateful Server
• Gatekeeper
– Manages the local list of active workers
– Spawns new workers
• Worker
– Manages the states for a player
– Optimistic locking
– Persist state after period of inactivity
Slide 22
Slide 22 text
Stateful Server
Game Server
Player A
Player B
S3
Worker C
Worker B
Gatekeeper
Request Handlers
Asynchronous
Slide 23
Slide 23 text
Stateful Server
Game Server
Player A
Player B
S3
Worker C
Worker B
Gatekeeper
Worker A
Request Handlers
Asynchronous
ok
Slide 24
Slide 24 text
Stateful Server
Game Server
Player A
Player B
S3
Worker C
Worker B
Gatekeeper
Worker A
Request Handlers
Asynchronous
Slide 25
Slide 25 text
Stateful Server
Game Server
Player A
Player B
S3
Worker C
Gatekeeper
Worker A
Request Handlers
Asynchronous
Slide 26
Slide 26 text
Stateful Server
Game Server
Player A
Player B
S3
Worker C
Worker A
Request Handlers
Gatekeeper
Asynchronous
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
MailboxProcessor
Slide 29
Slide 29 text
Async
Slide 30
Slide 30 text
switch state
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
Why F#?
• Time to Market
• Efficiency
• Correctness
• Complexity
Slide 34
Slide 34 text
Why F#?
• Agents
– No locks
– Asynchronous message passing
– Each actor is self-contained and easier to reason with
• Pattern matching
– Clear and concise way to handle all branch conditions
Slide 35
Slide 35 text
Why F#?
• Async Workflows
– Non-blocking IO
– Convert synchronous code into asynchronous code with
minimal code changes
– Similar to C# 5’s async-await feature, but available in F#
since 2007!
Slide 36
Slide 36 text
Thank You!
Slide 37
Slide 37 text
JackpotJoy Slots
http://apps.facebook.com/jackpotjoyslots
Bingo Lane
http://apps.facebook.com/bingolane
Here Be Monsters
http://apps.facebook.com/herebemonsters
Building a MMORPG
http://bit.ly/1hjqoL8
http://slidesha.re/18MD4XY
Google I/O 2013 – Here Be BigQuery
http://bit.ly/1fHjbce