Halo 4: High Demand
with Low Latency & High
Availability
▪SurgeConf 2014
Slide 2
Slide 2 text
Caitie McCaffrey
Distributed Systems
Engineer
@CaitieM20
CaitieM.com
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
Halo:CE - 6.43 million
Halo 2 - 8.49 million
Halo 3 - 11.87 million
Halo 3: ODST - 6.22 million
Halo Reach - 9.52 million
Slide 6
Slide 6 text
Presence
Statistics
Title Files
Cheat Detection
User Generated Content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
$220 million in sales
!
1 million players online
Day One
Slide 9
Slide 9 text
$300 million in sales
!
4 million players online
!
31.4 million hours
Week One
Slide 10
Slide 10 text
11.6 million players
!
1.5 billion games
!
270 million hours
Overall
Slide 11
Slide 11 text
Architectural
Challenges
Slide 12
Slide 12 text
Load Patterns
Slide 13
Slide 13 text
Cloud
Azure Worker Roles
Azure Table
Azure Blob
Azure Service Bus
Slide 14
Slide 14 text
Latency &
Concurrency
Slide 15
Slide 15 text
Stateless 3 Tier
Architecture
Slide 16
Slide 16 text
Latency Issues
Slide 17
Slide 17 text
Add A Cache
Add A Cache
Slide 18
Slide 18 text
Concurrency
Issues
Concurrency
Issues
Slide 19
Slide 19 text
Data Locality
19
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
Actor Model
A framework & basis for reasoning about concurrency
23
Slide 24
Slide 24 text
Orleans
“Orleans is a runtime and
programming model for
building distributed systems,
based on the actor model”
eXtreme Computing Group at Microsoft Research
Slide 25
Slide 25 text
Orleans Programming Model
Slide 26
Slide 26 text
• .NET Framework
• Actor Interfaces
• Promises
• Actor References
• Turns
• Persistence
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
• .NET Framework
• Actor Interfaces
• Promises
• Actor References
• Turns
• Persistence
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
• .NET Framework
• Actor Interfaces
• Promises
• Actor References
• Turns
• Persistence
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
Orleans is a Distributed
Actor Runtime
Slide 33
Slide 33 text
• Silos Provide
!
Messaging
!
Hosting
!
Execution
Slide 34
Slide 34 text
Virtual Actors
“An Orleans actor always exists, virtually. It
cannot be explicitly created or destroyed.”
Slide 35
Slide 35 text
V
Virtual Actors
Perpetual Existence
Automatic Instantiation
Location Transparency
Automatic Scale Out
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
Halo 4
Statistics Service
37
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
Player Grain
Slide 40
Slide 40 text
Game
Grain
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
Performance &
Scalability
43
Slide 44
Slide 44 text
“Orleans applications run at very high CPU
Utilization. We have run load tests with full
saturation … at 90%+ CPU utilization without
any instability.”
Slide 45
Slide 45 text
Performance & Scalability
Slide 46
Slide 46 text
Performance & Scalability
Slide 47
Slide 47 text
Failure Domains
47
Slide 48
Slide 48 text
At Least Once Messaging
Slide 49
Slide 49 text
Machine Failures
Slide 50
Slide 50 text
TOO!
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
TOO!
Slide 53
Slide 53 text
TOO!
Slide 54
Slide 54 text
Orleans is AP
Slide 55
Slide 55 text
preview: http://aka.ms/orleans
samples: https://orleans.codeplex.com/
Get Orleans