Slide 1
Slide 1 text
Designing for Concurrency
Susan Potter
Finsignia
August 2010
Slide 2
Slide 2 text
Types of Clients
Hedge firms
(e.g. Stark, CIG, CS)
Investment banks
(e.g. BofA)
Trading technology (SaaS/ASP) firms
Slide 3
Slide 3 text
Concurrent Applications
market data
trading systems (front office)
risk management (middle office)
accounts/party service (back office)
Slide 4
Slide 4 text
Traditional Approaches
Task-based thread [pools]
(e.g. database connections, server sockets)
Hand coded locks
(to access shared memory "safely")
Data: sharding or replication
(to increase throughput on data access)
Slide 5
Slide 5 text
Less Traditional Approaches
Actor-based processes
(e.g. message passing like in Erlang)
Software Transactional Memory (STM)
(consistent and safe way to access shared state)
Data: decentralized datastores
(run map/reduce queries on many nodes at once)
Slide 6
Slide 6 text
Task-based vs Actor-based
task threads access shared
state in objects
task threads compete for
locks on objects
synchronous operations
within task thread
limited task scheduling (e.
g. wait, notify)
mailboxes buffer incoming
messages
actors do not share state,
thus not competing for locks
messages sent
asynchronously
actors react to messages sent
to them
Slide 7
Slide 7 text
Actor-based Processes
Slide 8
Slide 8 text
When might actors be better?
complexity of the task-based model becomes bottleneck
(debugging race conditions, deadlocks, livelocks,
starvation). Depends on your use case.
system is event-driven conceptually. Easier to translate
to high level abstraction in actor-based models.
Slide 9
Slide 9 text
Locks vs STM
Flexibility: fine vs coarse
grained choice
Pessimistic locking
Locking semantic need
to be hand coded
Composable operations
are not well supported
Analogous to database
transaction recording
each txn as log entry
Optimistic reading
Atomic transaction
Supports composable
operations
Slide 10
Slide 10 text
Software Transactional Memory (STM)
Source: Armstrong on Software
Slide 11
Slide 11 text
When to use STM?
Using more cores/processors (STM=performance++) on
larger numbers of cores/processors (~>=4)
Hand coding and debugging locking semantics for
application becomes your bottleneck to prevent
deadlocks and livelocks
Priority inversion often hinders performance
BUT YOU CAN'T use STM when operation on shared
state cannot be undone. Must be undoable!
Slide 12
Slide 12 text
Replication vs Decentralized
Can improve throughput
Some flexibility: replication
strategies for a few use cases
Requires full replica(s) of data
set on each node
Improve throughput,
performance of complex
queries using map/reduce
Flexibility to optimize two of
three: Consistency,
Availability, Partition tolerance
(CAP Theorem)
Does not require full replica(s)
of data set
Slide 13
Slide 13 text
When to use decentralized data?
Large data set you want distribute without
creating/managing your own sharding scheme
Want to optimize two of CAP
Run distributed map/reduce complex queries
BUT datastore should satisfy your other needs first.
Usually key-value/bucket lookup, not RDBMS!
Slide 14
Slide 14 text
Other Approaches...(not in production)
Compiler parallel optimizations
e.g. Haskell sparks
Persistent data structures
to aid concurrency throughput by better API design
Slide 15
Slide 15 text
General Tips
Use SLA metrics/measures to optimize relevant parts of
your concurrent system judiciously
Ensure your applications fit use case(s) for approach
Test your hypothesis by benchmarking
NEVER assume your changes have made the impact you expect.
There is no silver bullet: think, implement and test!
Slide 16
Slide 16 text
Questions
Twitter: @SusanPotter
GitHub: http://github.com/mbbx6spp
Email: [email protected]