Designing for Concurrency

A high-level overview of ways to abstract over concurrency in software applications and services.

Transcript

1. Designing for Concurrency Susan Potter Finsignia August 2010
2. Types of Clients Hedge firms (e.g. Stark, CIG, CS) Investment banks (e.g. BofA) Trading technology (SaaS/ASP) firms
3. Concurrent Applications market data trading systems (front office) risk management (middle office) accounts/party service (back office)
4. Traditional Approaches Task-based thread [pools] (e.g. database connections, server sockets) Hand coded locks (to access shared memory "safely") Data: sharding or replication (to increase throughput on data access)
5. Less Traditional Approaches Actor-based processes (e.g. message passing like in Erlang) Software Transactional Memory (STM) (consistent and safe way to access shared state) Data: decentralized datastores (run map/reduce queries on many nodes at once)
6. Task-based vs Actor-based task threads access shared state in objects task threads compete for locks on objects synchronous operations within task thread limited task scheduling (e. g. wait, notify) mailboxes buffer incoming messages actors do not share state, thus not competing for locks messages sent asynchronously actors react to messages sent to them
7. Actor-based Processes
8. When might actors be better? complexity of the task-based model becomes bottleneck (debugging race conditions, deadlocks, livelocks, starvation). Depends on your use case. system is event-driven conceptually. Easier to translate to high level abstraction in actor-based models.
9. Locks vs STM Flexibility: fine vs coarse grained choice Pessimistic locking Locking semantic need to be hand coded Composable operations are not well supported Analogous to database transaction recording each txn as log entry Optimistic reading Atomic transaction Supports composable operations
10. Software Transactional Memory (STM) Source: Armstrong on Software
11. When to use STM? Using more cores/processors (STM=performance++) on larger numbers of cores/processors (~>=4) Hand coding and debugging locking semantics for application becomes your bottleneck to prevent deadlocks and livelocks Priority inversion often hinders performance BUT YOU CAN'T use STM when operation on shared state cannot be undone. Must be undoable!
12. Replication vs Decentralized Can improve throughput Some flexibility: replication strategies for a few use cases Requires full replica(s) of data set on each node Improve throughput, performance of complex queries using map/reduce Flexibility to optimize two of three: Consistency, Availability, Partition tolerance (CAP Theorem) Does not require full replica(s) of data set
13. When to use decentralized data? Large data set you want distribute without creating/managing your own sharding scheme Want to optimize two of CAP Run distributed map/reduce complex queries BUT datastore should satisfy your other needs first. Usually key-value/bucket lookup, not RDBMS!
14. Other Approaches...(not in production) Compiler parallel optimizations e.g. Haskell sparks Persistent data structures to aid concurrency throughput by better API design
15. General Tips Use SLA metrics/measures to optimize relevant parts of your concurrent system judiciously Ensure your applications fit use case(s) for approach Test your hypothesis by benchmarking NEVER assume your changes have made the impact you expect. There is no silver bullet: think, implement and test!
16. Questions Twitter: @SusanPotter GitHub: http://github.com/mbbx6spp Email: susan@finsignia.com