Jürgen Höller on Concurrency

Transcript

1. Concurrency in Application Frameworks Jürgen Höller VP & Distinguished Engineer SpringSource
2. Agenda • Common Myths • Thread Safety • Kinds of State • Scoped Objects
3. Common Myths • It's too hard for a regular developer to make shared objects thread-safe. • Concurrent programming is all about using the 'synchronized' keyword. • Shared objects are not allowed to contain 'state'. • Setter injection has a problem with visibility of variables. • Pooling is a relevant option for application objects.
4. Thread Safety (1) • First of all, what does "thread-safe" really mean? – What we usually mean is that there are no unintended side effects when reentrant method calls come into the same instance. • What guarantees does the Java runtime provide already? – Local variables within a method are always fully isolated between calls and hence between threads. – Modifications to instance variable state will affect shared state for all callers. • But: the problem of 'visibility' between threads… • Also called "happens-before" relationship…
5. Thread Safety (2) public class OrderService { private int orderCount; private int itemCount; public void processOrder(Order order) { … this.orderCount++; this.itemCount += order.getItemCount(); } }
6. Thread Safety (3) • Danger: In a concurrent environment, modifications to shared variable state will be performed in a processor's register and not in main memory. – Publishing the modification back to main memory will happen eventually • but not immediately due to a processor's memory caching – Other threads will pick up the current state from main memory – once it is there… • might see stale values in the meantime • The application can enforce immediate publication of state changes – through using a memory barrier ('synchronized') – or through using the 'volatile' keyword on the variable – 'final' variables are safely visible to all threads as well
7. Thread Safety (4) public class OrderService { private final Object statisticsMonitor = new Object(); private int orderCount; private int itemCount; public void processOrder(Order order) { … synchronized (this.statisticsMonitor) { this.orderCount++; this.itemCount += order.getItemCount(); } } }
8. Thread Safety (5) • Some recommendations – Mark instance variables as 'final' as far as possible • accept and/or calculate values on construction – When modifying shared instance variables, use either 'volatile' or synchronization. – For modifying complex object state, always use synchronization. • Otherwise the nested contents of your object might not be fully visible when you assign the reference. – Synchronization is quite cheap on modern VMs. • Don't optimize prematurely by trying to avoid synchronization where it doesn't matter. • Of course, don't over-use synchronization where it isn't necessary either.
9. Thread Safety (6) • Shared Collections or Maps – Plain ArrayList, HashSet, HashMap are not thread- safe – Collections.synchronizedList/Set/Map is quite costly • always locks the entire collection, even just for simple retrieval operations • not very recommendable on Java 5 and above anymore – Modern solution in java.util.concurrent: CopyOnWriteArrayList/Set and ConcurrentHashMap • optimized for frequent reads and less frequent writes • ConcurrentHashMap locks specific hash buckets only instead of the entire Map
10. Thread Safety (7) public class OrderService { private final Map<String, Country> countryCache = new ConcurrentHashMap<String, Country>(); public void processOrder(Order order) { String countryCode = order.getCountryCode(); Country country = this.countryCache.get(); if (country == null) { country = new Country(countryCode); this.countryCache.put(countryCode, country); } … } }
11. Kinds of State (1) • Now, which state to hold in shared instance variables in the first place? – e.g. in Spring singletons or in Servlets • NO: – clearly not user-specific state • from the perspective of a user, such shared objects are 'stateless' • user-specific or request-specific state to be passed in as method arguments only – no single-threaded resources either • such as JDBC Connections or Hibernate Sessions
12. Kinds of State (2) • Now, which state to hold in shared instance variables in the first place? – e.g. in Spring singletons or in Servlets • YES: – thread-safe resource factories • such as JDBC DataSources – generally: configuration state • also settings, flags, etc • to be covered in more detail in a minute – also: counters, statistics, etc • use synchronization accordingly
13. Kinds of State (3) • How to set configuration state into a shared object? – constructors ("constructor injection") – bean properties ("setter injection") – annotation-driven method/field injection • Constructors have the advantage of being able to set 'final' fields – state safely visible to any thread – no accidental modification possible – e.g. a final DataSource field in a DAO • Bean properties and config methods/fields can effectively only be populated in a second step
14. Kinds of State (4) public class OrderService { private final OrderRepository orderRepository; public OrderService(OrderRepository repo) { this.orderRepository = repo; } public void processOrder(Order order) { … this.orderRepository.storeOrder(order); } }
15. Kinds of State (5) • What about setter injection? – can't use 'final' variables – unclear visibility situation – potentially repeated calls • Need to differentiate between configuration- time setters and runtime setters – configuration time: will only be called before the object is put into service • i.e. before the object's service methods will be called – runtime: can be called while the object is in service • e.g. through JMX
16. Kinds of State (6) public class OrderService { private OrderRepository orderRepository; public void setRepository(OrderRepository repo) { this.orderRepository = repo; } public void processOrder(Order order) { … this.orderRepository.storeOrder(order); } }
17. Kinds of State (7) • "Thread-safe publishing" – putting an object into service only after its configuration-time setters have been called – only exposing it to the public at that time • Using synchronization for the configuration + publishing step enforces happens-before – between the object's creation phase and the service phase – all of its configuration state guaranteed to be visible in the service phase • This effectively makes setter injection as reliable as constructor injection – since frameworks such as Spring usually perform that level of synchronization under the hood
18. Kinds of State (8) • Runtime setters – as used for JMX and other forms of management – need to use synchronization when modifying the underlying instance variable(s) – consider using 'volatile' for simple types of state • such as String or Integer • Make sure that you're accessing such state in a thread-safe way as well – using synchronization on the same monitor – no problem when using 'volatile' which applies both ways anyway
19. Kinds of State (9) • Keep it simple for configuration state – Constructor injection: use 'final' • no synchronization or 'volatile' necessary – Setter injection: plain instance variables • no synchronization or 'volatile' necessary either • at least when managed by a framework • Use synchronization for runtime state – e.g. caches: use ConcurrentHashMap – e.g. statistics: use 'synchronized' – e.g. attributes through setters: use 'volatile'
20. Scoped Objects (1) • What about objects in other scopes: request, session, conversation? – that's dependent on the specific scope… • Request scope is generally only ever accessed by a single thread at the same time – no precautions to take for concurrent access • Session scope and conversation scope may be accessed from several requests at the same time – consider AJAX and multiple browser windows – same precautions to take as for singleton objects
21. Scoped Objects (2) • Generally, design your scoped objects such that they are fully thread-safe – be prepared for concurrent access – it's usually simple enough • Modern VMs optimize synchronization heavily if an object is proven to never leave the current thread – effectively removing synchronization at runtime if proven to be unnecessary – very useful if the same class will be used in shared as well as non-shared scenarios
22. Scoped Objects (3) • What about pooling? – Pooled objects are different from scoped objects • can't obtain the same instance when performing retrieval in the same contextual 'scope' – Pooling effectively locks the target instance for each method invocation • keep multiple objects in the pool in order to remain scalable • Pooling doesn't make a difference for configuration state – e.g. each pooled object referring to same JDBC DataSource – doesn't make sense for counter, statistics, etc either • You would have individual counters in each pooled object! – possibly keep a JDBC Connection or Hibernate Session in an instance variable? • Prefer the use of DataSource or SessionFactory's getCurrentSession(), also for transaction participation.
23. Scoped Objects (4) • If you feel that you need pooled application objects, reconsider your needs… – For simple resource accessors, use 'singleton' scope and reobtain the current resource (e.g. Hibernate Session) for every call. • Underlying resource factory reuses resources within transactions anyway (using a thread-local). • Underlying physical resources (e.g. JDBC Connections) will get pooled by the resource factory already. – For throwaway objects, consider the use of 'prototype' scope or 'request' scope. • new instance on every retrieval or for every request
24. Scoped Objects (5) • For longer-lived objects, consider 'session' or 'conversation' scope – Remember: You'll have to make session-scoped objects thread-safe as well. – can keep user-specific / session-specific state next to a long-lived Hibernate Session, for example • What about pooling for the purpose of limiting access to your component? – that typically happens at the system entry point level • e.g. limited thread pool attached to HTTP socket – or at the resource factory level • e.g. limited amount of JDBC Connections in DataSource – or at the interceptor level • e.g. Spring's ConcurrencyThrottleInterceptor