Writing concurrent libraries for all Ruby runtimes

Have you ever wondered how to use all of your cores?

The talk will take you on a path of writing a simple concurrent class. We'll start with a basic implementation and gradually improve it based on presented problems and newly learned facts. Our final solution will behave well in the face of concurrency and execute consistently on all Ruby implementations.

We’ll investigate various Ruby runtime differences that we’ve abstracted away with the synchronization layer of the concurrent-ruby gem. We'll go down to JRuby extensions, even volatile fields and compare-and-swap operations.

Benchmark source code and examples can be found in https://github.com/ruby-concurrency/concurrent-ruby/tree/master/benchmarks.

Video can be found here http://confreaks.tv/videos/rubyconf2015-writing-concurrent-libraries-for-all-ruby-runtimes

These slides contain updated JRuby results with enabled invokedynamic, gain was about 40% compared to original numbers which can be seen in the video.

Transcript

2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Writing concurrent libraries for all Ruby runtimes Petr Chalupa Principal Member of Technical Staf Oracle Labs November 16, 2015
3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to provide some insight into a line of research in Oracle Labs. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Oracle reserves the right to alter its development plans and practices at any time, and the development, release, and timing of any features or functionality described in connection with any Oracle product or service remains at the sole discretion of Oracle. Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle. 3
4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Project introductions 4
5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Concurrent Ruby gem • Unopinionated • Supports MRI, JRuby, Rubinius, (Jruby+Truffle) • Used by: Rails 5, Sucker Punch, Dynflow, … – High-level: Future, Channels, Actor, Agent, STM, … – References: AtomicInteger, AtomicReference, Delay, ThreadLocalVar, … – Data structures: Array, Hash, Cache, Tuple, ImmutableStruct, … – Low-level: CountDownLatch, Semaphore, CyclicBarrier, ReadWriteLock, … 5 The Concurrent Ruby logo was designed by David Jones. It is Copyright © 2014 Jerry D'Antonio. All Rights Reserved.
6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | JRuby+Truffle • Part of JRuby • Self-Optimizing AST interpreter • Compiled by Graal • Whole Ruby without restrictions including fast debugging, set_trace_func, ObjectSpace • Language 93%, Core 91%, Standard library 87% • Open source, you can try it 'jruby -X+T' • It runs: concurrent-ruby, psd.rb, chunky_png, arel, roda, slim, sinatra, redis- rb, open-weather2, activesupport 6 The JRuby logo is copyright (c) Tony Price 2011, licensed under the terms of Creative Commons Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0)
7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Implementing a concurrent abstraction 7
8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Why is concurrency difficult? • Processor or compiler are free to reorder – As long as it keeps sequential consistency – Unless we tell it not to reorder – Desirable • Another thread may see strange value • All possible orders have to be considered • A framework is needed to reason about the orders • Has greater impact on JRuby+Truffle 8
9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Reordering Example 9
10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Future • Reference of a value which is not yet computed • API – .new – #fulfill(value) – #value => value (blocking) – #complete? => true || false (non-blocking) 10
11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Usage and MutexFuture Example 11
12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MutexFuture comparison 12 Mutex Mutex Mutex Mutex MRI JRuby JRuby+Truffle Rubinius 0 1 2 3 4 5 6 7 8 9 fulfill value complete? Seconds
13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | MRI and Global VM lock • Synchronization is expensive • There is a mutex in GVL • All changes made by GVL releasing Thread are made visible to GVL acquiring Thread • MRI instance variable are efectively volatile • What is Volatile? • Undocumented behavior! 13
14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | GVLFuture Example 14
15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Faster complete? and value without synchronization 15 Mutex GVL MRI 0 1 2 3 4 5 6 7 8 9 fulfill value complete?
16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | JRuby, JRuby+Truffle and Rubinius • Instance variables are non-volatile • Method call is not protected (including #initialize) • MutexFuture#initialize is incorrect • Final instance variable is needed 16
17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | RBXFuture and JRubyFuture Example 17
18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Faster complete? and value without synchronization 18 Mutex GVL Mutex JRuby Mutex RBX Mutex RBX MRI JRuby JRuby+Truffle Rubinius 0 1 2 3 4 5 6 7 8 9 fulfill value complete?
19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Getting rid of multiple implementations • Multiple implementations are error prone – Many of the concurrent-ruby abstractions would have multiple implementations • To have a single Implementation we need a layer • Memory model • Memory model extensions 19
20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Memory model • Provides framework to reason about program behavior in concurrent environment • Based on current implementations • Used in concurrent-ruby • Defines Atomicity, Volatility, Serializability of variables – Local variables have only atomicity – Instance variables have atomicity and serializability 20
21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Memory model extensions • Provided by a layer in concurrent-ruby – Concurrent::Synchronization::Object – .safe_initialization! – #attr_volatile :instance_variable_name – #attr_atomic :instance_variable_name • Has diferent implementation for each Ruby runtime • Can adapt as Ruby runtimes evolve 21
22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | CRFuture Example 22
23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Concurrent Ruby layer has same performance as specific implementations 23 GVL CR JRuby CR RBX CR RBX CR MRI JRuby JRuby+Truffle Rubinius 0 1 2 3 4 5 6 7 8 fulfill value complete?
24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Compare and set • attr_atomic :count • Adds following atomic operations – compare_and_set_count(expected_count, new_count) => true | false – swap_count(new_count) => old_count • They are essential for building lock-free abstractions 24
25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | CRCasFuture Example 25
26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Reduced time of fulfill with compare-and-set 26 CR CAS CR CAS CR CAS CR CAS MRI JRuby JRuby+Truffle Rubinius 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 fulfill
27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Final comparison 27 Mut. GVL CR CAS Mut. JRuby CR CAS Mut. RBX CR CAS Mut. RBX CR CAS MRI JRuby JRuby+Truffle Rubinius 0 1 2 3 4 5 6 7 8 9 fulfill value complete?
28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Average speedup for CRCasFuture CAS 0 2 4 6 8 10 12 14 16 18 1 3.71 15.61 2.46 MRI JRuby JRuby+Truffle Rubinius 28
29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Conclusion • If you need concurrent tool, look in concurrent-ruby first • If you don't find what you need, let us know • We can help with adding a new one • The layer supports writing fast portable concurrent abstractions • Keep an eye on JRuby+Truffle 29
30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Links • http://concurrent-ruby.com • https://github.com/jruby/jruby/wiki/Truffle • http://chrisseaton.com/rubytruffle/ • @pitr_ch 30