JRuby 9.2 and Rails 5.x

(For RubyKaigi 2018) https://rubykaigi.org/2018/presentations/tom_enebo.html#jun02

JRuby 9.2 has been released. 9.2 supports Ruby 2.5 compatibility and it also runs Rails 5.x well. This talk will discuss some of the more interesting apects of JRuby 9.2:

Performance updates
Graal integration
IR instr refactoring
Object shaping
Full encoding support ( @@かいぎ ||= $🐻🌻.send :┬─┬ノº_ºノ' )
Improved Windows support

It will also give an update on the state of running Rails 5.x on JRuby. This talk will go over updates we have made to ActiveRecord-JDBC and show a real world use-case of getting Discourse running. Get up to date on the state of JRuby!

Transcript

1. JRuby 9.2.x & Rails 5.x Thomas E. Enebo @tom_enebo jruby.org Red Hat Inc.
2. Whoami • Thomas Enebo • JRuby co-lead • Red Hat Employee • Likes Beer, Running, and running with beer • Long time Java and Ruby guy
3. 1.7.27 jruby-1_7 1.9.x, Java 7+, End of Life JRuby Roadmap master ... jruby-9.1 ...2.3.x, Java 7+ 2.4.x, Java 8+ RubyKaigi 2017 -> Today JRuby 1.7.x retired 9.1.17.0 A few 9.1.x point releases JRuby 9.2.0.0 Released Last week with 2.5 support O_o
4. Procrastination? 10/16 10/16: 2.4 features work starts 05/17 05/17: 2.4 features 90% complete 09/17 09/17: Rubykaigi 90% complete 12/17: 2.5 released, 2.4 98% complete 12/17 05/18 05/18: JRuby 9.2 released, 2.5 support
5. JRuby 9.2.0.0 • Ruby 2.5 • non-ASCII identifiers and symbols
6. 2.5 Support Outline made from MRI NEWS file
7. New Features == New Contributors • New contributors love new features • isolated methods to implement • PRs are fun low commitment path
8. !ASCII • :ݴ༿ • ˏ(›°□°)›ớᵲᴸᵲ • @@ • $ • …local variables, methods, constants…
9. Mysterious Errors $ jruby throw.rb Calm down, bro ArgumentError: invalid byte sequence i inspect at org/jruby/RubySymbol.java inspect at org/jruby/RubySymbol.java inspect at org/jruby/RubyArray.java: p at org/jruby/RubyKernel.java <main> at ../snippets/throw.rb:10 9.1.17.0 $ jruby throw.rb Calm down, bro [:method_missing, :(›°□°)›ớᵲᴸᵲ, :pu 9.2.0.0 # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods
10. 9.2.0.0 $ jruby vowel.rb undefined local variable or method `öÖa' for main:Object 9.1.17.0 $ jruby vowel.rb undefined local variable or method `\366\326a’ for main:Objec Mangled Output # encoding: utf-8 begin öÖa rescue => e puts e.message end
11. Reflection Woes # coding: utf-8 def ݴ༿ puts "word" end self.class.send :ݴ༿ $ jruby wordy.rb NoMethodError: undefined method `ݴ༿' for method_missing at org/jruby/RubyBasicObje <main> at ../snippets/throw3.rb:6 9.1.17.0 9.2.0.0 $ jruby wordy.rb word
12. note: technically m17n came in 2008 — 10 years
13. INTERNALLY ALL KEY IDENTIFIERS ARE java.lang.String tl;dr No explicit knowledge of encoding ~5000 lines of changes…
14. Old Way # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods parsed into bytes make j.l.String from bytes store method with j.l.String as key get list of keys try to make symbols from keys WTF encoding is it????
15. New Way # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods parsed into bytes make Symbol from bytes store method with j.l.String as key get list of keys look up Symbols from “binary” Strings make “binary/iso8859_1” j.l.String +sym to symtable with String as key
16. Minor Incompatibility • Two same byte sequenced identifiers with different encodings
17. Performance
18. Graal • JIT Compiler for Java • Written in Java • Shipped in Java 9+ • Not on by default
19. Can Replace C2 Client Compiler Server Compiler Graal Compiler Compiler Interface JVMCI Hotspot C1 C2 Graal Use Graal Instead
20. Graal with JRuby • Sometimes significantly faster • Usually same or a little slower • Larger -> slower • Escape Analysis shines!
21. JRuby’s Math Problem long value = 12; 12 + 2 Internally RubyFixnums 12_obj:RubyFixnum “boxed value" 12_obj.value() + 2_obj.value() Unlike MRI which is a numeric primitive 12 + 2
22. Escape Analysis • Proves an object never leaves a scope in a particular lifetime • In fixnum case we know 2 and 12 do not escape it’s usage. • Compiler sees we only use the long field • Don’t allocate Ruby fixnum. just use field types directly 12_obj.value() + 2_obj.value() 12 + 2
23. Mandlebrot 0 0.45 0.9 1.35 1.8 Mandlebrot (s) 0.071 0.073 0.085 0.74 1.3 1.78 1.79 CRuby head CRuby head +MJIT JRuby JRuby +indy JRuby +indy +Graal JRuby +indy +Graal -fixnum Ruby Truffle Ruby Default JRuby JRuby Invokedynamic++ JRuby +Graal JRuby +Graal disable fixnum cache Truffle Ruby
24. Graal? • Keeps getting better • May replace C2? • Is an option today for JRuby users -J-XX+UnlockExperimentalVMOptions -J-XX:+EnableJVMCI -J-XX:+JVMCICompiler
25. Splitting up Call Frames
26. Call Frames {lastline($_), backref($~), visibility, block, self, name, class} We maintain extra info about each ruby method call: Special methods need to access to these: block_given? Saving these values has a cost
27. Partial Frames If we only need $~, then only save $~ def foo(a) a[0] end call 1_000_000 times 0i/s 30i/s 60i/s 90i/s 120i/s 1M calls 116M calls/s 68M calls/s 17M calls/s 9.1 9.2 9.2+graal
28. keyword arguments improvement
29. Reduce allocation size • Partial solution • Positional arguments opt coming • Storing in a real hash • Single bucket hashes used • noticed not using internal set methods
30. def foo(a: 1, b: 2) b + a end 1_000_000.times do foo(a: 2, b: 3) end 0.00k i/s 1.08k i/s 2.15k i/s 3.23k i/s 4.30k i/s kwargs call 4.3 3.82 9.1.17.0 9.2.0.0
31. JRuby on Rails
32. JRuby on Rails • Working since 2006 • Used by many • Support lagged during JRuby 9000, Ruby 2.3-2.4 work • Rails 5.0+ • ActiveRecord suffered the most
33. ActiveRecord JDBC “Our Journey?”
34. Compatibility is Hard
35. Compatibility AR to Java DataBase Connectivity ARJDBC Activerecord Rails JDBC JRuby JDBC Java Ruby DBs(SQLite, MySQL, PostgreSQL, msql, …)
36. Compatibility ActiveRecordJDBC 1.3.x JRuby 1.6.x, 1.7.x, 9.0.x, 9.1.x Java 6,7,8 ActiveRecord 4.2, … JDBC (many) databases Multiple versions Multiple distinct features (generic, h2, heal, mssql, MySQL, PostgreSQL, SQLite, MariaDB, DB2, Firebird, Oracle, ???)
37. ARJDB Compatibility • Too much to support / Not enough resources • Too much required domain knowledge • “Can I remove this line now?” • Repeats JRuby 1.7.x mistake • multi-mode compatibility is a mistake
38. ARJDBC Support Strategy • SQLite, MySQL/MariaDB, PostgreSQL • end of life == rails end of life 50.x Rails 5.0.x 51.x 52.x Rails 5.1.x Rails 5.2.x … 50-stable 51-stable master
39. Looking Promising… • Use case: MySQL support • old adapter: 1602 lines of Ruby • 50.0 adapter: 284 lines (still some old stuff which needs pruning) • Java code is smaller • 51.0 adapter: 6 lines of change!!!
40. Rails 5.1.6 actioncable: 139 runs, 733 assertions, 10 failures, 2 errors actionpack: 3063 runs, 14947 assertions, 2 failures, 0 errors actionmailer: 204 runs, 456 assertions, 0 failures, 0 errors actionview: 1957 runs, 4303 assertions, 3 failures, 4 errors activejob: 137 runs, 302 assertions, 0 failures, 0 errors activemodel: 713 runs, 2017 assertions, 0 failures, 0 errors activerecord: 4991 runs, 13902 assertions, 0 failures, 0 errors activesupport: 3671 runs, 760486 assertions, 14 failures, 0 errors railties: 40 runs, 73 assertions, 0 failures, 1 errors 30F, 6E & 797,219 assertions 99.995% passing
41. Mostly time/date error? Failure: TimeWithZoneTest#test_minus_with_time_precision [/home/ enebo/work/rails/activesupport/test/core_ext/ time_with_zone_test.rb:340]: Expected: 86399.999999998 Actual: 86399.99999999799
42. Rails 5.2.0 actioncable: something broken bootstrapping actionpack: 3148 runs, 15832 assertions, 1 failures, 0 errors actionmailer: 204 runs, 457 assertions, 0 failures, 0 errors actionview: 1990 runs, 4395 assertions, 4 failures, 4 errors activejob: 173 runs, 401 assertions, 0 failures, 0 errors activemodel: 803 runs, 2231 assertions, 0 failures, 0 errors activerecord: 5226 runs, 14665 assertions, 8 failures, 6 errors activesupport: 4135 runs, 762864 assertions, 17 failures, 2 errors railties: uses fork()
43. Rails 5.2 • Simple Scaffolded App works • SQLite • MySQL/PostgreSQL update needed
44. 5.1 arjdbc bench • used internally for point to point comparisons • “mild” idea of performance
45. .create x5000 Seconds 0 9.75 19.5 29.25 39 .create(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1
46. .find x5000 Seconds 0 0.325 0.65 0.975 1.3 .find(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1
47. Transcoding!!! JRuby DB JDBC Ruby UTF-8 UTF-16LE UTF-8 MRI DB Ruby UTF-8 UTF-8 2 transcodes 0 transcodes
48. .select x5000 Seconds 0 0.45 0.9 1.35 1.8 .select(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1
49. .find_all x5000 Seconds 0 0.8 1.6 2.4 3.2 .first(10) JRuby JRuby+indy JRuby+graal MRI 2.5.1
50. .update x5000 Seconds 0 0.5 1 1.5 2 .update(string) JRuby JRuby+indy JRuby+graal MRI 2.5.1
51. Scaffolded App Bench • Simple Scaffolded App • Rails 5.1.6, Postgresql • EC2 c4.xLarge: 4 vCPUs, 7.5G • Bench, DB, App on same instance
52. Requests per second 0 325 650 975 1300 10k 20k 30k 40k 50k 60k 70k 80k 90k 100k CRuby 2.5 JRuby JRuby +Indy 100k requests. Client Concurrency 10
53. Future • Try to push more Ruby from activerecord-jdbc -> active record • Be more engaged in Rails core development (!fork (›°□°)›ớᵲᴸᵲ )
54. Discourse? • Does not currently work with JRuby • “Standard” for comparing performance • Noah Gibbs (@appfolio) has harness • Good testbed for compatibility
55. Discourse Issues • NATIVE C EXTENSIONS!!!! • cppjieba -> jieba-analysis (JI bindings) • libv8/mini_racer -> therubyrhino (Gemfile) • nokogumbo -> ffi bindings • oj -> java native extension 70% done • pg -> jruby-pg (Gemfile) • rinku -> autolink-java (JI bindings)
56. Dealing with Cexts • Work with gem authors • Identify solution • ffi • Java integration • pure Ruby • java native extension • Do our own gem (least appealing)
57. Dealing with Cexts • Working with gem authors • Making our own gem gem ‘oj’ gem ‘therubyrhino’, platform: :jruby gem ‘miniracer’, platform: :ruby
58. Thank You • @tom_enebo • https://www.jruby.org • https://github.com/jruby/activerecord-jdbc • https://github.com/jruby/jruby • http://www.redhat.com