JRuby 9.2 and Rails 5.x

Slide 1

Slide 1 text

JRuby 9.2.x & Rails 5.x Thomas E. Enebo @tom_enebo jruby.org Red Hat Inc.

Slide 2

Slide 2 text

Whoami • Thomas Enebo • JRuby co-lead • Red Hat Employee • Likes Beer, Running, and running with beer • Long time Java and Ruby guy

Slide 3

Slide 3 text

1.7.27 jruby-1_7 1.9.x, Java 7+, End of Life JRuby Roadmap master ... jruby-9.1 ...2.3.x, Java 7+ 2.4.x, Java 8+ RubyKaigi 2017 -> Today JRuby 1.7.x retired 9.1.17.0 A few 9.1.x point releases JRuby 9.2.0.0 Released Last week with 2.5 support O_o

Slide 4

Slide 4 text

Procrastination? 10/16 10/16: 2.4 features work starts 05/17 05/17: 2.4 features 90% complete 09/17 09/17: Rubykaigi 90% complete 12/17: 2.5 released, 2.4 98% complete 12/17 05/18 05/18: JRuby 9.2 released, 2.5 support

Slide 5

Slide 5 text

JRuby 9.2.0.0 • Ruby 2.5 • non-ASCII identiﬁers and symbols

Slide 6

Slide 6 text

2.5 Support Outline made from MRI NEWS ﬁle

Slide 7

Slide 7 text

New Features == New Contributors • New contributors love new features • isolated methods to implement • PRs are fun low commitment path

Slide 8

Slide 8 text

!ASCII • :ݴ༿ • ˏ(›°□°)›ớᵲᴸᵲ • @@ • $ • …local variables, methods, constants…

Slide 9

Slide 9 text

Mysterious Errors $ jruby throw.rb Calm down, bro ArgumentError: invalid byte sequence i inspect at org/jruby/RubySymbol.java inspect at org/jruby/RubySymbol.java inspect at org/jruby/RubyArray.java: p at org/jruby/RubyKernel.java at ../snippets/throw.rb:10 9.1.17.0 $ jruby throw.rb Calm down, bro [:method_missing, :(›°□°)›ớᵲᴸᵲ, :pu 9.2.0.0 # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods

Slide 10

Slide 10 text

9.2.0.0 $ jruby vowel.rb undefined local variable or method `öÖa' for main:Object 9.1.17.0 $ jruby vowel.rb undefined local variable or method `\366\326a’ for main:Objec Mangled Output # encoding: utf-8 begin öÖa rescue => e puts e.message end

Slide 11

Slide 11 text

Reﬂection Woes # coding: utf-8 def ݴ༿ puts "word" end self.class.send :ݴ༿ $ jruby wordy.rb NoMethodError: undefined method `ݴ༿' for method_missing at org/jruby/RubyBasicObje at ../snippets/throw3.rb:6 9.1.17.0 9.2.0.0 $ jruby wordy.rb word

Slide 12

Slide 12 text

note: technically m17n came in 2008 — 10 years

Slide 13

Slide 13 text

INTERNALLY ALL KEY IDENTIFIERS ARE java.lang.String tl;dr No explicit knowledge of encoding ~5000 lines of changes…

Slide 14

Slide 14 text

Old Way # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods parsed into bytes make j.l.String from bytes store method with j.l.String as key get list of keys try to make symbols from keys WTF encoding is it????

Slide 15

Slide 15 text

New Way # coding: utf-8 def (›°□°)›ớᵲᴸᵲ puts "Calm down, bro" end (›°□°)›ớᵲᴸᵲ p self.class.methods parsed into bytes make Symbol from bytes store method with j.l.String as key get list of keys look up Symbols from “binary” Strings make “binary/iso8859_1” j.l.String +sym to symtable with String as key

Slide 16

Slide 16 text

Minor Incompatibility • Two same byte sequenced identiﬁers with different encodings

Slide 17

Slide 17 text

Performance

Slide 18

Slide 18 text

Graal • JIT Compiler for Java • Written in Java • Shipped in Java 9+ • Not on by default

Slide 19

Slide 19 text

Can Replace C2 Client Compiler Server Compiler Graal Compiler Compiler Interface JVMCI Hotspot C1 C2 Graal Use Graal Instead

Slide 20

Slide 20 text

Graal with JRuby • Sometimes signiﬁcantly faster • Usually same or a little slower • Larger -> slower • Escape Analysis shines!

Slide 21

Slide 21 text

JRuby’s Math Problem long value = 12; 12 + 2 Internally RubyFixnums 12_obj:RubyFixnum “boxed value" 12_obj.value() + 2_obj.value() Unlike MRI which is a numeric primitive 12 + 2

Slide 22

Slide 22 text

Escape Analysis • Proves an object never leaves a scope in a particular lifetime • In fixnum case we know 2 and 12 do not escape it’s usage. • Compiler sees we only use the long field • Don’t allocate Ruby fixnum. just use field types directly 12_obj.value() + 2_obj.value() 12 + 2

Slide 23

Slide 23 text

Mandlebrot 0 0.45 0.9 1.35 1.8 Mandlebrot (s) 0.071 0.073 0.085 0.74 1.3 1.78 1.79 CRuby head CRuby head +MJIT JRuby JRuby +indy JRuby +indy +Graal JRuby +indy +Graal -fixnum Ruby Truffle Ruby Default JRuby JRuby Invokedynamic++ JRuby +Graal JRuby +Graal disable fixnum cache Truffle Ruby

Slide 24

Slide 24 text

Graal? • Keeps getting better • May replace C2? • Is an option today for JRuby users -J-XX+UnlockExperimentalVMOptions -J-XX:+EnableJVMCI -J-XX:+JVMCICompiler

Slide 25

Slide 25 text

Splitting up Call Frames

Slide 26

Slide 26 text

Call Frames {lastline($_), backref($~), visibility, block, self, name, class} We maintain extra info about each ruby method call: Special methods need to access to these: block_given? Saving these values has a cost

Slide 27

Slide 27 text

Partial Frames If we only need $~, then only save $~ def foo(a) a[0] end call 1_000_000 times 0i/s 30i/s 60i/s 90i/s 120i/s 1M calls 116M calls/s 68M calls/s 17M calls/s 9.1 9.2 9.2+graal

Slide 28

Slide 28 text

keyword arguments improvement

Slide 29

Slide 29 text

Reduce allocation size • Partial solution • Positional arguments opt coming • Storing in a real hash • Single bucket hashes used • noticed not using internal set methods

Slide 30

Slide 30 text

def foo(a: 1, b: 2) b + a end 1_000_000.times do foo(a: 2, b: 3) end 0.00k i/s 1.08k i/s 2.15k i/s 3.23k i/s 4.30k i/s kwargs call 4.3 3.82 9.1.17.0 9.2.0.0

Slide 31

Slide 31 text

JRuby on Rails

Slide 32

Slide 32 text

JRuby on Rails • Working since 2006 • Used by many • Support lagged during JRuby 9000, Ruby 2.3-2.4 work • Rails 5.0+ • ActiveRecord suffered the most

Slide 33

Slide 33 text

ActiveRecord JDBC “Our Journey?”

Slide 34

Slide 34 text

Compatibility is Hard

Slide 35

Slide 35 text

Compatibility AR to Java DataBase Connectivity ARJDBC Activerecord Rails JDBC JRuby JDBC Java Ruby DBs(SQLite, MySQL, PostgreSQL, msql, …)

Slide 36

Slide 36 text

Compatibility ActiveRecordJDBC 1.3.x JRuby 1.6.x, 1.7.x, 9.0.x, 9.1.x Java 6,7,8 ActiveRecord 4.2, … JDBC (many) databases Multiple versions Multiple distinct features (generic, h2, heal, mssql, MySQL, PostgreSQL, SQLite, MariaDB, DB2, Firebird, Oracle, ???)

Slide 37

Slide 37 text

ARJDB Compatibility • Too much to support / Not enough resources • Too much required domain knowledge • “Can I remove this line now?” • Repeats JRuby 1.7.x mistake • multi-mode compatibility is a mistake

Slide 38

Slide 38 text

ARJDBC Support Strategy • SQLite, MySQL/MariaDB, PostgreSQL • end of life == rails end of life 50.x Rails 5.0.x 51.x 52.x Rails 5.1.x Rails 5.2.x … 50-stable 51-stable master

Slide 39

Slide 39 text

Looking Promising… • Use case: MySQL support • old adapter: 1602 lines of Ruby • 50.0 adapter: 284 lines (still some old stuff which needs pruning) • Java code is smaller • 51.0 adapter: 6 lines of change!!!

Slide 40

Slide 40 text

Rails 5.1.6 actioncable: 139 runs, 733 assertions, 10 failures, 2 errors actionpack: 3063 runs, 14947 assertions, 2 failures, 0 errors actionmailer: 204 runs, 456 assertions, 0 failures, 0 errors actionview: 1957 runs, 4303 assertions, 3 failures, 4 errors activejob: 137 runs, 302 assertions, 0 failures, 0 errors activemodel: 713 runs, 2017 assertions, 0 failures, 0 errors activerecord: 4991 runs, 13902 assertions, 0 failures, 0 errors activesupport: 3671 runs, 760486 assertions, 14 failures, 0 errors railties: 40 runs, 73 assertions, 0 failures, 1 errors 30F, 6E & 797,219 assertions 99.995% passing

Slide 41

Slide 41 text

Mostly time/date error? Failure: TimeWithZoneTest#test_minus_with_time_precision [/home/ enebo/work/rails/activesupport/test/core_ext/ time_with_zone_test.rb:340]: Expected: 86399.999999998 Actual: 86399.99999999799

Slide 42

Slide 42 text

Rails 5.2.0 actioncable: something broken bootstrapping actionpack: 3148 runs, 15832 assertions, 1 failures, 0 errors actionmailer: 204 runs, 457 assertions, 0 failures, 0 errors actionview: 1990 runs, 4395 assertions, 4 failures, 4 errors activejob: 173 runs, 401 assertions, 0 failures, 0 errors activemodel: 803 runs, 2231 assertions, 0 failures, 0 errors activerecord: 5226 runs, 14665 assertions, 8 failures, 6 errors activesupport: 4135 runs, 762864 assertions, 17 failures, 2 errors railties: uses fork()

Slide 43

Slide 43 text

Rails 5.2 • Simple Scaffolded App works • SQLite • MySQL/PostgreSQL update needed

Slide 44

Slide 44 text

5.1 arjdbc bench • used internally for point to point comparisons • “mild” idea of performance

Slide 45

Slide 45 text

.create x5000 Seconds 0 9.75 19.5 29.25 39 .create(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1

Slide 46

Slide 46 text

.ﬁnd x5000 Seconds 0 0.325 0.65 0.975 1.3 .ﬁnd(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1

Slide 47

Slide 47 text

Transcoding!!! JRuby DB JDBC Ruby UTF-8 UTF-16LE UTF-8 MRI DB Ruby UTF-8 UTF-8 2 transcodes 0 transcodes

Slide 48

Slide 48 text

.select x5000 Seconds 0 0.45 0.9 1.35 1.8 .select(id) JRuby JRuby+indy JRuby+graal MRI 2.5.1

Slide 49

Slide 49 text

.ﬁnd_all x5000 Seconds 0 0.8 1.6 2.4 3.2 .ﬁrst(10) JRuby JRuby+indy JRuby+graal MRI 2.5.1

Slide 50

Slide 50 text

.update x5000 Seconds 0 0.5 1 1.5 2 .update(string) JRuby JRuby+indy JRuby+graal MRI 2.5.1

Slide 51

Slide 51 text

Scaffolded App Bench • Simple Scaffolded App • Rails 5.1.6, Postgresql • EC2 c4.xLarge: 4 vCPUs, 7.5G • Bench, DB, App on same instance

Slide 52

Slide 52 text

Requests per second 0 325 650 975 1300 10k 20k 30k 40k 50k 60k 70k 80k 90k 100k CRuby 2.5 JRuby JRuby +Indy 100k requests. Client Concurrency 10

Slide 53

Slide 53 text

Future • Try to push more Ruby from activerecord-jdbc -> active record • Be more engaged in Rails core development (!fork (›°□°)›ớᵲᴸᵲ )

Slide 54

Slide 54 text

Discourse? • Does not currently work with JRuby • “Standard” for comparing performance • Noah Gibbs (@appfolio) has harness • Good testbed for compatibility

Slide 55

Slide 55 text

Discourse Issues • NATIVE C EXTENSIONS!!!! • cppjieba -> jieba-analysis (JI bindings) • libv8/mini_racer -> therubyrhino (Gemfile) • nokogumbo -> ffi bindings • oj -> java native extension 70% done • pg -> jruby-pg (Gemfile) • rinku -> autolink-java (JI bindings)

Slide 56

Slide 56 text

Dealing with Cexts • Work with gem authors • Identify solution • fﬁ • Java integration • pure Ruby • java native extension • Do our own gem (least appealing)

Slide 57

Slide 57 text

Dealing with Cexts • Working with gem authors • Making our own gem gem ‘oj’ gem ‘therubyrhino’, platform: :jruby gem ‘miniracer’, platform: :ruby

Slide 58

Slide 58 text

Thank You • @tom_enebo • https://www.jruby.org • https://github.com/jruby/activerecord-jdbc • https://github.com/jruby/jruby • http://www.redhat.com