Types and Ruby Programming Language

RubyConf 2017, New Orleans

Transcript

1. Types and
 Ruby
 Programming
 Language Soutaro Matsumoto (@soutaro)
2. @soutaro • CTO at SideCI • Working for program analysis (type checking) since 2005 • For PhD thesis • For product improvement
3. Type Checking Benefits • Finds more bugs • Verifiable Documentation • Helps refactoring • Faster execution (out of scope)
4. Type Checking and Ruby • Humans have tried to type check Ruby programs for 10 years • Four type checkers 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
5. Difficulties • Type inference • Dynamically typed objects (duck typing) • Metaprogramming
6. Dynamically Typed Objects • The class of the receiver does not matter • Different from popular statically typed languages (Java, C#, ...) write(File.open(path, "w")) write(StringIO.new) write("foo") write([]) def write(io) io << "Hello World" true end
7. Static Type Inference for Ruby
 [Furr and Foster, 2009] • Based on structural subtyping • Annotation for polymorphic methods • Implementation is still available
 https:/ /github.com/stereobooster/diamondback-ruby (fork)
8. Structural Subtyping • Subtyping with structural subtyping relation identification def write(io) io << "Hello World" true end write: (< <<: (String) -> 'a >) -> TrueClass
9. write: (< <<: (String) -> 'a >) -> TrueClass
10. write: (< <<: (String) -> 'a >) -> TrueClass write("") String = < <<: (String) -> String, length: -> Integer, ... >
11. write: (< <<: (String) -> 'a >) -> TrueClass write("") String = < <<: (String) -> String, length: -> Integer, ... >
12. write: (< <<: (String) -> 'a >) -> TrueClass StringIO = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
13. write: (< <<: (String) -> 'a >) -> TrueClass StringIO = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
14. write(true) TrueClass = < &: (Object) -> Object, |: (Object) -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
15. write(true) TrueClass = < &: (Object) -> Object, |: (Object) -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
16. write(true) TrueClass = < &: (Object) -> Object, |: (Object) -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
17. write(true) TrueClass = < &: (Object) -> Object, |: (Object) -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
18. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
19. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
20. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" [ERROR] instance Fixnum does not support methods to_str in method call id at ./foo.rb:8 in creating instance of Fixnum at ./foo.rb:8 in typing expression 1 at ./foo.rb:8 in typing actual argument 1 at ./foo.rb:8 in method call + at ./foo.rb:8 DRuby analysis complete.
21. Type Inference for Ruby Programs based on Polymorphic Record Types [Matsumoto and Minamide, 2007] • Infers polymorphism based on ML type inference • Object typing by polymorphic record types • Cannot type some Ruby builtin • Array#map
22. Dynamically Typed Objects • Approximation with structural subtyping/polymorphic record types • Type inference does not work well • Structural subtyping cannot be combined with polymorphism • ML based type inference cannot type Ruby builtins • Do we really need type inference?
23. Local Type Inference • Infer types using only local information • Local variable types • Polymorphic method calls • Programmers should write method types • Easily integrated with existing languages (Scala, C++, Swift, ...)
24. def foo(array) size = array.size array.map {|x| x.to_s * size } end No type annotation (completely inferred or dynamically typed)
25. def foo(array) size = array.size array.map {|x| x.to_s * size } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference)
26. def foo(array) size = array.size array.map {|x| x.to_s * size } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) def foo(array: Array<Integer>) -> Array<String> size = array.size array.map {|x| x.to_s * size } end Local type inference
27. Answer • Structural subtyping + local type inference • Structural subtyping to support Ruby's semantics • Local type inference to minimize # of key strokes • (Similar to TypeScript)
28. Type Checking Difficulties • Type inference → Local type inference • Dynamically typed objects (duck typing) → Structural subtyping • Metaprogramming
29. Metaprogramming • Modify class/method definitions during execution • define_method, Class.new Person = Class.new do define_method :name do instance_variable_get :@name end end class Person def name @name end end
30. Metaprogramming • attr_reader, attr_accessor
31. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to, ...
32. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to, ... • require
33. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to, ... • require require 'pp' if rand(10) > 3
34. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to, ... • require • Open class
35. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to, ... • require • Open class if rand(10) > 3 class Object # ... end end
36. Metaprogramming • How can the type checker know the actual type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end
37. Metaprogramming • How can the type checker know the actual type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end class Object def self.attr_reader(x) attr_accessor x end end
38. Solutions • Query type structure at runtime • Let programmers give type structures
39. Just-in-Time Static Type Checking for
 Dynamic Languages [Ren and Foster, 2016] • Run static type checking during execution • The implementation is called Hammingbird • Built on top of RDL: https:/ /github.com/plum-umd/rdl
40. • Ruby • No error detected (happens) • With JIT type checking (Hammingbird) • Type error detected! • Static type check runs when execution reaches on the beginning of f type '(String) -> %any' def f(x, y) x.foo if y end f("", false)
41. type '(String) -> %any' def f(x, y) x.foo if y end f("", false) class String type '() -> %bool' def foo # ... end end • With JIT type checking (Hamming-bird) • Type check passes because foo method is now defined
42. Just-in-Time Static Type Checking for
 Dynamic Languages • Pros: detect more errors in unit test • Cons: type checking depends on runtime behavior
43. Solutions • Query type structure at runtime • Let programmers give type structures
44. Steep [Matsumoto, 2017] • Gradual typing for Ruby: https:/ /github.com/soutaro/steep • $ gem install steep --pre • Type definition by another limited language
45. Type Definition • Written by programmers in .rbi file • The type definition language has limited expressiveness to give precise type definitions statically • No open class, no require, no metaprogramming class Contact def initialize: (name: String) -> any def name: -> String end
46. class Contact def initialize: (name: String) -> any def name: -> String end
47. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
48. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
49. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
50. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
51. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
52. class Contact def initialize: (name: String) -> any def name: -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
53. Steep • Some Ruby programs cannot be typed statically • Who cares? class A def foo def foo "string" end 3 end end x = A.new.foo + 1 y = A.new.foo + "a"
54. Type Checking and Ruby • Focus on dynamically typed objects; tries to infer types 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby • Focus on metaprogramming support 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
55. Recap • Introduced four type checkers for Ruby • Two historical type checkers (2007-2009) • Two new type checkers (2016-) • Conclusion? • Not yet
56. • BC. Pierce, Local Type Inference, 1997 • S. Matsumoto and Y. Minamide. Type Inference for Ruby Programs based on Polymorphic Record Types, 2007 • M. Furr, J. hoon (David) An, J. S. Foster, and M. Hicks. Static Type Inference for Ruby, 2009 • B. M. Ren and J. S. Foster. Just-in-Time Static Type Checking for Dynamic Languages, 2016 • S. Matsumoto, Type Checking Ruby Programs with Annotations, 2017