Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Types and Ruby Programming Language

Types and Ruby Programming Language

RubyConf 2017, New Orleans

Soutaro Matsumoto

November 16, 2017
Tweet

More Decks by Soutaro Matsumoto

Other Decks in Programming

Transcript

  1. @soutaro • CTO at SideCI • Working for program analysis

    (type checking) since 2005 • For PhD thesis • For product improvement
  2. Type Checking Benefits • Finds more bugs • Verifiable Documentation

    • Helps refactoring • Faster execution (out of scope)
  3. Type Checking and Ruby • Humans have tried to type

    check Ruby programs for 10 years • Four type checkers 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  4. Dynamically Typed Objects • The class of the receiver does

    not matter • Different from popular statically typed languages (Java, C#, ...) write(File.open(path, "w")) write(StringIO.new) write("foo") write([]) def write(io) io << "Hello World" true end
  5. Static Type Inference for Ruby
 [Furr and Foster, 2009] •

    Based on structural subtyping • Annotation for polymorphic methods • Implementation is still available
 https:/ /github.com/stereobooster/diamondback-ruby (fork)
  6. Structural Subtyping • Subtyping with structural subtyping relation identification def

    write(io) io << "Hello World" true end write: (< <<: (String) -> 'a >) -> TrueClass
  7. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  8. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  9. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  10. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  11. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  12. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  13. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  14. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  15. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  16. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  17. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" [ERROR] instance Fixnum does not support methods to_str in method call id at ./foo.rb:8 in creating instance of Fixnum at ./foo.rb:8 in typing expression 1 at ./foo.rb:8 in typing actual argument 1 at ./foo.rb:8 in method call + at ./foo.rb:8 DRuby analysis complete.
  18. Type Inference for Ruby Programs based on Polymorphic Record Types

    [Matsumoto and Minamide, 2007] • Infers polymorphism based on ML type inference • Object typing by polymorphic record types • Cannot type some Ruby builtin • Array#map
  19. Dynamically Typed Objects • Approximation with structural subtyping/polymorphic record types

    • Type inference does not work well • Structural subtyping cannot be combined with polymorphism • ML based type inference cannot type Ruby builtins • Do we really need type inference?
  20. Local Type Inference • Infer types using only local information

    • Local variable types • Polymorphic method calls • Programmers should write method types • Easily integrated with existing languages (Scala, C++, Swift, ...)
  21. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed)
  22. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference)
  23. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) def foo(array: Array<Integer>) -> Array<String> size = array.size array.map {|x| x.to_s * size } end Local type inference
  24. Answer • Structural subtyping + local type inference • Structural

    subtyping to support Ruby's semantics • Local type inference to minimize # of key strokes • (Similar to TypeScript)
  25. Type Checking Difficulties • Type inference → Local type inference

    • Dynamically typed objects (duck typing) → Structural subtyping • Metaprogramming
  26. Metaprogramming • Modify class/method definitions during execution • define_method, Class.new

    Person = Class.new do define_method :name do instance_variable_get :@name end end class Person def name @name end end
  27. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require • Open class if rand(10) > 3 class Object # ... end end
  28. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end
  29. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end class Object def self.attr_reader(x) attr_accessor x end end
  30. Just-in-Time Static Type Checking for
 Dynamic Languages [Ren and Foster,

    2016] • Run static type checking during execution • The implementation is called Hammingbird • Built on top of RDL: https:/ /github.com/plum-umd/rdl
  31. • Ruby • No error detected (happens) • With JIT

    type checking (Hammingbird) • Type error detected! • Static type check runs when execution reaches on the beginning of f type '(String) -> %any' def f(x, y) x.foo if y end f("", false)
  32. type '(String) -> %any' def f(x, y) x.foo if y

    end f("", false) class String type '() -> %bool' def foo # ... end end • With JIT type checking (Hamming-bird) • Type check passes because foo method is now defined
  33. Just-in-Time Static Type Checking for
 Dynamic Languages • Pros: detect

    more errors in unit test • Cons: type checking depends on runtime behavior
  34. Steep [Matsumoto, 2017] • Gradual typing for Ruby: https:/ /github.com/soutaro/steep

    • $ gem install steep --pre • Type definition by another limited language
  35. Type Definition • Written by programmers in .rbi file •

    The type definition language has limited expressiveness to give precise type definitions statically • No open class, no require, no metaprogramming class Contact def initialize: (name: String) -> any def name: -> String end
  36. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  37. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  38. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  39. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  40. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  41. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  42. Steep • Some Ruby programs cannot be typed statically •

    Who cares? class A def foo def foo "string" end 3 end end x = A.new.foo + 1 y = A.new.foo + "a"
  43. Type Checking and Ruby • Focus on dynamically typed objects;

    tries to infer types 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby • Focus on metaprogramming support 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  44. Recap • Introduced four type checkers for Ruby • Two

    historical type checkers (2007-2009) • Two new type checkers (2016-) • Conclusion? • Not yet
  45. • BC. Pierce, Local Type Inference, 1997 • S. Matsumoto

    and Y. Minamide. Type Inference for Ruby Programs based on Polymorphic Record Types, 2007 • M. Furr, J. hoon (David) An, J. S. Foster, and M. Hicks. Static Type Inference for Ruby, 2009 • B. M. Ren and J. S. Foster. Just-in-Time Static Type Checking for Dynamic Languages, 2016 • S. Matsumoto, Type Checking Ruby Programs with Annotations, 2017