Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Types and Ruby Programming Language

Types and Ruby Programming Language

RubyConf 2017, New Orleans

1fab9d01b25e99522f3dfd01e3d4cb51?s=128

Soutaro Matsumoto

November 16, 2017
Tweet

Transcript

  1. Types and
 Ruby
 Programming
 Language Soutaro Matsumoto (@soutaro)

  2. @soutaro • CTO at SideCI • Working for program analysis

    (type checking) since 2005 • For PhD thesis • For product improvement
  3. Type Checking Benefits • Finds more bugs • Verifiable Documentation

    • Helps refactoring • Faster execution (out of scope)
  4. Type Checking and Ruby • Humans have tried to type

    check Ruby programs for 10 years • Four type checkers 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  5. Difficulties • Type inference • Dynamically typed objects (duck typing)

    • Metaprogramming
  6. Dynamically Typed Objects • The class of the receiver does

    not matter • Different from popular statically typed languages (Java, C#, ...) write(File.open(path, "w")) write(StringIO.new) write("foo") write([]) def write(io) io << "Hello World" true end
  7. Static Type Inference for Ruby
 [Furr and Foster, 2009] •

    Based on structural subtyping • Annotation for polymorphic methods • Implementation is still available
 https:/ /github.com/stereobooster/diamondback-ruby (fork)
  8. Structural Subtyping • Subtyping with structural subtyping relation identification def

    write(io) io << "Hello World" true end write: (< <<: (String) -> 'a >) -> TrueClass
  9. write: (< <<: (String) -> 'a >) -> TrueClass

  10. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  11. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  12. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  13. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  14. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  15. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  16. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  17. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  18. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  19. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  20. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" [ERROR] instance Fixnum does not support methods to_str in method call id at ./foo.rb:8 in creating instance of Fixnum at ./foo.rb:8 in typing expression 1 at ./foo.rb:8 in typing actual argument 1 at ./foo.rb:8 in method call + at ./foo.rb:8 DRuby analysis complete.
  21. Type Inference for Ruby Programs based on Polymorphic Record Types

    [Matsumoto and Minamide, 2007] • Infers polymorphism based on ML type inference • Object typing by polymorphic record types • Cannot type some Ruby builtin • Array#map
  22. Dynamically Typed Objects • Approximation with structural subtyping/polymorphic record types

    • Type inference does not work well • Structural subtyping cannot be combined with polymorphism • ML based type inference cannot type Ruby builtins • Do we really need type inference?
  23. Local Type Inference • Infer types using only local information

    • Local variable types • Polymorphic method calls • Programmers should write method types • Easily integrated with existing languages (Scala, C++, Swift, ...)
  24. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed)
  25. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference)
  26. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) def foo(array: Array<Integer>) -> Array<String> size = array.size array.map {|x| x.to_s * size } end Local type inference
  27. Answer • Structural subtyping + local type inference • Structural

    subtyping to support Ruby's semantics • Local type inference to minimize # of key strokes • (Similar to TypeScript)
  28. Type Checking Difficulties • Type inference → Local type inference

    • Dynamically typed objects (duck typing) → Structural subtyping • Metaprogramming
  29. Metaprogramming • Modify class/method definitions during execution • define_method, Class.new

    Person = Class.new do define_method :name do instance_variable_get :@name end end class Person def name @name end end
  30. Metaprogramming • attr_reader, attr_accessor

  31. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ...
  32. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require
  33. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require require 'pp' if rand(10) > 3
  34. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require • Open class
  35. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require • Open class if rand(10) > 3 class Object # ... end end
  36. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end
  37. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end class Object def self.attr_reader(x) attr_accessor x end end
  38. Solutions • Query type structure at runtime • Let programmers

    give type structures
  39. Just-in-Time Static Type Checking for
 Dynamic Languages [Ren and Foster,

    2016] • Run static type checking during execution • The implementation is called Hammingbird • Built on top of RDL: https:/ /github.com/plum-umd/rdl
  40. • Ruby • No error detected (happens) • With JIT

    type checking (Hammingbird) • Type error detected! • Static type check runs when execution reaches on the beginning of f type '(String) -> %any' def f(x, y) x.foo if y end f("", false)
  41. type '(String) -> %any' def f(x, y) x.foo if y

    end f("", false) class String type '() -> %bool' def foo # ... end end • With JIT type checking (Hamming-bird) • Type check passes because foo method is now defined
  42. Just-in-Time Static Type Checking for
 Dynamic Languages • Pros: detect

    more errors in unit test • Cons: type checking depends on runtime behavior
  43. Solutions • Query type structure at runtime • Let programmers

    give type structures
  44. Steep [Matsumoto, 2017] • Gradual typing for Ruby: https:/ /github.com/soutaro/steep

    • $ gem install steep --pre • Type definition by another limited language
  45. Type Definition • Written by programmers in .rbi file •

    The type definition language has limited expressiveness to give precise type definitions statically • No open class, no require, no metaprogramming class Contact def initialize: (name: String) -> any def name: -> String end
  46. class Contact def initialize: (name: String) -> any def name:

    -> String end
  47. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  48. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  49. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  50. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  51. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  52. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  53. Steep • Some Ruby programs cannot be typed statically •

    Who cares? class A def foo def foo "string" end 3 end end x = A.new.foo + 1 y = A.new.foo + "a"
  54. Type Checking and Ruby • Focus on dynamically typed objects;

    tries to infer types 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby • Focus on metaprogramming support 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  55. Recap • Introduced four type checkers for Ruby • Two

    historical type checkers (2007-2009) • Two new type checkers (2016-) • Conclusion? • Not yet
  56. • BC. Pierce, Local Type Inference, 1997 • S. Matsumoto

    and Y. Minamide. Type Inference for Ruby Programs based on Polymorphic Record Types, 2007 • M. Furr, J. hoon (David) An, J. S. Foster, and M. Hicks. Static Type Inference for Ruby, 2009 • B. M. Ren and J. S. Foster. Just-in-Time Static Type Checking for Dynamic Languages, 2016 • S. Matsumoto, Type Checking Ruby Programs with Annotations, 2017