Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Types and Ruby Programming Language

Types and Ruby Programming Language

RubyConf 2017, New Orleans

Soutaro Matsumoto

November 16, 2017
Tweet

More Decks by Soutaro Matsumoto

Other Decks in Programming

Transcript

  1. Types and
 Ruby
 Programming
 Language Soutaro Matsumoto (@soutaro)

  2. @soutaro • CTO at SideCI • Working for program analysis

    (type checking) since 2005 • For PhD thesis • For product improvement
  3. Type Checking Benefits • Finds more bugs • Verifiable Documentation

    • Helps refactoring • Faster execution (out of scope)
  4. Type Checking and Ruby • Humans have tried to type

    check Ruby programs for 10 years • Four type checkers 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  5. Difficulties • Type inference • Dynamically typed objects (duck typing)

    • Metaprogramming
  6. Dynamically Typed Objects • The class of the receiver does

    not matter • Different from popular statically typed languages (Java, C#, ...) write(File.open(path, "w")) write(StringIO.new) write("foo") write([]) def write(io) io << "Hello World" true end
  7. Static Type Inference for Ruby
 [Furr and Foster, 2009] •

    Based on structural subtyping • Annotation for polymorphic methods • Implementation is still available
 https:/ /github.com/stereobooster/diamondback-ruby (fork)
  8. Structural Subtyping • Subtyping with structural subtyping relation identification def

    write(io) io << "Hello World" true end write: (< <<: (String) -> 'a >) -> TrueClass
  9. write: (< <<: (String) -> 'a >) -> TrueClass

  10. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  11. write: (< <<: (String) -> 'a >) -> TrueClass write("")

    String = < <<: (String) -> String, length: -> Integer, ... >
  12. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  13. write: (< <<: (String) -> 'a >) -> TrueClass StringIO

    = < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... >
  14. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  15. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  16. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  17. write(true) TrueClass = < &: (Object) -> Object, |: (Object)

    -> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass
  18. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  19. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar"
  20. Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def

    id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" [ERROR] instance Fixnum does not support methods to_str in method call id at ./foo.rb:8 in creating instance of Fixnum at ./foo.rb:8 in typing expression 1 at ./foo.rb:8 in typing actual argument 1 at ./foo.rb:8 in method call + at ./foo.rb:8 DRuby analysis complete.
  21. Type Inference for Ruby Programs based on Polymorphic Record Types

    [Matsumoto and Minamide, 2007] • Infers polymorphism based on ML type inference • Object typing by polymorphic record types • Cannot type some Ruby builtin • Array#map
  22. Dynamically Typed Objects • Approximation with structural subtyping/polymorphic record types

    • Type inference does not work well • Structural subtyping cannot be combined with polymorphism • ML based type inference cannot type Ruby builtins • Do we really need type inference?
  23. Local Type Inference • Infer types using only local information

    • Local variable types • Polymorphic method calls • Programmers should write method types • Easily integrated with existing languages (Scala, C++, Swift, ...)
  24. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed)
  25. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference)
  26. def foo(array) size = array.size array.map {|x| x.to_s * size

    } end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) def foo(array: Array<Integer>) -> Array<String> size = array.size array.map {|x| x.to_s * size } end Local type inference
  27. Answer • Structural subtyping + local type inference • Structural

    subtyping to support Ruby's semantics • Local type inference to minimize # of key strokes • (Similar to TypeScript)
  28. Type Checking Difficulties • Type inference → Local type inference

    • Dynamically typed objects (duck typing) → Structural subtyping • Metaprogramming
  29. Metaprogramming • Modify class/method definitions during execution • define_method, Class.new

    Person = Class.new do define_method :name do instance_variable_get :@name end end class Person def name @name end end
  30. Metaprogramming • attr_reader, attr_accessor

  31. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ...
  32. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require
  33. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require require 'pp' if rand(10) > 3
  34. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require • Open class
  35. Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,

    ... • require • Open class if rand(10) > 3 class Object # ... end end
  36. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end
  37. Metaprogramming • How can the type checker know the actual

    type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end class Object def self.attr_reader(x) attr_accessor x end end
  38. Solutions • Query type structure at runtime • Let programmers

    give type structures
  39. Just-in-Time Static Type Checking for
 Dynamic Languages [Ren and Foster,

    2016] • Run static type checking during execution • The implementation is called Hammingbird • Built on top of RDL: https:/ /github.com/plum-umd/rdl
  40. • Ruby • No error detected (happens) • With JIT

    type checking (Hammingbird) • Type error detected! • Static type check runs when execution reaches on the beginning of f type '(String) -> %any' def f(x, y) x.foo if y end f("", false)
  41. type '(String) -> %any' def f(x, y) x.foo if y

    end f("", false) class String type '() -> %bool' def foo # ... end end • With JIT type checking (Hamming-bird) • Type check passes because foo method is now defined
  42. Just-in-Time Static Type Checking for
 Dynamic Languages • Pros: detect

    more errors in unit test • Cons: type checking depends on runtime behavior
  43. Solutions • Query type structure at runtime • Let programmers

    give type structures
  44. Steep [Matsumoto, 2017] • Gradual typing for Ruby: https:/ /github.com/soutaro/steep

    • $ gem install steep --pre • Type definition by another limited language
  45. Type Definition • Written by programmers in .rbi file •

    The type definition language has limited expressiveness to give precise type definitions statically • No open class, no require, no metaprogramming class Contact def initialize: (name: String) -> any def name: -> String end
  46. class Contact def initialize: (name: String) -> any def name:

    -> String end
  47. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  48. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end
  49. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  50. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end
  51. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  52. class Contact def initialize: (name: String) -> any def name:

    -> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end
  53. Steep • Some Ruby programs cannot be typed statically •

    Who cares? class A def foo def foo "string" end 3 end end x = A.new.foo + 1 y = A.new.foo + "a"
  54. Type Checking and Ruby • Focus on dynamically typed objects;

    tries to infer types 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby • Focus on metaprogramming support 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations
  55. Recap • Introduced four type checkers for Ruby • Two

    historical type checkers (2007-2009) • Two new type checkers (2016-) • Conclusion? • Not yet
  56. • BC. Pierce, Local Type Inference, 1997 • S. Matsumoto

    and Y. Minamide. Type Inference for Ruby Programs based on Polymorphic Record Types, 2007 • M. Furr, J. hoon (David) An, J. S. Foster, and M. Hicks. Static Type Inference for Ruby, 2009 • B. M. Ren and J. S. Foster. Just-in-Time Static Type Checking for Dynamic Languages, 2016 • S. Matsumoto, Type Checking Ruby Programs with Annotations, 2017