Share
Embed
Transcript
56 slides
-
@soutaro • CTO at SideCI • Working for program analysis
(type checking) since 2005 • For PhD thesis • For product improvement -
Type Checking Benefits • Finds more bugs • Verifiable Documentation
• Helps refactoring • Faster execution (out of scope) -
Type Checking and Ruby • Humans have tried to type
check Ruby programs for 10 years • Four type checkers 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations -
Difficulties • Type inference • Dynamically typed objects (duck typing)
• Metaprogramming -
Dynamically Typed Objects • The class of the receiver does
not matter • Different from popular statically typed languages (Java, C#, ...) write(File.open(path, "w")) write(StringIO.new) write("foo") write([]) def write(io) io << "Hello World" true end -
Static Type Inference for Ruby [Furr and Foster, 2009] •
Based on structural subtyping • Annotation for polymorphic methods • Implementation is still available https:/ /github.com/stereobooster/diamondback-ruby (fork) -
Structural Subtyping • Subtyping with structural subtyping relation identification def
write(io) io << "Hello World" true end write: (< <<: (String) -> 'a >) -> TrueClass -
write: (< <<: (String) -> 'a >) -> TrueClass write("")
String = < <<: (String) -> String, length: -> Integer, ... > -
write: (< <<: (String) -> 'a >) -> TrueClass write("")
String = < <<: (String) -> String, length: -> Integer, ... > -
write: (< <<: (String) -> 'a >) -> TrueClass StringIO
= < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... > -
write: (< <<: (String) -> 'a >) -> TrueClass StringIO
= < <<: (String) -> StringIO, gets: -> String, ... > write(StringIO.new) write("") String = < <<: (String) -> String, length: -> Integer, ... > -
write(true) TrueClass = < &: (Object) -> Object, |: (Object)
-> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass -
write(true) TrueClass = < &: (Object) -> Object, |: (Object)
-> Object, ... > write: (< <<: (String) -> 'a >) -> TrueClass -
write(true) TrueClass = < &: (Object) -> Object, |: (Object)
-> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass -
write(true) TrueClass = < &: (Object) -> Object, |: (Object)
-> Object, ... > write(43) Integer = < +: (Numeric) -> Numeric, <<: (Integer) -> Integer, ... > write: (< <<: (String) -> 'a >) -> TrueClass -
Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def
id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" -
Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def
id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" -
Limitation • DiamondbackRuby cannot infer polymorphic methods class ID def
id(x) x end end ID.new.id(3) + 3 ID.new.id("foo") + "bar" [ERROR] instance Fixnum does not support methods to_str in method call id at ./foo.rb:8 in creating instance of Fixnum at ./foo.rb:8 in typing expression 1 at ./foo.rb:8 in typing actual argument 1 at ./foo.rb:8 in method call + at ./foo.rb:8 DRuby analysis complete. -
Type Inference for Ruby Programs based on Polymorphic Record Types
[Matsumoto and Minamide, 2007] • Infers polymorphism based on ML type inference • Object typing by polymorphic record types • Cannot type some Ruby builtin • Array#map -
Dynamically Typed Objects • Approximation with structural subtyping/polymorphic record types
• Type inference does not work well • Structural subtyping cannot be combined with polymorphism • ML based type inference cannot type Ruby builtins • Do we really need type inference? -
Local Type Inference • Infer types using only local information
• Local variable types • Polymorphic method calls • Programmers should write method types • Easily integrated with existing languages (Scala, C++, Swift, ...) -
def foo(array) size = array.size array.map {|x| x.to_s * size
} end No type annotation (completely inferred or dynamically typed) -
def foo(array) size = array.size array.map {|x| x.to_s * size
} end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) -
def foo(array) size = array.size array.map {|x| x.to_s * size
} end No type annotation (completely inferred or dynamically typed) def foo(array: Array<Integer>) -> Array<String> size: Integer = array.size array.map<String> {|x: Integer| -> String x.to_s * size } end Fully annotated (without any inference) def foo(array: Array<Integer>) -> Array<String> size = array.size array.map {|x| x.to_s * size } end Local type inference -
Answer • Structural subtyping + local type inference • Structural
subtyping to support Ruby's semantics • Local type inference to minimize # of key strokes • (Similar to TypeScript) -
Type Checking Difficulties • Type inference → Local type inference
• Dynamically typed objects (duck typing) → Structural subtyping • Metaprogramming -
Metaprogramming • Modify class/method definitions during execution • define_method, Class.new
Person = Class.new do define_method :name do instance_variable_get :@name end end class Person def name @name end end -
Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,
... • require require 'pp' if rand(10) > 3 -
Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,
... • require • Open class -
Metaprogramming • attr_reader, attr_accessor • ActiveRecord: DB columns, has_many, belongs_to,
... • require • Open class if rand(10) > 3 class Object # ... end end -
Metaprogramming • How can the type checker know the actual
type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end -
Metaprogramming • How can the type checker know the actual
type structure? • We can give special support for attr_reader, but what if attr_reader is overridden? class Person attr_reader :name end class Object def self.attr_reader(x) attr_accessor x end end -
Solutions • Query type structure at runtime • Let programmers
give type structures -
Just-in-Time Static Type Checking for Dynamic Languages [Ren and Foster,
2016] • Run static type checking during execution • The implementation is called Hammingbird • Built on top of RDL: https:/ /github.com/plum-umd/rdl -
• Ruby • No error detected (happens) • With JIT
type checking (Hammingbird) • Type error detected! • Static type check runs when execution reaches on the beginning of f type '(String) -> %any' def f(x, y) x.foo if y end f("", false) -
type '(String) -> %any' def f(x, y) x.foo if y
end f("", false) class String type '() -> %bool' def foo # ... end end • With JIT type checking (Hamming-bird) • Type check passes because foo method is now defined -
Just-in-Time Static Type Checking for Dynamic Languages • Pros: detect
more errors in unit test • Cons: type checking depends on runtime behavior -
Solutions • Query type structure at runtime • Let programmers
give type structures -
Steep [Matsumoto, 2017] • Gradual typing for Ruby: https:/ /github.com/soutaro/steep
• $ gem install steep --pre • Type definition by another limited language -
Type Definition • Written by programmers in .rbi file •
The type definition language has limited expressiveness to give precise type definitions statically • No open class, no require, no metaprogramming class Contact def initialize: (name: String) -> any def name: -> String end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end -
class Contact def initialize: (name: String) -> any def name:
-> String end class Contact # @implements Contact def initialize(name:) @name = name end def name; @name; end end class Contact # @implements Contact # @dynamic name attr_reader :name def initialize(name:) @name = name end end class Contact # @implements Contact # @dynamic name def initialize(name:); @name = name; end def method_missing(f) f == :name ? @name : super end end -
Steep • Some Ruby programs cannot be typed statically •
Who cares? class A def foo def foo "string" end 3 end end x = A.new.foo + 1 y = A.new.foo + "a" -
Type Checking and Ruby • Focus on dynamically typed objects;
tries to infer types 1. 2007, Type Inference for Ruby Programs based on Polymorphic Record Types 2. 2009, Static Type Inference for Ruby • Focus on metaprogramming support 3. 2016, Just-in-Time Static Type Checking for Dynamic Languages 4. 2017, Type Checking Ruby Programs with Annotations -
Recap • Introduced four type checkers for Ruby • Two
historical type checkers (2007-2009) • Two new type checkers (2016-) • Conclusion? • Not yet -
• BC. Pierce, Local Type Inference, 1997 • S. Matsumoto
and Y. Minamide. Type Inference for Ruby Programs based on Polymorphic Record Types, 2007 • M. Furr, J. hoon (David) An, J. S. Foster, and M. Hicks. Static Type Inference for Ruby, 2009 • B. M. Ren and J. S. Foster. Just-in-Time Static Type Checking for Dynamic Languages, 2016 • S. Matsumoto, Type Checking Ruby Programs with Annotations, 2017