Slide 1
Slide 1 text
The challenges behind
Ruby type checking
Soutaro Matsumoto
@soutaro
Slide 2
Slide 2 text
Steep changelog
• 10 releases since last RubyKaigi. (0.10.0, Mar 3, 2019)
• Many bugfixes and type system improvements.
• User experience improvements: $ steep watch
• Working for editor integration using LSP.
• Done: On the fly type checking, expression type hover.
• WIP: Jump to definition/declaration, completion.
$ gem install steep
Slide 3
Slide 3 text
Outline
• Ruby type checking difficulties review.
• Introduction to the Ruby signature language.
Slide 4
Slide 4 text
Duck typing
Slide 5
Slide 5 text
Duck typing
rubber duck
Slide 6
Slide 6 text
def write(io, message)
io << message
end
Slide 7
Slide 7 text
def write(io, message)
io << message
end
write(StringIO.new, "Hello World")
write(STDOUT, "Hello World")
Slide 8
Slide 8 text
def write(io, message)
io << message
end
write(StringIO.new, "Hello World")
write(STDOUT, "Hello World")
write([], "Hello World")
write(1, 8)
Slide 9
Slide 9 text
Duck typing
• An untyped programming.
• Assumes an interface (a set of methods) implicitly.
• An object conforms to an interface if the object has all of the methods
listed in the interface.
Slide 10
Slide 10 text
Duck Scale
Interface Implementation Languages
Declared explicitly Declare in class definition Java
Declared explicitly Declare after class definition Objective C, Swift, Haskell, Sorbet, ...
Declared explicitly Conforms implicitly TypeScript, Steep
Declared implicitly Conforms implicitly OCaml, C++, JavaScript, Ruby
Less Duck
More Duck
Slide 11
Slide 11 text
Metaprogramming
It means that a program can be designed to read,
generate, analyze or transform other programs, and
even modify itself while running.
Metaprogramming, Wikipedia
Slide 12
Slide 12 text
Metaprogramming
eval(gets)
Slide 13
Slide 13 text
Metaprogramming
eval(gets)
define_method :foo do ... end
Slide 14
Slide 14 text
Metaprogramming
class User < ApplicationRecord
has_many :articles
end
eval(gets)
define_method :foo do ... end
Slide 15
Slide 15 text
Metaprogramming
class User < ApplicationRecord
has_many :articles
end
include Enumerable
eval(gets)
define_method :foo do ... end
Slide 16
Slide 16 text
Metaprogramming
class User < ApplicationRecord
has_many :articles
end
include Enumerable
private
eval(gets)
define_method :foo do ... end
Slide 17
Slide 17 text
Metaprogramming
require "pathname"
load "./entrypoint.rb"
class User < ApplicationRecord
has_many :articles
end
include Enumerable
private
eval(gets)
define_method :foo do ... end
Slide 18
Slide 18 text
Metaprogramming
require "pathname"
load "./entrypoint.rb"
StrongJSON.new do
let :user, object(name: string,
email: string?)
end
class User < ApplicationRecord
has_many :articles
end
include Enumerable
private
eval(gets)
define_method :foo do ... end
Slide 19
Slide 19 text
• The metaprogramming primitives in Ruby are methods, which is subject to
re-definition while execution.
• What can we do?
• Let users write annotations.
• Simulate metaprogramming primitives in type checkers.
• Using runtime information.
• Typing rules plug-in?
Slide 20
Slide 20 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
Slide 21
Slide 21 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
a = [1, "2", :3, 4.0]
x, y*, z = a
Slide 22
Slide 22 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
a = [1, "2", :3, 4.0]
x, y*, z = a
spawn("rm", "-rf", "/")
spawn("rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf .", chdir: "/")
Slide 23
Slide 23 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
a = [1, "2", :3, 4.0]
x, y*, z = a
1.instance_eval do
self + 3
end
spawn("rm", "-rf", "/")
spawn("rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf .", chdir: "/")
Slide 24
Slide 24 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
a = [1, "2", :3, 4.0]
x, y*, z = a
1.instance_eval do
self + 3
end
1.tap do
break "foo"
end
spawn("rm", "-rf", "/")
spawn("rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf .", chdir: "/")
Slide 25
Slide 25 text
case x
when String
puts "x is a String"
when Integer
puts "x is a Integer"
end
a = [1, "2", :3, 4.0]
x, y*, z = a
1.instance_eval do
self + 3
end
1.tap do
break "foo"
end
[a,b].sort
[1,2,3].map!(&:to_s)
spawn("rm", "-rf", "/")
spawn("rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf /")
spawn({"ENV"=>"/bin"}, "rm -rf .", chdir: "/")
Slide 26
Slide 26 text
• We extend the type system.
• Union types.
• Tuple types, record types.
• Method overloading.
• Adding self type on block types.
• Adding break type on block types?
• Conditional types on method types??
Slide 27
Slide 27 text
No Ruby library has type
definition.
Slide 28
Slide 28 text
Overview
• Define the standard language to describe signature of Ruby program.
• Ruby will ship with stdlib signatures.
• Gems will ship with their signatures.
• Type checkers will use the signature to know the type of libraries.
Slide 29
Slide 29 text
Goals
1. Define the syntax of signature language.
2. Define the semantics of the signature language.
3. Implement a library to manipulate Ruby signatures.
4. Provide the signature of Ruby standard library.
5. Encourage another type checker development.
Slide 30
Slide 30 text
https://github.com/soutaro/ruby-signature
Slide 31
Slide 31 text
$ rbi list
$ rbi methods --singleton ::Object
$ rbi -r pathname method ::Object Pathname
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
class Array[A]
def map: [X] { (A) -> X } -> Array[X]
| -> Enumerable[A, self]
...
end
Slide 34
Slide 34 text
Signature language
• Defines structure of Ruby programs statically.
• No program execution, no metaprogramming.
• Expressive enough to describe type of Ruby methods.
• Easy to read/write and intuitive.
Slide 35
Slide 35 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 36
Slide 36 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 37
Slide 37 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 38
Slide 38 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 39
Slide 39 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 40
Slide 40 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 41
Slide 41 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 42
Slide 42 text
class Set[A]
def initialize: (_Each[A] objects) -> void
def add: (A) -> self
def add?: (A) -> self?
include Enumerable[A, void]
def each: { (A) -> void } -> Set[A]
def self.`[]`: [X] (*X) -> Set[X]
...
end
interface _Each[A]
def each: { (A) -> void } -> any
end
Slide 43
Slide 43 text
Ruby 㱻 Signature
class ClassInstanceType
include Application
def sub(s)
self.class.new(
name: name,
args: args.map {|arg| arg.sub(s) }
)
end
end
class ClassInstanceType
include Application
def sub: (_Substitution) -> self
end
interface _Substitution
def sub: (type) -> type
end
Slide 44
Slide 44 text
Mixin?
• include/prepend/extend in rbi are syntax, not method calls.
• No confusion by re-definition.
• Almost compatible semantics with Ruby (Module).
• We have attr_reader/attr_writer/attr_accessor, and they are
syntax too.
• And private/public
Slide 45
Slide 45 text
Types
Integer
singleton(Integer)
Class instance/singleton
Slide 46
Slide 46 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Slide 47
Slide 47 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
Slide 48
Slide 48 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
Slide 49
Slide 49 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
nil
bool
void
Base types
Slide 50
Slide 50 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
nil
bool
void
Base types
Integer?
Optional type
Slide 51
Slide 51 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
nil
bool
void
Base types
Integer?
Optional type
1
:hello
"world"
Singleton types
Slide 52
Slide 52 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
nil
bool
void
Base types
Integer?
Optional type
1
:hello
"world"
Singleton types
[Integer, String]
{ name: String, email: String? }
Tuple and record types
Slide 53
Slide 53 text
Types
Integer
singleton(Integer)
Class instance/singleton
Array[Integer]
Hash[Symbol, String]
Generic class type
Integer | String
Union types
any
Dynamic type
nil
bool
void
Base types
Integer?
Optional type
1
:hello
"world"
Singleton types
[Integer, String]
{ name: String, email: String? }
Tuple and record types
^(Integer, String) -> void
Proc type
Slide 54
Slide 54 text
Method types
(Integer, ?bool, *String, any, name: String, ?email: String?, **String) -> S
(String name, ?email: String? email) -> void
Slide 55
Slide 55 text
Method types
(Integer, ?bool, *String, any, name: String, ?email: String?, **String) -> S
(String name, ?email: String? email) -> void
(Integer) { (Integer) -> void } -> String
(Integer) ?{ (Integer) -> void } -> String
Slide 56
Slide 56 text
Method types
(Integer, ?bool, *String, any, name: String, ?email: String?, **String) -> S
(String name, ?email: String? email) -> void
(Integer) { (Integer) -> void } -> String
(Integer) ?{ (Integer) -> void } -> String
[X] { (A) -> X } -> Array[X]
| -> Enumerable[A, self]
Slide 57
Slide 57 text
def self.open: (String | Integer path,
?String | Integer mode,
?Integer perm,
?external_encoding: Encoding | String,
?internal_encoding: Encoding | String,
?encoding: Encoding | String,
?mode: String | Integer,
?textmode: bool,
?binmode: bool,
?autoclose: bool) -> IO
| [X] (String | Integer path,
?String | Integer mode,
?Integer perm,
?external_encoding: Encoding | String,
?internal_encoding: Encoding | String,
?encoding: Encoding | String,
?mode: String | Integer,
?textmode: bool,
?binmode: bool,
?autoclose: bool) { (IO) -> X } -> X
Slide 58
Slide 58 text
Open class
• Add methods to existing classes/modules using extension construct.
• Keep class/module declarations closed in signatures.
• Does this define new class, or modify an existing class?
extension Kernel (Pathname)
def Pathname: (String) -> Pathname
end
Slide 59
Slide 59 text
Local things
• You can declare the instance variables
and private methods in signature.
• Ruby allows accessing them through
mixin/inheritance.
• If it is a part of external API, write them.
class User
@name: String
@email: String?
@phone: String?
def send: (String) -> void
private
def send_email: (String)-> void
def send_sms: (String) -> void
end
Slide 60
Slide 60 text
Integration with type checkers
• There are at least four type checking tools with incompatible type
systems.
• Sorbet, RDL: Nominal subtyping
• Steep: Structural subtyping.
• type-profiler: No subtyping yet.
• Parametrized type checking semantics with axioms.
Slide 61
Slide 61 text
T <: void
T <: bool
S <: T | S
T <: T | S
[T, S] <: ::Array[T | S]
S <: T
Axiom of union types
Axiom of void and bool types
Axiom of tuple types?
Type checkers will define their own subtyping relations,
but should follow the axioms.
S <: S?
nil <: S?
Axiom of optional types
::String <: ::Object
Axiom of subclassing
Slide 62
Slide 62 text
T <: void
T <: bool
S <: T | S
T <: T | S
[T, S] <: ::Array[T | S]
S <: T
Axiom of union types
Axiom of void and bool types
Axiom of tuple types?
Type checkers will define their own subtyping relations,
but should follow the axioms.
S <: S?
nil <: S?
Axiom of optional types
::String <: ::Object
Axiom of subclassing
Slide 63
Slide 63 text
Missing features
• Mechanism to reuse parameter types.
• Mechanism to share type of existing gems.
Slide 64
Slide 64 text
Sharing gem types
• What can we do for existing gems without signatures? (if the authors don't
ship with signatures?)
• TypeScript (DefinitelyTyped)
• Community managed type definitions of npm packages.
source "https://ruby-signatures.org" do
gem "rails"
end
Slide 65
Slide 65 text
Sharing gem types
• What can we do for existing gems without signatures? (if the authors don't
ship with signatures?)
• TypeScript (DefinitelyTyped)
• Community managed type definitions of npm packages.
source "https://ruby-signatures.org" do
gem "rails"
end
A prefix to avoid name conflict.ʢୄʣ
Slide 66
Slide 66 text
Recap
• Type signatures of existing Ruby libraries are essential.
• We define the standard type signature for Ruby libraries.
• You can write types of your library.
• Type checking is optional, but the signature helps developers to
understand precisely how the API of your library is.
• Give us your feedbacks!