Tweet
Tweet

Slide 1

Slide 1 text

Automated
 Type Contracts Generation — Valentin Fondaratov Hiroshima, September 2017 A tale about better code analysis @valich @fondarat

Slide 2

Slide 2 text

Why Types Matter An IDE Perspective

Slide 3

Slide 3 text

An IDE Perspective — • Where does the method go? (aka Resolution)

Slide 4

Slide 4 text

An IDE Perspective — • Where does the method go? (aka Resolution) • Bug prediction (aka NameError)

Slide 5

Slide 5 text

An IDE Perspective — • Where does the method go? (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed)

Slide 6

Slide 6 text

An IDE Perspective — • Where does the method go? (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed) • Rename refactoring (aka Safety),

Slide 7

Slide 7 text

An IDE Perspective — • Where does the method go? (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed) • Rename refactoring (aka Safety),

Slide 8

Slide 8 text

Why Types Matter A non-IDE Perspective

Slide 9

Slide 9 text

RuboCop
 is an industry standard solution — © http://batsov.com/articles/2014/09/05/rubocop-logo/ Inspecting 2184 files ....................................................................................................................... ..............................................................................................................W........ ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................W.................................................................................. ....................................................................................................................... ........................ Offenses: actionpack/lib/action_dispatch/system_test_case.rb:109:7: C: Use 2 (not 11) spaces for indentation. SystemTesting::Browser.new(using, screen_size) ^^^^^^^^^^^ actionpack/lib/action_dispatch/system_test_case.rb:112:16: W: end at 112, 15 is not aligned with driver = if at 108, 6. end ^^^ guides/rails_guides/markdown/renderer.rb:104:18: W: end at 104, 17 is not aligned with path = case at 97, 10. end ^^^ guides/rails_guides/markdown/renderer.rb:106:1: C: Extra blank line detected. 2184 files inspected, 4 offenses detected

Slide 10

Slide 10 text

Missed errors — RuboCop does its job quite well suggesting following Ruby Code Style.
 
 The real error on line 5 is missed, though.
 (downcase is not a method of Hash) Inspecting 1 file C Offenses: rubocop_fails.rb:1:1: C: Missing frozen string literal comment. x = "123" ^ rubocop_fails.rb:1:5: C: Prefer single-quoted strings when you don't need string interpolation or special symbols. x = "123" ^^^^^ rubocop_fails.rb:4:5: C: Space inside { missing. x = {:a => '1', :b => '2', :c => '3'} ^ rubocop_fails.rb:4:6: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:17: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:28: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:37: C: Space inside } missing. x = {:a => '1', :b => '2', :c => '3'} ^ 1 file inspected, 7 offenses detected

Slide 11

Slide 11 text

Static analysis can detect more — RubyMine, for example, can detect such errors. Notice that this unresolved warning 
 is not screaming red. We’ll see why.

Slide 12

Slide 12 text

Ruby DSLs
 are hard
 to analyse — Ruby core features, readability and elegance, have a price. Inability
 to properly verify the programs 
 is one of the compromises.
 What type does @photo variable have?

Slide 13

Slide 13 text

“Beware of bugs in the above code;
 I have only proved it correct, not tried it” Donald E. Knuth

Slide 14

Slide 14 text

“Beware of bugs in the above code;
 I have only proved it correct, not tried it” Donald E. Knuth “Program testing can be used 
 to show the presence of bugs, 
 but never to show their absence!” Edsger W. Dijkstra

Slide 15

Slide 15 text

Coverage is a lie — Any composition of 2+ branching methods requires cross product of branches tests. Without static analysis and proper inspections we are limited by • Running tests (uncaught bugs we call “regressions”), • Looking through the code with debugger and verifying manually (after each change??), • Testing with users in production :)

Slide 16

Slide 16 text

There is so much more
 we can have by running
 the tests Than just checking the answers

Slide 17

Slide 17 text

Analysing RSpec::Matchers —

Slide 18

Slide 18 text

Analysing RSpec::Matchers —

Slide 19

Slide 19 text

Analysing RSpec::Matchers —

Slide 20

Slide 20 text

//demo/gif —

Slide 21

Slide 21 text

//demo/gif —

Slide 22

Slide 22 text

How it works — The process behind the magic:

Slide 23

Slide 23 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls,

Slide 24

Slide 24 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts,

Slide 25

Slide 25 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.

Slide 26

Slide 26 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.

Slide 27

Slide 27 text

Retrieving the data with TracePoint API — TracePoint is a class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding

Slide 28

Slide 28 text

Retrieving the data with TracePoint API — TracePoint is a class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding

Slide 29

Slide 29 text

Retrieving the data with TracePoint API — TracePoint is a class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding

Slide 30

Slide 30 text

Retrieving the data with TracePoint API — TracePoint is a class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding

Slide 31

Slide 31 text

Retrieving the data with TracePoint API — TracePoint is a class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding

Slide 32

Slide 32 text

Unspecified arguments — One can’t distinguish default parameter values from the passed ones.
 If a method is defined dynamically, there is no way to derive which types
 will be passed.
 
 What type does foo() return? (Int) -> Int (String) -> String foo() = ?

Slide 33

Slide 33 text

Optional parameters — YARV compiles code into the bytecode. Note that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.

Slide 34

Slide 34 text

Optional parameters — YARV compiles code into the bytecode. Note that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.

Slide 35

Slide 35 text

Optional parameters — YARV compiles code into the bytecode. Note that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.

Slide 36

Slide 36 text

Optional parameters — YARV compiles code into the bytecode. Note that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.

Slide 37

Slide 37 text

Optional parameters — rb_control_frame_t const VALUE *pc; const rb_iseq_t *iseq; PC

Slide 38

Slide 38 text

Optional parameters — rb_control_frame_t const VALUE *pc; const rb_iseq_t *iseq; PC

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Resources — http://patshaughnessy.net/ruby-under-a-microscope
 
 https://silverhammermba.github.io/emberb/c/
 
 https://github.com/ruby/ruby

Slide 42

Slide 42 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.

Slide 43

Slide 43 text

Type Tuples — str.split(pattern=nil, [limit]) -> anArray

Slide 44

Slide 44 text

Type Tuples — str.split(pattern=nil, [limit]) -> anArray

Slide 45

Slide 45 text

Type Tuples — str.split(pattern=nil, [limit]) -> anArray Method calls Type tuples

Slide 46

Slide 46 text

Too much data — str.split(pattern=nil, [limit]) -> anArray

Slide 47

Slide 47 text

Too much data — str.split(pattern=nil, [limit]) -> anArray

Slide 48

Slide 48 text

Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 49

Slide 49 text

Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 50

Slide 50 text

Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 51

Slide 51 text

Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 52

Slide 52 text

Too much data — str.split(pattern=nil, [limit]) -> anArray split(, nil) ? ! https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 53

Slide 53 text

Too much data — str.split(pattern=nil, [limit]) -> anArray ? ? split(, ) https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

Slide 54

Slide 54 text

Too much data —

Slide 55

Slide 55 text

Too much data —

Slide 56

Slide 56 text

Too much data —

Slide 57

Slide 57 text

A worse example —

Slide 58

Slide 58 text

Template automatons —

Slide 59

Slide 59 text

Equality masks —

Slide 60

Slide 60 text

Equality masks — Param0 Param1 Param2 Equals to Param0 Equals to Param1 {} {1} {11}

Slide 61

Slide 61 text

Equality masks —

Slide 62

Slide 62 text

Merge —

Slide 63

Slide 63 text

Merge —

Slide 64

Slide 64 text

Merge —

Slide 65

Slide 65 text

Merge — + Merge

Slide 66

Slide 66 text

Merge — + Merge Quack inference?

Slide 67

Slide 67 text

//demo/ —

Slide 68

Slide 68 text

//demo/ —

Slide 69

Slide 69 text

How it works — The process behind the magic: 1. Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Q. How do I collect the data?

Slide 72

Slide 72 text

Q. How do I collect the data? A. Run tests.

Slide 73

Slide 73 text

Q. How do I collect the data? A. Run tests.

Slide 74

Slide 74 text

Q. How do I collect the data? A. Run tests. Q. How do I collect more data?

Slide 75

Slide 75 text

Q. How do I collect the data? A. Run tests. Q. How do I collect more data? A. Run more tests.

Slide 76

Slide 76 text

Q. How do I collect the data? A. Run tests. Q. How do I collect more data? A. Run more tests.

Slide 77

Slide 77 text

Q. How do I collect the data? A. Run tests. Q. How do I collect more data? A. Run more tests. Q. How do I collect so much data that the type contracts obtain exhaustiveness, i.e. become true?

Slide 78

Slide 78 text

Q. How do I collect the data? A. Run tests. Q. How do I collect more data? A. Run more tests. Q. How do I collect so much data that the type contracts obtain exhaustiveness, i.e. become true? A. Cooperate with others to run even MORE TESTS.

Slide 79

Slide 79 text

Community effort — Project1 Project2 … ProjectN

Slide 80

Slide 80 text

Community effort — Project1 Project2 … ProjectN

Slide 81

Slide 81 text

Community effort — Project1 Project2 … ProjectN Contracts1 Contracts2 ContractsN Spec Spec Spec

Slide 82

Slide 82 text

Community effort — Project1 Project2 … ProjectN Contracts1 Contracts2 ContractsN Spec Spec Spec “Devise Annotated”, ‘4.2’ M E R G E

Slide 83

Slide 83 text

Community effort — Contract
 Diffs Rails 5 4.2 4.1 capybara
 +selenium 2.12.0 2.11.0 “Ruby weekly”.tar.gz Rubyists Cloud Storage

Slide 84

Slide 84 text

Community effort — Contract
 Diffs Rails 5 4.2 4.1 capybara
 +selenium 2.12.0 2.11.0 “Ruby weekly”.tar.gz Rubyists Cloud Storage

Slide 85

Slide 85 text

A single team may not have
 100% test coverage. A community is likely to have.

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

Tooling —

Slide 88

Slide 88 text

Tooling — • IDE goodness

Slide 89

Slide 89 text

Tooling — • IDE goodness • Code verification

Slide 90

Slide 90 text

Tooling — • IDE goodness • Code verification • Guided Optimization

Slide 91

Slide 91 text

Contribute — This might be a viable alternative to explicit
 type annotations which is contradictory.
 
 We have a chance to make Ruby 
 much more “static” for analysis
 while preserving its
 power and beauty.


Slide 92

Slide 92 text

jetbrains.com Thank you for your attention — jetbrains.com/ruby/ github.com/valich github.com/jetbrains/ruby-type-inference