Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automated Type Contracts Generation

Automated Type Contracts Generation

Beauty and power of Ruby and Rails pays us back when it comes to finding bugs in large codebases. Static analysis is hindered by magic DSLs and patches.

We may annotate the code with YARD which also enables improved tooling such as code completion. Sadly, the benefits of this process rarely compensate for the effort.

In this session we’ll see a new approach to type annotations generation.
We'll learn how to obtain this data from runtime, to cope with DSLs and monkey patching, propose some tooling beyond YARD and create contracts like `(String, T) -> T`

YARV hacking and minimized DFAs included.

3cb281372ce7485bdc5f261b5589f91f?s=128

Valentin Fondaratov

September 19, 2017
Tweet

Transcript

  1. Automated
 Type Contracts Generation — Valentin Fondaratov Hiroshima, September 2017

    A tale about better code analysis @valich @fondarat
  2. Why Types Matter An IDE Perspective

  3. An IDE Perspective — • Where does the method go?

    (aka Resolution)
  4. An IDE Perspective — • Where does the method go?

    (aka Resolution) • Bug prediction (aka NameError)
  5. An IDE Perspective — • Where does the method go?

    (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed)
  6. An IDE Perspective — • Where does the method go?

    (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed) • Rename refactoring (aka Safety),
  7. An IDE Perspective — • Where does the method go?

    (aka Resolution) • Bug prediction (aka NameError) • IDE goodness (aka Speed) • Rename refactoring (aka Safety),
  8. Why Types Matter A non-IDE Perspective

  9. RuboCop
 is an industry standard solution — © http://batsov.com/articles/2014/09/05/rubocop-logo/ Inspecting

    2184 files ....................................................................................................................... ..............................................................................................................W........ ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................................................................................................... ....................................W.................................................................................. ....................................................................................................................... ........................ Offenses: actionpack/lib/action_dispatch/system_test_case.rb:109:7: C: Use 2 (not 11) spaces for indentation. SystemTesting::Browser.new(using, screen_size) ^^^^^^^^^^^ actionpack/lib/action_dispatch/system_test_case.rb:112:16: W: end at 112, 15 is not aligned with driver = if at 108, 6. end ^^^ guides/rails_guides/markdown/renderer.rb:104:18: W: end at 104, 17 is not aligned with path = case at 97, 10. end ^^^ guides/rails_guides/markdown/renderer.rb:106:1: C: Extra blank line detected. 2184 files inspected, 4 offenses detected
  10. Missed errors — RuboCop does its job quite well suggesting

    following Ruby Code Style.
 
 The real error on line 5 is missed, though.
 (downcase is not a method of Hash) Inspecting 1 file C Offenses: rubocop_fails.rb:1:1: C: Missing frozen string literal comment. x = "123" ^ rubocop_fails.rb:1:5: C: Prefer single-quoted strings when you don't need string interpolation or special symbols. x = "123" ^^^^^ rubocop_fails.rb:4:5: C: Space inside { missing. x = {:a => '1', :b => '2', :c => '3'} ^ rubocop_fails.rb:4:6: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:17: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:28: C: Use the new Ruby 1.9 hash syntax. x = {:a => '1', :b => '2', :c => '3'} ^^^^^ rubocop_fails.rb:4:37: C: Space inside } missing. x = {:a => '1', :b => '2', :c => '3'} ^ 1 file inspected, 7 offenses detected
  11. Static analysis can detect more — RubyMine, for example, can

    detect such errors. Notice that this unresolved warning 
 is not screaming red. We’ll see why.
  12. Ruby DSLs
 are hard
 to analyse — Ruby core features,

    readability and elegance, have a price. Inability
 to properly verify the programs 
 is one of the compromises.
 What type does @photo variable have?
  13. “Beware of bugs in the above code;
 I have only

    proved it correct, not tried it” Donald E. Knuth
  14. “Beware of bugs in the above code;
 I have only

    proved it correct, not tried it” Donald E. Knuth “Program testing can be used 
 to show the presence of bugs, 
 but never to show their absence!” Edsger W. Dijkstra
  15. Coverage is a lie — Any composition of 2+ branching

    methods requires cross product of branches tests. Without static analysis and proper inspections we are limited by • Running tests (uncaught bugs we call “regressions”), • Looking through the code with debugger and verifying manually (after each change??), • Testing with users in production :)
  16. There is so much more
 we can have by running


    the tests Than just checking the answers
  17. Analysing RSpec::Matchers —

  18. Analysing RSpec::Matchers —

  19. Analysing RSpec::Matchers —

  20. //demo/gif —

  21. //demo/gif —

  22. How it works — The process behind the magic:

  23. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls,
  24. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts,
  25. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.
  26. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.
  27. Retrieving the data with TracePoint API — TracePoint is a

    class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding
  28. Retrieving the data with TracePoint API — TracePoint is a

    class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding
  29. Retrieving the data with TracePoint API — TracePoint is a

    class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding
  30. Retrieving the data with TracePoint API — TracePoint is a

    class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding
  31. Retrieving the data with TracePoint API — TracePoint is a

    class allowing 
 to hook several Ruby VM events
 like method calls and returns 
 and get any data through Binding
  32. Unspecified arguments — One can’t distinguish default parameter values from

    the passed ones.
 If a method is defined dynamically, there is no way to derive which types
 will be passed.
 
 What type does foo() return? (Int) -> Int (String) -> String foo() = ?
  33. Optional parameters — YARV compiles code into the bytecode. Note

    that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.
  34. Optional parameters — YARV compiles code into the bytecode. Note

    that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.
  35. Optional parameters — YARV compiles code into the bytecode. Note

    that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.
  36. Optional parameters — YARV compiles code into the bytecode. Note

    that instructions for filling in 
 the default values are present, 
 independent on usages. When optional parameters are passed,
 VM just skips these instructions.
  37. Optional parameters — rb_control_frame_t const VALUE *pc; const rb_iseq_t *iseq;

    PC
  38. Optional parameters — rb_control_frame_t const VALUE *pc; const rb_iseq_t *iseq;

    PC
  39. None
  40. None
  41. Resources — http://patshaughnessy.net/ruby-under-a-microscope
 
 https://silverhammermba.github.io/emberb/c/
 
 https://github.com/ruby/ruby

  42. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.
  43. Type Tuples — str.split(pattern=nil, [limit]) -> anArray

  44. Type Tuples — str.split(pattern=nil, [limit]) -> anArray

  45. Type Tuples — str.split(pattern=nil, [limit]) -> anArray Method calls Type

    tuples
  46. Too much data — str.split(pattern=nil, [limit]) -> anArray

  47. Too much data — str.split(pattern=nil, [limit]) -> anArray

  48. Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

  49. Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

  50. Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

  51. Too much data — str.split(pattern=nil, [limit]) -> anArray https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg

  52. Too much data — str.split(pattern=nil, [limit]) -> anArray split(<String>, nil)

    ? ! https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg
  53. Too much data — str.split(pattern=nil, [limit]) -> anArray ? ?

    split(<String>, <String>) https://boardgamegeek.com/image/1955740/game-goose http://www.megahowto.com/wp-content/uploads/2010/11/how-to-make-board-games.jpg
  54. Too much data —

  55. Too much data —

  56. Too much data —

  57. A worse example —

  58. Template automatons —

  59. Equality masks —

  60. Equality masks — Param0 Param1 Param2 Equals to Param0 Equals

    to Param1 {} {1} {11}
  61. Equality masks —

  62. Merge —

  63. Merge —

  64. Merge —

  65. Merge — + Merge

  66. Merge — + Merge Quack inference?

  67. //demo/ —

  68. //demo/ —

  69. How it works — The process behind the magic: 1.

    Attach to Ruby VM to collect the types for all calls, 2. Transform raw call data into type contracts, 3. Collect and share the data.
  70. None
  71. Q. How do I collect the data?

  72. Q. How do I collect the data? A. Run tests.

  73. Q. How do I collect the data? A. Run tests.

  74. Q. How do I collect the data? A. Run tests.

    Q. How do I collect more data?
  75. Q. How do I collect the data? A. Run tests.

    Q. How do I collect more data? A. Run more tests.
  76. Q. How do I collect the data? A. Run tests.

    Q. How do I collect more data? A. Run more tests.
  77. Q. How do I collect the data? A. Run tests.

    Q. How do I collect more data? A. Run more tests. Q. How do I collect so much data that the type contracts obtain exhaustiveness, i.e. become true?
  78. Q. How do I collect the data? A. Run tests.

    Q. How do I collect more data? A. Run more tests. Q. How do I collect so much data that the type contracts obtain exhaustiveness, i.e. become true? A. Cooperate with others to run even MORE TESTS.
  79. Community effort — Project1 Project2 … ProjectN

  80. Community effort — Project1 Project2 … ProjectN

  81. Community effort — Project1 Project2 … ProjectN Contracts1 Contracts2 ContractsN

    Spec Spec Spec
  82. Community effort — Project1 Project2 … ProjectN Contracts1 Contracts2 ContractsN

    Spec Spec Spec “Devise Annotated”, ‘4.2’ M E R G E
  83. Community effort — Contract
 Diffs Rails 5 4.2 4.1 capybara


    +selenium 2.12.0 2.11.0 “Ruby weekly”.tar.gz Rubyists Cloud Storage
  84. Community effort — Contract
 Diffs Rails 5 4.2 4.1 capybara


    +selenium 2.12.0 2.11.0 “Ruby weekly”.tar.gz Rubyists Cloud Storage
  85. A single team may not have
 100% test coverage. A

    community is likely to have.
  86. None
  87. Tooling —

  88. Tooling — • IDE goodness

  89. Tooling — • IDE goodness • Code verification

  90. Tooling — • IDE goodness • Code verification • Guided

    Optimization
  91. Contribute — This might be a viable alternative to explicit


    type annotations which is contradictory.
 
 We have a chance to make Ruby 
 much more “static” for analysis
 while preserving its
 power and beauty.

  92. jetbrains.com Thank you for your attention — jetbrains.com/ruby/ github.com/valich github.com/jetbrains/ruby-type-inference