Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Regression test for RuboCop

7bc6612fa20296bf652f6b0357db81c1?s=47 pocke
December 14, 2019

Regression test for RuboCop

7bc6612fa20296bf652f6b0357db81c1?s=128

pocke

December 14, 2019
Tweet

Transcript

  1. Regression Test for RuboCop 平成 Ruby 会議 01 14th Dec.

    2019
  2. pocke.display • Masataka “Pocke” Kuwabara • Work for Bit Journey,

    Inc. We develop Kibela. • rurema team member • RuboCop core team member
  3. Kibela • Documentation SaaS ◦ Access https://kibe.la! • We use

    ◦ Ruby on Rails ◦ RuboCop ◦ GraphQL ◦ TypeScript ◦ React ◦ etc...
  4. RuboCop core team member • Many patches (recently a few

    patches) • The author of ◦ rubocop-rake ◦ rubocop-extention-generator ◦ mry ◦ gry ◦ rubocop-rubycw ◦ rubocop-typed ◦ rubocop-regression-test ← Today’s topic
  5. rurema team member • The Ruby Reference Manual project. ◦

    Work under https://github.com/rurema • I write documents and review pull requests.
  6. Rurema reviewing online meetup • We review rurema pull requests

    every Tuesday! ◦ Next meetup is 17th Dec. • Everyone can join the meetup! ◦ connpass: https://rurema-review.connpass.com/event/159182 ◦ slack: https://ruby-jp.github.io/ #rurema channel
  7. Agenda • Why does RuboCop need the regression test? •

    What’s the regression test? • How to work. • How to contribute to RuboCop. • Future work.
  8. Why does RuboCop need the regression test?

  9. RuboCop breaks easily, with Ruby syntax • Ruby has too

    complex syntax. • Actually we do not understand Ruby syntax perfectly. • A cop, implemented by us, will overlooks edge cases of Ruby syntax. • Example: omitting parens
  10. RuboCop breaks easily, with the configuration • RuboCop has many

    options. You can configure RuboCop for your style. • But it introduces complexity. • Not default option is not maintained enough.
  11. Conclusion of this section • RuboCop breaks easily ◦ with

    Ruby syntax ◦ with configurations • The regression test is necessary to find bugs before releasing RuboCop.
  12. What’s the regression test?

  13. Developing under here • rubocop-hq/rubocop#6805 • https://github.com/pocke/rubocop-regression-test

  14. What’s the regression test? • In short, it executes RuboCop

    ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  15. Example • In project-a ◦ rubocop --config .rubocop.1.yml ◦ rubocop

    --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-b ◦ rubocop --config .rubocop.1.yml ◦ rubocop --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-c ◦ ...
  16. What does check the test? • Check raised exceptions. ◦

    Grep “An error occurred while ...” message. ◦ Do not check false positives.
  17. How to work

  18. What’s the regression test? (page 14) • In short, it

    executes RuboCop ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  19. with all option combinations

  20. with all option combinations • Many cops has EnforcedStyle option

    to specify style. • The style candidates is available as SupportedStyles.
  21. Example of default configuration Style/StringLiterals: EnforcedStyle: single_quotes SupportedStyles: - single_quotes

    - double_quotes Layout/DotPosition: EnforcedStyle: leading SupportedStyles: - leading - trailing
  22. Example of all options combinations # .rubocop.1.yml Style/StringLiterals: EnforcedStyle: single_quotes

    Layout/DotPosition: EnforcedStyle: leading # .rubocop.2.yml Style/StringLiterals: EnforcedStyle: double_quotes Layout/DotPosition: EnforcedStyle: trailing
  23. in real Ruby projects

  24. It tests RuboCop in real Ruby projects • Unit test

    contains only a few code. ◦ It may overlook. • Real Ruby projects contain many codes. ◦ We can test cop with many patterns.
  25. It tests RuboCop in real Ruby projects • Now it

    tests with: ◦ tric/trick2013, 2015 and 2018 ◦ rubocop-hq/rubocop • In the future, I plan adding test with: ◦ ruby/spec (pocke/rubocop-regressiontest#16) ◦ ruby/ruby ◦ rails/rails ◦ gitlabhq/gitlabhq ◦ discourse/discourse
  26. How to contribute to RuboCop

  27. How to contribute to RuboCop • We have two contributing

    ways. ◦ Contribute to RuboCop ◦ Contribute to the regression test
  28. Contribute to RuboCop

  29. Contribute to RuboCop • Find bugs with the test on

    your codebase ↓ If you find bags, you can report them to RuboCop!
  30. Execute the following command $ git clone https://github.com/pocke/rubocop-regression-t est $

    cd rubocop-regression-test # Clone repo from GitHub and run the test $ ruby main.rb check OWNER/REPO # You can specify local directory also $ ruby main.rb check path/to/your/project/dir
  31. Contribute to rubocop-regression -test

  32. Contribute to RuboCop Regression Test • You can add more

    repositories to the test. ◦ Add them to CircleCI configuration. ◦ If it is green on CI, I can merge it. • Work for TODOs ◦ See the next section, “Future work”.
  33. Future work

  34. Test infinite loop in auto correction • Currently it ignores

    infinite loop in auto correction. ◦ Because it is too noisy. • But it also should detect and fix infinite loops.
  35. More test for --auto-correct • Auto correction should generate code

    that have same meaning if the cop is marked as “safe”. • In short, test `rubocop -a && rspec`.
  36. Test plugins • RuboCop has many plugins under the rubocop-hq

    organization. It should test them also. ◦ rubocop-rails ◦ rubocop-performance ◦ rubocop-rspec ◦ rubocop-rake ◦ and so on.