Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Regression test for RuboCop

pocke
December 14, 2019

Regression test for RuboCop

pocke

December 14, 2019
Tweet

More Decks by pocke

Other Decks in Programming

Transcript

  1. Regression Test for RuboCop 平成 Ruby 会議 01 14th Dec.

    2019
  2. pocke.display • Masataka “Pocke” Kuwabara • Work for Bit Journey,

    Inc. We develop Kibela. • rurema team member • RuboCop core team member
  3. Kibela • Documentation SaaS ◦ Access https://kibe.la! • We use

    ◦ Ruby on Rails ◦ RuboCop ◦ GraphQL ◦ TypeScript ◦ React ◦ etc...
  4. RuboCop core team member • Many patches (recently a few

    patches) • The author of ◦ rubocop-rake ◦ rubocop-extention-generator ◦ mry ◦ gry ◦ rubocop-rubycw ◦ rubocop-typed ◦ rubocop-regression-test ← Today’s topic
  5. rurema team member • The Ruby Reference Manual project. ◦

    Work under https://github.com/rurema • I write documents and review pull requests.
  6. Rurema reviewing online meetup • We review rurema pull requests

    every Tuesday! ◦ Next meetup is 17th Dec. • Everyone can join the meetup! ◦ connpass: https://rurema-review.connpass.com/event/159182 ◦ slack: https://ruby-jp.github.io/ #rurema channel
  7. Agenda • Why does RuboCop need the regression test? •

    What’s the regression test? • How to work. • How to contribute to RuboCop. • Future work.
  8. Why does RuboCop need the regression test?

  9. RuboCop breaks easily, with Ruby syntax • Ruby has too

    complex syntax. • Actually we do not understand Ruby syntax perfectly. • A cop, implemented by us, will overlooks edge cases of Ruby syntax. • Example: omitting parens
  10. RuboCop breaks easily, with the configuration • RuboCop has many

    options. You can configure RuboCop for your style. • But it introduces complexity. • Not default option is not maintained enough.
  11. Conclusion of this section • RuboCop breaks easily ◦ with

    Ruby syntax ◦ with configurations • The regression test is necessary to find bugs before releasing RuboCop.
  12. What’s the regression test?

  13. Developing under here • rubocop-hq/rubocop#6805 • https://github.com/pocke/rubocop-regression-test

  14. What’s the regression test? • In short, it executes RuboCop

    ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  15. Example • In project-a ◦ rubocop --config .rubocop.1.yml ◦ rubocop

    --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-b ◦ rubocop --config .rubocop.1.yml ◦ rubocop --config .rubocop.2.yml ◦ rubocop --config .rubocop.3.yml ◦ … • In project-c ◦ ...
  16. What does check the test? • Check raised exceptions. ◦

    Grep “An error occurred while ...” message. ◦ Do not check false positives.
  17. How to work

  18. What’s the regression test? (page 14) • In short, it

    executes RuboCop ◦ with all option combinations, and ◦ in real Ruby projects. • Check exceptions.
  19. with all option combinations

  20. with all option combinations • Many cops has EnforcedStyle option

    to specify style. • The style candidates is available as SupportedStyles.
  21. Example of default configuration Style/StringLiterals: EnforcedStyle: single_quotes SupportedStyles: - single_quotes

    - double_quotes Layout/DotPosition: EnforcedStyle: leading SupportedStyles: - leading - trailing
  22. Example of all options combinations # .rubocop.1.yml Style/StringLiterals: EnforcedStyle: single_quotes

    Layout/DotPosition: EnforcedStyle: leading # .rubocop.2.yml Style/StringLiterals: EnforcedStyle: double_quotes Layout/DotPosition: EnforcedStyle: trailing
  23. in real Ruby projects

  24. It tests RuboCop in real Ruby projects • Unit test

    contains only a few code. ◦ It may overlook. • Real Ruby projects contain many codes. ◦ We can test cop with many patterns.
  25. It tests RuboCop in real Ruby projects • Now it

    tests with: ◦ tric/trick2013, 2015 and 2018 ◦ rubocop-hq/rubocop • In the future, I plan adding test with: ◦ ruby/spec (pocke/rubocop-regressiontest#16) ◦ ruby/ruby ◦ rails/rails ◦ gitlabhq/gitlabhq ◦ discourse/discourse
  26. How to contribute to RuboCop

  27. How to contribute to RuboCop • We have two contributing

    ways. ◦ Contribute to RuboCop ◦ Contribute to the regression test
  28. Contribute to RuboCop

  29. Contribute to RuboCop • Find bugs with the test on

    your codebase ↓ If you find bags, you can report them to RuboCop!
  30. Execute the following command $ git clone https://github.com/pocke/rubocop-regression-t est $

    cd rubocop-regression-test # Clone repo from GitHub and run the test $ ruby main.rb check OWNER/REPO # You can specify local directory also $ ruby main.rb check path/to/your/project/dir
  31. Contribute to rubocop-regression -test

  32. Contribute to RuboCop Regression Test • You can add more

    repositories to the test. ◦ Add them to CircleCI configuration. ◦ If it is green on CI, I can merge it. • Work for TODOs ◦ See the next section, “Future work”.
  33. Future work

  34. Test infinite loop in auto correction • Currently it ignores

    infinite loop in auto correction. ◦ Because it is too noisy. • But it also should detect and fix infinite loops.
  35. More test for --auto-correct • Auto correction should generate code

    that have same meaning if the cop is marked as “safe”. • In short, test `rubocop -a && rspec`.
  36. Test plugins • RuboCop has many plugins under the rubocop-hq

    organization. It should test them also. ◦ rubocop-rails ◦ rubocop-performance ◦ rubocop-rspec ◦ rubocop-rake ◦ and so on.