Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Regression testing for RuboCop

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
pocke
March 28, 2019

Transcript

23 slides
  1. Because RuboCop is buggy, especially with: • NOT being a

    default setting. ◦ The core team and many users use the default settings, but other settings are used by less people. • complex Ruby code. ◦ Ruby is a complex language. There are mysterious codes in the world. ◦ Cop’s author often overlooks edge cases.
  2. What is the regression test? • It executes RuboCop ◦

    with all setting combinations, and ◦ in real Ruby projects, that have many / mysterious / cool Ruby code.
  3. With all setting combinations • Many cops have EnforcedStyle and

    SupportedStyles attributes. • The test configures all options. • It is based on pocke/gry.
  4. Combination hell: AlignHash Layout/AlignHash: EnforcedHashRocketStyle: key SupportedHashRocketStyles: [key, separator, table]

    EnforcedColonStyle: key SupportedColonStyles: [key, separator, table] EnforcedLastArgumentHashStyle: always_inspect SupportedLastArgumentHashStyles: [always_inspect, always_ignore, ignore_implicit, ignore_explicit] 3 * 3 * 4 = 36 patterns
  5. Real Ruby projects • Test with edge cases ◦ tric/trick2013,

    2015 and 2018. ◦ ruby/spec • Test with large code base ◦ ruby/ruby ◦ gitlabhq/gitlabhq ◦ etc.
  6. rubocop-hq/rubocop#6804, #6803 • Bug of AutoCorrection ◦ -␣10000 => -10_000

    ◦ 12345e6 => 12_345e6 • It was found with tric/trick2015/yoshi-taka/entry.rb
  7. Current State • The test passed on trick2013, 2015 and

    2018. • It ran only on my local PC.
  8. Production problems • It takes a long time. ◦ Parallelize?

    Optimize? Machine power? • CI configuration. • How to report errors.
  9. In The Future • Test AutoCorrection dynamically. ◦ Currently it

    tests only SyntaxError, but I hope to make it test RuntimeError. ◦ e.g. $ rubocop -a && rspec # $? == 0 Thank you for listening!

More Decks by pocke

Other Decks in Programming