Effective Testing or how to sleep when you are on call

Light talk on how to power up your testing with mutations

Transcript

1. EFFECTIVE TESTING or how to sleep when you are on call
2. FAST WHAT MAKES A TEST A GOOD TEST?
3. WHAT MAKES A TEST A GOOD TEST? “Sensitivity means that, of all mistakes that actually happen when writing your system, a high percentage are caught by the test suite, and a high percentage of those are caught early in the run.” RELIABILITY
5. class Group def accessible?(user) return true if public? owner_or_admin(user) || is_member(user) end spec describe “#accessible?" do let(:user) { create(:user) } context "when the group is private" do let(:owner) { create(:user) } let(:group) { create(:group, :private, user: owner.id) } context "when the user is a member" do before do group.group_members << create(:group_member, user: user) end it { expect(group.accessible?(user)).to be true } end end end
6. Code coverage is checking that your code has been called somewhere, but what does it really says about the quality of your tests?
8. ➤ The goal of mutation tests is to make your tests fail. ➤ Mutations are going to slightly change your code based on a set of defined operators. For each discrete change, the tests for that code are going to be run. ➤ Value ➤ Decision ➤ Statement ➤ If your tests kill all the mutants, then your code is notably more safe than just with line coverage MUTATION TESTING
9. ➤ https://github.com/mbj/mutant* ➤ Supports RSpec and minitest * there might be some cases when the mutants are all killed but still your code is not 100% covered TOOLS
10. MUTANTS group.rb def accessible(user) return true if public? owner_or_admin(user) || is_member(user) end
11. MUTANT OPERATORS Source https://github.com/mbj/mutant/blob/master/meta/if.rb
12. MUTANT OPERATORS Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/node/ if.rb
13. MUTANT OPERATORS Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/ node.rb#L102
14. MUTANT OPERATORS Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/node/ if.rb
15. WHAT DOES MUTANT TELL US ABOUT OUR 100% CODE COVERAGE?
16. describe “#accessible?" do let(:user) { create(:user) } context "when the group is private" do let(:owner) { create(:user) } let(:group) { create(:group, :private, user: owner.id) } context "when the user is an admin of the website or the owner of the group" do it “returns true" do admin = create(:user, :admin) expect(group.accessible?(admin)).to be true expect(group.accessible?(owner)).to be true end end context "when the user is a member" do it “returns true" do user = create(:user) group.group_members << create(:group_member, user: user) expect(group.accessible?(user)).to be true end end context "when the user is not a member" do it { expect(group.accessible?(create(:user)).to be false } end end context "when the group is public" do it “returns true" do expect(group.accessible?(create(:user)).to be true end end end
17. group.rb def accessible(user) return true if public? owner_or_admin(user) || is_member(user) end MUTANTS
18. DOWNSIDES ➤ cRuby/MRI - Ruby 2.5.x-2.6 ➤ As you can imagine, mutation tests can be extremely time consuming, so it's almost impossible to run them on the whole codebase with the same frequency you HAVE TO run your usual tests ➤ If you deviate from the MVC pattern, it’s a tad more difficult to make mutant run all the appropriate tests for the class you want to test
19. SOLUTIONS ➤ mutant allows you to narrow the mutations down to a single method -> run mutant after you finished writing the tests for your method to ensure that your code is fully covered ➤ run mutant continuously just to cover the most sensitive part of your application ➤ the —since REVISION option will make mutant mutate only the code that has been modified since the specified point, thus cutting down execution time and scope ➤ add a top level namespace that eventually allows mutant to match example groups with: Foo::Bar#baz, Foo::Bar, Foo
20. ➤ http://www.sitepoint.com/mutation-testing-mutant/ ➤ http://solnic.eu/2013/01/23/mutation-testing-with- mutant.html ➤ http://en.wikipedia.org/wiki/Mutation_testing ➤ https://www.youtube.com/watch?v=WccaOMuf01Y ➤ (RailsConf 2014 keynote - live coding around 25 minutes in) ➤ https://www.youtube.com/watch?v=cLh3yMKEqHc (5 minutes introduction to mutation testing)
21. Special thanks to @mbj for having reviewed this presentation
22. @KFMolli on twitter nirnaeth on github and Speaker Deck monica.giambitto@protonmail.ch THANKS! @KFMOLLI GITHUB.COM/NIRNAETH SPEAKERDECK.COM/NIRNAETH MONICA.GIAMBITTO@PM.ME