WHAT MAKES A TEST A GOOD TEST? “Sensitivity means that, of all mistakes that actually happen when writing your system, a high percentage are caught by the test suite, and a high percentage of those are caught early in the run.” RELIABILITY
class Group def accessible?(user) return true if public? owner_or_admin(user) || is_member(user) end spec describe “#accessible?" do let(:user) { create(:user) } context "when the group is private" do let(:owner) { create(:user) } let(:group) { create(:group, :private, user: owner.id) } context "when the user is a member" do before do group.group_members << create(:group_member, user: user) end it { expect(group.accessible?(user)).to be true } end end end
➤ The goal of mutation tests is to make your tests fail. ➤ Mutations are going to slightly change your code based on a set of defined operators. For each discrete change, the tests for that code are going to be run. ➤ Value ➤ Decision ➤ Statement ➤ If your tests kill all the mutants, then your code is notably more safe than just with line coverage MUTATION TESTING
➤ https://github.com/mbj/mutant* ➤ Supports RSpec and minitest * there might be some cases when the mutants are all killed but still your code is not 100% covered TOOLS
describe “#accessible?" do let(:user) { create(:user) } context "when the group is private" do let(:owner) { create(:user) } let(:group) { create(:group, :private, user: owner.id) } context "when the user is an admin of the website or the owner of the group" do it “returns true" do admin = create(:user, :admin) expect(group.accessible?(admin)).to be true expect(group.accessible?(owner)).to be true end end context "when the user is a member" do it “returns true" do user = create(:user) group.group_members << create(:group_member, user: user) expect(group.accessible?(user)).to be true end end context "when the user is not a member" do it { expect(group.accessible?(create(:user)).to be false } end end context "when the group is public" do it “returns true" do expect(group.accessible?(create(:user)).to be true end end end
DOWNSIDES ➤ cRuby/MRI - Ruby 2.5.x-2.6 ➤ As you can imagine, mutation tests can be extremely time consuming, so it's almost impossible to run them on the whole codebase with the same frequency you HAVE TO run your usual tests ➤ If you deviate from the MVC pattern, it’s a tad more difficult to make mutant run all the appropriate tests for the class you want to test
SOLUTIONS ➤ mutant allows you to narrow the mutations down to a single method -> run mutant after you finished writing the tests for your method to ensure that your code is fully covered ➤ run mutant continuously just to cover the most sensitive part of your application ➤ the —since REVISION option will make mutant mutate only the code that has been modified since the specified point, thus cutting down execution time and scope ➤ add a top level namespace that eventually allows mutant to match example groups with: Foo::Bar#baz, Foo::Bar, Foo