Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effective Testing or how to sleep when you are on call

Effective Testing or how to sleep when you are on call

Light talk on how to power up your testing with mutations

Monica Giambitto

April 30, 2020
Tweet

More Decks by Monica Giambitto

Other Decks in Programming

Transcript

  1. EFFECTIVE TESTING
    or how to sleep when you are on call

    View Slide

  2. FAST
    WHAT MAKES A TEST A GOOD TEST?

    View Slide

  3. WHAT MAKES A TEST A GOOD TEST?
    “Sensitivity means that, of all
    mistakes that actually happen
    when writing your system, a high
    percentage are caught by the test
    suite, and a high percentage of
    those are caught early in the run.”
    RELIABILITY

    View Slide

  4. View Slide

  5. class Group
    def accessible?(user)
    return true if public?
    owner_or_admin(user) || is_member(user)
    end
    spec
    describe “#accessible?" do
    let(:user) { create(:user) }
    context "when the group is private" do
    let(:owner) { create(:user) }
    let(:group) { create(:group, :private, user: owner.id) }
    context "when the user is a member" do
    before do
    group.group_members << create(:group_member,
    user: user)
    end
    it { expect(group.accessible?(user)).to be true }
    end
    end
    end

    View Slide

  6. Code coverage is checking that your code has been called
    somewhere, but what does it really says about the quality of
    your tests?

    View Slide

  7. View Slide

  8. ➤ The goal of mutation tests is to make your tests fail.
    ➤ Mutations are going to slightly change your code based on a
    set of defined operators. For each discrete change, the tests
    for that code are going to be run.
    ➤ Value
    ➤ Decision
    ➤ Statement
    ➤ If your tests kill all the mutants, then your code is notably
    more safe than just with line coverage
    MUTATION TESTING

    View Slide

  9. ➤ https://github.com/mbj/mutant*
    ➤ Supports RSpec and minitest
    * there might be some cases when the mutants are all killed but still your code is not 100% covered
    TOOLS

    View Slide

  10. MUTANTS
    group.rb
    def accessible(user)
    return true if public?
    owner_or_admin(user) || is_member(user)
    end

    View Slide

  11. MUTANT OPERATORS
    Source https://github.com/mbj/mutant/blob/master/meta/if.rb

    View Slide

  12. MUTANT OPERATORS
    Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/node/
    if.rb

    View Slide

  13. MUTANT OPERATORS
    Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/
    node.rb#L102

    View Slide

  14. MUTANT OPERATORS
    Source https://github.com/mbj/mutant/blob/master/lib/mutant/mutator/node/
    if.rb

    View Slide

  15. WHAT DOES MUTANT TELL US ABOUT OUR 100% CODE COVERAGE?

    View Slide

  16. describe “#accessible?" do
    let(:user) { create(:user) }
    context "when the group is private" do
    let(:owner) { create(:user) }
    let(:group) { create(:group, :private, user: owner.id) }
    context "when the user is an admin of the website or the owner of the group" do
    it “returns true" do
    admin = create(:user, :admin)
    expect(group.accessible?(admin)).to be true
    expect(group.accessible?(owner)).to be true
    end
    end
    context "when the user is a member" do
    it “returns true" do
    user = create(:user)
    group.group_members << create(:group_member, user: user)
    expect(group.accessible?(user)).to be true
    end
    end
    context "when the user is not a member" do
    it { expect(group.accessible?(create(:user)).to be false }
    end
    end
    context "when the group is public" do
    it “returns true" do
    expect(group.accessible?(create(:user)).to be true
    end
    end
    end

    View Slide

  17. group.rb
    def accessible(user)
    return true if public?
    owner_or_admin(user) || is_member(user)
    end
    MUTANTS

    View Slide

  18. DOWNSIDES
    ➤ cRuby/MRI - Ruby 2.5.x-2.6
    ➤ As you can imagine, mutation tests can be extremely time
    consuming, so it's almost impossible to run them on the
    whole codebase with the same frequency you HAVE TO run
    your usual tests
    ➤ If you deviate from the MVC pattern, it’s a tad more difficult
    to make mutant run all the appropriate tests for the class you
    want to test

    View Slide

  19. SOLUTIONS
    ➤ mutant allows you to narrow the mutations down to a single
    method -> run mutant after you finished writing the tests for
    your method to ensure that your code is fully covered
    ➤ run mutant continuously just to cover the most sensitive part
    of your application
    ➤ the —since REVISION option will make mutant mutate only
    the code that has been modified since the specified point, thus
    cutting down execution time and scope
    ➤ add a top level namespace that eventually allows mutant to
    match example groups with: Foo::Bar#baz, Foo::Bar,
    Foo

    View Slide

  20. ➤ http://www.sitepoint.com/mutation-testing-mutant/
    ➤ http://solnic.eu/2013/01/23/mutation-testing-with-
    mutant.html
    ➤ http://en.wikipedia.org/wiki/Mutation_testing
    ➤ https://www.youtube.com/watch?v=WccaOMuf01Y
    ➤ (RailsConf 2014 keynote - live coding around 25 minutes in)
    ➤ https://www.youtube.com/watch?v=cLh3yMKEqHc (5
    minutes introduction to mutation testing)

    View Slide

  21. Special thanks to @mbj for having reviewed this presentation

    View Slide

  22. @KFMolli on twitter
    nirnaeth on github and Speaker Deck
    [email protected]
    THANKS!
    @KFMOLLI
    GITHUB.COM/NIRNAETH
    SPEAKERDECK.COM/NIRNAETH
    [email protected]

    View Slide