Hack with Me: Unit and Behavioral Tests

Unit test hackday intro for WordPress Portland meetup group.

Transcript

1. Hack With Me:
   Unit and Behavioral
   Tests
   PDXWP
   APRIL 27, 2013
   

   View Slide

2. Who am I?
   @MAXCUTLER
   HTTP://WWW.MAXCUTLER.COM
   

   View Slide

3. Why write tests?
   Verify functionality
   Validate technical design
   Protect against regressions
   Refactor with confidence
   Documentation
   

   View Slide

4. Unit testing
   

   View Slide

5. What is a unit?
   

   View Slide

6. But, what about the rest?
   Integration testing
   Acceptance testing
   Behavioral testing
   

   View Slide

7. Code coverage
   How much of the non-test code is
   exercised by test cases
   100% coverage?
   Depends on project priorities
   Pragmatic, not dogmatic
   WordPress core has plenty of
   room for improvement
   ◦ Patches!
   

   View Slide

8. Code coverage
   

   View Slide

9. Frameworks
   PHP: PHPUnit, SimpleTest, Behat
   JavaScript: QUnit, Jasmine, Mocha
   And many more, for almost every programming language
   For UI automation: Selenium, Watir
   For headless browser testing: Phantom.js
   

   View Slide

10. PHPUnit
    HTTP://PHPUNIT.DE
    

    View Slide

11. Anatomy of a test
    Class extending
    PHPUnit_Framework_TestCase
    At least one test method
    Optional setUp and tearDown
    ◦ Shared among all test methods in
    the class
    setUp
    testMethod
    •Assertion 1
    •Assertion 2
    •…
    tearDown
    

    View Slide

12. Anatomy of a test
    

    View Slide

13. Assertions
    assertArrayHasKey()
    assertClassHasAttribute()
    assertClassHasStaticAttribute()
    assertContains()
    assertContainsOnly()
    assertContainsOnlyInstancesOf()
    assertCount()
    assertEmpty()
    assertEqualXMLStructure()
    assertEquals()
    assertFalse()
    assertFileEquals()
    assertFileExists()
    assertGreaterThan()
    assertGreaterThanOrEqual()
    assertInstanceOf()
    assertInternalType()
    assertJsonFileEqualsJsonFile()
    assertJsonStringEqualsJsonFile()
    assertJsonStringEqualsJsonString()
    assertLessThan()
    assertLessThanOrEqual()
    assertNull()
    assertObjectHasAttribute()
    assertRegExp()
    assertStringMatchesFormat()
    assertStringMatchesFormatFile()
    assertSame()
    assertSelectCount()
    assertSelectEquals()
    assertSelectRegExp()
    assertStringEndsWith()
    assertStringEqualsFile()
    assertStringStartsWith()
    assertTag()
    assertThat()
    assertTrue()
    assertXmlFileEqualsXmlFile()
    assertXmlStringEqualsXmlFile()
    assertXmlStringEqualsXmlString()
    expectOutputString()
    setExpectedException()
    

    View Slide

14. Data providers
    

    View Slide

15. Stubs & Mocks
    When testing a unit, replace any
    dependencies with fakes that are
    fully under control of the test
    Stubs
    ◦ Replaces all methods with stubs
    ◦ You configure return values of any
    methods
    Mocks
    ◦ Verify the behavior of interactions
    between unit and other objects
    ◦ Beware of coupling test to
    implementation
    

    View Slide

16. Running tests
    Usage: phpunit [switches] UnitTest [UnitTest.php]
    phpunit [switches]
    --filter Filter which tests to run.
    --group ... Only runs tests from the specified group(s).
    --exclude-group ... Exclude tests from the specified group(s).
    --list-groups List available test groups.
    --repeat Runs the test(s) repeatedly.
    --colors Use colors in output.
    --stderr Write to STDERR instead of STDOUT.
    --stop-on-error Stop execution upon first error.
    --stop-on-failure Stop execution upon first error or failure.
    --stop-on-skipped Stop execution upon first skipped test.
    --stop-on-incomplete Stop execution upon first incomplete test.
    --strict Run tests in strict mode.
    -v|--verbose Output more verbose information.
    --debug Display debbuging information during test execution.
    --process-isolation Run each test in a separate PHP process.
    --bootstrap A "bootstrap" PHP file that is run before the tests.
    -c|--configuration Read configuration from XML file.
    --no-configuration Ignore default configuration file (phpunit.xml).
    --include-path Prepend PHP's include_path with given path(s).
    -d key[=value] Sets a php.ini value.
    -h|--help Prints this usage information.
    --version Prints the version and exits.
    --debug Output debugging information.
    

    View Slide

17. Running tests
    Command-line
    IDE Integrations
    ◦ PHPStorm
    ◦ Eclipse
    ◦ Netbeans
    ◦ Sublime Text 2
    ◦ …
    Continuous Integration Servers
    ◦ Jenkins
    ◦ Travis
    ◦ …
    

    View Slide

18. WordPress testing
    

    View Slide

19. WordPress Core Unit Tests
    http://make.wordpress.org/core/handbook/automated-testing/
    On WP Core trac
    ◦ “Unit Tests” component
    ◦ “needs-unit-tests” keyword
    Jenkins CI
    ◦ http://jenkins.kpayne.co:8080
    ◦ Automatically runs unit tests after commits
    ◦ Provides pass/fail history, code coverage, automatic code documentation,
    and more
    

    View Slide

20. Organization
    Base classes
    ◦ WP_UnitTestCase
    ◦ WP_Ajax_UnitTestCase
    ◦ WP_XMLRPC_UnitTestCase
    WP_UnitTest_Factory
    ◦ Use to create data for tests
    ◦ WP_UnitTest_Factory_For_*
    ◦ Post, Attachment, User, Term, Blog,
    Comment
    ◦ create, create_and_get, create_many
    Annotations:
    ◦ @group
    ◦ @ticket
    Mocks:
    ◦ MockAction
    ◦ MockPHPMailer
    ◦ WP_Image_Editor_Mock
    

    View Slide

21. WP_UnitTestCase
    setUp
    ◦ Open database connection
    ◦ Create WP_UnitTest_Factory
    ◦ Clean up global scope
    ◦ Clear $_GET, $_POST, flush object cache
    ◦ Beware global scope – avoid constants (see multisite, ajax)
    ◦ Start a DB transaction
    tearDown
    ◦ Rollback the DB transaction
    Correlates @ticket annotation to Trac tickets and skips tests for known issues
    ◦ Works for Core, Unit Tests, and Plugin tickets
    

    View Slide

22. How can you help?
    Fix failing test cases
    Improve code coverage
    Write patches for core tickets that are causing skipped/failed tests
    ◦ needs-unit-tests tickets
    ◦ Tests for any core bugs
    Does your plugin use a core API or action? Write a test for it so that core
    devs do not unintentionally break your use-case.
    Write tests for your plugins!
    

    View Slide