Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The new Testing Landscape: Panther, Foundry and More!

The new Testing Landscape: Panther, Foundry and More!

Symfony comes with a number of tools for testing, like BrowserKit & DomCrawler as well as test classes for booting the kernel, accessing services, and using a bunch of built-in assertions.

But what do you do if you need to test the JavaScript on your page? And what's the best approach for setting up your database? Should you load fixtures? Clear the data before each test? And how can I run my tests in parallel?

In this talk, we'll explore a set of new tools & clear patterns (Arrange, Act, Assert) for testing, as well as tools like Panther, Foundry, Browser and more!

weaverryan

April 02, 2021
Tweet

More Decks by weaverryan

Other Decks in Programming

Transcript

  1. The New Testing Landscape


    Panther, Foundry & More
    by your friend Ryan Weaver
    @weaverryan

    View Slide

  2. > Member of the Symfony docs team

    > Some Dude at


    SymfonyCasts.com
    > Husband of the talented and


    beloved @leannapelham
    symfonycasts.com


    twitter.com/weaverryan
    Yo! I’m Ryan!
    > Father to my much more


    charming son, Beckett

    View Slide

  3. Testing:


    A "From Space" Overview
    Part 1
    @weaverryan

    View Slide

  4. Test Types
    Unit tests
    Integration tests (i.e. kernel tests)
    Functional tests (i.e. e2e / browser tests)
    Directly test classes


    Uses mocks, Symfony is not involved at all
    Directly test classes


    Uses real services from the container
    Test your site's pages/endpoints


    Uses your fully-functioning Symfony app

    View Slide

  5. This Presentation:
    PHPUnit Basics
    @weaverryan
    TDD
    Kick-butt testing with Symfony

    View Slide

  6. @weaverryan
    Full code & step-by-step commits


    github.com/weaverryan/s
    fl
    ive_2021_testing/commits/
    fi
    nished

    View Slide

  7. PHPUnit & phpunit-bridge
    Part 2
    @weaverryan

    View Slide

  8. Installing PHPUnit
    @weaverryan
    $ composer require phpunit --dev
    This installs symfony/test-pack. Today, that includes:
    • symfony/browser-kit


    • symfony/css-selector


    • symfony/dom-crawler


    • symfony/phpunit-bridge

    View Slide

  9. $ php bin/phpunit
    We execute PHPUnit indirectly through phpunit-bridge.


    Why?

    View Slide

  10. @weaverryan
    PHPUnit Bridge gives us:
    A. Deprecations reports after tests


    B. Clock mocking & other features


    C. Run tests against multiple PHPUnit versions


    D. PHPUnit's dependencies are separate from your app's
    But only C and D require you to run phpunit through phpunit-bridge

    View Slide

  11. Takeaway: Install PHPUnit Directly
    $ composer require phpunit/phpunit --dev
    $ php vendor/bin/phpunit
























    View Slide

  12. Recipe will:


    • install phpunit/phpunit always via the test-pack


    • Make bin/phpunit use phpunit directly (if available)

    View Slide

  13. Unit Tests
    Part 3
    @weaverryan

    View Slide

  14. $ php bin/console make:test
    @weaverryan

    View Slide

  15. // tests/Util/CalculatorTest.php


    namespace App\Tests\Util;


    use App\Util\Calculator;


    use PHPUnit\Framework\TestCase;


    class CalculatorTest extends TestCase


    {


    public function testAdd(): void


    {


    $calculator = new Calculator();


    $this->assertSame(42, $calculator->add(20, 22));


    }


    }


    @weaverryan
    TestCase === no Symfony

    View Slide

  16. $ php vendor/bin/phpunit
    @weaverryan

    View Slide

  17. Integration (Kernel) Tests
    Part 4
    @weaverryan

    View Slide

  18. $ php bin/console make:test

    View Slide

  19. // tests/Repository/ProductRepositoryTest.php


    namespace App\Tests\Repository;


    use App\Repository\ProductRepository;


    use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;


    class ProductRepositoryTest extends KernelTestCase


    {


    public function testSearch(): void


    {


    $kernel = self::bootKernel();


    $productRepository = self::$container->get(ProductRepository::class);


    }


    }


    KernelTestCase === the container is available
    A)
    B)
    A) Booting makes the container available
    B) self::$container is a special container where all services are public

    View Slide

  20. But wait!
    How do I "seed" my database before
    making the query?
    @weaverryan

    View Slide

  21. Hello zenstruck/foundry
    Part 4
    Putting the FoUNdry back into data
    fi
    xtures!

    View Slide

  22. $ composer require zenstruck/foundry --dev
    $ php bin/console make:factory

    View Slide

  23. // src/Factory/ProductFactory.php


    namespace App\Factory;


    use App\Entity\Product;


    use Zenstruck\Foundry\ModelFactory;


    final class ProductFactory extends ModelFactory


    {


    protected function getDefaults(): array


    {


    return [


    // TODO add your default values here


    ];


    }


    protected static function getClass(): string


    {


    return Product::class;


    }


    }


    @weaverryan

    View Slide

  24. protected function getDefaults(): array


    {


    return [


    'name' => self::faker()->words(3, true),


    'description' => self::faker()->paragraph,


    'brand' => self::faker()->company,


    'price' => self::faker()->numberBetween(1000, 10000),


    // will create a random category


    'category' => CategoryFactory::new(),


    ];


    }


    @weaverryan

    View Slide

  25. ProductFactory::createOne(['name' => 'floppy disk']);


    // create 5 all with the same 1 Category


    ProductFactory::createMany(5, ['category' => CategoryFactory::new()]);


    // create 4 all using an existing, random Category


    ProductFactory::createMany(5, ['category' => CategoryFactory::random()]);


    Now from a test or Doctrine Fixtures class
    @weaverryan

    View Slide

  26. class ProductRepositoryTest extends KernelTestCase


    {


    use Factories;


    public function testSearch(): void


    {


    $kernel = self::bootKernel();


    $productRepository = self::$container->get(ProductRepository::class);


    ProductFactory::createOne(['name' => 'floppy disk']);


    ProductFactory::createOne(['name' => 'popcorn']);


    ProductFactory::createOne(['description' => 'A CD (compact disk)']);


    $results = $productRepository->search('disk');


    $this->assertCount(2, $results);


    }


    }


    @weaverryan

    View Slide

  27. Setting up the test Database
    @weaverryan
    # .env.test


    # ...


    DATABASE_URL="mysql://[email protected]:3306/app_test?serverVersion=5.7"


    Now run php bin/console doctrine:database:create --env=test right?

    View Slide

  28. use Zenstruck\Foundry\Test\Factories;


    use Zenstruck\Foundry\Test\ResetDatabase;


    class ProductRepositoryTest extends KernelTestCase


    {


    use Factories;


    use ResetDatabase;


    // ...


    }


    Nope!
    ResetDatabase automatically drops and recreates your


    database before your tests begin + recreates your schema


    before each test!

    View Slide

  29. $ php vendor/bin/phpunit
    @weaverryan

    View Slide

  30. Functional Tests
    Part 5
    (without JavaScript)
    @weaverryan

    View Slide

  31. $ php bin/console make:test

    View Slide

  32. // tests/Controller/ProductControllerTest.php


    namespace App\Tests\Controller;


    use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;


    class ProductControllerTest extends WebTestCase


    {


    public function testSomething(): void


    {


    $client = static::createClient();


    $crawler = $client->request('GET', '/');


    $this->assertResponseIsSuccessful();


    $this->assertSelectorTextContains('h1', 'Hello World');


    }


    }


    WebTestCase === the container is available

    and you can "browse" the site


    with a fake, "in php" browser

    View Slide

  33. // tests/Controller/ProductControllerTest.php


    namespace App\Tests\Controller;


    use Zenstruck\Foundry\Test\Factories;


    use Zenstruck\Foundry\Test\ResetDatabase;


    class ProductControllerTest extends WebTestCase


    {


    use Factories, ResetDatabase;


    // ...


    }


    @weaverryan

    View Slide

  34. View Slide

  35. public function testEditProduct(): void


    {


    $client = static::createClient();


    $product = ProductFactory::createOne(['name' => 'Brand new popcorn']);


    $crawler = $client->request(


    'GET',


    sprintf('/product/%s/edit', $product->getId())


    );


    $client->submitForm('Update', [


    'product[name]' => 'Slightly old popcorn'


    ]);


    $client->followRedirect();


    $this->assertResponseIsSuccessful();


    $this->assertSame('Slightly old popcorn', $product->getName());


    }


    View Slide

  36. $ php vendor/bin/phpunit

    View Slide

  37. public function testEditProduct(): void


    {


    $client = static::createClient();


    $product = ProductFactory::createOne(['name' => 'Brand new popcorn'])


    ->enableAutoRefresh();


    $crawler = $client->request(


    'GET',


    sprintf('/product/%s/edit', $product->getId())


    );


    $client->submitForm('Update', [


    'product[name]' => 'Slightly old popcorn'


    ]);


    $client->followRedirect();


    $this->assertResponseIsSuccessful();


    $this->assertSame('Slightly old popcorn', $product->getName());


    }


    View Slide

  38. $ php vendor/bin/phpunit
    @weaverryan

    View Slide

  39. Functional/E2E Tests
    Part 6
    (testing WITH JavaScript)
    @weaverryan

    View Slide

  40. (I built this with a Stimulus controller in 40 lines
    including debouncing & "click outside" to close)

    View Slide

  41. public function testSearchAutoSuggestion(): void


    {


    $client = static::createClient();


    ProductFactory::createOne(['name' => 'Floppy Disk']);


    ProductFactory::createOne(['name' => 'Compact Disk']);


    $crawler = $client->request('GET', '/');


    $searchForm = $crawler->selectButton('Search')->form();


    $searchForm->setValues(['q' => 'dis']);


    $this->assertCount(


    2,


    $crawler->filter('.search-preview .list-group-item')


    );


    }


    View Slide

  42. $ php vendor/bin/phpunit
    @weaverryan

    View Slide

  43. Hello Panther
    @weaverryan
    Panther tests with a "headless" Chrome or Firefox
    Uses the same API as the normal static::createClient()

    View Slide

  44. $ composer require symfony/panther --dev

    View Slide

  45. $ composer require --dev dbrekelmans/bdi
    $ ./vendor/bin/bdi detect drivers

    View Slide

  46. public function testSearchAutoSuggestion(): void


    {


    $client = static::createPantherClient();


    ProductFactory::createOne(['name' => 'Floppy Disk']);


    ProductFactory::createOne(['name' => 'Compact Disk']);


    $crawler = $client->request('GET', '/');


    $searchForm = $crawler->selectButton('Search')->form();


    $searchForm->setValues(['q' => 'dis']);


    $client->takeScreenshot(__DIR__.'/../../screenshot.png');


    $this->assertCount(


    2,


    $crawler->filter('.search-preview .list-group-item')


    );


    }


    View Slide

  47. $ php vendor/bin/phpunit
    @weaverryan

    View Slide

  48. But we have a screenshot!!!

    View Slide

  49. Black Magic!
    A) Panther automatically starts a local PHP web server


    before the test - e.g. http://localhost:9080
    B) Panther then communicates to Chrome/Firefox to visit


    that site (without opening a real browser**)
    ** there is an option to open a real browser:


    it's great for debugging a problem

    View Slide

  50. But we have 2 Panther Problems
    @weaverryan
    1. Panther isn't using our test database!
    2. We need to "wait" until the suggestions box appears
    Panther launches the web server in a "panther"


    (it can't use "test" as that has fake session handling)

    View Slide

  51. Using our Test Database
    # .env.panther


    # duplicate your .env.test config


    DATABASE_URL="mysql://[email protected]:3306/testing?serverVersion=5.7"
    The duplication is not ideal. There is a pull request to DoctrineBundle


    to make overriding the database name easier for tests
    https://github.com/doctrine/DoctrineBundle/pull/1290

    View Slide

  52. Automatic Screenshots on Failure





















    View Slide

  53. Waiting for the Search Preview Box
    public function testSearchAutoSuggestion(): void


    {


    // ...


    $crawler = $client->request('GET', '/');


    $searchForm = $crawler->selectButton('Search')->form();


    $searchForm->setValues(['q' => 'dis']);


    // wait for the AJAX to load


    $client->waitForElementToContain('.search-preview', 'Disk');


    $this->assertCount(


    2,


    $crawler->filter('.search-preview .list-group-item')


    );


    }


    View Slide

  54. $ php vendor/bin/phpunit
    https://symfony.com/blog/announcing-symfony-panther-1-0
    @weaverryan

    View Slide

  55. Hello zenstruck/browser
    Part 7
    <— Yea, this guy again!
    (wrapper around BrowserKit & Panther)

    View Slide

  56. The truth
    I
    fi
    nd DomCrawler's API


    (
    fi
    nding elements,
    fi
    lling out form
    fi
    elds, clicking links)


    … to be a bit clunky
    @weaverryan

    View Slide

  57. $ composer require zenstruck/browser --dev
    The "
    fi
    ne print"


    zenstruck/browser is currently 0.4


    It has not reached a stable 1.0 release yet,


    But there are plans to do this in the coming months
    @weaverryan

    View Slide

  58. Automatic Screenshots, HTML dump and


    console.log dump on Failure












    -


    +








    @weaverryan

    View Slide

  59. Refactor the non-JS test:
    public function testEditProduct(): void


    {


    $product = ProductFactory::createOne(['name' => 'Brand new popcorn'])


    ->enableAutoRefresh();


    $this->browser()


    ->visit(sprintf('/product/%s/edit', $product->getId()))


    ->assertSuccessful()


    ->fillField('Name', 'Slightly old popcorn')


    ->click('Update')


    ->followRedirect()


    ->assertSuccessful();


    $this->assertSame('Slightly old popcorn', $product->getName());


    }


    Find & interact with


    elements by their "text"

    View Slide

  60. Refactor the Panther test:
    public function testSearchAutoSuggestion(): void


    {


    ProductFactory::createOne(['name' => 'Floppy Disk']);


    ProductFactory::createOne(['name' => 'Compact Disk']);


    $this->pantherBrowser()


    ->visit('/')


    ->fillField('q', 'dis')


    ->waitUntilSeeIn('.search-preview', 'Floppy Disk')


    ->assertElementCount('.search-preview .list-group-item', 2);


    }


    @weaverryan

    View Slide

  61. As you fail, browser leaves you clues

    View Slide

  62. $browser


    ->visit('/my/page')


    ->follow('A link')


    ->fillField('Name', 'Kevin')


    ->checkField('Accept Terms')


    ->selectFieldOption('Type', 'Employee') // single option select


    ->attachFile('Photo', '/path/to/photo.jpg')


    ->click('Submit')


    ->assertSeeIn('h1', 'some text')


    ->assertNotSeeIn('h1', 'some text')


    // the following use symfony/var-dumper's dump() function and continue


    ->dump() // raw response body or array if json


    ->dump('h1') // html element


    ->dump('foo') // if json response, array key


    ->dump('foo.*.baz') // if json response, JMESPath notation can be used


    ;


    View Slide

  63. ParaTest
    Part 8
    (Run your tests in Parallel Processes)
    @weaverryan

    View Slide

  64. $ composer require brianium/paratest
    $ ./vendor/bin/paratest

    View Slide

  65. Unique Database Per Process
    # .env.test and .env.panther


    DATABASE_URL="mysql://[email protected]:3306/testing${TEST_TOKEN}?serverVersion=5.7"


    View Slide

  66. The Fine Print!
    1. If you write to the
    fi
    lesystem, you may need a
    fi
    lesystem


    abstraction (e.g. Flysystem) so you can use di
    ff
    erent


    paths for each test process
    2. Panther currently needs a hack to work (otherwise it tries to


    multiple chrome-driver on the same port.


    See: https://github.com/symfony/panther/issues/436
    @weaverryan

    View Slide

  67. So…


    My 2021 testing stack is…
    @weaverryan

    View Slide

  68. Executing


    phpunit/phpunit


    directly
    @weaverryan

    View Slide

  69. zenstruck/foundry


    for data
    fi
    xtures


    and test data
    @weaverryan

    View Slide

  70. zenstruck/browser


    for a beautiful functional
    testing API on top of
    Panther and BrowserKit
    @weaverryan

    View Slide

  71. And I ❤ it
    @weaverryan

    View Slide

  72. @kbond
    THANK YOU!
    @wouterj @nyholm
    @weaverryan

    View Slide

  73. SCREENCASTS
    Stimulus:


    https://symfonycasts.com/screencast/stimulus


    Foundry & Data Fixtures (basics):


    https://symfonycasts.com/screencast/symfony-doctrine


    Testing:


    Upcoming…
    @weaverryan

    View Slide