Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The new Testing Landscape: Panther, Foundry and More!

The new Testing Landscape: Panther, Foundry and More!

Symfony comes with a number of tools for testing, like BrowserKit & DomCrawler as well as test classes for booting the kernel, accessing services, and using a bunch of built-in assertions.

But what do you do if you need to test the JavaScript on your page? And what's the best approach for setting up your database? Should you load fixtures? Clear the data before each test? And how can I run my tests in parallel?

In this talk, we'll explore a set of new tools & clear patterns (Arrange, Act, Assert) for testing, as well as tools like Panther, Foundry, Browser and more!



April 02, 2021


  1. The New Testing Landscape Panther, Foundry & More by your

    friend Ryan Weaver @weaverryan
  2. > Member of the Symfony docs team 
 > Some

    Dude at SymfonyCasts.com > Husband of the talented and beloved @leannapelham symfonycasts.com twitter.com/weaverryan Yo! I’m Ryan! > Father to my much more charming son, Beckett
  3. Testing: A "From Space" Overview Part 1 @weaverryan

  4. Test Types Unit tests Integration tests (i.e. kernel tests) Functional

    tests (i.e. e2e / browser tests) Directly test classes Uses mocks, Symfony is not involved at all Directly test classes Uses real services from the container Test your site's pages/endpoints Uses your fully-functioning Symfony app
  5. This Presentation: PHPUnit Basics @weaverryan TDD Kick-butt testing with Symfony

  6. @weaverryan Full code & step-by-step commits github.com/weaverryan/s fl ive_2021_testing/commits/ fi

  7. PHPUnit & phpunit-bridge Part 2 @weaverryan

  8. Installing PHPUnit @weaverryan $ composer require phpunit --dev This installs

    symfony/test-pack. Today, that includes: • symfony/browser-kit • symfony/css-selector • symfony/dom-crawler • symfony/phpunit-bridge
  9. $ php bin/phpunit We execute PHPUnit indirectly through phpunit-bridge. Why?

  10. @weaverryan PHPUnit Bridge gives us: A. Deprecations reports after tests

    B. Clock mocking & other features C. Run tests against multiple PHPUnit versions D. PHPUnit's dependencies are separate from your app's But only C and D require you to run phpunit through phpunit-bridge
  11. Takeaway: Install PHPUnit Directly $ composer require phpunit/phpunit --dev $

    php vendor/bin/phpunit <!-- phpunit.xml.dist --> <phpunit> <!-- activates phpunit-bridge features --> <!-- this already comes in the recipe --> <listeners> <listener class="Symfony\Bridge\PhpUnit\SymfonyTestsListener" /> </listeners> </phpunit>
  12. Recipe will: • install phpunit/phpunit always via the test-pack •

    Make bin/phpunit use phpunit directly (if available)
  13. Unit Tests Part 3 @weaverryan

  14. $ php bin/console make:test @weaverryan

  15. // tests/Util/CalculatorTest.php namespace App\Tests\Util; use App\Util\Calculator; use PHPUnit\Framework\TestCase; class CalculatorTest

    extends TestCase { public function testAdd(): void { $calculator = new Calculator(); $this->assertSame(42, $calculator->add(20, 22)); } } @weaverryan TestCase === no Symfony
  16. $ php vendor/bin/phpunit @weaverryan

  17. Integration (Kernel) Tests Part 4 @weaverryan

  18. $ php bin/console make:test

  19. // tests/Repository/ProductRepositoryTest.php namespace App\Tests\Repository; use App\Repository\ProductRepository; use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase; class ProductRepositoryTest

    extends KernelTestCase { public function testSearch(): void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); } } KernelTestCase === the container is available A) B) A) Booting makes the container available B) self::$container is a special container where all services are public
  20. But wait! How do I "seed" my database before making

    the query? @weaverryan
  21. Hello zenstruck/foundry Part 4 Putting the FoUNdry back into data

    fi xtures!
  22. $ composer require zenstruck/foundry --dev $ php bin/console make:factory

  23. // src/Factory/ProductFactory.php namespace App\Factory; use App\Entity\Product; use Zenstruck\Foundry\ModelFactory; final class

    ProductFactory extends ModelFactory { protected function getDefaults(): array { return [ // TODO add your default values here ]; } protected static function getClass(): string { return Product::class; } } @weaverryan
  24. protected function getDefaults(): array { return [ 'name' => self::faker()->words(3,

    true), 'description' => self::faker()->paragraph, 'brand' => self::faker()->company, 'price' => self::faker()->numberBetween(1000, 10000), // will create a random category 'category' => CategoryFactory::new(), ]; } @weaverryan
  25. ProductFactory::createOne(['name' => 'floppy disk']); // create 5 all with the

    same 1 Category ProductFactory::createMany(5, ['category' => CategoryFactory::new()]); // create 4 all using an existing, random Category ProductFactory::createMany(5, ['category' => CategoryFactory::random()]); Now from a test or Doctrine Fixtures class @weaverryan
  26. class ProductRepositoryTest extends KernelTestCase { use Factories; public function testSearch():

    void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); ProductFactory::createOne(['name' => 'floppy disk']); ProductFactory::createOne(['name' => 'popcorn']); ProductFactory::createOne(['description' => 'A CD (compact disk)']); $results = $productRepository->search('disk'); $this->assertCount(2, $results); } } @weaverryan
  27. Setting up the test Database @weaverryan # .env.test # ...

    DATABASE_URL="mysql://root@" Now run php bin/console doctrine:database:create --env=test right?
  28. use Zenstruck\Foundry\Test\Factories; use Zenstruck\Foundry\Test\ResetDatabase; class ProductRepositoryTest extends KernelTestCase { use

    Factories; use ResetDatabase; // ... } Nope! ResetDatabase automatically drops and recreates your database before your tests begin + recreates your schema before each test!
  29. $ php vendor/bin/phpunit @weaverryan

  30. Functional Tests Part 5 (without JavaScript) @weaverryan

  31. $ php bin/console make:test

  32. // tests/Controller/ProductControllerTest.php namespace App\Tests\Controller; use Symfony\Bundle\FrameworkBundle\Test\WebTestCase; class ProductControllerTest extends WebTestCase

    { public function testSomething(): void { $client = static::createClient(); $crawler = $client->request('GET', '/'); $this->assertResponseIsSuccessful(); $this->assertSelectorTextContains('h1', 'Hello World'); } } WebTestCase === the container is available 
 and you can "browse" the site with a fake, "in php" browser
  33. // tests/Controller/ProductControllerTest.php namespace App\Tests\Controller; use Zenstruck\Foundry\Test\Factories; use Zenstruck\Foundry\Test\ResetDatabase; class ProductControllerTest

    extends WebTestCase { use Factories, ResetDatabase; // ... } @weaverryan
  34. None
  35. public function testEditProduct(): void { $client = static::createClient(); $product =

    ProductFactory::createOne(['name' => 'Brand new popcorn']); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
  36. $ php vendor/bin/phpunit

  37. public function testEditProduct(): void { $client = static::createClient(); $product =

    ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
  38. $ php vendor/bin/phpunit @weaverryan

  39. Functional/E2E Tests Part 6 (testing WITH JavaScript) @weaverryan

  40. (I built this with a Stimulus controller in 40 lines

    including debouncing & "click outside" to close)
  41. public function testSearchAutoSuggestion(): void { $client = static::createClient(); ProductFactory::createOne(['name' =>

    'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  42. $ php vendor/bin/phpunit @weaverryan

  43. Hello Panther @weaverryan Panther tests with a "headless" Chrome or

    Firefox Uses the same API as the normal static::createClient()
  44. $ composer require symfony/panther --dev

  45. $ composer require --dev dbrekelmans/bdi $ ./vendor/bin/bdi detect drivers

  46. public function testSearchAutoSuggestion(): void { $client = static::createPantherClient(); ProductFactory::createOne(['name' =>

    'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $client->takeScreenshot(__DIR__.'/../../screenshot.png'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  47. $ php vendor/bin/phpunit @weaverryan

  48. But we have a screenshot!!!

  49. Black Magic! A) Panther automatically starts a local PHP web

    server before the test - e.g. http://localhost:9080 B) Panther then communicates to Chrome/Firefox to visit that site (without opening a real browser**) ** there is an option to open a real browser: it's great for debugging a problem
  50. But we have 2 Panther Problems @weaverryan 1. Panther isn't

    using our test database! 2. We need to "wait" until the suggestions box appears Panther launches the web server in a "panther" (it can't use "test" as that has fake session handling)
  51. Using our Test Database # .env.panther # duplicate your .env.test

    config DATABASE_URL="mysql://root@" The duplication is not ideal. There is a pull request to DoctrineBundle to make overriding the database name easier for tests https://github.com/doctrine/DoctrineBundle/pull/1290
  52. Automatic Screenshots on Failure <!-- phpunit.xml.dist --> <phpunit> <!-- ...

    --> <extensions> <extension class="Symfony\Component\Panther\ServerExtension" /> </extensions> </phpunit>
  53. Waiting for the Search Preview Box public function testSearchAutoSuggestion(): void

    { // ... $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); // wait for the AJAX to load $client->waitForElementToContain('.search-preview', 'Disk'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  54. $ php vendor/bin/phpunit https://symfony.com/blog/announcing-symfony-panther-1-0 @weaverryan

  55. Hello zenstruck/browser Part 7 <— Yea, this guy again! (wrapper

    around BrowserKit & Panther)
  56. The truth I fi nd DomCrawler's API ( fi nding

    elements, fi lling out form fi elds, clicking links) … to be a bit clunky @weaverryan
  57. $ composer require zenstruck/browser --dev The " fi ne print"

    zenstruck/browser is currently 0.4 It has not reached a stable 1.0 release yet, But there are plans to do this in the coming months @weaverryan
  58. Automatic Screenshots, HTML dump and console.log dump on Failure <!--

    phpunit.xml.dist --> <phpunit> <!-- ... --> <extensions> - <extension class="Symfony\Component\Panther\ServerExtension" /> + <extension class="Zenstruck\Browser\Test\BrowserExtension" /> </extensions> </phpunit> @weaverryan
  59. Refactor the non-JS test: public function testEditProduct(): void { $product

    = ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $this->browser() ->visit(sprintf('/product/%s/edit', $product->getId())) ->assertSuccessful() ->fillField('Name', 'Slightly old popcorn') ->click('Update') ->followRedirect() ->assertSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); } Find & interact with elements by their "text"
  60. Refactor the Panther test: public function testSearchAutoSuggestion(): void { ProductFactory::createOne(['name'

    => 'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $this->pantherBrowser() ->visit('/') ->fillField('q', 'dis') ->waitUntilSeeIn('.search-preview', 'Floppy Disk') ->assertElementCount('.search-preview .list-group-item', 2); } @weaverryan
  61. As you fail, browser leaves you clues

  62. $browser ->visit('/my/page') ->follow('A link') ->fillField('Name', 'Kevin') ->checkField('Accept Terms') ->selectFieldOption('Type', 'Employee')

    // single option select ->attachFile('Photo', '/path/to/photo.jpg') ->click('Submit') ->assertSeeIn('h1', 'some text') ->assertNotSeeIn('h1', 'some text') // the following use symfony/var-dumper's dump() function and continue ->dump() // raw response body or array if json ->dump('h1') // html element ->dump('foo') // if json response, array key ->dump('foo.*.baz') // if json response, JMESPath notation can be used ;
  63. ParaTest Part 8 (Run your tests in Parallel Processes) @weaverryan

  64. $ composer require brianium/paratest $ ./vendor/bin/paratest

  65. Unique Database Per Process # .env.test and .env.panther DATABASE_URL="mysql://root@${TEST_TOKEN}?serverVersion=5.7"

  66. The Fine Print! 1. If you write to the fi

    lesystem, you may need a fi lesystem abstraction (e.g. Flysystem) so you can use di ff erent paths for each test process 2. Panther currently needs a hack to work (otherwise it tries to multiple chrome-driver on the same port. See: https://github.com/symfony/panther/issues/436 @weaverryan
  67. So… My 2021 testing stack is… @weaverryan

  68. Executing phpunit/phpunit directly @weaverryan

  69. zenstruck/foundry for data fi xtures and test data @weaverryan

  70. zenstruck/browser for a beautiful functional testing API on top of

    Panther and BrowserKit @weaverryan
  71. And I ❤ it @weaverryan

  72. @kbond THANK YOU! @wouterj @nyholm @weaverryan

  73. SCREENCASTS Stimulus: https://symfonycasts.com/screencast/stimulus Foundry & Data Fixtures (basics): https://symfonycasts.com/screencast/symfony-doctrine Testing:

    Upcoming… @weaverryan