Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The new Testing Landscape: Panther, Foundry and More!

The new Testing Landscape: Panther, Foundry and More!

Symfony comes with a number of tools for testing, like BrowserKit & DomCrawler as well as test classes for booting the kernel, accessing services, and using a bunch of built-in assertions.

But what do you do if you need to test the JavaScript on your page? And what's the best approach for setting up your database? Should you load fixtures? Clear the data before each test? And how can I run my tests in parallel?

In this talk, we'll explore a set of new tools & clear patterns (Arrange, Act, Assert) for testing, as well as tools like Panther, Foundry, Browser and more!


April 02, 2021

More Decks by weaverryan

Other Decks in Programming


  1. > Member of the Symfony docs team 
 > Some

    Dude at SymfonyCasts.com > Husband of the talented and beloved @leannapelham symfonycasts.com twitter.com/weaverryan Yo! I’m Ryan! > Father to my much more charming son, Beckett
  2. Test Types Unit tests Integration tests (i.e. kernel tests) Functional

    tests (i.e. e2e / browser tests) Directly test classes Uses mocks, Symfony is not involved at all Directly test classes Uses real services from the container Test your site's pages/endpoints Uses your fully-functioning Symfony app
  3. Installing PHPUnit @weaverryan $ composer require phpunit --dev This installs

    symfony/test-pack. Today, that includes: • symfony/browser-kit • symfony/css-selector • symfony/dom-crawler • symfony/phpunit-bridge
  4. @weaverryan PHPUnit Bridge gives us: A. Deprecations reports after tests

    B. Clock mocking & other features C. Run tests against multiple PHPUnit versions D. PHPUnit's dependencies are separate from your app's But only C and D require you to run phpunit through phpunit-bridge
  5. Takeaway: Install PHPUnit Directly $ composer require phpunit/phpunit --dev $

    php vendor/bin/phpunit <!-- phpunit.xml.dist --> <phpunit> <!-- activates phpunit-bridge features --> <!-- this already comes in the recipe --> <listeners> <listener class="Symfony\Bridge\PhpUnit\SymfonyTestsListener" /> </listeners> </phpunit>
  6. Recipe will: • install phpunit/phpunit always via the test-pack •

    Make bin/phpunit use phpunit directly (if available)
  7. // tests/Util/CalculatorTest.php namespace App\Tests\Util; use App\Util\Calculator; use PHPUnit\Framework\TestCase; class CalculatorTest

    extends TestCase { public function testAdd(): void { $calculator = new Calculator(); $this->assertSame(42, $calculator->add(20, 22)); } } @weaverryan TestCase === no Symfony
  8. // tests/Repository/ProductRepositoryTest.php namespace App\Tests\Repository; use App\Repository\ProductRepository; use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase; class ProductRepositoryTest

    extends KernelTestCase { public function testSearch(): void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); } } KernelTestCase === the container is available A) B) A) Booting makes the container available B) self::$container is a special container where all services are public
  9. // src/Factory/ProductFactory.php namespace App\Factory; use App\Entity\Product; use Zenstruck\Foundry\ModelFactory; final class

    ProductFactory extends ModelFactory { protected function getDefaults(): array { return [ // TODO add your default values here ]; } protected static function getClass(): string { return Product::class; } } @weaverryan
  10. protected function getDefaults(): array { return [ 'name' => self::faker()->words(3,

    true), 'description' => self::faker()->paragraph, 'brand' => self::faker()->company, 'price' => self::faker()->numberBetween(1000, 10000), // will create a random category 'category' => CategoryFactory::new(), ]; } @weaverryan
  11. ProductFactory::createOne(['name' => 'floppy disk']); // create 5 all with the

    same 1 Category ProductFactory::createMany(5, ['category' => CategoryFactory::new()]); // create 4 all using an existing, random Category ProductFactory::createMany(5, ['category' => CategoryFactory::random()]); Now from a test or Doctrine Fixtures class @weaverryan
  12. class ProductRepositoryTest extends KernelTestCase { use Factories; public function testSearch():

    void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); ProductFactory::createOne(['name' => 'floppy disk']); ProductFactory::createOne(['name' => 'popcorn']); ProductFactory::createOne(['description' => 'A CD (compact disk)']); $results = $productRepository->search('disk'); $this->assertCount(2, $results); } } @weaverryan
  13. Setting up the test Database @weaverryan # .env.test # ...

    DATABASE_URL="mysql://[email protected]:3306/app_test?serverVersion=5.7" Now run php bin/console doctrine:database:create --env=test right?
  14. use Zenstruck\Foundry\Test\Factories; use Zenstruck\Foundry\Test\ResetDatabase; class ProductRepositoryTest extends KernelTestCase { use

    Factories; use ResetDatabase; // ... } Nope! ResetDatabase automatically drops and recreates your database before your tests begin + recreates your schema before each test!
  15. // tests/Controller/ProductControllerTest.php namespace App\Tests\Controller; use Symfony\Bundle\FrameworkBundle\Test\WebTestCase; class ProductControllerTest extends WebTestCase

    { public function testSomething(): void { $client = static::createClient(); $crawler = $client->request('GET', '/'); $this->assertResponseIsSuccessful(); $this->assertSelectorTextContains('h1', 'Hello World'); } } WebTestCase === the container is available 
 and you can "browse" the site with a fake, "in php" browser
  16. public function testEditProduct(): void { $client = static::createClient(); $product =

    ProductFactory::createOne(['name' => 'Brand new popcorn']); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
  17. public function testEditProduct(): void { $client = static::createClient(); $product =

    ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
  18. (I built this with a Stimulus controller in 40 lines

    including debouncing & "click outside" to close)
  19. public function testSearchAutoSuggestion(): void { $client = static::createClient(); ProductFactory::createOne(['name' =>

    'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  20. Hello Panther @weaverryan Panther tests with a "headless" Chrome or

    Firefox Uses the same API as the normal static::createClient()
  21. public function testSearchAutoSuggestion(): void { $client = static::createPantherClient(); ProductFactory::createOne(['name' =>

    'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $client->takeScreenshot(__DIR__.'/../../screenshot.png'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  22. Black Magic! A) Panther automatically starts a local PHP web

    server before the test - e.g. http://localhost:9080 B) Panther then communicates to Chrome/Firefox to visit that site (without opening a real browser**) ** there is an option to open a real browser: it's great for debugging a problem
  23. But we have 2 Panther Problems @weaverryan 1. Panther isn't

    using our test database! 2. We need to "wait" until the suggestions box appears Panther launches the web server in a "panther" (it can't use "test" as that has fake session handling)
  24. Using our Test Database # .env.panther # duplicate your .env.test

    config DATABASE_URL="mysql://[email protected]:3306/testing?serverVersion=5.7" The duplication is not ideal. There is a pull request to DoctrineBundle to make overriding the database name easier for tests https://github.com/doctrine/DoctrineBundle/pull/1290
  25. Automatic Screenshots on Failure <!-- phpunit.xml.dist --> <phpunit> <!-- ...

    --> <extensions> <extension class="Symfony\Component\Panther\ServerExtension" /> </extensions> </phpunit>
  26. Waiting for the Search Preview Box public function testSearchAutoSuggestion(): void

    { // ... $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); // wait for the AJAX to load $client->waitForElementToContain('.search-preview', 'Disk'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
  27. The truth I fi nd DomCrawler's API ( fi nding

    elements, fi lling out form fi elds, clicking links) … to be a bit clunky @weaverryan
  28. $ composer require zenstruck/browser --dev The " fi ne print"

    zenstruck/browser is currently 0.4 It has not reached a stable 1.0 release yet, But there are plans to do this in the coming months @weaverryan
  29. Automatic Screenshots, HTML dump and console.log dump on Failure <!--

    phpunit.xml.dist --> <phpunit> <!-- ... --> <extensions> - <extension class="Symfony\Component\Panther\ServerExtension" /> + <extension class="Zenstruck\Browser\Test\BrowserExtension" /> </extensions> </phpunit> @weaverryan
  30. Refactor the non-JS test: public function testEditProduct(): void { $product

    = ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $this->browser() ->visit(sprintf('/product/%s/edit', $product->getId())) ->assertSuccessful() ->fillField('Name', 'Slightly old popcorn') ->click('Update') ->followRedirect() ->assertSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); } Find & interact with elements by their "text"
  31. Refactor the Panther test: public function testSearchAutoSuggestion(): void { ProductFactory::createOne(['name'

    => 'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $this->pantherBrowser() ->visit('/') ->fillField('q', 'dis') ->waitUntilSeeIn('.search-preview', 'Floppy Disk') ->assertElementCount('.search-preview .list-group-item', 2); } @weaverryan
  32. $browser ->visit('/my/page') ->follow('A link') ->fillField('Name', 'Kevin') ->checkField('Accept Terms') ->selectFieldOption('Type', 'Employee')

    // single option select ->attachFile('Photo', '/path/to/photo.jpg') ->click('Submit') ->assertSeeIn('h1', 'some text') ->assertNotSeeIn('h1', 'some text') // the following use symfony/var-dumper's dump() function and continue ->dump() // raw response body or array if json ->dump('h1') // html element ->dump('foo') // if json response, array key ->dump('foo.*.baz') // if json response, JMESPath notation can be used ;
  33. The Fine Print! 1. If you write to the fi

    lesystem, you may need a fi lesystem abstraction (e.g. Flysystem) so you can use di ff erent paths for each test process 2. Panther currently needs a hack to work (otherwise it tries to multiple chrome-driver on the same port. See: https://github.com/symfony/panther/issues/436 @weaverryan