The new Testing Landscape: Panther, Foundry and More!

Symfony comes with a number of tools for testing, like BrowserKit & DomCrawler as well as test classes for booting the kernel, accessing services, and using a bunch of built-in assertions.

But what do you do if you need to test the JavaScript on your page? And what's the best approach for setting up your database? Should you load fixtures? Clear the data before each test? And how can I run my tests in parallel?

In this talk, we'll explore a set of new tools & clear patterns (Arrange, Act, Assert) for testing, as well as tools like Panther, Foundry, Browser and more!

Transcript

1. The New Testing Landscape Panther, Foundry & More by your friend Ryan Weaver @weaverryan
2. > Member of the Symfony docs team 
 > Some Dude at SymfonyCasts.com > Husband of the talented and beloved @leannapelham symfonycasts.com twitter.com/weaverryan Yo! I’m Ryan! > Father to my much more charming son, Beckett
3. Testing: A "From Space" Overview Part 1 @weaverryan
4. Test Types Unit tests Integration tests (i.e. kernel tests) Functional tests (i.e. e2e / browser tests) Directly test classes Uses mocks, Symfony is not involved at all Directly test classes Uses real services from the container Test your site's pages/endpoints Uses your fully-functioning Symfony app
5. This Presentation: PHPUnit Basics @weaverryan TDD Kick-butt testing with Symfony
6. @weaverryan Full code & step-by-step commits github.com/weaverryan/s fl ive_2021_testing/commits/ fi nished
7. PHPUnit & phpunit-bridge Part 2 @weaverryan
8. Installing PHPUnit @weaverryan $ composer require phpunit --dev This installs symfony/test-pack. Today, that includes: • symfony/browser-kit • symfony/css-selector • symfony/dom-crawler • symfony/phpunit-bridge
9. $ php bin/phpunit We execute PHPUnit indirectly through phpunit-bridge. Why?
10. @weaverryan PHPUnit Bridge gives us: A. Deprecations reports after tests B. Clock mocking & other features C. Run tests against multiple PHPUnit versions D. PHPUnit's dependencies are separate from your app's But only C and D require you to run phpunit through phpunit-bridge
11. Takeaway: Install PHPUnit Directly $ composer require phpunit/phpunit --dev $ php vendor/bin/phpunit <!-- phpunit.xml.dist --> <phpunit> <!-- activates phpunit-bridge features --> <!-- this already comes in the recipe --> <listeners> <listener class="Symfony\Bridge\PhpUnit\SymfonyTestsListener" /> </listeners> </phpunit>
12. Recipe will: • install phpunit/phpunit always via the test-pack • Make bin/phpunit use phpunit directly (if available)
13. Unit Tests Part 3 @weaverryan
14. $ php bin/console make:test @weaverryan
15. // tests/Util/CalculatorTest.php namespace App\Tests\Util; use App\Util\Calculator; use PHPUnit\Framework\TestCase; class CalculatorTest extends TestCase { public function testAdd(): void { $calculator = new Calculator(); $this->assertSame(42, $calculator->add(20, 22)); } } @weaverryan TestCase === no Symfony
16. $ php vendor/bin/phpunit @weaverryan
17. Integration (Kernel) Tests Part 4 @weaverryan
18. $ php bin/console make:test
19. // tests/Repository/ProductRepositoryTest.php namespace App\Tests\Repository; use App\Repository\ProductRepository; use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase; class ProductRepositoryTest extends KernelTestCase { public function testSearch(): void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); } } KernelTestCase === the container is available A) B) A) Booting makes the container available B) self::$container is a special container where all services are public
20. But wait! How do I "seed" my database before making the query? @weaverryan
21. Hello zenstruck/foundry Part 4 Putting the FoUNdry back into data fi xtures!
22. $ composer require zenstruck/foundry --dev $ php bin/console make:factory
23. // src/Factory/ProductFactory.php namespace App\Factory; use App\Entity\Product; use Zenstruck\Foundry\ModelFactory; final class ProductFactory extends ModelFactory { protected function getDefaults(): array { return [ // TODO add your default values here ]; } protected static function getClass(): string { return Product::class; } } @weaverryan
24. protected function getDefaults(): array { return [ 'name' => self::faker()->words(3, true), 'description' => self::faker()->paragraph, 'brand' => self::faker()->company, 'price' => self::faker()->numberBetween(1000, 10000), // will create a random category 'category' => CategoryFactory::new(), ]; } @weaverryan
25. ProductFactory::createOne(['name' => 'floppy disk']); // create 5 all with the same 1 Category ProductFactory::createMany(5, ['category' => CategoryFactory::new()]); // create 4 all using an existing, random Category ProductFactory::createMany(5, ['category' => CategoryFactory::random()]); Now from a test or Doctrine Fixtures class @weaverryan
26. class ProductRepositoryTest extends KernelTestCase { use Factories; public function testSearch(): void { $kernel = self::bootKernel(); $productRepository = self::$container->get(ProductRepository::class); ProductFactory::createOne(['name' => 'floppy disk']); ProductFactory::createOne(['name' => 'popcorn']); ProductFactory::createOne(['description' => 'A CD (compact disk)']); $results = $productRepository->search('disk'); $this->assertCount(2, $results); } } @weaverryan
27. Setting up the test Database @weaverryan # .env.test # ... DATABASE_URL="mysql://root@127.0.0.1:3306/app_test?serverVersion=5.7" Now run php bin/console doctrine:database:create --env=test right?
28. use Zenstruck\Foundry\Test\Factories; use Zenstruck\Foundry\Test\ResetDatabase; class ProductRepositoryTest extends KernelTestCase { use Factories; use ResetDatabase; // ... } Nope! ResetDatabase automatically drops and recreates your database before your tests begin + recreates your schema before each test!
29. $ php vendor/bin/phpunit @weaverryan
30. Functional Tests Part 5 (without JavaScript) @weaverryan
31. $ php bin/console make:test
32. // tests/Controller/ProductControllerTest.php namespace App\Tests\Controller; use Symfony\Bundle\FrameworkBundle\Test\WebTestCase; class ProductControllerTest extends WebTestCase { public function testSomething(): void { $client = static::createClient(); $crawler = $client->request('GET', '/'); $this->assertResponseIsSuccessful(); $this->assertSelectorTextContains('h1', 'Hello World'); } } WebTestCase === the container is available 
 and you can "browse" the site with a fake, "in php" browser
33. // tests/Controller/ProductControllerTest.php namespace App\Tests\Controller; use Zenstruck\Foundry\Test\Factories; use Zenstruck\Foundry\Test\ResetDatabase; class ProductControllerTest extends WebTestCase { use Factories, ResetDatabase; // ... } @weaverryan
35. public function testEditProduct(): void { $client = static::createClient(); $product = ProductFactory::createOne(['name' => 'Brand new popcorn']); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
36. $ php vendor/bin/phpunit
37. public function testEditProduct(): void { $client = static::createClient(); $product = ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $crawler = $client->request( 'GET', sprintf('/product/%s/edit', $product->getId()) ); $client->submitForm('Update', [ 'product[name]' => 'Slightly old popcorn' ]); $client->followRedirect(); $this->assertResponseIsSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); }
38. $ php vendor/bin/phpunit @weaverryan
39. Functional/E2E Tests Part 6 (testing WITH JavaScript) @weaverryan
40. (I built this with a Stimulus controller in 40 lines including debouncing & "click outside" to close)
41. public function testSearchAutoSuggestion(): void { $client = static::createClient(); ProductFactory::createOne(['name' => 'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
42. $ php vendor/bin/phpunit @weaverryan
43. Hello Panther @weaverryan Panther tests with a "headless" Chrome or Firefox Uses the same API as the normal static::createClient()
44. $ composer require symfony/panther --dev
45. $ composer require --dev dbrekelmans/bdi $ ./vendor/bin/bdi detect drivers
46. public function testSearchAutoSuggestion(): void { $client = static::createPantherClient(); ProductFactory::createOne(['name' => 'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); $client->takeScreenshot(__DIR__.'/../../screenshot.png'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
47. $ php vendor/bin/phpunit @weaverryan
48. But we have a screenshot!!!
49. Black Magic! A) Panther automatically starts a local PHP web server before the test - e.g. http://localhost:9080 B) Panther then communicates to Chrome/Firefox to visit that site (without opening a real browser**) ** there is an option to open a real browser: it's great for debugging a problem
50. But we have 2 Panther Problems @weaverryan 1. Panther isn't using our test database! 2. We need to "wait" until the suggestions box appears Panther launches the web server in a "panther" (it can't use "test" as that has fake session handling)
51. Using our Test Database # .env.panther # duplicate your .env.test config DATABASE_URL="mysql://root@127.0.0.1:3306/testing?serverVersion=5.7" The duplication is not ideal. There is a pull request to DoctrineBundle to make overriding the database name easier for tests https://github.com/doctrine/DoctrineBundle/pull/1290
52. Automatic Screenshots on Failure <!-- phpunit.xml.dist --> <phpunit> <!-- ... --> <extensions> <extension class="Symfony\Component\Panther\ServerExtension" /> </extensions> </phpunit>
53. Waiting for the Search Preview Box public function testSearchAutoSuggestion(): void { // ... $crawler = $client->request('GET', '/'); $searchForm = $crawler->selectButton('Search')->form(); $searchForm->setValues(['q' => 'dis']); // wait for the AJAX to load $client->waitForElementToContain('.search-preview', 'Disk'); $this->assertCount( 2, $crawler->filter('.search-preview .list-group-item') ); }
54. $ php vendor/bin/phpunit https://symfony.com/blog/announcing-symfony-panther-1-0 @weaverryan
55. Hello zenstruck/browser Part 7 <— Yea, this guy again! (wrapper around BrowserKit & Panther)
56. The truth I fi nd DomCrawler's API ( fi nding elements, fi lling out form fi elds, clicking links) … to be a bit clunky @weaverryan
57. $ composer require zenstruck/browser --dev The " fi ne print" zenstruck/browser is currently 0.4 It has not reached a stable 1.0 release yet, But there are plans to do this in the coming months @weaverryan
58. Automatic Screenshots, HTML dump and console.log dump on Failure <!-- phpunit.xml.dist --> <phpunit> <!-- ... --> <extensions> - <extension class="Symfony\Component\Panther\ServerExtension" /> + <extension class="Zenstruck\Browser\Test\BrowserExtension" /> </extensions> </phpunit> @weaverryan
59. Refactor the non-JS test: public function testEditProduct(): void { $product = ProductFactory::createOne(['name' => 'Brand new popcorn']) ->enableAutoRefresh(); $this->browser() ->visit(sprintf('/product/%s/edit', $product->getId())) ->assertSuccessful() ->fillField('Name', 'Slightly old popcorn') ->click('Update') ->followRedirect() ->assertSuccessful(); $this->assertSame('Slightly old popcorn', $product->getName()); } Find & interact with elements by their "text"
60. Refactor the Panther test: public function testSearchAutoSuggestion(): void { ProductFactory::createOne(['name' => 'Floppy Disk']); ProductFactory::createOne(['name' => 'Compact Disk']); $this->pantherBrowser() ->visit('/') ->fillField('q', 'dis') ->waitUntilSeeIn('.search-preview', 'Floppy Disk') ->assertElementCount('.search-preview .list-group-item', 2); } @weaverryan
61. As you fail, browser leaves you clues
62. $browser ->visit('/my/page') ->follow('A link') ->fillField('Name', 'Kevin') ->checkField('Accept Terms') ->selectFieldOption('Type', 'Employee') // single option select ->attachFile('Photo', '/path/to/photo.jpg') ->click('Submit') ->assertSeeIn('h1', 'some text') ->assertNotSeeIn('h1', 'some text') // the following use symfony/var-dumper's dump() function and continue ->dump() // raw response body or array if json ->dump('h1') // html element ->dump('foo') // if json response, array key ->dump('foo.*.baz') // if json response, JMESPath notation can be used ;
63. ParaTest Part 8 (Run your tests in Parallel Processes) @weaverryan
64. $ composer require brianium/paratest $ ./vendor/bin/paratest
65. Unique Database Per Process # .env.test and .env.panther DATABASE_URL="mysql://root@127.0.0.1:3306/testing${TEST_TOKEN}?serverVersion=5.7"
66. The Fine Print! 1. If you write to the fi lesystem, you may need a fi lesystem abstraction (e.g. Flysystem) so you can use di ff erent paths for each test process 2. Panther currently needs a hack to work (otherwise it tries to multiple chrome-driver on the same port. See: https://github.com/symfony/panther/issues/436 @weaverryan
67. So… My 2021 testing stack is… @weaverryan
68. Executing phpunit/phpunit directly @weaverryan
69. zenstruck/foundry for data fi xtures and test data @weaverryan
70. zenstruck/browser for a beautiful functional testing API on top of Panther and BrowserKit @weaverryan
71. And I ❤ it @weaverryan
72. @kbond THANK YOU! @wouterj @nyholm @weaverryan
73. SCREENCASTS Stimulus: https://symfonycasts.com/screencast/stimulus Foundry & Data Fixtures (basics): https://symfonycasts.com/screencast/symfony-doctrine Testing: Upcoming… @weaverryan