The new Testing Landscape: Panther, Foundry and More!

Symfony comes with a number of tools for testing, like BrowserKit & DomCrawler as well as test classes for booting the kernel, accessing services, and using a bunch of built-in assertions.

But what do you do if you need to test the JavaScript on your page? And what's the best approach for setting up your database? Should you load fixtures? Clear the data before each test? And how can I run my tests in parallel?

In this talk, we'll explore a set of new tools & clear patterns (Arrange, Act, Assert) for testing, as well as tools like Panther, Foundry, Browser and more!

Transcript

1. The New Testing Landscape
   
   
   Panther, Foundry & More
   by your friend Ryan Weaver
   @weaverryan
   

   View Slide

2. > Member of the Symfony docs team
   

   > Some Dude at
   
   
   SymfonyCasts.com
   > Husband of the talented and
   
   
   beloved @leannapelham
   symfonycasts.com
   
   
   twitter.com/weaverryan
   Yo! I’m Ryan!
   > Father to my much more
   
   
   charming son, Beckett
   

   View Slide

3. Testing:
   
   
   A "From Space" Overview
   Part 1
   @weaverryan
   

   View Slide

4. Test Types
   Unit tests
   Integration tests (i.e. kernel tests)
   Functional tests (i.e. e2e / browser tests)
   Directly test classes
   
   
   Uses mocks, Symfony is not involved at all
   Directly test classes
   
   
   Uses real services from the container
   Test your site's pages/endpoints
   
   
   Uses your fully-functioning Symfony app
   

   View Slide

5. This Presentation:
   PHPUnit Basics
   @weaverryan
   TDD
   Kick-butt testing with Symfony
   

   View Slide

6. @weaverryan
   Full code & step-by-step commits
   
   
   github.com/weaverryan/s
   fl
   ive_2021_testing/commits/
   fi
   nished
   

   View Slide

7. PHPUnit & phpunit-bridge
   Part 2
   @weaverryan
   

   View Slide

8. Installing PHPUnit
   @weaverryan
   $ composer require phpunit --dev
   This installs symfony/test-pack. Today, that includes:
   • symfony/browser-kit
   
   
   • symfony/css-selector
   
   
   • symfony/dom-crawler
   
   
   • symfony/phpunit-bridge
   

   View Slide

9. $ php bin/phpunit
   We execute PHPUnit indirectly through phpunit-bridge.
   
   
   Why?
   

   View Slide

10. @weaverryan
    PHPUnit Bridge gives us:
    A. Deprecations reports after tests
    
    
    B. Clock mocking & other features
    
    
    C. Run tests against multiple PHPUnit versions
    
    
    D. PHPUnit's dependencies are separate from your app's
    But only C and D require you to run phpunit through phpunit-bridge
    

    View Slide

11. Takeaway: Install PHPUnit Directly
    $ composer require phpunit/phpunit --dev
    $ php vendor/bin/phpunit
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    View Slide

12. Recipe will:
    
    
    • install phpunit/phpunit always via the test-pack
    
    
    • Make bin/phpunit use phpunit directly (if available)
    

    View Slide

13. Unit Tests
    Part 3
    @weaverryan
    

    View Slide

14. $ php bin/console make:test
    @weaverryan
    

    View Slide

15. // tests/Util/CalculatorTest.php
    
    
    namespace App\Tests\Util;
    
    
    use App\Util\Calculator;
    
    
    use PHPUnit\Framework\TestCase;
    
    
    class CalculatorTest extends TestCase
    
    
    {
    
    
    public function testAdd(): void
    
    
    {
    
    
    $calculator = new Calculator();
    
    
    $this->assertSame(42, $calculator->add(20, 22));
    
    
    }
    
    
    }
    
    
    @weaverryan
    TestCase === no Symfony
    

    View Slide

16. $ php vendor/bin/phpunit
    @weaverryan
    

    View Slide

17. Integration (Kernel) Tests
    Part 4
    @weaverryan
    

    View Slide

18. $ php bin/console make:test
    

    View Slide

19. // tests/Repository/ProductRepositoryTest.php
    
    
    namespace App\Tests\Repository;
    
    
    use App\Repository\ProductRepository;
    
    
    use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
    
    
    class ProductRepositoryTest extends KernelTestCase
    
    
    {
    
    
    public function testSearch(): void
    
    
    {
    
    
    $kernel = self::bootKernel();
    
    
    $productRepository = self::$container->get(ProductRepository::class);
    
    
    }
    
    
    }
    
    
    KernelTestCase === the container is available
    A)
    B)
    A) Booting makes the container available
    B) self::$container is a special container where all services are public
    

    View Slide

20. But wait!
    How do I "seed" my database before
    making the query?
    @weaverryan
    

    View Slide

21. Hello zenstruck/foundry
    Part 4
    Putting the FoUNdry back into data
    fi
    xtures!
    

    View Slide

22. $ composer require zenstruck/foundry --dev
    $ php bin/console make:factory
    

    View Slide

23. // src/Factory/ProductFactory.php
    
    
    namespace App\Factory;
    
    
    use App\Entity\Product;
    
    
    use Zenstruck\Foundry\ModelFactory;
    
    
    final class ProductFactory extends ModelFactory
    
    
    {
    
    
    protected function getDefaults(): array
    
    
    {
    
    
    return [
    
    
    // TODO add your default values here
    
    
    ];
    
    
    }
    
    
    protected static function getClass(): string
    
    
    {
    
    
    return Product::class;
    
    
    }
    
    
    }
    
    
    @weaverryan
    

    View Slide

24. protected function getDefaults(): array
    
    
    {
    
    
    return [
    
    
    'name' => self::faker()->words(3, true),
    
    
    'description' => self::faker()->paragraph,
    
    
    'brand' => self::faker()->company,
    
    
    'price' => self::faker()->numberBetween(1000, 10000),
    
    
    // will create a random category
    
    
    'category' => CategoryFactory::new(),
    
    
    ];
    
    
    }
    
    
    @weaverryan
    

    View Slide

25. ProductFactory::createOne(['name' => 'floppy disk']);
    
    
    // create 5 all with the same 1 Category
    
    
    ProductFactory::createMany(5, ['category' => CategoryFactory::new()]);
    
    
    // create 4 all using an existing, random Category
    
    
    ProductFactory::createMany(5, ['category' => CategoryFactory::random()]);
    
    
    Now from a test or Doctrine Fixtures class
    @weaverryan
    

    View Slide

26. class ProductRepositoryTest extends KernelTestCase
    
    
    {
    
    
    use Factories;
    
    
    public function testSearch(): void
    
    
    {
    
    
    $kernel = self::bootKernel();
    
    
    $productRepository = self::$container->get(ProductRepository::class);
    
    
    ProductFactory::createOne(['name' => 'floppy disk']);
    
    
    ProductFactory::createOne(['name' => 'popcorn']);
    
    
    ProductFactory::createOne(['description' => 'A CD (compact disk)']);
    
    
    $results = $productRepository->search('disk');
    
    
    $this->assertCount(2, $results);
    
    
    }
    
    
    }
    
    
    @weaverryan
    

    View Slide

27. Setting up the test Database
    @weaverryan
    # .env.test
    
    
    # ...
    
    
    DATABASE_URL="mysql://root@127.0.0.1:3306/app_test?serverVersion=5.7"
    
    
    Now run php bin/console doctrine:database:create --env=test right?
    

    View Slide

28. use Zenstruck\Foundry\Test\Factories;
    
    
    use Zenstruck\Foundry\Test\ResetDatabase;
    
    
    class ProductRepositoryTest extends KernelTestCase
    
    
    {
    
    
    use Factories;
    
    
    use ResetDatabase;
    
    
    // ...
    
    
    }
    
    
    Nope!
    ResetDatabase automatically drops and recreates your
    
    
    database before your tests begin + recreates your schema
    
    
    before each test!
    

    View Slide

29. $ php vendor/bin/phpunit
    @weaverryan
    

    View Slide

30. Functional Tests
    Part 5
    (without JavaScript)
    @weaverryan
    

    View Slide

31. $ php bin/console make:test
    

    View Slide

32. // tests/Controller/ProductControllerTest.php
    
    
    namespace App\Tests\Controller;
    
    
    use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
    
    
    class ProductControllerTest extends WebTestCase
    
    
    {
    
    
    public function testSomething(): void
    
    
    {
    
    
    $client = static::createClient();
    
    
    $crawler = $client->request('GET', '/');
    
    
    $this->assertResponseIsSuccessful();
    
    
    $this->assertSelectorTextContains('h1', 'Hello World');
    
    
    }
    
    
    }
    
    
    WebTestCase === the container is available
    

    and you can "browse" the site
    
    
    with a fake, "in php" browser
    

    View Slide

33. // tests/Controller/ProductControllerTest.php
    
    
    namespace App\Tests\Controller;
    
    
    use Zenstruck\Foundry\Test\Factories;
    
    
    use Zenstruck\Foundry\Test\ResetDatabase;
    
    
    class ProductControllerTest extends WebTestCase
    
    
    {
    
    
    use Factories, ResetDatabase;
    
    
    // ...
    
    
    }
    
    
    @weaverryan
    

    View Slide

34. View Slide

35. public function testEditProduct(): void
    
    
    {
    
    
    $client = static::createClient();
    
    
    $product = ProductFactory::createOne(['name' => 'Brand new popcorn']);
    
    
    $crawler = $client->request(
    
    
    'GET',
    
    
    sprintf('/product/%s/edit', $product->getId())
    
    
    );
    
    
    $client->submitForm('Update', [
    
    
    'product[name]' => 'Slightly old popcorn'
    
    
    ]);
    
    
    $client->followRedirect();
    
    
    $this->assertResponseIsSuccessful();
    
    
    $this->assertSame('Slightly old popcorn', $product->getName());
    
    
    }
    
    
    

    View Slide

36. $ php vendor/bin/phpunit
    

    View Slide

37. public function testEditProduct(): void
    
    
    {
    
    
    $client = static::createClient();
    
    
    $product = ProductFactory::createOne(['name' => 'Brand new popcorn'])
    
    
    ->enableAutoRefresh();
    
    
    $crawler = $client->request(
    
    
    'GET',
    
    
    sprintf('/product/%s/edit', $product->getId())
    
    
    );
    
    
    $client->submitForm('Update', [
    
    
    'product[name]' => 'Slightly old popcorn'
    
    
    ]);
    
    
    $client->followRedirect();
    
    
    $this->assertResponseIsSuccessful();
    
    
    $this->assertSame('Slightly old popcorn', $product->getName());
    
    
    }
    
    
    

    View Slide

38. $ php vendor/bin/phpunit
    @weaverryan
    

    View Slide

39. Functional/E2E Tests
    Part 6
    (testing WITH JavaScript)
    @weaverryan
    

    View Slide

40. (I built this with a Stimulus controller in 40 lines
    including debouncing & "click outside" to close)
    

    View Slide

41. public function testSearchAutoSuggestion(): void
    
    
    {
    
    
    $client = static::createClient();
    
    
    ProductFactory::createOne(['name' => 'Floppy Disk']);
    
    
    ProductFactory::createOne(['name' => 'Compact Disk']);
    
    
    $crawler = $client->request('GET', '/');
    
    
    $searchForm = $crawler->selectButton('Search')->form();
    
    
    $searchForm->setValues(['q' => 'dis']);
    
    
    $this->assertCount(
    
    
    2,
    
    
    $crawler->filter('.search-preview .list-group-item')
    
    
    );
    
    
    }
    
    
    

    View Slide

42. $ php vendor/bin/phpunit
    @weaverryan
    

    View Slide

43. Hello Panther
    @weaverryan
    Panther tests with a "headless" Chrome or Firefox
    Uses the same API as the normal static::createClient()
    

    View Slide

44. $ composer require symfony/panther --dev
    

    View Slide

45. $ composer require --dev dbrekelmans/bdi
    $ ./vendor/bin/bdi detect drivers
    

    View Slide

46. public function testSearchAutoSuggestion(): void
    
    
    {
    
    
    $client = static::createPantherClient();
    
    
    ProductFactory::createOne(['name' => 'Floppy Disk']);
    
    
    ProductFactory::createOne(['name' => 'Compact Disk']);
    
    
    $crawler = $client->request('GET', '/');
    
    
    $searchForm = $crawler->selectButton('Search')->form();
    
    
    $searchForm->setValues(['q' => 'dis']);
    
    
    $client->takeScreenshot(__DIR__.'/../../screenshot.png');
    
    
    $this->assertCount(
    
    
    2,
    
    
    $crawler->filter('.search-preview .list-group-item')
    
    
    );
    
    
    }
    
    
    

    View Slide

47. $ php vendor/bin/phpunit
    @weaverryan
    

    View Slide

48. But we have a screenshot!!!
    

    View Slide

49. Black Magic!
    A) Panther automatically starts a local PHP web server
    
    
    before the test - e.g. http://localhost:9080
    B) Panther then communicates to Chrome/Firefox to visit
    
    
    that site (without opening a real browser**)
    ** there is an option to open a real browser:
    
    
    it's great for debugging a problem
    

    View Slide

50. But we have 2 Panther Problems
    @weaverryan
    1. Panther isn't using our test database!
    2. We need to "wait" until the suggestions box appears
    Panther launches the web server in a "panther"
    
    
    (it can't use "test" as that has fake session handling)
    

    View Slide

51. Using our Test Database
    # .env.panther
    
    
    # duplicate your .env.test config
    
    
    DATABASE_URL="mysql://root@127.0.0.1:3306/testing?serverVersion=5.7"
    The duplication is not ideal. There is a pull request to DoctrineBundle
    
    
    to make overriding the database name easier for tests
    https://github.com/doctrine/DoctrineBundle/pull/1290
    

    View Slide

52. Automatic Screenshots on Failure
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    View Slide

53. Waiting for the Search Preview Box
    public function testSearchAutoSuggestion(): void
    
    
    {
    
    
    // ...
    
    
    $crawler = $client->request('GET', '/');
    
    
    $searchForm = $crawler->selectButton('Search')->form();
    
    
    $searchForm->setValues(['q' => 'dis']);
    
    
    // wait for the AJAX to load
    
    
    $client->waitForElementToContain('.search-preview', 'Disk');
    
    
    $this->assertCount(
    
    
    2,
    
    
    $crawler->filter('.search-preview .list-group-item')
    
    
    );
    
    
    }
    
    
    

    View Slide

54. $ php vendor/bin/phpunit
    https://symfony.com/blog/announcing-symfony-panther-1-0
    @weaverryan
    

    View Slide

55. Hello zenstruck/browser
    Part 7
    <— Yea, this guy again!
    (wrapper around BrowserKit & Panther)
    

    View Slide

56. The truth
    I
    fi
    nd DomCrawler's API
    
    
    (
    fi
    nding elements,
    fi
    lling out form
    fi
    elds, clicking links)
    
    
    … to be a bit clunky
    @weaverryan
    

    View Slide

57. $ composer require zenstruck/browser --dev
    The "
    fi
    ne print"
    
    
    zenstruck/browser is currently 0.4
    
    
    It has not reached a stable 1.0 release yet,
    
    
    But there are plans to do this in the coming months
    @weaverryan
    

    View Slide

58. Automatic Screenshots, HTML dump and
    
    
    console.log dump on Failure
    
    
    
    
    
    
    
    
    
    
    
    
    -
    
    
    +
    
    
    
    
    
    
    
    
    @weaverryan
    

    View Slide

59. Refactor the non-JS test:
    public function testEditProduct(): void
    
    
    {
    
    
    $product = ProductFactory::createOne(['name' => 'Brand new popcorn'])
    
    
    ->enableAutoRefresh();
    
    
    $this->browser()
    
    
    ->visit(sprintf('/product/%s/edit', $product->getId()))
    
    
    ->assertSuccessful()
    
    
    ->fillField('Name', 'Slightly old popcorn')
    
    
    ->click('Update')
    
    
    ->followRedirect()
    
    
    ->assertSuccessful();
    
    
    $this->assertSame('Slightly old popcorn', $product->getName());
    
    
    }
    
    
    Find & interact with
    
    
    elements by their "text"
    

    View Slide

60. Refactor the Panther test:
    public function testSearchAutoSuggestion(): void
    
    
    {
    
    
    ProductFactory::createOne(['name' => 'Floppy Disk']);
    
    
    ProductFactory::createOne(['name' => 'Compact Disk']);
    
    
    $this->pantherBrowser()
    
    
    ->visit('/')
    
    
    ->fillField('q', 'dis')
    
    
    ->waitUntilSeeIn('.search-preview', 'Floppy Disk')
    
    
    ->assertElementCount('.search-preview .list-group-item', 2);
    
    
    }
    
    
    @weaverryan
    

    View Slide

61. As you fail, browser leaves you clues
    

    View Slide

62. $browser
    
    
    ->visit('/my/page')
    
    
    ->follow('A link')
    
    
    ->fillField('Name', 'Kevin')
    
    
    ->checkField('Accept Terms')
    
    
    ->selectFieldOption('Type', 'Employee') // single option select
    
    
    ->attachFile('Photo', '/path/to/photo.jpg')
    
    
    ->click('Submit')
    
    
    ->assertSeeIn('h1', 'some text')
    
    
    ->assertNotSeeIn('h1', 'some text')
    
    
    // the following use symfony/var-dumper's dump() function and continue
    
    
    ->dump() // raw response body or array if json
    
    
    ->dump('h1') // html element
    
    
    ->dump('foo') // if json response, array key
    
    
    ->dump('foo.*.baz') // if json response, JMESPath notation can be used
    
    
    ;
    
    
    

    View Slide

63. ParaTest
    Part 8
    (Run your tests in Parallel Processes)
    @weaverryan
    

    View Slide

64. $ composer require brianium/paratest
    $ ./vendor/bin/paratest
    

    View Slide

65. Unique Database Per Process
    # .env.test and .env.panther
    
    
    DATABASE_URL="mysql://root@127.0.0.1:3306/testing${TEST_TOKEN}?serverVersion=5.7"
    
    
    

    View Slide

66. The Fine Print!
    1. If you write to the
    fi
    lesystem, you may need a
    fi
    lesystem
    
    
    abstraction (e.g. Flysystem) so you can use di
    ff
    erent
    
    
    paths for each test process
    2. Panther currently needs a hack to work (otherwise it tries to
    
    
    multiple chrome-driver on the same port.
    
    
    See: https://github.com/symfony/panther/issues/436
    @weaverryan
    

    View Slide

67. So…
    
    
    My 2021 testing stack is…
    @weaverryan
    

    View Slide

68. Executing
    
    
    phpunit/phpunit
    
    
    directly
    @weaverryan
    

    View Slide

69. zenstruck/foundry
    
    
    for data
    fi
    xtures
    
    
    and test data
    @weaverryan
    

    View Slide

70. zenstruck/browser
    
    
    for a beautiful functional
    testing API on top of
    Panther and BrowserKit
    @weaverryan
    

    View Slide

71. And I ❤ it
    @weaverryan
    

    View Slide

72. @kbond
    THANK YOU!
    @wouterj @nyholm
    @weaverryan
    

    View Slide

73. SCREENCASTS
    Stimulus:
    
    
    https://symfonycasts.com/screencast/stimulus
    
    
    Foundry & Data Fixtures (basics):
    
    
    https://symfonycasts.com/screencast/symfony-doctrine
    
    
    Testing:
    
    
    Upcoming…
    @weaverryan
    

    View Slide