Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web scraping with Symfony Panther

Web scraping with Symfony Panther

Talk presented at Darkmira Tour PHP 2019

Tools like Guzzle and DomCrawler make it easy to create bots to navigate and get data from other systems in pure HTML, but the web has evolved and we have SPAs built with Javascript and executed client-side. Interacting with alerts and other dynamic elements has become a problem for bots.

This presentation features Symfony Panther, e2e test tool and web crawler with support for running Javascript and interacting with visual elements, simulating a user with extreme fidelity.


June 09, 2019

More Decks by raphaeldealmeida

Other Decks in Technology


  1. References • https://github.com/symfony/panther • https://martinfowler.com/bliki/PageObject.html • https://symfony.com/doc/current/components/dom_crawler.html • http://docs.guzzlephp.org/en/stable/ •

    http://wttr.in/sao%20paulo • https://developers.google.com/web/updates/2017/04/headless-chrome • https://vue-pokemon-memory-game.vinicius73.dev/ • https://mmonit.com/monit/ • https://github.com/Seldaek/monolog • https://pt.wikipedia.org/wiki/Willis_Carrier