We’re bad at web integration testing

Video available at http://www.youtube.com/watch?v=fLP3NKUsx3k

Lightning introduction to https://speakerdeck.com/mattisg/state-of-web-integration-testing

Transcript

1. We’re bad at Matti Schneider @matti_sg // #dotJS web integration testing Good afternoon. I will be talking —briefly— about web integration testing.
2. web integration testing That’s what you do when you start a browser, interact with your web application like users would, and check the resulting presented information against some baseline. And that’s what I think we, as a community, are currently rather bad at automating.
3. I guess many of you have worked with Selenium, the web automation framework. It works well but it is slow and has an ugly API.
4. Java Ruby .NET Node.js PHP Safari driver IE driver Chrome driver Firefox driver Opera driver Watir Watij Watin Intern Mink The worst thing, in my opinion, is the complexity and fragility of its stack. The reliability of your tests depends largely on the environment of execution, and the rate at which browsers update.
5. PhantomJS In the JavaScript community, PhantomJS is often considered the successor to Selenium. It is indeed faster than Selenium, and has a nicer API to work with if you’re used to Javascript. But it also makes us lose critical things along the way. The most important one is probably that we’re not testing what our users see anymore. Phantom: specific WebKit version, specific JS engine. None of your users are going to access your application through a browser that has no GUI.
6. PhantomJS TrifleJS SlimerJS The worst thing is probably that we are not even going to simplify the stack if we want to do cross-browser testing, as recent developments have shown. At some point, we should admit that each browser, each rendering engine needs its own strategy for automation. Now, I don’t mean to diminish any of these tools. They do a great job. But they all have a common weakness when used to do integration testing.
7. Old web They are made for the old web. The web of websites, not the one of web applications. If you're developing a webapp today, you think in terms of components and binding them through APIs. But our testing tools still make us write code that walks the DOM and checks HTML attributes! How are we supposed to enjoy writing tests when the code feels like what we wrote five years ago? How are we supposed to write maintainable tests when they have no domain knowledge? So, today, I'd like to put forward a proposal on the future of web integration testing.
8. Component Fixture • — • — • — • — Scenario What about leveraging our know-how in components and injecting it in our testing code? You can view this as a “human API specification”, just like components are expected to have unit tests to check their developer API. We would then describe how we expect our users to interact with these elements through scenarios of usage that would drive the automation. Finally, we would isolate this logic from the data we are using and checking against. This would allow black-box testing of web applications, even when they are made of the combination of several third-party components with specific code, just like we do for other levels of testing. The end vision would be that every public Web Component includes its automation description. This idea is of course not entirely new. But I have something to offer to the community today.
9. Component Fixture • — • — • — • — Scenario github.com/MattiSG/Watai Here is the URL to a tool I’ve been refining and using in production for over a year. So, if you’re interested in improving the quality of the web, have a look at this tool, and let’s find out where we want to go, together. Thanks for your attention.