#LDNSE 12 : Speedy Web Automation at Huddle

Phil Mullan : Test Engineering Manager at Huddle talk about Web automation strategies at Huddle
Event Details : http://www.meetup.com/seleniumlondon/events/220601189/

Transcript

2. Phil Mullan Test Engineering Manager Speedy Web Automation at Huddle
3. Farewell Internet Explorer!
4. Lay of the Land at Huddle • Legacy Monolithic Application • Released once per month • Requires long regression phase • Lots of Selenium Tests • Few Unit tests • UI coupled to backend
5. Lay of the Land at Huddle (cont…) • New Autonomous Components • Release on demand • Regression == Smoke test • Few Selenium Tests • Lots of Unit/Acceptance tests • UI is a separate component tested with PhantomCSS (more from James Cryer later)
6. Legacy Selenium Suite • Test execution time around 4.5 hours • High test complexity (10+ steps per test) • Intermittent failures due to test executors • Use of data builders coupled to SQL Schema
7. Buying Time for Re-Architecture • 12-18 months from ditching monolith • Re-writing entire test framework too expensive • Need to make existing tests faster • Need to make existing tests more reliable • All with minimal code change / effort
8. The Solution? • Stabilize test execution environment • Introduce Parallelism • Throw machines at the problem
9. Stabilize Test Execution Environment • Build new test execution VMs from scratch • Ensures all pre-requisites are available • Ensures specific versions of browsers installed • Disposable / Rebuildable Executors
10. Introducing Parallelism
11. Introducing Parallelism
12. Introducing Parallelism • Configurable degrees of parallelism • 3 times as fast as before with 5 concurrent processes running tests • Only limitation is machine spec
13. Now Throw Machines at the Problem • Run all test suites at the same time • Run each test fixture in parallel within each test suite • Create more granular test suites to decrease time spent in queue and maximise the load on each test executor • Test execution decreased from 4.5 hours to 15 minutes
14. Throw MORE Machines at the Problem • Build a Selenium Grid • Uses the same mechanism as the execution builders • Benefit from double fan-out parallelism • Balances load from specific tests • Can increase parallelism on test executors as they no longer launch browsers
15. Summary • Internet Explorer is dead to me • Parallelism is paramount in getting fast feedback on failure • You can alleviate short term pain by throwing hardware at a software problem