Evolution of Test Automation

Transcript

1. Evolution of Test Automation Travel through ages by Dmitry Buzdin
2. About Myself • Software Engineer with 10 years of experience • Close relations with test automation • Test Automation Framework in Tieto • Test Automation Framework in Ericsson
3. Evolution of Testing From testers and developers point of view
4. Dinosaur Age ~1990 - 1999
5. Waterfall Approach
6. Typical QA department
7. State of Testing • Most of the testing is manual • First custom scripts for automation • Running automated tests manually… • Release cycle 6 months
8. Very few open source testing tools
9. GUI Testing • Commercial automation tools • Record and playback mostly • Testing on Windows 95/98
10. Custom Scripting Languages • “Even non-programmers understand” • Works only inside commercial tool
11. Something Like This http://lolcode.org/ HAI CAN HAS STDIO? PLZ OPEN FILE "LOLCATS.TXT"? AWSUM THX VISIBLE FILE O NOES VISIBLE "ERROR!" KTHXBYE
12. Summary • Most of test automations projects fail • Test automation is not working
13. Developers and Testing
14. Iron Age 1999 - 2004
15. 1999 2002 Agile Revolution
16. Agile Methodologies • Small teams • Cross-functional • Short development cycles • Release cycle ~ 3 months
17. Apache JMeter 2001 JUnit.org 2000
18. State of Testing • Some automation achieved for core test cases • Mostly commercial tools for testing
19. Developers and Testing • Developers are writing their first tests • TDD is invented! • But covers unit testing only • Developers and QA are not sharing tests
20. Test-Driven Development
21. Developers and Testing
22. Middle Ages 2004 - 2008
23. The Webolution • Everything is a Web app now • Ajax • Browser wars
24. New Tools Selenium 2004 First functional open source tools, which everyone uses FitNesse 2005
25. Continuous Integration 2007 Multiple integrations per day
26. Continuous Integration • Continuous Integration becomes popular • Tests are are executed every night • Release cycle - one sprint
27. State of Testing • Moderate test automation?
28. Developers and Testing • Developers learned integration testing • Talking to QA from time to time • Using mock frameworks
29. Age of Enlightenment 2008 - 2014
31. Agile is the only option!
32. GitHub 2008 2008 Open Source Everywhere! Ruby
33. Mobile Revolution How do you even test this? …
34. Continuous Delivery
35. DevOps Tooling
36. State of Testing • Tests are executed after every commit • Release cycle - one week or less
37. Developers and Testing • Everyone writes automated tests • QA and development working together
38. Cyberpunk Age 2015 - now
39. Age of the Cloud
40. Open Source • Mix and match interoperable tools • New testing tool every month • Open source Testing middleware
41. State of Testing • End-to-end delivery pipeline automation • No time for manual testing • Release cycle - less than one day
42. Testing Architecture • Testing is behind development for about 5 years • We already experienced this change in Dev • We are working on making Test change happen
43. Test Automation At Scale Test Management System Test Data System Test Execution System Test Reporting System Test Environment Manager Testing Cloud Test Scheduling System Requirement Management System Defect Tracking System
44. Developers and Testers
45. Summary • Embrace the change • Pushing the limit of test automation • Testing is more complicated than ever