Automation Isn't All Shiny Toys

Transcript

1. Automation Isn’t All Shiny Toys Thursday, October 6, 11

2. Jim Holmes @aJimHolmes [email protected] http:/ /blogs.telerik.com/JimHolmes Thursday, October 6, 11

3. Automation two weeks in: Thursday, October 6, 11

4. “UNICORNS and PUPPIES!” Thursday, October 6, 11

5. Automation two months in: Thursday, October 6, 11

6. “Wow, this is harder than we thought.” Thursday, October 6,

   11

7. Automation six months in: Thursday, October 6, 11

8. “All of a sudden those 3,000 manual test cases aren’t

   looking so bad.” Thursday, October 6, 11

9. What happens? Thursday, October 6, 11

10. Automation fails for two or three main reasons Thursday, October

    6, 11

11. #1 Tests take too long - no short feedback loop

    Thursday, October 6, 11

12. #2 & #3: Brittle tests, too hard to maintain Thursday,

    October 6, 11

13. What to do? Thursday, October 6, 11

14. Some planning required Thursday, October 6, 11

15. A) What’s the size of the effort? Small, medium, large

    Thursday, October 6, 11

16. Medium & Large efforts need extra care Thursday, October 6,

    11

17. Set expectations Thursday, October 6, 11

18. You’ll need people, hardware, and time Thursday, October 6, 11

19. Problem #1: Long Running Test Suites Thursday, October 6, 11

20. 800 tests * 30 sec/test == 24000 secs (400 mins

    or SIX HOURS) Thursday, October 6, 11

21. Why? Thursday, October 6, 11

22. You’re using the browser too much Thursday, October 6, 11

23. Nav to app Log on Validate logged on Click New

    Contact Enter valid data Click Submit Verify Saved Perform Test Open new contact 30 seconds Thursday, October 6, 11

24. Setup in Browser == #FAIL Thursday, October 6, 11

25. Tip: Plan ahead. Create a backing API or framework Thursday,

    October 6, 11

26. Nav to app Log on Validate logged on Click New

    Contact Enter valid data Click Submit Verify Saved Perform Test Open new contact Helper: Create User 20 seconds Thursday, October 6, 11

27. Can you bypass steps? Thursday, October 6, 11

28. Tip: Cut all nav where possible Thursday, October 6, 11

29. Nav to app Log on Validate logged on Click New

    Contact Enter valid data Click Submit Verify Saved Perform Test Open new contact Helper: Create User Bypass logon 15 seconds Thursday, October 6, 11

30. Nav to app Log on Validate logged on Click New

    Contact Enter valid data Click Submit Verify Saved Perform Test Open new contact Helper: Create User Bypass logon 10 seconds Nav direct to form Thursday, October 6, 11

31. You’re testing TOO MUCH! Thursday, October 6, 11

32. (Automating too much , at least) Thursday, October 6, 11

33. Focus on value for your automation Thursday, October 6, 11

34. Skip fluff Thursday, October 6, 11

35. Tip: Focus on your high-value tests (CRUD, security, main functionality)

    Thursday, October 6, 11

36. Low value == style images alignment layout colors Thursday, October

    6, 11

37. Perform Test Helper: Create User Bypass logon 12 secs to

    run How long to write? Nav direct to form Plus Visual Checks Thursday, October 6, 11

38. Tip: Use manual tests for visual comparisons Thursday, October 6,

    11

39. Tip: Use partial sets (BVTs) Thursday, October 6, 11

40. Infrastructure matters Thursday, October 6, 11

41. Get appropriate hardware at the start Thursday, October 6, 11

42. Sale out Thursday, October 6, 11

43. (and do it early) Thursday, October 6, 11

44. Thursday, October 6, 11

45. Thursday, October 6, 11

46. Problems #2a or #3: Test Maintenance Thursday, October 6, 11

47. Locators, Locators, Locators Thursday, October 6, 11

48. Define them once, and only once Thursday, October 6, 11

49. Tip: use Page Objects pattern or an element repository Thursday,

    October 6, 11

50. Pick your locators carefully! Thursday, October 6, 11

51. (XPath kills kittens, folks) Thursday, October 6, 11

52. Tip: Get friendly with your UI devs Thursday, October 6,

    11

53. Treat your test code like production code Thursday, October 6,

    11

54. (BECAUSE IT IS!) Thursday, October 6, 11

55. Tip: Use the DRY principle Thursday, October 6, 11

56. Tip: Refactor your tests & infrastructure Thursday, October 6, 11

57. Tip: Get your devs to help review your tests Thursday,

    October 6, 11

58. Don’t be afraid to throw out a bad sub-optimal approach

    Thursday, October 6, 11

59. Spike approaches Thursday, October 6, 11

60. CONSTANT review of your automation efforts Thursday, October 6, 11

61. Wrapping Up Thursday, October 6, 11

62. Automation is debt Thursday, October 6, 11

63. It’s also value Thursday, October 6, 11

64. (a LOT of value!) Thursday, October 6, 11

65. Choose your battles carefully Thursday, October 6, 11

66. Keep run times as short as possible Thursday, October 6,

    11

67. Fight complexity Thursday, October 6, 11

68. Jim Holmes @aJimHolmes [email protected] http:/ /blogs.telerik.com/JimHolmes Thursday, October 6, 11