Test Automation for Packaged Systems: Yes You Can!

Presented at STAR East, Orlando, 2013

Transcript

1. Test  Automa+on   For  Packaged   Systems: Yes  You  Can! Chris  Bushell Lead  Consultant,  ThoughtWorks
4. What  I  do • Developer/Tech  Lead/Architect/ Agile  coach • Custom  so8ware  development • Focus  on  building  the  right  thing • Focus  on  high  quality • Test  driven
5. How  I  do  it  –  TDD/BDD
6. How  I  do  it  –  Revision  Control
7. How  I  do  it  –  Con+nuous   Integra+on
8. How  I  do  it  –  Open  Source JUnit • Light-­‐weight,  powerful • Extensible • Excellent  community • Examples:  xUnit,  Selenium/ WebDriver,  RSpec,  Concordion,   Cucumber  etc.
9. November  2011
10. In  the  beginning....
11. The  COTS  Con! Highly extensible software “configured” in powerful development environments, comparable in complexity, and susceptibility to defects as anything in the custom software development world
12. The  land  that  +me  forgot
13. Complete  lack  of  test   automa+on “Computers  are  designed  to  do  simple  repeKKve   tasks.  The  second  you  have  humans  doing   repeKKve  tasks,  all  the  computers  get  together   late  at  night  and  laugh  at  you” “Every  Kme  you  do  something  for  the  third  Kme,   automate  it.  You’ll  be  doing  it  a  million  Kmes” -­‐  Neal  Ford
14. Surely  someone  must  be  doing   this  already? • Commercial  tools  exist,  but  tend  to  be  a  poor  fit  for   Agile  environments -­‐ TesKng  viewed  as  a  luxury  (an  add-­‐on) -­‐ Available  to  few  ($$$$  licensing) -­‐ Discourages  collecKve  ownership -­‐ Difficult  to  use  good  engineering  pracKces -­‐ Difficult  to  integrate  with  ConKnuous  IntegraKon   process  (lack  of  scriptability/headless  mode)
15. Lessons  learned  -­‐  what  worked   and  what  didn’t
16. Oracle  Siebel  CRM
17. But  first,  browser  automa+on   basics • HTML  +  DOM • Selectors -­‐  Id,  class,  xpath • AutomaKon  API -­‐ e.g.  Selenium,  WebDriver,  Sahi,  WaKr  etc.
18. Our  biggest  problem • AcKveX • <object> • Impervious  to   tradiKonal   automaKon   toolset
19. All  is  not  lost • Not  all  parts  of  the  applicaKon  used  AcKveX • Some  controls  were  regular  HTML  (albeit  machine   generated) -­‐ WebDriver • Perhaps  we  could  use  some  kind  of  hybrid  approach   to  test  automaKon? -­‐ WebDriver  +  X,  Y  or  Z?
20. Introducing  Sikuli  -­‐  automa+on   using  images
21. An  example  using  Google
22. package com.tw.xconf; import org.sikuli.script.FindFailed; import org.sikuli.script.Screen; import org.sikuli.script.Settings; public class Demo { public static void main(String[] args) { Screen screen = new Screen(); try { screen.click("images/criteria.png"); screen.type("ThoughtWorks"); screen.click("images/search.png"); } catch (FindFailed e) { e.printStackTrace(); } } }
23. Drawbacks  of  Sikuli • Difficult  to  manage  large  numbers  of  images • Can  be  confused  when  different  regions  of  the  screen   look  the  same • Tests  easily  broken  by  small  changes  in  the  UI • ComparaKvely  slow  to  execute  
24. The  story  so  far.... • Ability  to  automate  the  UI -­‐ WebDriver  for  the  regular  HTML -­‐ Sikuli  for  the  AcKveX  controls
25. The  last  hurdle • How  do  we  extract  values  from  the  page  so  we  can   perform  asserKons  on  them?
26. Tried  but  failed • OpKcal  Character  RecogniKon  (OCR) -­‐ Tesseract • White • AutoIt • Win32-­‐Autogui
27. Plan  B •  Oracle  ApplicaKon  TesKng  Suite  (OATS) -­‐Record  +  playback -­‐Proprietary  test  automaKon  API
28. Test  automa+on  is  soYware   development • Code  duplicaKon  is  bad,  resulKng  in  fragile  test  suites -­‐ Small  changes  can  break  many  tests  which  requires   significant  maintenance -­‐ Frequently  see  test  automaKon  abandoned  as  a   result  of  this • Record  +  playback  tools  make  it  difficult  to  introduce   abstracKons  to  remove  duplicaKon
29. Oracle  Applica+on  Tes+ng  Suite • A  process  of  learning  through  experimentaKon • Generates  Java  code  under  the  covers  (for  both  Seibel   and  HTML  automaKon) -­‐ Liberate  the  API  from  tooling • Creates  executable  JWG  files  as  test  arKfacts -­‐ Understand  the  format  of  these  files  so  we  can   create  them  programmaKcally
30. Gains •  Ability  to  more  easily  use  good  programming   pracKces  (abstracKons,  code  refactoring  etc) • Ability  to  integrate  test  arKfacts  with  ConKnuous   IntegraKon  process • A  maintainable  test  suite  for  an  Agile  team
32. Remedy  +cket   management   system
33. Great  news! • A  more  tradiKonal  webapp  (albeit  with  many  quirks) • AutomaKon  possible  with  familiar  toolset   (WebDriver,  in  our  case)  
34. Example  quirk  #1:  windows
35. Solu+on:  model  the  windows  a   stack • WindowManagingWebDriver • Allowed  us  to  keep  tests  focused • Also  responsible  for  maximizing  browser  windows   (quirk  1.1)
36. Example  quirk  #2:  mouse   ac+ons  alone  not  enough                public  void  navigateToRelationshipsTab()  {          findElement(RELATIONSHIPS).sendKeys(Keys.ENTER);          findElement(RELATIONSHIPS).click();        }
37. The  importance  of  abstrac+ons • Page  model • Domain  objects  e.g.  Incident • Builders  for  domain  objects • Page  level  gekers/sekers  e.g.  IncidentSeker,   IncidentGeker • Workflows  to  model  sequences  of  interacKons
38. Data  management
39. Data  management
40. Solu+on:  APIs  (again) • AcKon  Request  System  (ARS)  API  allows  creaKon  of   state  e.g.  Incidents  programmaKcally • Simple  key/value  interface • Gave  us  the  ability  to  replace  the  use  of  some  of  our   Workflows  with  faster  running  API  based  Workflows • Objects  created  by  API  based  workflows  could  be   looked  up  in  the  browser
41. The  story  so  far.... • Ability  to  automate  the  UI  using  WebDriver • AbstracKons  in  place  for  robust  tesKng • Ability  to  setup  state  using  fast  API • Growing  regression  test  suite
42. Reverse  Con+nuous  Integra+on
43. Summary • COTS  test  automaKon  is  possible! -­‐ Be  prepared  to  find  challenges/quirks  along  the  way • ConKnuous  IntegraKon  is  possible • APIs  are  your  friends  -­‐  seek  them  out • AbstracKons  are  crucial,  especially  with  machine   generated  code  and  environmental  differences
44. Ques+ons
45. Email:  chris.bushell@thoughtworks.com Twi`er:  @chrisbushelloz Github:  www.github.com/cbushell My  talks:  www.speakerdeck.com/cbushell LinkedIn:  www.linkedin.com/in/chrisbushell ancisco, California 94104 6 F: +1 415 986 2964 ancisco, California 94104 6 F: +1 415 986 2964 315 Montgomery Street, 16th Floor San Francisco, California 94104 T: +1 415 273 1389 C: +1 415 568 8186 F: +1 415 986 2964 E: chris.bushell@thoughtworks.com 315 Montgomery Street, 16th Floor San Francisco, California 94104 T: +1 415 273 1389 C: +1 415 568 8186 F: +1 415 986 2964 E: chris.bushell@thoughtworks.com 315 Montgomery Street, 16th Floo T: +1 415 273 1389 C: +1 415 E: chris.bushell@thoughtworks.com 315 Montgomery Street, 16th Floo T: +1 415 273 1389 C: +1 415 E: chris.bushell@thoughtworks.com Chris Bushell Lead Consultant Chris Bushell Lead Consultant Chris Bushell Chris Bushell Lead Consultant Chris Bushell Lead Consultant Chris Bushell
46. Credits • Ross  Macbeth,  J3474  :  Old  night  safe,  Belfast,  licensed   under  CreaKve  Commons,  hkp://www.geograph.ie/ photo/2261863 • Alan  Chia,  A pile of Lego blocks, of assorted colours and sizes, licensed  under  CreaKve   Commons,  hkp://en.wikipedia.org/wiki/ File:Lego_Color_Bricks.jpg