Agile Test Automation

Prepared for large Australian Telco client

Transcript

1. Agile  Test  Automa.on   ThoughtWorks  
2. Waterfall  So5ware  Delivery   Planning   Analysis   Design   Code   Test   Deploy  
3. Planning   Analysis   Design   Code   Test   Deploy   Cost  of  Change   $ Time   Barry  Boehm:  “….the  average  cost  to  address  a  defect  rises  exponen.ally  the   longer  it  takes  you  to  find  it”  
4. Feedback  Cycles   •  How  long  does  it  take  to  validate  change?   –  Have  we  built  something  to  specifica.on?   –  Have  we  built  something  free  from  defects?   –  Have  we  built  something  which  is  relevant  to  our  users?       Agile  techniques  reduce  feedback  cycles  from   months  and  weeks  to  days,  hours  and  minutes      
5. Agile  Tes.ng  Prac.ces   •  Team  collabora.on   •  Test  automa.on   •  Con.nuous  Integra.on   •  Test  Driven  Development   – Acceptance  tests   – Unit  tests  
6. Agile  Team  Collabora.on   QA
7. Agile  Team  Collabora.on   QA DEV BA CUSTOMER BeQer  domain  understanding   BeQer  acceptance  tests  
8. Agile  Team  Collabora.on   QA DEV BA CUSTOMER BeQer  Test  Automa.on  code   BeQer  Unit  Tests  
9. Agile  Team  Collabora.on   QA DEV BA CUSTOMER BeQer  domain  understanding   BeQer  acceptance  tests   BeQer  end-­‐to-­‐end  tests  
10. Rethinking  The  QA  Role   Tradi+onal  Tester  Role   •  Separate  Test  Team   •  Tes.ng  happens  at  the  end  of   development     •  Testers  work  alone   •  Testers  act  as  gatekeepers   •  No  or  liQle  contact  with   business   •  Tests  wriQen  and  automated   a5er  development   Agile  QA  Role   •  Part  of  an  en.re  team   •  Tes.ng  happens  parallel  to   development   •  Testers  pair  with  BAs,  Devs   and  other  testers   •  Testers  highlight  risk   •  Direct  contact  with  Business   •  Tests  wriQen  and  automated   before  and  during   development  
11. Test  Automa.on   •  Myth:  we  should  automate  everything   •  Reality:  Value  based  tes.ng   – How  heavily  used  is  this  func.onality?   – Is  this  func.onality  undergoing  change?   – What’s  the  cost  to  automate  this  test?   – What’s  the  risk  if  this  func.onality  breaks?   – What’s  the  value  of  this  test?  
12. Myth:  Test  Automa.on  Is  Not   So5ware  Development   Technical  skill/ap.tude/interest   Non  technical  skill/ap.tude/interest   Developer   High   Low   High   Automa.on  tester   Business  analyst  
13. Con.nuous  Integra.on   •  Every  change  to  so5ware  has  the  poten.al  to   introduce  new  defect(s)   •  The  earlier  we  detect  a  defect  the  cheaper  it  is   to  fix  
14. Con.nuous  Integra.on   •  Mul.ple  streams  of  work  all  share  a  common   versioned  code  repository   •  Developers  commit  small  changes  to  this   repository  o5en   •  Each  commit  triggers  the  test  suite  run  against   the  new  version  of  the  so5ware   •  Effort  immediately  switches  to  fixing  any   failing  test(s)  
15. Con.nuous  Integra.on   Test  suite  is  run  several  .mes  each  day,  and   probably  or  hundreds  or  thousands  of  .mes   over  the  course  of  a  project      
16. The  Agile  Manifesto   “….we  have  come  to  value….   Individuals  and  interac.ons  over  processes  and   tools….”  
17. Characteris.cs  of  Agile  Test  Tools   Characteris+c   Benefit   Supports  star.ng  test  automa.on  effort   immediately   Fast  feedback   Separate  test  intent  from  implementa.on   details   Allows  for  beQer  collabora.on  from   technical  and  non-­‐technical  team   members   Support  and  encourage  good   programming  prac.ces  for  the  test  code   Tests  are  cheaper  to  maintain   Support  wri.ng  test  automa.on  code   using  real  languages  and  IDEs   Tests  are  cheaper  to  maintain  and  more   powerful   Fosters  collabora.on   Lowers  risk  and  ensures  the  right  thing  is   built   Easily  integrated  with  Con.nuous   Integra.on  process   Fast  feedback   Extensible   To  handle  frequent  change  and  new   scenarios  
18. Tool  Selec.on   Test  Inten.on   DSL   Test  Runner   Plain  English  Test   Specifica.on   Re-­‐usable  automa.on   components     Technical  implementa.on  of   test  components   Execute  and  report  on  test   results   Framework   Concordion   Java   jUnit   Technology   specific   Tool   Purpose  
19. Concordion   •  For  wri.ng  acceptance  tests  in  human  readable   form  and  integra.ng  with  automa.on   –  Versions  available  for  Java,  .Net,  Ruby,  Python,  Scala   •  Allows  for  highly  readable  tests   –  Tests  double  up  as  system  documenta.on   •  Extremely  well  suited  to  cross  team  collabora.on    
20. WebDriver   •  Programma.c  control  of  web  browser   – Internet  Explorer,  Firefox,  Chrome,  Safari,  Opera,   iPhone,  Android   •  Easily  extended   – To  handle  popups,  for  example  
21. Jenkins   •  Con.nuous  Integra.on   •  Scheduled  runs  of  en.re  test  suite   – On  demand     – On  change  of  applica.on   – At  a  set  interval   •  Repor.ng  dashboard  
22. Why  Test  Automa.on  Fails   •  Automa.on  is  not  treated  as  so5ware  development   •  Unsuitable  tool  selec.on   –  Available  only  to  a  select  few   –  Produce  briQle  tests   –  Not  extensible   –  Unsuitable  for  Con.nuous  Integra.on   •  Sub  op.mal  development  prac.ces   •  Inappropriate  es.ma.on  and  resourcing   •  Over  engineering  of  automa.on  code   •  Trying  to  test  everything  with  automa.on  
23. Q&A