Behat from zero to hero - A practical guide to symfony integration and usage

Starting from behat introduction, we have explained some good practices for BDD testing in a symfony software. This presentation should be a quick "guide" for users that starts with Behat and also for those who want to fix their tests problems.

Transcript

1. Behat from zero to hero “A practical guide to symfony integration and usage” Matteo Moretti - Samuele Lilli @SymfonyDayIt - 20 Ottobre 2017 - Milano
2. Matteo Moretti | Samuele Lilli WHO ARE WE? Matteo Moretti Samuele Lilli
3. Matteo Moretti | Samuele Lilli OUR EXPERIENCE
4. Matteo Moretti | Samuele Lilli OUR EXPERIENCE ON nuvola.madisoft.it https://symfony.com/showcase/191
5. Matteo Moretti | Samuele Lilli BEHAVIOUR DRIVEN DEVELOPMENT
6. Matteo Moretti | Samuele Lilli BEHAT
7. Matteo Moretti | Samuele Lilli A LONG TIME AGO ...
8. Matteo Moretti | Samuele Lilli
9. Matteo Moretti | Samuele Lilli
10. Matteo Moretti | Samuele Lilli TESTS ARE GREEN!
12. Matteo Moretti | Samuele Lilli BUT, SUDDENLY...
13. Matteo Moretti | Samuele Lilli AS SOON AS YOU ADD MORE TESTS...
14. Matteo Moretti | Samuele Lilli HARD TO MAINTAIN
15. Matteo Moretti | Samuele Lilli RANDOM FAILURES
16. Matteo Moretti | Samuele Lilli SLOW TESTS
18. Matteo Moretti | Samuele Lilli “WHY THE HELL THIS TEST IS BROKEN? I DIDN’T MODIFY THAT CODE”
19. Matteo Moretti | Samuele Lilli “MMM, 10 MINUTES TO ADD A FEATURE. 2 HOURS TO FIX TESTS.”
21. Matteo Moretti | Samuele Lilli
22. Matteo Moretti | Samuele Lilli
23. Matteo Moretti | Samuele Lilli WHAT IS BEHAT?
24. Matteo Moretti | Samuele Lilli IT’S A BDD TOOL!
25. Matteo Moretti | Samuele Lilli IT’S A STORY BDD TOOL!
26. Matteo Moretti | Samuele Lilli WHAT IS A STORY?
27. Matteo Moretti | Samuele Lilli HI! I’M A STORY!
28. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
29. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
30. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
31. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
32. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
33. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
34. Matteo Moretti | Samuele Lilli EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT ACCEPTANCE TEST DOCUMENTATION THAT NEVER GOES OUT TO DATE WRITTEN IN GHERKIN EXECUTABLE SPECIFICATION A STORY IS ...
35. Matteo Moretti | Samuele Lilli GHERKIN
36. Matteo Moretti | Samuele Lilli IT’S A LANGUAGE!
37. Matteo Moretti | Samuele Lilli IT’S A BUSINESS READABLE DOMAIN SPECIFIC LANGUAGE!
39. Matteo Moretti | Samuele Lilli LIKE YAML INDENTATION RULES PARSING
40. Matteo Moretti | Samuele Lilli STORIES ARE ORGANIZED IN FEATURES FILES (.feature)
41. Matteo Moretti | Samuele Lilli GIVEN - WHEN - THEN (AND, BUT) ARE KEYWORDS THAT MATCH PHP CODE AND EXECUTE IT. STEPS
42. Matteo Moretti | Samuele Lilli
44. Matteo Moretti | Samuele Lilli REGEX
45. Matteo Moretti | Samuele Lilli
46. Matteo Moretti | Samuele Lilli
47. Matteo Moretti | Samuele Lilli
48. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
49. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
50. Matteo Moretti | Samuele Lilli HOW A GOOD STORY LOOKS LIKE?
51. Matteo Moretti | Samuele Lilli FOCUS ON OUTPUT NOT INPUT X
52. Matteo Moretti | Samuele Lilli √ FOCUS ON OUTPUT NOT INPUT
53. Matteo Moretti | Samuele Lilli DON’T USE UI REFERENCES X
54. Matteo Moretti | Samuele Lilli DON’T USE UI REFERENCES √
55. Matteo Moretti | Samuele Lilli USE BACKGROUND
56. Matteo Moretti | Samuele Lilli USE TRANSFORMERS
57. Matteo Moretti | Samuele Lilli USE TRANSFORMERS
58. Matteo Moretti | Samuele Lilli COLLAPSE STEPS AS MUCH AS POSSIBLE UNLESS THESE GIVES YOU BUSINESS/STORY VALUE
59. Matteo Moretti | Samuele Lilli
60. Matteo Moretti | Samuele Lilli HOW MUCH DETAILS?
61. Matteo Moretti | Samuele Lilli “What many people don’t realize is that different levels of details are appropriate for different scenarios in the same system - sometimes in the same feature - depending on what it is they’re describing” (The cucumber book - Matt Wynne, Aslak Hellesoy)
62. Matteo Moretti | Samuele Lilli BEHAT → GHERKIN → FEATURES → STORIES (SCENARIOS) → STEPS → CONTEXTS
63. Matteo Moretti | Samuele Lilli BEHAT → GHERKIN → FEATURES → STORIES (SCENARIOS) → STEPS → CONTEXTS
64. Matteo Moretti | Samuele Lilli DIVIDE ET IMPERA
65. Matteo Moretti | Samuele Lilli ONE FEATURE PER .FEATURE FILE
66. Matteo Moretti | Samuele Lilli DIVIDE FEATURES AND CONTEXTS BY DOMAIN SUBJECT
67. Matteo Moretti | Samuele Lilli DIVIDE FEATURES AND CONTEXTS BY ACTIONS TO BE PERFORMED ON DOMAIN SUBJECTS
68. Matteo Moretti | Samuele Lilli
69. Matteo Moretti | Samuele Lilli
70. Matteo Moretti | Samuele Lilli UI
71. Matteo Moretti | Samuele Lilli HEADLESS BROWSER EMULATOR VS BROWSER CONTROLLER
72. Matteo Moretti | Samuele Lilli HEADLESS BROWSER EMULATOR • pure HTTP spec implementations • very simple to run and configure • no need of a real browser: speed • usually no JS/AJAX support
73. Matteo Moretti | Samuele Lilli BROWSER CONTROLLER • aim to control a real browser • complex to configure • slow • JS/AJAX support
75. Matteo Moretti | Samuele Lilli YOU MAY NEED BOTH OF THEM
76. Matteo Moretti | Samuele Lilli MINK http://mink.behat.org/en/latest/
77. Matteo Moretti | Samuele Lilli BEHAT PARSES SCENARIOS + MINK BROWSES THE WEB
79. Matteo Moretti | Samuele Lilli
80. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
81. Matteo Moretti | Samuele Lilli HOW TO USE MINK FROM CONTEXT CLASSES?
82. Matteo Moretti | Samuele Lilli PAGE OBJECT PATTERN https://martinfowler.com/bliki/PageObject.html
85. Matteo Moretti | Samuele Lilli https://github.com/sensiolabs/BehatPageObjectExtension BEHAT PAGE OBJECT EXTENSION
86. Matteo Moretti | Samuele Lilli ASSERTIONS
88. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
89. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
90. Matteo Moretti | Samuele Lilli AJAX
91. Matteo Moretti | Samuele Lilli DON’T PULL YOUR TESTS WITH SLEEP() CALLS
93. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
94. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
95. Matteo Moretti | Samuele Lilli UI END-TO-END TESTS • BRITTLE • EXPENSIVE TO WRITE • TIME CONSUMING TO RUN • MORE AFFECTED BY NON-DETERMINISM
96. Matteo Moretti | Samuele Lilli
97. Matteo Moretti | Samuele Lilli https://martinfowler.com/bliki/TestPyramid.html
98. Matteo Moretti | Samuele Lilli DO YOU REALLY NEED A REAL JS BROWSER? https://blog.eleven-labs.com/en/behat-structure-functional-tests/
99. Matteo Moretti | Samuele Lilli STICK TO A BROWSER EMULATOR USE @javascript ONLY WHEN NEEDED
100. Matteo Moretti | Samuele Lilli FIXTURES DATA • NO FIXTURES (INTEGRATION TESTS) • LOAD FIXTURES AT RUNTIME • IN MEMORY DBMS (SQLITE)
101. Matteo Moretti | Samuele Lilli PARALLELISM • SPEED UP PERFORMANCE • COMPLEX SETUP AND DEBUG DO YOU NEED IT?
102. Matteo Moretti | Samuele Lilli FASTEST https://github.com/liuggio/fastest bin/behat --list-scenarios --config tests/behat/behat.yml | bin/fastest -vv --process=4 "bin/behat --stop-on-failure --config tests/behat/behat.yml {}"
103. Matteo Moretti | Samuele Lilli ISSUES HARD TO MAINTAIN RANDOM FAILURES SLOW TESTS
104. Matteo Moretti | Samuele Lilli SAVE OUTPUT FOR FAILED TESTS
106. Matteo Moretti | Samuele Lilli DEBUG
107. Matteo Moretti | Samuele Lilli CLI DEBUG • BEHAT CODE • APP CODE IF INTEGRATION TEST (NO UI) • APP CODE IF UI TESTS WITH BROWSER EMULATOR (BrowserKit)
108. Matteo Moretti | Samuele Lilli WEB DEBUG FOR APP CODE IF BROWSER CONTROLLER (SELENIUM)
109. Matteo Moretti | Samuele Lilli RECAP
110. Matteo Moretti | Samuele Lilli TREAT YOUR TEST CODE AS YOUR APP (PROD) CODE
111. Matteo Moretti | Samuele Lilli WRITE CLEAN, CONCISE AND WELL ORGANIZED FEATURES
112. Matteo Moretti | Samuele Lilli SPLIT YOUR CONTEXT CLASSES AND COMPOSE / REUSE THEM
113. Matteo Moretti | Samuele Lilli REMEMBER TEST PYRAMID FOCUS ON INTEGRATION LEVEL
114. Matteo Moretti | Samuele Lilli USE @javascript WISELY USE PAGE OBJECTS FOR UI
115. Matteo Moretti | Samuele Lilli THANK YOU
116. Matteo Moretti | Samuele Lilli WE ARE HIRING! (wanna join? ask us or visit our website) labs.madisoft.it
117. Matteo Moretti | Samuele Lilli https://joind.in/talk/707a1
118. QUESTIONS?