Behat from zero to hero - A practical guide to symfony integration and usage

Starting from behat introduction, we have explained some good practices for BDD testing in a symfony software. This presentation should be a quick "guide" for users that starts with Behat and also for those who want to fix their tests problems.

Transcript

1. Behat from zero to hero
   “A practical guide to symfony integration and usage”
   Matteo Moretti - Samuele Lilli
   @SymfonyDayIt - 20 Ottobre 2017 - Milano
   

   View Slide

2. Matteo Moretti | Samuele Lilli
   WHO ARE WE?
   Matteo Moretti Samuele Lilli
   

   View Slide

3. Matteo Moretti | Samuele Lilli
   OUR
   EXPERIENCE
   

   View Slide

4. Matteo Moretti | Samuele Lilli
   OUR EXPERIENCE
   ON nuvola.madisoft.it
   https://symfony.com/showcase/191
   

   View Slide

5. Matteo Moretti | Samuele Lilli
   BEHAVIOUR
   DRIVEN
   DEVELOPMENT
   

   View Slide

6. Matteo Moretti | Samuele Lilli
   BEHAT
   

   View Slide

7. Matteo Moretti | Samuele Lilli
   A LONG TIME
   AGO ...
   

   View Slide

8. Matteo Moretti | Samuele Lilli
   

   View Slide

9. Matteo Moretti | Samuele Lilli
   

   View Slide

10. Matteo Moretti | Samuele Lilli
    TESTS ARE GREEN!
    

    View Slide

11. View Slide

12. Matteo Moretti | Samuele Lilli
    BUT,
    SUDDENLY...
    

    View Slide

13. Matteo Moretti | Samuele Lilli
    AS SOON AS
    YOU ADD MORE
    TESTS...
    

    View Slide

14. Matteo Moretti | Samuele Lilli
    HARD TO MAINTAIN
    

    View Slide

15. Matteo Moretti | Samuele Lilli
    RANDOM
    FAILURES
    

    View Slide

16. Matteo Moretti | Samuele Lilli
    SLOW TESTS
    

    View Slide

17. View Slide

18. Matteo Moretti | Samuele Lilli
    “WHY THE HELL THIS
    TEST IS BROKEN?
    I DIDN’T MODIFY THAT
    CODE”
    

    View Slide

19. Matteo Moretti | Samuele Lilli
    “MMM, 10 MINUTES TO
    ADD A FEATURE. 2
    HOURS TO FIX TESTS.”
    

    View Slide

20. View Slide

21. Matteo Moretti | Samuele Lilli
    

    View Slide

22. Matteo Moretti | Samuele Lilli
    

    View Slide

23. Matteo Moretti | Samuele Lilli
    WHAT IS
    BEHAT?
    

    View Slide

24. Matteo Moretti | Samuele Lilli
    IT’S A BDD TOOL!
    

    View Slide

25. Matteo Moretti | Samuele Lilli
    IT’S A
    STORY
    BDD TOOL!
    

    View Slide

26. Matteo Moretti | Samuele Lilli
    WHAT IS A
    STORY?
    

    View Slide

27. Matteo Moretti | Samuele Lilli
    HI! I’M A STORY!
    

    View Slide

28. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

29. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

30. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

31. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

32. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

33. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

34. Matteo Moretti | Samuele Lilli
    EXAMPLE OF HOW THE SYSTEM SHOULD BEHAVE
    DESCRIPTION OF A GIVEN REQUIREMENT IN UBIQUITOUS NATURAL LANGUAGE
    UNDERSTANDABLE BY ALL STAKEHOLDERS OF THE PROJECT
    ACCEPTANCE TEST
    DOCUMENTATION THAT NEVER GOES OUT TO DATE
    WRITTEN IN GHERKIN
    EXECUTABLE SPECIFICATION
    A STORY IS ...
    

    View Slide

35. Matteo Moretti | Samuele Lilli
    GHERKIN
    

    View Slide

36. Matteo Moretti | Samuele Lilli
    IT’S A LANGUAGE!
    

    View Slide

37. Matteo Moretti | Samuele Lilli
    IT’S A BUSINESS
    READABLE
    DOMAIN SPECIFIC
    LANGUAGE!
    

    View Slide

38. View Slide

39. Matteo Moretti | Samuele Lilli
    LIKE YAML
    INDENTATION
    RULES PARSING
    

    View Slide

40. Matteo Moretti | Samuele Lilli
    STORIES ARE ORGANIZED IN
    FEATURES FILES (.feature)
    

    View Slide

41. Matteo Moretti | Samuele Lilli
    GIVEN - WHEN - THEN
    (AND, BUT) ARE KEYWORDS
    THAT MATCH PHP CODE
    AND EXECUTE IT.
    STEPS
    

    View Slide

42. Matteo Moretti | Samuele Lilli
    

    View Slide

43. View Slide

44. Matteo Moretti | Samuele Lilli
    REGEX
    

    View Slide

45. Matteo Moretti | Samuele Lilli
    

    View Slide

46. Matteo Moretti | Samuele Lilli
    

    View Slide

47. Matteo Moretti | Samuele Lilli
    

    View Slide

48. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

49. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

50. Matteo Moretti | Samuele Lilli
    HOW A GOOD
    STORY LOOKS
    LIKE?
    

    View Slide

51. Matteo Moretti | Samuele Lilli
    FOCUS ON OUTPUT NOT INPUT
    X
    

    View Slide

52. Matteo Moretti | Samuele Lilli
    √
    FOCUS ON OUTPUT NOT INPUT
    

    View Slide

53. Matteo Moretti | Samuele Lilli
    DON’T USE UI REFERENCES
    X
    

    View Slide

54. Matteo Moretti | Samuele Lilli
    DON’T USE UI REFERENCES
    √
    

    View Slide

55. Matteo Moretti | Samuele Lilli
    USE BACKGROUND
    

    View Slide

56. Matteo Moretti | Samuele Lilli
    USE TRANSFORMERS
    

    View Slide

57. Matteo Moretti | Samuele Lilli
    USE TRANSFORMERS
    

    View Slide

58. Matteo Moretti | Samuele Lilli
    COLLAPSE STEPS AS MUCH AS
    POSSIBLE
    UNLESS THESE GIVES YOU
    BUSINESS/STORY VALUE
    

    View Slide

59. Matteo Moretti | Samuele Lilli
    

    View Slide

60. Matteo Moretti | Samuele Lilli
    HOW
    MUCH
    DETAILS?
    

    View Slide

61. Matteo Moretti | Samuele Lilli
    “What many people don’t realize is that
    different levels of details are appropriate
    for different scenarios in the same system
    - sometimes in the same feature -
    depending on what it is they’re describing”
    (The cucumber book - Matt Wynne, Aslak Hellesoy)
    

    View Slide

62. Matteo Moretti | Samuele Lilli
    BEHAT → GHERKIN → FEATURES →
    STORIES (SCENARIOS) → STEPS →
    CONTEXTS
    

    View Slide

63. Matteo Moretti | Samuele Lilli
    BEHAT → GHERKIN → FEATURES →
    STORIES (SCENARIOS) → STEPS →
    CONTEXTS
    

    View Slide

64. Matteo Moretti | Samuele Lilli
    DIVIDE ET IMPERA
    

    View Slide

65. Matteo Moretti | Samuele Lilli
    ONE FEATURE
    PER .FEATURE FILE
    

    View Slide

66. Matteo Moretti | Samuele Lilli
    DIVIDE FEATURES
    AND CONTEXTS
    BY DOMAIN SUBJECT
    

    View Slide

67. Matteo Moretti | Samuele Lilli
    DIVIDE FEATURES
    AND CONTEXTS
    BY ACTIONS TO BE
    PERFORMED ON
    DOMAIN SUBJECTS
    

    View Slide

68. Matteo Moretti | Samuele Lilli
    

    View Slide

69. Matteo Moretti | Samuele Lilli
    

    View Slide

70. Matteo Moretti | Samuele Lilli
    UI
    

    View Slide

71. Matteo Moretti | Samuele Lilli
    HEADLESS BROWSER EMULATOR
    VS
    BROWSER CONTROLLER
    

    View Slide

72. Matteo Moretti | Samuele Lilli
    HEADLESS BROWSER EMULATOR
    ● pure HTTP spec implementations
    ● very simple to run and configure
    ● no need of a real browser: speed
    ● usually no JS/AJAX support
    

    View Slide

73. Matteo Moretti | Samuele Lilli
    BROWSER CONTROLLER
    ● aim to control a real browser
    ● complex to configure
    ● slow
    ● JS/AJAX support
    

    View Slide

74. View Slide

75. Matteo Moretti | Samuele Lilli
    YOU MAY NEED BOTH OF THEM
    

    View Slide

76. Matteo Moretti | Samuele Lilli
    MINK
    http://mink.behat.org/en/latest/
    

    View Slide

77. Matteo Moretti | Samuele Lilli
    BEHAT PARSES SCENARIOS
    +
    MINK BROWSES THE WEB
    

    View Slide

78. View Slide

79. Matteo Moretti | Samuele Lilli
    

    View Slide

80. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

81. Matteo Moretti | Samuele Lilli
    HOW TO USE MINK
    FROM CONTEXT
    CLASSES?
    

    View Slide

82. Matteo Moretti | Samuele Lilli
    PAGE OBJECT PATTERN
    https://martinfowler.com/bliki/PageObject.html
    

    View Slide

83. View Slide

84. View Slide

85. Matteo Moretti | Samuele Lilli
    https://github.com/sensiolabs/BehatPageObjectExtension
    BEHAT PAGE OBJECT EXTENSION
    

    View Slide

86. Matteo Moretti | Samuele Lilli
    ASSERTIONS
    

    View Slide

87. View Slide

88. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

89. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

90. Matteo Moretti | Samuele Lilli
    AJAX
    

    View Slide

91. Matteo Moretti | Samuele Lilli
    DON’T PULL YOUR
    TESTS WITH SLEEP()
    CALLS
    

    View Slide

92. View Slide

93. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

94. Matteo Moretti | Samuele Lilli
    ISSUES
    HARD TO MAINTAIN
    RANDOM FAILURES
    SLOW TESTS
    

    View Slide

95. Matteo Moretti | Samuele Lilli
    UI END-TO-END TESTS
    ● BRITTLE
    ● EXPENSIVE TO WRITE
    ● TIME CONSUMING TO RUN
    ● MORE AFFECTED BY NON-DETERMINISM
    

    View Slide

96. Matteo Moretti | Samuele Lilli
    

    View Slide

97. Matteo Moretti | Samuele Lilli
    https://martinfowler.com/bliki/TestPyramid.html
    

    View Slide

98. Matteo Moretti | Samuele Lilli
    DO YOU REALLY NEED A REAL JS BROWSER?
    https://blog.eleven-labs.com/en/behat-structure-functional-tests/
    

    View Slide

99. Matteo Moretti | Samuele Lilli
    STICK TO A BROWSER EMULATOR
    USE @javascript ONLY WHEN NEEDED
    

    View Slide

100. Matteo Moretti | Samuele Lilli
     FIXTURES DATA
     ● NO FIXTURES (INTEGRATION TESTS)
     ● LOAD FIXTURES AT RUNTIME
     ● IN MEMORY DBMS (SQLITE)
     

     View Slide

101. Matteo Moretti | Samuele Lilli
     PARALLELISM
     ● SPEED UP PERFORMANCE
     ● COMPLEX SETUP AND DEBUG
     DO YOU NEED IT?
     

     View Slide

102. Matteo Moretti | Samuele Lilli
     FASTEST
     https://github.com/liuggio/fastest
     bin/behat --list-scenarios --config
     tests/behat/behat.yml | bin/fastest -vv --process=4
     "bin/behat --stop-on-failure --config
     tests/behat/behat.yml {}"
     

     View Slide

103. Matteo Moretti | Samuele Lilli
     ISSUES
     HARD TO MAINTAIN
     RANDOM FAILURES
     SLOW TESTS
     

     View Slide

104. Matteo Moretti | Samuele Lilli
     SAVE OUTPUT FOR
     FAILED TESTS
     

     View Slide

105. View Slide

106. Matteo Moretti | Samuele Lilli
     DEBUG
     

     View Slide

107. Matteo Moretti | Samuele Lilli
     CLI DEBUG
     ● BEHAT CODE
     ● APP CODE IF INTEGRATION TEST (NO UI)
     ● APP CODE IF UI TESTS WITH BROWSER
     EMULATOR (BrowserKit)
     

     View Slide

108. Matteo Moretti | Samuele Lilli
     WEB DEBUG
     FOR APP CODE IF BROWSER CONTROLLER
     (SELENIUM)
     

     View Slide

109. Matteo Moretti | Samuele Lilli
     RECAP
     

     View Slide

110. Matteo Moretti | Samuele Lilli
     TREAT YOUR TEST
     CODE AS YOUR
     APP (PROD) CODE
     

     View Slide

111. Matteo Moretti | Samuele Lilli
     WRITE CLEAN,
     CONCISE AND
     WELL ORGANIZED
     FEATURES
     

     View Slide

112. Matteo Moretti | Samuele Lilli
     SPLIT YOUR
     CONTEXT CLASSES
     AND
     COMPOSE / REUSE
     THEM
     

     View Slide

113. Matteo Moretti | Samuele Lilli
     REMEMBER TEST PYRAMID
     FOCUS ON INTEGRATION LEVEL
     

     View Slide

114. Matteo Moretti | Samuele Lilli
     USE @javascript WISELY
     USE PAGE OBJECTS FOR UI
     

     View Slide

115. Matteo Moretti | Samuele Lilli
     THANK YOU
     

     View Slide

116. Matteo Moretti | Samuele Lilli
     WE ARE HIRING!
     (wanna join? ask us or visit our website)
     labs.madisoft.it
     

     View Slide

117. Matteo Moretti | Samuele Lilli
     https://joind.in/talk/707a1
     

     View Slide

118. QUESTIONS?
     

     View Slide