Developing an Ember Test Strategy - EmberConf 2019

Taking lessons from the classic writings on Software Testing, we'll walk through what are industry tried and true approaches for developing a robust and effective test suite.

Many of the Testing philosophies and strategies today have their origins as far back as the 60's, and really got their legs during the advent of "Extreme Programming" and other early "Agile" methodologies from the 1990s.

In this talk we'll take many of these tried and true test strategies and show how one would apply them in a real way to an Ember project.

Transcript

1. Developing an Ember Test Strategy Todd Jordan
2. About Me
3. About Me
4. About Me
5. About Me
6. About Me @tddjordan
7. ❤
8. Testing in Ember The Hidden Levels
9. With Some Origin Stories Mixed In
10. Speed and Feedback Loops
11. Speed and Feedback Loops
12. https://deming.org/deming/timeline
13. Act Check Do Plan Deming Circle
15. https://blog.deming.org/2016/10/toyotas-management-history/
16. https://www.amazon.com/Implementing-Lean-Software-Development-Concept/dp/0321437381/
17. Boehm’s Law 0 $ $$ $$$ Requirments Design Development Test Prod https://docs.google.com/viewer?url=https%3A%2F%2Fwww.cs.umd.edu%2Fprojects%2FSoftEng%2FESEG%2Fpapers%2F82.78.pdf
18. The faster your feedback, the more successful you’ll be
19. The faster your feedback, the more successful you’ll be
20. Test Types What Tests to Write and When to Use Them https://www.flickr.com/photos/chiffheed/6889521843
21. http://wiki.c2.com/?WardAndKent
22. https://agilemanifesto.org
23. Business Facing Technology Facing Support Programming Critique Product Performance Security Usability Exploratory Acceptance End To End Rendering Unit http://www.exampler.com/old-blog/2003/08/21.1.html#agile-testing-project-1 https://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468
24. Business Facing Technology Facing Support Programming Critique Product Performance Security Usability Exploratory Acceptance End To End Rendering Unit http://www.exampler.com/old-blog/2003/08/21.1.html#agile-testing-project-1 https://www.amazon.com/Agile-Testing-Practical-Guide-Testers/dp/0321534468
25. UI Service Unit fast slow pricey cheap https://www.amazon.com/Succeeding-Agile-Software-Development-Using/dp/0321579364 https://martinfowler.com/bliki/TestPyramid.html
26. Ember Testing Ember Unit Tests - Call functions directly (10-100ms) Ember Rendering Tests - Call components via the template compiler (200-600 ms) Ember Application Tests - Interact with the entire Ember application through the browser (700-3000ms with stubbed HTTP)
27. 1000 Tests Unit Rendering Application Minutes 0 10 20 30 40 33.3 6.7 1.7 Assumes 100ms Unit, 400ms Rendering, 2000ms Application
28. 1000 tests: Distributed Minutes 0 3.5 7 10.5 14 33/33/33 40/40/20 40/50/10 Unit Rendering Application
29. * With Doubles Application* Unit/Rendering E2E Service
30. Test Workflow A.K.A a nondogmatic discussion of Test First
33. “The original description of TDD was in an ancient book about programming. It said you take the input tape, manually type in the output tape you expect, then program until the actual output tape matches the expected output. ” -Kent Beck: OG Extreme Programmer, TDD Rediscoverer
35. TDD Write a Failing Test Make it Pass Refactor Repeat!!
36. Look Familiar? Test 1 … Test n Works!
37. Look Familiar? Test 1 … Test n Works!
38. -Everyone at some point, probably “Won’t working this way slow you down?”
39. Code http://geepawhill.org/tdd-and-the-lump-of-coding-fallacy/
40. Code http://geepawhill.org/tdd-and-the-lump-of-coding-fallacy/
41. Typing
42. Typing
43. Typing Waiting on NPM
44. Typing Waiting on NPM Reading Code
45. Typing Waiting on NPM Googling Reading Code
46. Typing Waiting on NPM Googling Reading Code Chatting
47. Typing Waiting on NPM Googling Reading Code Chatting Running Stuff
48. Typing Debugging Waiting on NPM Googling Reading Code Chatting Running Stuff
49. Typing Debugging Waiting on NPM Googling Reading Code Chatting Running Stuff Automated Tests
50. Typing Debugging Waiting on NPM Googling Reading Code Chatting Running Stuff Automated Tests
51. Typing Debugging Waiting on NPM Googling Reading Code Chatting Running Stuff Automated Tests
52. Classic TDD API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
53. Classic TDD Test API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
54. Classic TDD Test API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
55. Classic TDD Test Test API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
56. Classic TDD Test Test API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
57. Classic TDD Test Your object tree forms as part of your refactor step Test API https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
58. Example in Ember: A Page Component Last Email Submit First Input: Object with First, Last, Email, Profile Pic Input: Save action Output: Rendered DOM Output: Action invoked Upload
59. Classic TDD API
60. Classic TDD Test API
61. Classic TDD Test Profile Page API
62. Classic TDD Test Profile Page API Test
63. Classic TDD Test Profile Page API Test Profile Picture
64. Classic TDD Test Profile Page API Test Profile Picture
65. Macro Inside Out
66. Macro Inside Out Stubs Tests Core Logic
67. Data Macro Inside Out Stubs Tests Core Logic
68. Data Macro Inside Out Tests API Stubs Tests Core Logic
69. Data UI E2E Macro Inside Out Tests API Stubs Tests Core Logic
70. TDD with Mock Objects Mocks assert the behavior of your object’s collaborators Enable a form of “Need Driven Development” See sinon.js and ember- sinon https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627 https://docs.google.com/viewer?url=http%3A%2F%2Fjmock.org%2Foopsla2004.pdf
71. TDD with Mocks http://www.natpryce.com/articles/000342.html API
72. TDD with Mocks http://www.natpryce.com/articles/000342.html Test API
73. TDD with Mocks http://www.natpryce.com/articles/000342.html Test API
74. TDD with Mocks http://www.natpryce.com/articles/000342.html Test Mock Mock API
75. TDD with Mocks http://www.natpryce.com/articles/000342.html Test Mock Mock API Test
76. TDD with Mocks http://www.natpryce.com/articles/000342.html Test Mock Mock API Test Mocks are your roles
77. TDD with Mocks http://www.natpryce.com/articles/000342.html Test Mock Mock API Test Your mocks become the APIs for your next test Mocks are your roles
78. TDD with Mocks http://www.natpryce.com/articles/000342.html Test Mock Mock API Test Your mocks become the APIs for your next test Keep going until you reach a “leaf”, or an external boundary Mocks are your roles
79. Example in Ember: Chat Service Hi! New phone, who dis? todd.jordan locks 1:00pm 1:05pm Fetches chat history Receives incoming messages in realtime Sends outgoing messages
80. TDD with Mocks API
81. TDD with Mocks Test API
82. TDD with Mocks Test Chat Service API
83. TDD with Mocks Test Chat Service API Realtime Service Receive Incoming Profile Service Fetch Profile Info History Service Past Chats
84. TDD with Mocks Test Chat Service Test API Realtime Service Receive Incoming Profile Service Fetch Profile Info History Service Past Chats
85. Macro Outside In (Need Driven Development)
86. E2E Macro Outside In (Need Driven Development)
87. E2E Macro Outside In (Need Driven Development) UI Mocks Tests
88. E2E Macro Outside In (Need Driven Development) Tests API Mocks UI Mocks Tests
89. E2E Tests Mocks Core Logic Macro Outside In (Need Driven Development) Tests API Mocks UI Mocks Tests
90. E2E Tests Mocks Core Logic Macro Outside In (Need Driven Development) Tests API Mocks Data UI Mocks Tests
91. E2E Tests Mocks Core Logic Macro Outside In (Need Driven Development) Tests API Mocks Data API UI UI Mocks Tests
92. E2E Tests Mocks Core Logic Macro Outside In (Need Driven Development) Tests API Mocks Data API UI UI Mocks Tests *Done Iteratively, feature by feature
93. So… Which Should I do?
94. So… Which Should I do? ¯\_(ツ)_/¯
95. - Nat Pryce, co-author: JMock, GOOS “Mock objects are just a tool. They are one of many tools that you need to use when doing TDD. Like any tool, they are designed to help solve a set of problems in a specific context.” “So, instead of treating mock objects, or any other technique or tool, as an all-or-nothing "golden hammer", it's more helpful to understand where that tool fits into the broader picture of TDD practice.” https://groups.yahoo.com/neo/groups/testdrivendevelopment/conversations/messages/29686
96. Some Parting Perspectives
97. Do I Have To? https://giphy.com/gifs/star-trek-riker-tng-tYelSVzQPtLdm
98. Do I Have To? https://giphy.com/gifs/star-trek-riker-tng-tYelSVzQPtLdm
99. The Flawed Theory Behind Unit Testing Classical TDD Mockist TDD Clean Room Design By Contract Michael Feathers https://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
100. The Flawed Theory Behind Unit Testing Classical TDD Mockist TDD Clean Room Design By Contract Michael Feathers https://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html All
101. –Michael Feathers “Quality is a function of thought and reflection - precise thought and reflection. That’s the magic. Techniques which reinforce that discipline invariably increase quality.”
102. Keep an eye on your feedback loops Test in a methodical way that drives design Thought and discipline are faster than quickly spitting out code
103. Thanks!!! @tddjordan toddjordan todd.jordan