Ogres and tests are like onions

http://2017.mobilization.pl/

Transcript

2. Ogres and tests are like onions Mobilization Łódź, 2017
3. Ewa Ludwiczak Senior Software Tester ewa.ludwiczak@allegrogroup.com @ewabielskapoz Aleksander Grzyb iOS Developer aleksander.grzyb@allegrogroup.com @aleksandergrzyb 3
4. 4 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
5. 5 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
6. 6 Problem 1:
7. 7 Problem 1:
8. 8 Problem 1: Crash on production caused by force unwrapping Solution: SwiftLint
9. 9 1. basic set of rules to begin with 2. using the same version of SwiftLint 3. discussion about formatting https://github.com/realm/SwiftLint Problem 1: Crash on production caused by force unwrapping Solution: SwiftLint
10. 10 Problem 1: Crash on production caused by force unwrapping Solution: SwiftLint
12. 12 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B Tests 8. Usability Tests
13. 13 http://cleanswifter.com/asynchronous-ios-unit-test-tutorial/
14. 14 http://cleanswifter.com/asynchronous-ios-unit-test-tutorial/
15. 15 http://cleanswifter.com/asynchronous-ios-unit-test-tutorial/ Problem 2: Evergreen unit tests
17. 17 Problem 2: Evergreen unit tests Solution: Mutation testing source code
18. 18 Problem 2: Evergreen unit tests Solution: Mutation testing source code mutant mutation
19. 19 Problem 2: Evergreen unit tests Solution: Mutation testing source code mutant mutation
21. 21 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
22. 22 Problem 3: Testing views & layouts
23. 23 Problem 3: Testing views & layouts Solution: Snapshot testing via FBSnapshotTestCase
24. 24 Compact Width Regular Width
25. 25 xSmall xxxLarge Large (Default)
26. 26 Mock size classes
27. 27 Mock dynamic font size
29. 29 https://rtyley.github.io/bfg-repo-cleaner/
30. 30 https://github.com/facebookarchive/ios-snapshot-test-case/issues/227#issuecomment-335175385
31. 31 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
32. 32 Problem 4: Predicting consequences of change
33. 33 Problem 4: Predicting consequences of change Solution: UI tests ➔ functional UI tests ➔ separate target ➔ flaky tests
34. 34 Problem 4: Predicting consequences of change Solution: UI tests ➔ stubbed UI tests ➔ scheme verification ➔ functional UI tests ➔ separate target ➔ flaky tests
35. 35 Problem 4: Predicting consequences of change Solution: UI tests JSON files JSON scheme configuration file validation
36. 36 ➔ functional UI tests ➔ separate target ➔ flaky tests Problem 4: Predicting consequences of change Solution: UI tests ➔ stubbed UI tests ➔ scheme verification ➔ slower growth of test coverage
37. 37 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
38. 38 Problem 5: Late feedback
39. 39 Problem 5: Late feedback
40. 40 Problem 5: Late feedback Solution: Continues delivery
42. 42 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
43. 43 Problem 6: Testing in laboratory condition
44. 44 Problem 6: Testing in laboratory condition ENG: “After latest release application crashes - soft 9.0” Elen Musk
45. 45 Problem 6: Testing in laboratory condition Elen Musk ENG: “After latest release application crashes - soft 9.0”
46. 46 Problem 6: Testing in laboratory condition Solution: Phased release
47. 47 Problem 6: Testing in laboratory condition Solution: Phased release https://www.smithsonianmag.com/smart-news/story-real-canary-coal-mine-180961570/
48. 48 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
49. 49 Problem 7: Decision making
50. 50 Problem 7: Decision making Solution: A/B testing
51. 51 https://twitter.com/lukew/status/918887971117441024 https://docs.google.com/document/d/19BoeoynEBtk0lQZREyrhHHemaMT9zP3CZL31xr6Xtvs/edit
52. 52 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Tests are like onions 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
53. 53 Problem 8: Do we have the right product? ENG: “This app is a toy” ENG: “It’s useless like whole Allegro” ENG: “After updates, only worse not better...” ENG: “Searches nothing - useless.” Elton Johny Heisenburg Kris Eidhov Taylor Swift
54. 54 Problem 8: Does the user know how to use our app? Solution: Usability testing One-way mirror
55. 55 1. Static analysis 2. Mutation testing 3. Snapshot tests 4. Functional tests Summary 5. Continuous delivery 6. Phased release 7. A/B tests 8. Usability tests
56. 56 Thank you for your attention! goo.gl/Cfzovd Ewa Ludwiczak Senior Software Tester ewa.ludwiczak@allegrogroup.com @ewabielskapoz Aleksander Grzyb iOS Developer aleksander.grzyb@allegrogroup.com @aleksandergrzyb