Effective Testing Strategy in Android

Testing is important, we know. There’s no other way to verify if our app works as it should. Unless we choose our final users as the testers… in production. Trust me, that’s not a good option 👀. If we want to deliver a reliable app with a great user experience and that fits our users needs and works well on their devices, we need to guide our development process by testing. However, “implementing tests is easy, we just need some mocks and fakes, right?” Not exactly! In this talk, I’m going to show you that specifying a Testing Strategy as a team, can help us to effectively test our apps. With the right approach, considering our app’s architecture and the goal of each type of test, we can have a really helpful test suite that goes beyond coverage percentage, and actually expresses quality to us.

Transcript

1. Effective Testing Strategy in Android Natan Ximenes at Chicago Roboto - September, 2021.
2. // Natan Ximenes Senior software engineer, Android at WE ARE HIRING -> league.com/us/careers/ - Rio de Janeiro, Brazil.🌴 - Living and surviving in Android world over the last 6 years. 🤖 - Passionate about Digital Products, Agile and Testing. 󰡷 - Big fan of (brazilian)Football, NBA, e-sports, Marvel and Star Wars 🏃 @natanximenesdev github.com/Senemix29 natanximenes29@gmail.com
3. Strategy
4. "Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat. " (Sun Tzu, 500 b.c)
5. "It's the art of planning the best way to gain an advantage or achieve success." (Collins Dictionary)
6. Strategy is about.. planning the best, with the resources we have, to achieve a goal.
7. Testing Strategy
8. untested software shouldn’t be delivered to production...
9. ...otherwise, the users might become the testers
10. ...otherwise, the users might become the testers So... Testing is really important!
11. Testing Strategy A good strategy can help us to test properly an application, when we plan: // what to test… // how to test it... // without personal bias influence... // focused on software specificities... // possible points of failure...
12. What to test
13. What to test It can be any component of an application: // Function // Class // Layer // Screen
14. How to test it
15. How to Test It’s a combination between: // Testing types(Unit, Integration, UI, ...) // Testing tools (JUnit, Roboletric, Espresso, Assertion Libraries,...) Depending on the application architecture and design.
16. Bias-independent decisions
17. Bias-independent decisions Defining what and how to test something should be: // Impersonal ⛔“Who did code it?” ✅“What does this code do?” // Based on facts, not on personal tastes ⛔ “I prefer Unit Tests” ⛔ “I prefer Integrated Tests” ✅ “Which testing types and testing tools would apply for our application at the moment?”
18. Quality can't be taken for granted, it needs to be ensured!
19. Bias-independent decisions should be documented Documentation should be a team agreement, containing guidelines about: // Naming // Style(BDD, TDD) // Which tools to use // Which testing types and how to apply it …it must be updated frequently!
20. Software Specificities
21. Software Specificities Things that might directly impact the tests:
22. Software Specificities Things that might directly impact the tests: // Frameworks(View-based, Compose, Epoxy, ...) // Concurrency(Coroutines, RxJava, Flows, AsyncTask, ...) // Architecture(MVC, MVP, MVVM, MVI) // Dependency Injection(Koin, Dagger/Hilt)
23. Software Specificities Things that might directly impact the tests: // Frameworks(View-based, Compose, Epoxy, ...) // Concurrency(Coroutines, RxJava, Flows, AsyncTask, ...) // Architecture(MVC, MVP, MVVM, MVI) // Dependency Injection(Koin, Dagger/Hilt) What might be impacted in the tests: // Configuration // Execution // Assertions
24. Software Specificities Only adopt a framework or library when it’s configuration for testing is known and well documented.
25. Points of Failure
26. Points of Failure // in the App // outside the App
27. Points of Failure in the App Examples: // Side effects in the core of the app // Crashes
28. Points of Failure in the App - What to do? // Smoke Tests at the CI
29. Points of Failure in the App - What to do? // Smoke Tests at the CI // Crash reports
30. Points of Failure in the App - What to do? // Smoke Tests at the CI // Crash reports // Feature Flags
31. Points of Failure in the App - What to do? // Smoke Tests at the CI // Crash reports // Feature Flags // Phased releases
32. Points of Failure outside the App Examples: // Internal backend failures // External services/SDKs failures // Unstable mobile network Sometimes, even automated tests won’t be enough
33. Points of Failure outside the app - What to do? // Expect external/internal services might fail // Provide enough user feedback // Incident postmortems
34. Testing Pyramid… is it a Testing Strategy?
35. Testing Pyramid... It’s an ideal testing types geometric distribution: 10% UI Testing 20% Integration Testing   70% Unit Testing // It’s a trade off, between Execution time, Fidelity, Maintenance and Debugging
36. Testing Pyramid - it’s not a strategy // ignores different testing types/approaches // ignores specificities // ignores points of failure // it doesn’t always apply
37. Testing Pyramid - isn’t the only one // Testing Trophy - by Kent C. Dodds
38. Testing Pyramid - isn’t the only one // The Bug Filter - by Noah Sussman
39. Testing Distribution - it’s about adding value Plan your strategy and follow what’s best, thinking about: // Execution time // Fidelity // Maintenance // Debugging
40. TL;DR
41. TL;DR // Testing is important… but it needs to be well structured // It’s not about you, it’s about the application // Testing isn’t enough... // Testing Pyramid it’s not a testing strategy
42. Further reading // Medium Blog: https://natanximenes.medium.com/ // References: https://cutt.ly/testing-strategy-talk-references
43. THANK YOU! Effective Testing Strategy in Android Natan Ximenes at Chicago Roboto - September, 2021. @natanximenesdev github.com/Senemix29 natanximenes29@gmail.com linkedin.com/in/natan-ximenes29/