This presentation gives a high-level overview of an array of different activities, processes and even entire disciplines that will strengthen the quality in your software while cutting down testing time and the impact of defects on production.
• Once on prod, it was usually detected by user complaints • Getting the complaints from the support agents to the engineers takes time • Developing and releasing the fix is slow • Updating depends on the users Defects’ Impact The Price of Quality People photo created by yanalya
• Effective monitoring alerts us of problems on production before users do • Changes can be released to small group of customers first, effectively using them as testers • Small release batches allow fast rollback to previous version without losing features • Automatic checks and deployments allow hotfixes to be very fast Mitigating Factors The Price of Quality
DOD & DOR Beyond Testing Definitions of Done/Ready Why ➔ DOR: To minimize rework or discussions about a feature too late in the SDLC ➔ DOD: To prevent accumulation of technical debt How Discussion and agreement within the team; No tools required Pitfalls DOR and DOD are only guidelines, they should never be applied as “Laws”
Code Traceability Why We should be able to connect any code change to a business requirement. Thus we will know exactly what we release or revert at any point in time. How Add JIRA issue ID to commit messages / pull requests JIRA plugins can point you to PRs from tickets Connects code to business
Code Reviews Beyond Testing Why ➔ Two heads think better than one ➔ QA engineers can check for test coverage, can suggest better/more test cases, will know where to focus testing efforts in later phases How Multiple tools depending on VC (version control) hosting Pitfalls ➔ Design discussions should happen before implementation, not during code reviews; ➔ Tools should not replace discussions or pair programming
Code Coverage Policy Beyond Testing Why ➔ Will guarantee you have some unit tests ➔ Will help the team align on the level of coverage they aim for How • CodeCov • CodeClimate • SonarQube Attention Code Coverage on its own is just a number; It shows lines of executed code, thus you could achieve 100% coverage with 0 asserts in your “tests” - it’s an easy-to- manipulate metric
Observability Beyond Testing Observability aims to provide highly granular insights into the behavior of systems along with rich context, perfect for debugging purposes. Monitoring should provide a bird’s eye view of the overall health of a system. In the event of a failure, it should immediately be able to answer “what” and “why” went wrong.
Monitoring Beyond Testing Why ➔ To know immediately when something goes wrong on production ➔ To identify root cause in chain-reaction incidents How - NewRelic - Sentry - Nagios Pitfalls Monitoring should be simple; Any alarm that is regularly ignored or metric that is not actionable should be removed
Observability Beyond Testing Why ➔ To get as much information about the application behaviour under test or on production ➔ Includes logging, tracing, metrics How - Logs: Graylog, Splunk - Time series database: Prometheus, InfluxDB - Errors: Sentry - Display: Grafana - Multitool: DataDog
Customer Feedback Beyond Testing Why Best insight into how is your product used How - Statistics based on Customer Support enquiries - NPS surveys - Customer Interviews Pitfalls If you base decisions on customer feedback, make sure you got it from a representative group and a segment that is big enough to draw conclusions