How Google Tests Software

Summary of learnings from the book.

Transcript

1. HOW GOOGLE TESTS SOFTWARE
2. OVERVIEW 1. Ideologies 2. Roles 3. Methodologies 4. Types of tests 5. Tools 6. Roles for us.
3. IDEOLOGIES Entire team is responsible for quality.
4. IDEOLOGIES Entire team is responsible for quality. Google engineers prefer quality over features.
5. IDEOLOGIES Entire team is responsible for quality. Google engineers prefer quality over features. Optimize for both quality and speed.
6. IDEOLOGIES Entire team is responsible for quality. Google engineers prefer quality over features. Optimize for both quality and speed. Quality != Test
7. IDEOLOGIES Entire team is responsible for quality. Google engineers prefer quality over features. Optimize for both quality and speed. Quality != Test Quality is not important until the software is important
8. ROLES Engineering Productivity Team Software Engineer Software Engineer in Test Test Engineer Test Director
9. ENGINEERING PRODUCTIVITY "overcome bias against testing and a company culture that favored heroic effort over engineering rigor."
10. SOFTWARE ENGINEER fault tolerant designs failure recovery TDD unit tests
11. SET Review designs Check code quality and risk Refactor code to make it more testable. Write testing frameworks and automation.
12. TE User focused Product experts Quality advisers and analyzers of risk. Write automation scripts. Coordinate with contract testers, dogfooders and beta users etc.
13. METHODOLOGIES Make testing a feature Centers around Code Review Single Repository CI with submit queue Test suite - Parallel on single machine.
14. TYPES OF TESTS Small Medium Large (Enormous)
15. SMALL All mocked, no http or database calls Ideally 100ms < 1 min 70%
16. MEDIUM Database, localhost calls Ideally 1 sec < 5 mins 20%
17. LARGE Allowed to call external service Focus on user's expectation < 1hr 10%
18. TOOLS Test Analytics ACC (Attribute Component Capability) Google Test Case Manager Manual test cases, Exploratory testing Buganizer Triage, accountability, reports, search before filing Record Playback Framework record and generate selenium tests.
19. TEST ANALYTICS
20. BUGS Severity: how much it affects functionality. Priority: how soon it should be fixed.
21. ROLES FOR US SWE == SET Team Lead =~ TE (Test Engineering Designer) Devops = Engineer Productivity + Tooling
22. TE DESIGNER Test design and coordination Identify resources required for testing.
23. ENCOURAGE TESTING OKRs Certification program (test coverage, test speeds, 0 flaky tests, automated smoke test etc).
24. NEXT Things we can do to improve quality More resources http://googletesting.blogspot.com/