How We Test in LINE TRAVEL That You’ll Love in the Future

How we test in LINE TRAVEL that you'll love in the future by Miki Liao on Test Corner#23 https://testcorner.kktix.cc/events/testcorner23

Transcript

1. Miki Liao 2019.09.03 How We Test in LINE TRAVEL That You’ll Love in the Future
2. 2 LINE Automation Engineer • LINE TRAVEL Team • Automated Test Development • Build CI/CD MIKI LIAO
3. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
4. LINE TRAVEL Entrance 1 2
5. Journey Planning
6. Affiliate Partners AGODA LION TRAVEL KKDAY TRIPRESSO
7. Search and Order
8. My Orders
9. POI Collections
10. Coupon
11. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
12. Which cases should be automated New Functionality Smoke Test
13. Search and Order
14. Automation Demo
15. My Orders
16. Automation Demo
17. Coupon
18. Automation Demo
19. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
20. Agile Testing
    We programmed in pairs, test-drove all our production code, practiced continuous integration, and also regularly explored.  Reference: Elisabeth Hendrickson.
    Explore It!
21. Agile Testing Lifecycle Test Case Design Acceptance Criteria Unit Test Regression Test Exploratory Test Test Case Execution Run Automation Automation Test Daily Standup Refinement Planning Dev & Test Regression Release Continuous Integration
22. Continuous Integration, Continuous Deployment Pull Request Unit Test Integration Test Analysis Build & Deploy Code Review Webhook Success Continuous Integration Merge Continuous Deployment
23. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
24. Flaky • Local run the test passes but failed after merged • The more flaky tests, the more time needs to do maintain • The flaky test makes problems invisible • When a test is failed, we always think 'Trigger again, then it would be correct' (→we do not trust the result)
25. Stability Test
27. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
28. Archive Videos to Check the Failure
31. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
32. Retry Mechanism • Purpose Quite obviously, even if we are writing Cypress tests idiomatically and correctly, there's no way to get around a flaky server or a flaky testing environment. • Factors of Unstable Tests • Poorly written • Accidentally rely on timing • Rely on other unstable services, but only use re-try on 3rd-party issues (v) • (v) Only when unstable APIs(3rd-party, networking) issues • (v) Not more than twice of retry • (x) Not on all test cases
33. Retry Mechanism Ref: https://github.com/Bkucera/cypress-plugin-retries
34. Agenda • Brief introduction of LINE TRAVEL • Test to be Automated • Agile Test • Automation Test Common Practices • Avoid Flaky Automation • Archive Videos to Check the Failure • Retry Mechanism • Monitoring
35. Monitoring
36. Reference • Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing • Jenkins archive artifact/save file in Pipeline • https://medium.com/@gustavo.guss/jenkins-archive-artifact-save-file-in-pipeline-ac6d8b569c2c • Retry Mechanism • https://github.com/Bkucera/cypress-plugin-retries • Archive Videos to Check the Failure • https://medium.com/@gustavo.guss/jenkins-archive-artifact-save-file-in-pipeline-ac6d8b569c2c