Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

QA Processes Supporting High-Frequency Releases

PayPay
October 01, 2024

QA Processes Supporting High-Frequency Releases

Speaker: Koichiro Takashima
Title: QA Processes Supporting High-Frequency Releases

PayPay

October 01, 2024
Tweet

More Decks by PayPay

Other Decks in Technology

Transcript

  1. CONFIDENTIAL © PayPay Self introduction 高嶋 康一郎 (Koichiro Takashima) 2023~

    PayPay - QA Engineering Manager - Working on QA of one of the product lines. 2014~ Rakuten - Test Engineer, Manager - Worked on various types of platform (e.g. Game console, Smartphone app, Web app) 2012~ SIer - Application Engineer - Application development in SIer (mainly C#) Career Experienced services, roles PayPay, Online Payment service, Flea market app, Internet supermarket, e-book, e-magazine, Messaging app, Game console app etc., QA team management, QA environment management, Test device management Hobbys Travel, Gadgets, Cooking, Bread making by いらすとや
  2. CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A

    Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans
  3. CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A

    Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans
  4. CONFIDENTIAL © PayPay Introduction of the PayPay QA Core App

    & Growth Financial Services & Billpayments Payments & MiniApp QA Service/QA Tech (Kaizen, Standalization, Tools) Merchant O2O Core App & Growth QA Financial Services & Billpayments QA Payments & MiniApp QA Merchant O2O QA Total 130+ members! QA Team Performance and Stability Assurance (PSA)
  5. CONFIDENTIAL © PayPay Introduction of the PayPay QA PayPay QA's

    History Manual & Automation separate team • Outsourcing and managed by dev Establishment of “In-house QA Team” Manual & Automation integration • Have 1 common regression test suite • Automation of the regression test PSA (Performance Test) Integration • Established PayPay India QA Team
  6. CONFIDENTIAL © PayPay Introduction of the PayPay QA Minimize Manual

    Maximize Auto Manual & Auto members Auto Script 1 Auto Script 2 Auto Script X Project Test Run & Report Coverage Complementary Maintenance Product Line Manual & Auto QA Test case 1 Test Case 2 Test Case X Integrated Process Auto Regression Test Suite Manual Regression Test Suite Coverage NOT shared Maintenance Manual QA Test case 1 Test Case 2 Test Case X Maintenance Auto QA Auto Script 1 Auto Script 2 Auto Script X Project Test Run & Report Test Run & Report X X Coverage NOT shared QA Legacy Challenge NOW Integrated Regression Test Suite
  7. CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A

    Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans
  8. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Release Note

    4.60.0 - iOS / Android / Web - 2024.09.04 Release Release Note 4.61.0 - iOS / Android / Web - 2024.09.11 Release Release Note 4.62.0 - iOS / Android / Web - 2024.09.18 Release ... Weekly Release!
  9. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 13 Regression

    test New feature test Time consuming factors - Manual test execution - Unoptimized process - Test estimation - Reporting by いらすとや
  10. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 14 Regression

    test New feature test Time consuming factors - Manual test execution - Unoptimized process - Test estimation - Reporting by いらすとや
  11. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases -70%! Manual

    effort reduction by Automation! • On the Regression Test. • Reduced 70% of manual test effort. • Execution can be done within a couple of days.
  12. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Project A

    Feature test Regression test Release Project C Feature test Regression test Release Project B Feature test Regression test Release Regression test on each project…?
  13. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Project A

    Feature test Regression test Release Project C Feature test Regression test Release Project B Feature test Regression test Release Regression test on each project…? - Need extra effort = cost - Quality risk (branch management)
  14. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases PayPay QA

    Regression Test • Separate from new feature test • Regular execution (Weekly) • Fixed members and resources
  15. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature

    A New feature E QA QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E
  16. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature

    A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F
  17. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature

    A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F STG QA
  18. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature

    A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F STG QA - For Feature Test - Each module version is based on each feature test - For Regression Test - Each module version is same as PROD
  19. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 25 Regression

    test New feature test Time consuming factors - Duplicate test - Manual test - Test estimation - Reporting by いらすとや
  20. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Test estimation

    Need at least several days (2~7 days) Test estimation tasks - Listing up test items - How many test cases - Number of test device - Resource allocation - Buffer So many tasks! By いらすとや
  21. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Test estimation

    Bottom-up estimation Analogical estimation 2~7 days less than a day by unsplash by unsplash
  22. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases QA sign-off

    report (Release judgement report) Manually need to gather data from different places. Tool A - Project info - Requirement - Specs Tool B - Test cases Tool C - Bug tickets By いらすとや
  23. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases QA sign-off

    report: Centralized reporting tool & Automation Atlassian tools - Project info - Requirement - Specs - Test cases - Bug tickets Integrated tools 2~3 days > A couple of hours Automated report creation tool (Google Extension) By いらすとや By Google
  24. CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Automation Test

    Report: Cetralized Reporting Tool Kibana (All test results status and Report) ReportPortal (Screenshots, Logs. For analysis.)
  25. CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A

    Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans
  26. CONFIDENTIAL © PayPay Collaboration and Culture of a Multinational Team

    Engineers from all over the world By unsplash • India • Japan • Taiwan • Indonesia • China • Vietnam • US • Russia • Mexico • Korea • Germany • Bangladesh
  27. CONFIDENTIAL © PayPay Collaboration and Culture of a Multinational Team

    Japanese Speaker Bilingual, Multi-lingual English Speaker How we are communicating? By いらすとや By いらすとや By いらすとや
  28. CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A

    Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans
  29. CONFIDENTIAL © PayPay Future Plans Advanced Testing Technologies Visual Testing

    
 CI/CD, DevOps 
 AI Utilization 
 By unsplash
  30. CONFIDENTIAL © PayPay Future Plans - PayPay SDET 40 SDET

    (Software Development Engineer in Test): A specialized role combining software development and testing skills to design, develop, and implement automated tests, ensuring robust and high-quality software delivery. Recent Contributions: • Impact Analysis Tool: Analyzes change impacts. • DevTestPro: This tool automates CI testing, enabling early bug detection and supporting Shift Left.. • Pipeline Optimization: Enhances testing pipeline efficiency. Benefits of Expanding the SDET Team: • Accelerate Automation: Speed up automation and support new tool development. • Enhance Testing Coverage: Improve back-end and integration testing. • Develop Frameworks: Create and refine testing frameworks. • Early Automation: Collaborate for early bug detection. • Tool Development: Implement new tools and APIs. Key Advantage: • Increased Efficiency: Boost automation capabilities and overall project quality. by OpenArt 
  31. CONFIDENTIAL © PayPay We are hiring! PayPay Groupでは新しい仲間を募集しています。ご興味のある方はこちらからご応募ください(現在、 PayPay 証券、PayPayカードで採用中です)

    PayPay Group is now looking for new members. If you are interested, please apply here (We are currently hiring for PayPay Securities and PayPay Card) PayPay Securities PayPay Card