Share
Embed
QA Processes Supporting High-Frequency Releases
Speaker: Koichiro Takashima
Title: QA Processes Supporting High-Frequency Releases
PayPay
October 01, 2024
Transcript
41 slides
-
CONFIDENTIAL © PayPay 2024/9/26 QA Processes Supporting High-Frequency Releases
1 Koichiro Takashima PayPay QA Team -
CONFIDENTIAL © PayPay Self introduction 高嶋 康一郎 (Koichiro Takashima) 2023~
PayPay - QA Engineering Manager - Working on QA of one of the product lines. 2014~ Rakuten - Test Engineer, Manager - Worked on various types of platform (e.g. Game console, Smartphone app, Web app) 2012~ SIer - Application Engineer - Application development in SIer (mainly C#) Career Experienced services, roles PayPay, Online Payment service, Flea market app, Internet supermarket, e-book, e-magazine, Messaging app, Game console app etc., QA team management, QA environment management, Test device management Hobbys Travel, Gadgets, Cooking, Bread making by いらすとや -
CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A
Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans -
CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A
Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans -
CONFIDENTIAL © PayPay Introduction of the PayPay QA Core App
& Growth Financial Services & Billpayments Payments & MiniApp QA Service/QA Tech (Kaizen, Standalization, Tools) Merchant O2O Core App & Growth QA Financial Services & Billpayments QA Payments & MiniApp QA Merchant O2O QA Total 130+ members! QA Team Performance and Stability Assurance (PSA) -
CONFIDENTIAL © PayPay Introduction of the PayPay QA PayPay QA's
History Manual & Automation separate team • Outsourcing and managed by dev Establishment of “In-house QA Team” Manual & Automation integration • Have 1 common regression test suite • Automation of the regression test PSA (Performance Test) Integration • Established PayPay India QA Team -
CONFIDENTIAL © PayPay Introduction of the PayPay QA Minimize Manual
Maximize Auto Manual & Auto members Auto Script 1 Auto Script 2 Auto Script X Project Test Run & Report Coverage Complementary Maintenance Product Line Manual & Auto QA Test case 1 Test Case 2 Test Case X Integrated Process Auto Regression Test Suite Manual Regression Test Suite Coverage NOT shared Maintenance Manual QA Test case 1 Test Case 2 Test Case X Maintenance Auto QA Auto Script 1 Auto Script 2 Auto Script X Project Test Run & Report Test Run & Report X X Coverage NOT shared QA Legacy Challenge NOW Integrated Regression Test Suite -
CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A
Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Release Note
4.60.0 - iOS / Android / Web - 2024.09.04 Release Release Note 4.61.0 - iOS / Android / Web - 2024.09.11 Release Release Note 4.62.0 - iOS / Android / Web - 2024.09.18 Release ... Weekly Release! -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 12 What
are the blockers of high frequency release? -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 13 Regression
test New feature test Time consuming factors - Manual test execution - Unoptimized process - Test estimation - Reporting by いらすとや -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 14 Regression
test New feature test Time consuming factors - Manual test execution - Unoptimized process - Test estimation - Reporting by いらすとや -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases -70%! Manual
effort reduction by Automation! • On the Regression Test. • Reduced 70% of manual test effort. • Execution can be done within a couple of days. -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Project A
Feature test Regression test Release Project C Feature test Regression test Release Project B Feature test Regression test Release Regression test on each project…? -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Project A
Feature test Regression test Release Project C Feature test Regression test Release Project B Feature test Regression test Release Regression test on each project…? - Need extra effort = cost - Quality risk (branch management) -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases PayPay QA
Regression Test • Separate from new feature test • Regular execution (Weekly) • Fixed members and resources -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature
A New feature E QA QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature
A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature
A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F STG QA -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases New feature
A New feature E QA QA New feature F (Agile) QA Regression Test (ver. 4.55.0) Regression Test (ver. 4.56.0) Regression Test (ver. 4.57.0) New feature B QA New feature C QA New feature D QA - New feature A - New feature B - New feature C - New feature D - New feature E - New feature F STG QA - For Feature Test - Each module version is based on each feature test - For Regression Test - Each module version is same as PROD -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases 25 Regression
test New feature test Time consuming factors - Duplicate test - Manual test - Test estimation - Reporting by いらすとや -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Test estimation
Need at least several days (2~7 days) Test estimation tasks - Listing up test items - How many test cases - Number of test device - Resource allocation - Buffer So many tasks! By いらすとや -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Test estimation
Bottom-up estimation Analogical estimation 2~7 days less than a day by unsplash by unsplash -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases QA sign-off
report (Release judgement report) Manually need to gather data from different places. Tool A - Project info - Requirement - Specs Tool B - Test cases Tool C - Bug tickets By いらすとや -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases QA sign-off
report: Centralized reporting tool & Automation Atlassian tools - Project info - Requirement - Specs - Test cases - Bug tickets Integrated tools 2~3 days > A couple of hours Automated report creation tool (Google Extension) By いらすとや By Google -
CONFIDENTIAL © PayPay QA Processes Supporting High-Frequency Releases Automation Test
Report: Cetralized Reporting Tool Kibana (All test results status and Report) ReportPortal (Screenshots, Logs. For analysis.) -
CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A
Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans -
CONFIDENTIAL © PayPay Collaboration and Culture of a Multinational Team
Engineers from all over the world By unsplash • India • Japan • Taiwan • Indonesia • China • Vietnam • US • Russia • Mexico • Korea • Germany • Bangladesh -
CONFIDENTIAL © PayPay Collaboration and Culture of a Multinational Team
Japanese Speaker Bilingual, Multi-lingual English Speaker How we are communicating? By いらすとや By いらすとや By いらすとや -
CONFIDENTIAL © PayPay Collaboration and Culture of a Multinational Team
Kiara (Slack) AI Interpreter -
CONFIDENTIAL © PayPay Agenda 1. PayPay QA Introduction 2. A
Processes Supporting High-Frequency Releases 3. Collaboration of a Multinational Team 4. Our Future Plans -
CONFIDENTIAL © PayPay Future Plans Advanced Testing Technologies Visual Testing
CI/CD, DevOps AI Utilization By unsplash -
CONFIDENTIAL © PayPay Future Plans Automation Expansion Increasing Automation coverage
SDET by unsplash By いらすとや -
CONFIDENTIAL © PayPay Future Plans - PayPay SDET 40 SDET
(Software Development Engineer in Test): A specialized role combining software development and testing skills to design, develop, and implement automated tests, ensuring robust and high-quality software delivery. Recent Contributions: • Impact Analysis Tool: Analyzes change impacts. • DevTestPro: This tool automates CI testing, enabling early bug detection and supporting Shift Left.. • Pipeline Optimization: Enhances testing pipeline efficiency. Benefits of Expanding the SDET Team: • Accelerate Automation: Speed up automation and support new tool development. • Enhance Testing Coverage: Improve back-end and integration testing. • Develop Frameworks: Create and refine testing frameworks. • Early Automation: Collaborate for early bug detection. • Tool Development: Implement new tools and APIs. Key Advantage: • Increased Efficiency: Boost automation capabilities and overall project quality. by OpenArt -
CONFIDENTIAL © PayPay We are hiring! PayPay Groupでは新しい仲間を募集しています。ご興味のある方はこちらからご応募ください(現在、 PayPay 証券、PayPayカードで採用中です)
PayPay Group is now looking for new members. If you are interested, please apply here (We are currently hiring for PayPay Securities and PayPay Card) PayPay Securities PayPay Card