Upgrade to Pro — share decks privately, control downloads, hide ads and more …

QA to Improve Development Process - Expansion of Domain from QC to QA

QA to Improve Development Process - Expansion of Domain from QC to QA

LINE DEVDAY 2021
PRO

November 11, 2021
Tweet

More Decks by LINE DEVDAY 2021

Other Decks in Technology

Transcript

  1. View Slide

  2. 2012 : System Integrator
    Improve Legacy System
    - Test Automation
    - Ticket Management
    2020/12 : LINE Fukuoka
    LINE Front-end Framework (LIFF)
    QA Engineer
    Hiroyuki Okuda
    LINE Fukuoka QA Engineering 1 Team

    View Slide

  3. What is Quality Assurance?

    View Slide

  4. QA ≠ Test

    View Slide

  5. LINE QA

    View Slide

  6. Agenda - Vision & Mission
    - Case Studies in Project
    - Learned from the Case Studies

    View Slide

  7. History
    LINE Fukuoka Quality Organization
    2014
    10Project
    2016
    40Project
    2019
    100Project

    View Slide

  8. History
    LINE Fukuoka Quality Organization
    2014
    10Project
    2016
    40Project
    2019
    100Project
    Service Test Center
    QA Engineering Dept.
    2020

    View Slide

  9. Vision & Mission
    Mission
    Service Test Center
    Mission
    QA Engineering Dept.
    Vision
    Improve the quality further and shorten the
    lead time by improving not only the product
    quality but also the project quality.
    Manage the entire test process during the
    development phase to ensure product
    quality efficiently
    Design Implementation Test
    Planning
    Establish the new quality assurance style of LINE
    to deliver services to users in the best conditions

    View Slide

  10. Project Quality
    Design Implementation Test
    Planning
    Product Quality
    Internal Quality
    Process Quality
    - Software
    - Document
    - Code
    - Procedure
    Product Quality & Project Quality
    Influences Influences

    View Slide

  11. Design Implementation Test
    Planning
    Project Quality Product Quality
    Quality Control(QC)
    Quality Assurance(QA)
    QA Activity Domain

    View Slide

  12. Design Implementation Test
    Planning
    Rework Cost
    Product Quality
    Rework Cost Rework Cost Rework Cost
    Need to Expand Quality Activities
    - Many Initial Bugs

    View Slide

  13. Design Implementation Test
    Planning
    Rework Cost
    Product Quality
    Rework Cost Rework Cost Rework Cost
    Need to Expand Quality Activities
    Influences
    - Few Initial Bugs
    Project Quality

    View Slide

  14. Task
    QC
    QA
    Design Implementation Test
    Planning
    Test
    No Specific Task

    View Slide

  15. Design Implementation Test
    Planning
    Project Quality Product Quality
    QC
    QA
    Summary
    Service Test Center
    QA Engineering Dept.

    View Slide

  16. Agenda - Vision & Mission
    - Case Studies in Project
    - Learned from the Case Studies

    View Slide

  17. Case Studies in Project
    Product Quality Goal
    Test Planning in Parallel with Design
    Optimization of Regression Test

    View Slide

  18. Design Implementation Test
    Planning
    PM Server Side
    Developer Team
    Front End
    Developer Team
    Native App
    Developer Team
    QA Team
    - E2E Manual Test
    - E2E Automation Test
    Project Structure

    View Slide

  19. Case Studies in Project
    Product Quality Goal
    Test Planning in Parallel with Design
    Optimization of Regression Test

    View Slide

  20. Product Quality Goal
    AS IS
    No Product Quality Goal
    TO BE
    Set Product Quality Goal
    Rate of Bug by Release
    Bug Level
    Critical
    Major
    Minor
    - Change Failure Rate

    View Slide

  21. Product Quality Goal
    AS IS
    No Product Quality Goal
    TO BE
    Set Product Quality Goal
    Rate of Bug by Release
    Bug Level
    Critical
    Major
    Minor
    - Change Failure Rate

    View Slide

  22. Product Quality Goal
    Detailed Bug Level
    AS IS TO BE
    Bug Level
    Critical As soon as possible
    Major Modify as soon as necessary
    Minor Released in the next
    Bug Level Function A Function B Function C
    Critical
    Major
    Minor

    View Slide

  23. Product Quality Goal
    Detailed Bug Level
    AS IS TO BE
    Bug Level
    Critical As soon as possible
    Major Modify as soon as necessary
    Minor Released in the next
    Bug Level Function A Function B Function C
    Critical
    Major
    Minor
    - Frequency
    - Impact Range

    View Slide

  24. Product Quality Goal
    Positive Effect
    Analysis Improvement
    - Review the bug level according to the bug
    level table
    - Analysis of the reasons for lowering the
    change failure rate
    - 33% of bugs occur with the same function
    - There is no regression test
    - Add regression test

    View Slide

  25. Product Quality Goal
    Positive Effect
    Analysis Improvement
    - Review the bug level according to the bug
    level table
    - Analysis of the reasons for lowering the
    change failure rate
    - 33% of bugs occur with the same function
    - There is no regression test
    - Add regression test

    View Slide

  26. Product Quality Goal
    Positive Effect
    Analysis Improvement
    - Review the bug level according to the bug
    level table
    - Analysis of the reasons for lowering the
    change failure rate
    - 33% of bugs occur with the same function
    - There is no regression test
    - Add regression test

    View Slide

  27. Case Studies in Project
    Product Quality Goal
    Test Planning in Parallel with Design
    Optimization of Regression Test

    View Slide

  28. Optimization of Regression Test
    Situation
    Function List Manual Test Automation Test
    Enough? Duplicates?

    View Slide

  29. Different Structure
    Function
    Function List Automation Test
    Manual Test
    Subcategory Function
    Category Subcategory Function
    AS IS
    Optimization of Regression Test
    Structural Review

    View Slide

  30. Optimization of Regression Test
    AS IS
    TO BE Same Structure
    Structural Review
    Function
    Function List Automation Test
    Manual Test
    Category Subcategory Function
    Category Subcategory Function
    Same Structure
    Function
    Function List Automation Test
    Manual Test
    Subcategory Function
    Category Subcategory Function

    View Slide

  31. Optimization of Regression Test
    Category Subcategory Function
    Function List Manual + Automation Test
    TO BE Same Structure
    Category Subcategory Function Type
    Automation
    Manual
    TO BE Same Table
    Structural Review
    Same Table
    Function
    Function List Automation Test
    Manual Test
    Category Subcategory Function
    Category Subcategory Function

    View Slide

  32. Optimization of Regression Test
    Category Subcategory Function
    Function List Manual + Automation Test
    TO BE Same Table
    Category Subcategory Function Type
    Automation
    Manual
    Function List Manual + Automation Test
    TO BE Accessible
    Category Subcategory Function Type
    Category Subcategory Function Test URL
    Accessible
    Structural Review

    View Slide

  33. Optimization of Regression Test
    Function List Manual + Automation Test
    TO BE
    Category Subcategory Function Type
    Automation
    Manual
    Category Subcategory Function Test URL
    Same Structure
    Structural Review
    Same Table
    Accessible

    View Slide

  34. Optimization of Regression Test
    Removed Test Cases
    Bug Level Function A Function B Function C
    Critical
    Major
    Minor
    Function Pattern
    Function A
    Function B
    Function C Pattern 1
    Pattern 2
    Pattern 3
    Pattern 4
    Manual + Automation Test
    - Frequency
    - Impact Range

    View Slide

  35. Case Studies in Project
    Product Quality Goal
    Test Planning in Parallel with Design
    Optimization of Regression Test

    View Slide

  36. Design Implementation
    Test Plan
    Test Run
    Planning
    Design Implementation Test Run
    Planning
    1 Test Planning in Parallel with Design
    2 Review the Test Plan by All
    3 Sharing E2E and Unit Test
    Test Plan
    1 Just Before the Test Starts
    2 Reviewed by Only Some Members
    3 Not Sharing E2E and Unit Test
    Test Planning in Parallel with Design
    AS IS
    TO BE

    View Slide

  37. AS IS
    TO BE
    1 Test Planning in Parallel with Design
    2 Review the Test Plan by All
    3 Sharing E2E and Unit Test
    Test Plan
    1 Just Before the Test Starts
    2 Reviewed by Only Some Members
    3 Not Sharing E2E and Unit Test
    Test Planning in Parallel with Design
    Design Implementation
    Test Plan
    Test Run
    Planning
    Design Implementation Test Run
    Planning

    View Slide

  38. Agenda - Vision & Mission
    - Case Studies in a Project
    - Learned from the Case Studies

    View Slide

  39. Challenge

    View Slide

  40. I don't know what to do

    View Slide

  41. Design Implementation Test
    Planning
    QA Team
    Project Structure
    QA Team
    Expectations from Project
    Mission QA Engineering Dept.
    Gap

    View Slide

  42. Steps to Achievements
    Imagining
    Success
    Find and Refine
    Challenge
    Improve
    Beyond the Test

    View Slide

  43. Steps to Achievements
    Imagining
    Success
    Find and Refine
    Challenge
    Improve
    Beyond the Test

    View Slide

  44. Imagining Success
    Mission
    Start What I need to know Success
    - Participate in the design
    - Discuss testing with developers
    - Improve Project Quality
    - What design work is being done
    - What the developers are testing

    View Slide

  45. Direction to Go

    View Slide

  46. Steps to Achievements
    Imagining
    Success
    Find and Refine
    Challenge
    Improve
    Beyond the Test

    View Slide

  47. Design Implementation Test
    Planning
    Quality Challenge List
    Find and Refine Challenge
    - Find Challenge
    - Refine Challenge

    View Slide

  48. Steps to Achievements
    Imagining
    Success
    Find and Refine
    Challenge
    Improve
    Beyond the Test

    View Slide

  49. Design Implementation Test
    Planning
    Quality Challenge List
    Improve Beyond the Test
    Optimization of
    Regression Test
    Test Planning in
    Parallel with Design
    Product Quality Goal

    View Slide

  50. Improve Beyond the Test
    AS IS
    - No Product Quality Goal
    TO BE
    - Set Product Quality Goal
    Quality Challenge List
    - Think About Quality by All
    Project Situation
    - Set Project KPI
    Product Quality Goal

    View Slide

  51. Design Implementation Test
    Planning
    Quality Challenge List
    Improve Beyond the Test
    Optimization of
    Regression Test
    Test Planning in
    Parallel with Design
    Product Quality Goal

    View Slide

  52. AS IS
    - Just Before the Test Starts
    - Reviewed by Only Some
    Members
    - Not Sharing E2E and Unit
    Test
    TO BE
    - Test Planning in Parallel with
    Design
    - Review the Test Plan by All
    - Sharing E2E and Unit Test
    Quality Challenge List
    - Participate in the design
    - Discuss testing with
    developers
    Project Situation
    - Lack of Unit test
    Test Planning in Parallel with Design
    Improve Beyond the Test

    View Slide

  53. Design Implementation Test
    Planning
    Quality Challenge List
    Improve Beyond the Test
    Project Situation
    Optimization of
    Regression Test
    Test Planning in
    Parallel with Design
    Product Quality Goal

    View Slide

  54. I want to do
    × Team Problem
    Quality Challenge List Project Situation
    - Team Member’s Perspective
    Improve Beyond the Test

    View Slide

  55. Steps to Achievements
    Imagining
    Success
    Find and Refine
    Challenge
    Improve
    Beyond the Test

    View Slide

  56. Design Implementation Test
    Planning
    Project Quality Product Quality
    QC
    QA
    Summary
    Silo

    View Slide

  57. Anxiety
    I don't know what to do
    Summary
    I don't know if the team will accept it
    Imagining Success Solve Team Problems
    Feature Another Person
    Uncertainty

    View Slide

  58. Thank you

    View Slide