Coverage Pragmatic. Hedonic What is Quality in Use? ü The concept that already existed ü Definition and measurement standard (ISO/IEC 25010 / 25022) ü It is not usability, but the overall quality of use.
we perform Quality in Use Quality In Use Develop phase QA phase ? Process Process quality Internal properties External properties Quality In use Software product Process measure Internal measure External measure Influences Influences Influences depends on depends on depends on Contexts of use
III. Select proper metrics I. Choice pilot service (LINE Pay, TH Bank) II. Task definition III. Weekly measurement IV. Indicator evaluation. I. Analysis result II. Analysis error code Search Measurement method Develop metric & Pilot Service measurement Analysis and improvement through results. Quality In Use standard & research
Method Source Effectiveness Task Completed Ratio = A/B A = Number of unique tasks completed B = Total number of unique tasks attempted API log GA Efficiency Task Time ratio = T/A A = Task time T = Ideal time (Step of Task * 2s) API log GA Freedom from risk Safety of VTFS affected by use of the system = 1-A/B A = Number of user put at Blocked System error by task B = Total number of user using service by task API log Satisfaction Proportion of user complaints about a task = 1- ((A/B) / (C/D)) A = Number of complaints for a particular task B = Total number of user using a particular task C = Number of complaints for a Service D = Total number of user using service VOC
ratio = tasks completed total tasks attempted Analysis ü Check each step drop-off ü Check Step Response code (exclude success code) ü Check Complete ratio trend by task ü Result = 0.312 ü # of tasks completed = 151,646 ü Total # tasks attempted = 488,944
their goal? Task time ratio = ideal time (step x 2s) task time * Ideal time = internal basis, each step(user action or screen transition) * 2s Analysis ü Check the steps that took the most time ü Check the back-end API performance ü Check task time ratio trend by task ü Result = 0.426 ü Task time = 18.8 ü Ideal time = 4*2
the service? Safety of user affected = facing error on a task total of user on a task ü Result = 0.992 ü # of user facing error = 1,279 ü Total # of user on task = 166,056 Analysis ü Analysis of the pattern of defined errors. ü Analysis of the types of system errors. ü Check the trend of errors by task.
the service? Proportion of user complaints about a task = 1 − (% & / ( ) ) A = Number of complaints for a particular task B = Total number of user using particular task C = Number of complaints for a Service D = Total number of user using service Analysis ü Analysis of VOC items by task. (In particular, it mainly deals with inquiries about use.) ü Check VOC Trend by task ü Result = 0.861 ü A = 51 ü B = 166,056 ü C = 2,788 ü D = 1,252,910
+ Freedom from Risk Number of Item QIU Score = Each measurement result are related. But we haven't found the relation yet. Metrics Result Effectiveness 31 Efficiency Freedom from Risk Satisfaction 99 QIU Score 64.5