Ң݊ም
• ࡡᓃ߅Ҧ௴፬ࠋபɛ
• Server Side Developer for 8 years.
Slide 5
Slide 5 text
Ң݊ም
• ࡡᓃ߅Ҧ௴፬ࠋபɛ
• Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile
Slide 6
Slide 6 text
Ң݊ም
• ࡡᓃ߅Ҧ௴፬ࠋபɛ
• Server Side Developer.
• QA Engineer @ Glu Mobile
• SDET @ Real Networks
Slide 7
Slide 7 text
Ң݊ም
• ࡡᓃ߅Ҧ௴፬ࠋபɛ
• Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile
• SDET @ Real Networks
• SDE @ Medio Systems.
Slide 8
Slide 8 text
TODAY’S TOPIC
Slide 9
Slide 9 text
Software Quality
Slide 10
Slide 10 text
Why Software Quality is
Important?
Slide 11
Slide 11 text
When I am busy working on features
Slide 12
Slide 12 text
and finding customers
Slide 13
Slide 13 text
Because it is a feature
Slide 14
Slide 14 text
Because it is a strategic weapon
Slide 15
Slide 15 text
Trust me, I can build a
data center for you
Slide 16
Slide 16 text
VMWare: We know our service.
Slide 17
Slide 17 text
software is all about trust.
Slide 18
Slide 18 text
Users need to trust the cloud won’t
explode and bring down their business
Slide 19
Slide 19 text
slashdot effect -
overnight success turns into a
disaster.
Slide 20
Slide 20 text
How to build trust?
Slide 21
Slide 21 text
Trust is faith, Faith is a religion.
But engineering is a science
not religion.
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
Trust is Quality
Slide 24
Slide 24 text
What is Quality
• Availability
• Manageability
• Performance
• Reliability
• Scalability
• Security
Slide 25
Slide 25 text
Availability
• of your software
• of your IT / cloud infrastructure
• of your network environment
• of your external dependencies
• of your client side network environment
Slide 26
Slide 26 text
Calculate Availability
source: Economics of High Availability for Telecommunications Systems. An Intel® Primer
Availability
• Service Level Agreement
• Scheduled down time
• Unscheduled down time
Slide 32
Slide 32 text
Manageability
• Expose Information for System Admin to check the
condition of the online system.
• Configurability
• Monitorability
Slide 33
Slide 33 text
Manageability
• Monitoring is must, every cloud Infrastructure fails
• Amazon
• Azure
• Google
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
Monitoring
• Service credits are not automatics.
• client has to file a ticket immediately.
• claim downtime of certain time period.
• proof / log of outrage and area.
Reliability
• Availability v.s. Reliability
• Availability: system uptime
• Reliability: number of errors in a given time.
Slide 40
Slide 40 text
Reliability
• Crash or become unresponsive occasionally.
• Output is inconsistent.
Slide 41
Slide 41 text
Scalability
• Slashdot Effect.
• Scale Up and Scale Out
• ERP system are much more sticky.
• A mid-size website may have ~200 active sessions
• But a ERP system used by a company of 200 employees
would have 200 active sessions.
Slide 42
Slide 42 text
Security
• Secure communication.
• User Authorization and Authentication.
• Data Management.
• Usage/Operation logging.
Slide 43
Slide 43 text
Trust is tangible evidences.
Slide 44
Slide 44 text
• Internal Audit
•DEV Test
•QA Process
• External Audit
• ISO/27001 - security management standard.
• [email protected] :)
Slide 45
Slide 45 text
Software Metrics
Slide 46
Slide 46 text
How do we know Service A
can handle X amount of traffic
before we launch the Service?
Slide 47
Slide 47 text
Load Test
Slide 48
Slide 48 text
Load Tests
• Performance Test
• Stress Test
• Longevity Test
Slide 49
Slide 49 text
Performance Test
• find
• best case performance.
• baseline performance.
Slide 50
Slide 50 text
Performance Test
• Key figures
• request per minutes
• number of concurrent requests
• number of machines
• overall throughput
Important Figures
• minimum
• maximum
• mean
• standard deviation
• 75 percentile
• 95 percentile
• 98 percentile
• 99 percentile
• 99.9 percentile
Slide 53
Slide 53 text
A Picture Worth A Thousand Words
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
Stress Test
• Test how system perform under extreme situation.
• ie: iphone5 is on sales.
Slide 56
Slide 56 text
Longevity Test
• evaluates a system's ability to handle a constant, moderate
work load for a long time.
• find memory leak
• find hidden routine that exhausts system resource.
Slide 57
Slide 57 text
Monitoring System Under Test
• OS Level: CPU, Memory, Disc I/O, and Network Usage
• JVM: Memory Usage, Object counts, GC Cycles.
• Application Level: request count, method call time, size of
queue, cache hit/miss ratio
Slide 58
Slide 58 text
Load Test is the external view of
the system.
Slide 59
Slide 59 text
we shall measure
the code from developer’s
point of view too.