Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Software Quality Metrics
Search
yunglin
June 27, 2012
1
100
Software Quality Metrics
yunglin
June 27, 2012
Tweet
Share
More Decks by yunglin
See All by yunglin
NoSQL 大腸花
yunglin
27
3.8k
Manage cloud server with open source tools
yunglin
1
120
Java Developer Day 2013 Scala Future API
yunglin
3
730
Java Developer Day 2012 Introduction to Actor Model
yunglin
1
150
Featured
See All Featured
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Producing Creativity
orderedlist
PRO
346
40k
Speed Design
sergeychernyshev
32
1k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Side Projects
sachag
455
42k
Stop Working from a Prison Cell
hatefulcrawdad
270
20k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Rebuilding a faster, lazier Slack
samanthasiow
82
9.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.8k
Scaling GitHub
holman
459
140k
Transcript
ழᐶࣖ ࡡᓃ߅Ҧ О͑
[email protected]
Ң݊ም • ࡡᓃ߅Ҧ௴፬ɛ
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer. • QA Engineer
@ Glu Mobile • SDET @ Real Networks
Ң݊ም • ࡡᓃ߅Ҧ௴፬ࠋபɛ • Server Side Developer for 8 years.
• QA Engineer @ Glu Mobile • SDET @ Real Networks • SDE @ Medio Systems.
TODAY’S TOPIC
Software Quality
Why Software Quality is Important?
When I am busy working on features
and finding customers
Because it is a feature
Because it is a strategic weapon
Trust me, I can build a data center for you
VMWare: We know our service.
software is all about trust.
Users need to trust the cloud won’t explode and bring
down their business
slashdot effect - overnight success turns into a disaster.
How to build trust?
Trust is faith, Faith is a religion. But engineering is
a science not religion.
None
Trust is Quality
What is Quality • Availability • Manageability • Performance •
Reliability • Scalability • Security
Availability • of your software • of your IT /
cloud infrastructure • of your network environment • of your external dependencies • of your client side network environment
Calculate Availability source: Economics of High Availability for Telecommunications Systems.
An Intel® Primer
Network Availability • 2001/02/09 ʕߕऎቊ୵ʵॎ • 2003/10/02 ʕߕऎቊ୵ʵॎ • 2006/12/26
ܩ݆ήቤ • 2009/08/13 זдჱࠬ • 2010/03/04 ৷ඪ̀͠ήቤ • 2011/11/14 ऎආБၪࡌʬ
Availability of External Dependencies
None
SOA in reality
Availability • Service Level Agreement • Scheduled down time •
Unscheduled down time
Manageability • Expose Information for System Admin to check the
condition of the online system. • Configurability • Monitorability
Manageability • Monitoring is must, every cloud Infrastructure fails •
Amazon • Azure • Google
None
None
None
Monitoring • Service credits are not automatics. • client has
to file a ticket immediately. • claim downtime of certain time period. • proof / log of outrage and area.
Performance • Response Time. • Throughput • Network Bandwidth
Reliability • Availability v.s. Reliability • Availability: system uptime •
Reliability: number of errors in a given time.
Reliability • Crash or become unresponsive occasionally. • Output is
inconsistent.
Scalability • Slashdot Effect. • Scale Up and Scale Out
• ERP system are much more sticky. • A mid-size website may have ~200 active sessions • But a ERP system used by a company of 200 employees would have 200 active sessions.
Security • Secure communication. • User Authorization and Authentication. •
Data Management. • Usage/Operation logging.
Trust is tangible evidences.
• Internal Audit •DEV Test •QA Process • External Audit
• ISO/27001 - security management standard. •
[email protected]
:)
Software Metrics
How do we know Service A can handle X amount
of traffic before we launch the Service?
Load Test
Load Tests • Performance Test • Stress Test • Longevity
Test
Performance Test • find • best case performance. • baseline
performance.
Performance Test • Key figures • request per minutes •
number of concurrent requests • number of machines • overall throughput
Performance Test • Ꮠ͜ό࿁ᏀɢٙˀᏐ݊νО • ஈଣඎ throughput ݊щึᎇഹ request ᅰᄣ̋Ͼᄣ̋d݊щί൴
ཀݔࡈᔷұᓃࣛdึක ˀᔷɨࠥ • ਂމ͊Ը൙П݊щცࠅᄣ̋ዚኜא٫݊ॴዚኜٙਿ࠽f • ཫПɪᇞ༈Դ͜εˇዚኜ ٙ࣬ኽf
Important Figures • minimum • maximum • mean • standard
deviation • 75 percentile • 95 percentile • 98 percentile • 99 percentile • 99.9 percentile
A Picture Worth A Thousand Words
None
Stress Test • Test how system perform under extreme situation.
• ie: iphone5 is on sales.
Longevity Test • evaluates a system's ability to handle a
constant, moderate work load for a long time. • find memory leak • find hidden routine that exhausts system resource.
Monitoring System Under Test • OS Level: CPU, Memory, Disc
I/O, and Network Usage • JVM: Memory Usage, Object counts, GC Cycles. • Application Level: request count, method call time, size of queue, cache hit/miss ratio
Load Test is the external view of the system.
we shall measure the code from developer’s point of view
too.
what we thought != reality
Shopping Cart • SQL Query • Page Rendering • Session
Replication
Shopping Cart • SQL Query • Page Rendering • Session
Replication It is slow!
Shopping Cart • SQL Query • Page Rendering • Session
Replication • Load test result: 300ms.
Shopping Cart • SQL Query • Page Rendering • Session
Replication we shall cache sql results!
Shopping Cart • SQL Query • Page Rendering • Session
Replication • 50 ms • 20 ms • 230 ms
We need to measure the code.
To make better decision.
•Obverse ᝈ࿀ •Orient ሜ •Decide Ӕഄ •Act Бਗ
• Perf4j • Java Simon • Google Guava Stopwatch •
Metrics by Coda Hale. TOOLS
Thank you • Any Questions?