Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Re-thinking Regression Testing
Search
Mario Gonzalez Macedo
March 21, 2014
Research
1
130
Re-thinking Regression Testing
Talk I gave at Mountain West Ruby Conf 2014 about mutation testing
Mario Gonzalez Macedo
March 21, 2014
Tweet
Share
Other Decks in Research
See All in Research
129 2 th
0325
0
250
システムから変える 自分と世界を変えるシステムチェンジの方法論 / Systems Change Approaches
dmattsun
3
900
Large Vision Language Model (LVLM) に関する最新知見まとめ (Part 1)
onely7
22
4.8k
国際会議ACL2024参加報告
chemical_tree
1
350
Geospecific View Generation - Geometry-Context Aware High-resolution Ground View Inference from Satellite Views
satai
2
130
TransformerによるBEV Perception
hf149
1
580
言語処理学会30周年記念事業留学支援交流会@YANS2024:「学生のための短期留学」
a1da4
1
270
20241115都市交通決起集会 趣旨説明・熊本事例紹介
trafficbrain
0
710
FOSS4G 山陰 Meetup 2024@砂丘 はじめの挨拶
wata909
1
120
最近のVisual Odometryと Depth Estimation
sgk
1
310
メタヒューリスティクスに基づく汎用線形整数計画ソルバーの開発
snowberryfield
3
620
20240918 交通くまもとーく 未来の鉄道網編(太田恒平)
trafficbrain
0
360
Featured
See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Faster Mobile Websites
deanohume
305
30k
RailsConf 2023
tenderlove
29
940
Rebuilding a faster, lazier Slack
samanthasiow
79
8.7k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
95
17k
GraphQLの誤解/rethinking-graphql
sonatard
67
10k
How to Think Like a Performance Engineer
csswizardry
22
1.2k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Designing for humans not robots
tammielis
250
25k
Building an army of robots
kneath
302
44k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
5
450
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
2
170
Transcript
Re-thinking regression testing Mario González Macedo @maramono ortask.com
http://www.softwaretestpro.com/Item/5020/Minor-Risk-Patches/Cartoon
regression code coverage test redundancy ortask.com
ortask.com
regression code coverage test redundancy ortask.com
mutation testing
…vs code coverage
ortask.com
ortask.com
ortask.com
ortask.com
http://www.softwaretestpro.com/Item/5020/Minor-Risk-Patches/Cartoon
ortask.com
ortask.com
mutation score
(0, 1) worst mutation-adequate
ok… but what?
a type of fault injection
no extra code
change the code
mutate existing source code changes to tests: none
None
None
score… how?
mutants total killed mutants score =
zombies total killed zombies score =
ninjas total killed ninjas score =
how many mutations?
one mutation?
two mutations?
100 mutations?
k = mutation qty
assumption
assumption (a valid hypothesis)
competent programmer hypothesis
competent programmer hypothesis Some software bugs are caused by syntactically
small errors
competent programmer hypothesis k = mutation qty
competent programmer hypothesis k = 1
k
if k == 1 then first-order
if k == 2 then second-order
k-order mutant k mutations
k-order for simplicity easy to communicate
if k == 1 then low-order or simply first-order
if k > 1 then high-order
high-order… but why?
competent programmer hypothesis
competent programmer hypothesis coupling effect
coupling effect Other software bugs are caused by interactions of
small errors
coupling effect 1 < k < 6
why it works
for regression analysis one developer on fixes and features competent
programmer hypothesis
for regression analysis many contributors on fixes and features competent
programmer hypothesis coupling effect
so… why better?
test redundancy mutation score correlation: code coverage & test redundancy
ortask.com
ortask.com
mutation score test redundancy mutation score correlation: code coverage &
ortask.com
code coverage correlation: likelihood of bugs & test redundancy ortask.com
http://www.sustainablebabysteps.com/inflating-tires.html
http://willyoubemyyokoono.blogspot.com/2012_09_01_archive.html
correlation: mutation score & test redundancy code coverage ortask.com
how to apply it in practice
heckle mutant muJava Jester NinjaTurtles
heckle: obsolete as of Ruby 1.9 mutant: did not use
(rc) muJava Jester NinjaTurtles
https://github.com/ortask/mutator research-strength fast cross-language
painless mutation analysis a few guidelines
guideline 1: passing tests http://vector-magz.com/logo/quality-control-logo-item-4/
guideline 2: mutants 50 low-order experiment with high-order
guideline 3: cycle mutate score improve test
guideline 4: coupling low coupling high coupling detection
Thank you! Mario González Macedo @maramono ortask.com
Other mutation testing sources http://slid.es/markusschirp/mutation-testing https://github.com/mbj/mutant/tree/master/l ib/mutant/mutator/node https://github.com/seattlerb/heckle http://vimeo.com/10863662
Other sources http://vimeo.com/1683910