Share
Embed
Lehman versus Lehman: Dealing with Debt
Keynote Presented at MTD 2015, the 7th IEEE International Workshop on Managing Technical Debt.
October 2, 2015, Bremen, Germany.
https://www.sei.cmu.edu/community/td2015/
Arie van Deursen
October 02, 2015
Transcript
61 slides
-
Lehman Versus Lehman: Dealing with Debt Arie van
Deursen Del6 University of Technology @avandeursen 1 Keynote Presented at MTD 2015, The 7th IEEE InternaJonal Workshop on Managing Technical Debt. October 2, 2015, Bremen, Germany flickr: kfcatles -
What Is Technical Debt? • Ward Cunningham: – “I
coined the debt metaphor to explain the refactoring that we were doing.” • Michael Feathers: – “The refactoring effort needed to add a feature non invasively” 2 hVps://www.youtube.com/watch?v=7hL6g1aTGvo hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor -
3 Disclaimer: I’m not a “technical debt” expert:
The term occurs in only two of my papers -
Why the Great Glitch of July 8th Should
Scare You hVps://medium.com/message/why-‐the-‐great-‐glitch-‐of-‐july-‐8th-‐should-‐scare-‐you-‐b791002fff03 • United flights grounded • NYSE down • Wall Street Journal website down 6 -
8 So6ware EvoluJon Belady & Lehman, 1976. Law
of Con)nuing Change A system that is used, undergoes con@nuing change un@l it becomes more economical to replace it by a new or restructured system -
9 So6ware EvoluJon Belady & Lehman, 1976. Law
of Increasing Entropy The entropy of a system increases with @me unless specific work is executed to maintain or reduce it. -
16 hVps://www.npmjs.com/, Numbers from September 27, 2015 So6ware
Reuse: Success of the 2010s -
Library Upgrading: From the Trenches. System • 2004:
First version. – Maven – Acegi Security v1.0 – Spring, Struts, JSP • 2004-‐2011 – Dependencies carved in stone • 2011: Single sign on needed – Use Atlassian Crowd Libraries Issues • Crowd ✖ Acegi • Acegi included in Spring 2 • Crowd needs Spring 3 • Upgrade to Spring 3 needed • Struts JSP 2 ✖ Spring 3 • Upgrade to JSP 3 needed • JSP3 syntax ✖ JSP2 syntax Total upgrade effort: 1 week 19 Steven Raemaekers, Arie van Deursen, Joost Visser. Measuring so6ware library stability through historical version analysis. ICSM 2012: 378-‐387 -
Studying Upgradeability in Maven’s Versioned Libraries • 150,000
released jar files; 100,000 with source • As in Maven Central on July 30, 2011 • With resolved usage data 20 Steven Raemaekers, Arie van Deursen, Joost Visser. The maven repository dataset of metrics, changes, and dependencies. MSR 2013: 221-‐224 Julius Davies, Daniel M. German, Michael W. Godfrey, and Abram Hindle. So6ware BerJllonage -‐ determining the provenance of so6ware development arJfacts. EMSE, 2013 -
Breaking Changes in APIs 21 1. // Version
1 of Lib1 1. public class Lib1 { 2. public void foo() {...} 3. public int doStuff() {...} 4. } 1. // method2 uses foo & doStuff 1. public class System1 { 2. public void method2() { 3. Lib1 c1 = new Lib1(); 4. c1.foo(); 5. int x = c1.doStuff(); 6. anUnrelatedChange(); 7. }} 1. // Version 2 of Lib1 1. public class Lib1 { 2. public void foo(int bar) {...} 3. public String doStuff() {...} 4. } 1. // method2 uses foo & doStuff 1. public class System1 { 2. public void method2() { 3. Lib1 c1 = new Lib1(); 4. c1.foo(); 5. int x = c1.doStuff(); 6. } 7. } -
Breaking Changes? Binary Incompa@bili@es Focus of study:
Removals: Interface, class, method, or field Changes: Nr of parameters, or change in field / parameter / return type 22 “Pre-‐exis@ng client binaries must link and run with new releases of the component without recompiling.” -‐-‐-‐ the CLIRR tool. flickr.com/photos/dullhunk/ -
Breaking Changes in Maven 23 Steven Raemaekers, Arie
van Deursen, Joost Visser. SemanJc Versioning versus Breaking Changes: A Study of the Maven Repository. SCAM 2014. -
24 0 20,000 40,000 60,000
80,000 100,000 120,000 140,000 160,000 180,000 200,000 MR CR FR PTC RTC IR NPC MAI FTC CFR Frequency of most common breaking changes -
Is Anything Being Broken? • Maven libraries depend on
... each other – S uses L; – L updated to L’; – Will S compile with L’? • Assess impact by step by step injec@on of breaking changes 25 Let’s do this for 48,000 (L,L’)-‐pairs where L is used in some S. Steven Raemaekers, Arie van Deursen, Joost Visser. InjecJng Breaking API Changes to Assess the Impact of Library Updates. TUD-‐SERG-‐2014-‐020. SubmiHed. -
26 0 500,000 1,000,000 1,500,000
2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 4,500,000 MR CR FR PTC RTC IR NPC MAI FTC CFR Frequency #Errors # CompilaJon errors a6er injecJng breaking changes The breaking API changes cause actual pain. -
SemanJc Versioning Major . Minor . Patch: •
Major: incremented when incompaJble API changes are made • Minor: incremented when funcJonality is added in backward compaJble manner • Patch: incremented when backward-‐compaJble bug fixes are made 27 “Based on pre-‐exis@ng widespread common prac@ces” “SemanJc Versioning” semver.org -
Versioning Schemes used in Maven 28 Steven Raemaekers,
Arie van Deursen, Joost Visser. SemanJc Versioning versus Breaking Changes: A Study of the Maven Repository. SCAM 2014. -
DeprecaJon to the Rescue? “Depreca@ng exis@ng func@onality is a
normal part of soJware development and is oJen required” (semver) DeprecaJon in semanJc versioning: 1. Update documentaJon 2. Introduce deprecaJon in minor release 3. A6er that remove method in major release 31 -
DeprecaJon to the Rescue? DeprecaFon paGern Frequency
% Update pairs Delete public method (no deprecaJon) 86449 33.0% Add public deprecaJon (no deleJon) 793 0.3% Delete deprecated method 0 0.0% 32 -
DeprecaJon to the Rescue? DeprecaFon paGern Frequency
% Update pairs Delete public method (no deprecaJon) 86449 33.0% Add public deprecaJon (no deleJon) 793 0.3% Delete deprecated method 0 0.0% 33 -
Update Lags 34 L1 uses Jan 1 May
1 Feb 1 Mar 1 Apr 1 S1 L2 S2 S3 next ver. L3 Aug 1 S3-L Update lag patch major minor One minor release lagging -
Update Lags in Maven CorrelaJon with number of breaking
changes? Hardly: 0.14 for minor releases 0.02 and 0.08 for patch and major releases 35 Number of major / minor / patch versions lagging behind Breaking changes do not explain update lags -
Disclosure Triggers AVacks Before we knew it: An
empirical study of zero-‐ day aVacks in the real world. Leyla Bilge and Tudor Dimitras, CCS 2012 37 -
Digital ID for The Dutch • Ruby on Rails
< 3.2.11: CVE-‐2013-‐0155/0156 – SQL InjecJon, code execuJon, bypass authenJcaJon, denial of service – Fix involved a regression • 2.0.x affected, not fixed – Must upgrade to 2.3 • 9M Dutch ciJzens • Tax payments • DigiD immediately upgraded – Site was taken off line for 9 hours – Mostly for regression tes@ng purposes 38 Arie van Deursen, “Design for Upgradability and the Rails DigiD Outage”, hVp://avandeursen.com, January 2013 -
39 Others on CVE-‐2013-‐0155/0156: “A vulnerability that
allows easy remote code execu@on / shell access and a security patch that breaks compa@bility. You are doing great Rails team! This is preHy much the death of Rails reputa@on” -
Act Fast (Or Else: Backdoor) 40 Automated aHacks
began compromising Drupal 7 websites that were not patched or updated to Drupal 7.32 within hours of the announcement of SA-‐CORE-‐2014-‐005 -‐ Drupal core -‐ SQL injec@on. You should proceed under the assump@on that every Drupal 7 website was compromised unless updated or patched before Oct 15th, 11pm UTC, that is 7 hours aJer the announcement. Simply upda@ng to Drupal 7.32 will not remove backdoors. -
OWASP Top 10 1. InjecJon 2. Broken authenJcaJon
3. Cross-‐site scripJng 4. Insecure direct object references 5. Security misconfiguraJon 6. SensiJve data exposure 7. Missing funcJon level access control 8. Cross site request forgery 9. Using components with known vulnerabiliJes 10. Unvalidated redirects and forwards 41 hVps://www.owasp.org/index.php/Top_10_2013-‐Top_10 -
Projects With At Least One Vulnerable Dependency
43 20% of 12,000 open source systems 70% of 75 commercial systems Cadariu, Bouwers, van Deursen, Visser. Tracking known security vulnerabiliJes in third-‐party components. SANER, 2015. -
Upgradeability, Security, and Availability • Third party
libraries come with vulnerabiliJes aJer system-‐go-‐live • Upon disclosure, fast upgrading is required – System is either exposed to security risk; – Or not available. • “Upgrading debt” may come with high interest 45 -
Technical Debt at Scale: Apocalypse Now?
En@re engineering organiza@ons can be brought to a stand-‐s@ll under the debt load of an unconsolidated implementa@on 46 Ward Cunningham, OOPSLA 1992 -
48 • 200 interviews conducted • 100 published
on Guardian blog • All interviews anonymous • Organized into book a6erwards -
Too Big To Fail?
Or Too Big to Exist? “Employees at the big banks themselves do not believe their top people know what's going on; the big banks have simply become too complex and too big to manage.” “This system is highly dysfunc@onal, deeply entrenched, and enormously abusive, both to its own workers and the society it operates in.” 49 hVp://www.theguardian.com/commenJsfree/joris-‐luyendijk-‐banking-‐blog/2013/jun/19/banking-‐britain-‐beyond-‐control -
“Most Terrifying” We rely upon self-‐declara@on, upon what is
presented to us by a bank's internal management. But oJen they don't know what's going on, because banks today are so vast and hugely complex. The real threat is not a bank's management hiding things from us: it's the management not knowing themselves what the risks are, either because nobody realizes it or because some people are keeping it from their bosses. 50 hVp://www.theguardian.com/commenJsfree/joris-‐luyendijk-‐banking-‐blog/2013/oct/01/10-‐best-‐quotes-‐financial-‐insiders-‐banking-‐blog -
Technical Debt at the Bank "I am sure your
readers would be shocked if they realized just how crap IT has been organized in many banks as well as corpora@ons and government ministries.” “Nobody has a complete and in-‐depth overview any longer.” 51 hVp://www.theguardian.com/commenJsfree/joris-‐luyendijk-‐banking-‐blog/2012/may/30/former-‐it-‐salesman-‐voices-‐of-‐finance -
Some@mes we get a glimpse, when a company is
unavailable for days due to 'computer problems'. Have you no@ced these cases always take longer than expected? This is not because repairs take long. Finding out what the problem is in the first place – 'root cause analysis' – that's nearly always the most @me-‐consuming. 52 -
“The next global financial panic will start with
an IT crash” 53 -
Joris Luyendijk / Ron Rosenbaum: Journalists are Freudians
Belief that: • What maVers most is hidden and obscured • InvesJgaJve research / therapy is needed to idenJfy underlying phenomena • Revealing shocking outcomes will lead to recovery 54 -
Conclusions I: On Metaphors Metaphor is one of our
most important tools for trying to comprehend par@ally what cannot be comprehended totally George Lakoff and Mark Johnson, Metaphors we Live By, 1980 55 hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor -
What’s Your Refactoring Metaphor? Health • Update hygiene
• Infected code • Design in good shape • Goto considered harmful • The design needs to recover from some intense hacking Religion • I am guilty of bypassing the layering • I confess violaJng the invariant • I have sinned against the SOLID principles • Please forgive me for this singleton 56 -
Conclusions II: Technical Debt is a Security Concern
• Zero-‐day vulnerability & high technical debt? – Longer fix Jme – Lower availability, or longer exposed to risk • Security risks may be more compelling to management than principals or interest rates 57 What are the security implica@ons of your technical debt research? -
Conclusions III: On Learning I am in
favor of wri@ng code to reflect your current understanding of a problem even if that understanding is par@al 58 hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor -
Debt is RelaJve • The refactoring effort needed to
add a feature (resolve an issue) non invasively – Debt depends on features and issues to solve • Systems are used and society progresses – New libraries and versions come available – Actual usage affects our understanding 59 Can we rephrase Technical Debt as a knowledge problem? -
Conclusions IV: How Technical is Technical Debt? •
Technical debt is a people problem – Managing it requires people management. • Calls for “Debt Confessions”: – How do best projects deal with debt? – How do people live with this huge pile of debt? – Mix qualitaJve research and benchmarks 60 Michaela Greiler, Arie van Deursen, Margaret-‐Anne Storey. Test Confessions: A Study of TesJng PracJces for Plug-‐In Systems. ICSE, 2012 -
Lehman Versus Lehman: Dealing with Debt Arie van
Deursen Del6 University of Technology @avandeursen 61 Keynote Presented at MTD 2015, The 7th IEEE InternaJonal Workshop on Managing Technical Debt. October 2, 2015, Bremen, Germany flickr: kfcatles