Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lehman versus Lehman: Dealing with Debt

Lehman versus Lehman: Dealing with Debt

Keynote Presented at MTD 2015, the 7th IEEE International Workshop on Managing Technical Debt.
October 2, 2015, Bremen, Germany.

https://www.sei.cmu.edu/community/td2015/

Arie van Deursen

October 02, 2015
Tweet

More Decks by Arie van Deursen

Other Decks in Technology

Transcript

  1. Lehman  Versus  Lehman:   Dealing  with  Debt   Arie  van

     Deursen   Del6  University  of  Technology   @avandeursen   1   Keynote  Presented  at  MTD  2015,   The  7th  IEEE  InternaJonal  Workshop  on     Managing  Technical  Debt.   October  2,  2015,  Bremen,  Germany   flickr:  kfcatles  
  2. What  Is  Technical  Debt?   •  Ward  Cunningham:   – “I

     coined  the  debt  metaphor  to   explain  the  refactoring  that  we   were  doing.”   •  Michael  Feathers:   – “The  refactoring  effort  needed  to   add  a  feature  non  invasively”   2   hVps://www.youtube.com/watch?v=7hL6g1aTGvo     hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor    
  3. 3   Disclaimer:  I’m  not  a  “technical  debt”  expert:  

    The  term  occurs  in  only  two  of  my  papers  
  4. Why  the  Great  Glitch  of     July  8th  Should

     Scare  You   hVps://medium.com/message/why-­‐the-­‐great-­‐glitch-­‐of-­‐july-­‐8th-­‐should-­‐scare-­‐you-­‐b791002fff03   •  United  flights  grounded   •  NYSE  down   •  Wall  Street  Journal     website  down   6  
  5. 8   So6ware  EvoluJon   Belady  &  Lehman,  1976.  Law

     of  Con)nuing  Change   A  system  that  is  used,  undergoes  con@nuing  change   un@l  it  becomes  more  economical     to  replace  it  by  a  new  or  restructured  system  
  6. 9   So6ware  EvoluJon   Belady  &  Lehman,  1976.  Law

     of  Increasing  Entropy   The  entropy  of  a  system  increases  with  @me  unless   specific  work  is  executed  to  maintain  or  reduce  it.  
  7. Library  Upgrading:  From  the  Trenches.   System   •  2004:

     First  version.   –  Maven   –  Acegi  Security  v1.0   –  Spring,  Struts,  JSP   •  2004-­‐2011   –  Dependencies  carved  in  stone   •  2011:  Single  sign  on  needed   –  Use  Atlassian  Crowd   Libraries  Issues   •  Crowd  ✖  Acegi   •  Acegi  included  in  Spring  2   •  Crowd  needs  Spring  3   •  Upgrade  to  Spring  3  needed   •  Struts  JSP  2    ✖  Spring  3   •  Upgrade  to  JSP  3  needed   •  JSP3  syntax  ✖  JSP2  syntax   Total  upgrade  effort:  1  week   19   Steven  Raemaekers,  Arie  van  Deursen,  Joost  Visser.  Measuring  so6ware  library   stability  through  historical  version  analysis.  ICSM  2012:  378-­‐387  
  8. Studying  Upgradeability  in   Maven’s  Versioned  Libraries   •  150,000

     released  jar  files;  100,000  with  source   •  As  in  Maven  Central  on  July  30,  2011   •  With  resolved  usage  data   20   Steven  Raemaekers,  Arie  van  Deursen,  Joost  Visser.   The  maven  repository  dataset  of  metrics,  changes,  and  dependencies.  MSR  2013:  221-­‐224   Julius  Davies,  Daniel  M.  German,  Michael  W.  Godfrey,  and  Abram  Hindle.     So6ware  BerJllonage  -­‐  determining  the  provenance  of  so6ware  development  arJfacts.  EMSE,  2013  
  9. Breaking  Changes  in  APIs   21   1. // Version

    1 of Lib1 1. public class Lib1 { 2. public void foo() {...} 3. public int doStuff() {...} 4. } 1. // method2 uses foo & doStuff 1. public class System1 { 2. public void method2() { 3. Lib1 c1 = new Lib1(); 4. c1.foo(); 5. int x = c1.doStuff(); 6. anUnrelatedChange(); 7. }} 1. // Version 2 of Lib1 1. public class Lib1 { 2. public void foo(int bar) {...} 3. public String doStuff() {...} 4. } 1. // method2 uses foo & doStuff 1. public class System1 { 2. public void method2() { 3. Lib1 c1 = new Lib1(); 4. c1.foo(); 5. int x = c1.doStuff(); 6. } 7. }
  10. Breaking  Changes?   Binary  Incompa@bili@es   Focus  of  study:  

    Removals:  Interface,  class,  method,  or  field   Changes:      Nr  of  parameters,  or  change  in                                            field  /  parameter  /  return  type   22   “Pre-­‐exis@ng  client  binaries  must  link  and  run  with  new  releases   of  the  component  without  recompiling.”  -­‐-­‐-­‐  the  CLIRR  tool.   flickr.com/photos/dullhunk/  
  11. Breaking  Changes  in  Maven   23   Steven  Raemaekers,  Arie

     van  Deursen,  Joost  Visser.     SemanJc  Versioning    versus  Breaking  Changes:    A  Study  of  the  Maven  Repository.  SCAM  2014.  
  12. 24   0   20,000   40,000   60,000  

    80,000   100,000   120,000   140,000   160,000   180,000   200,000   MR   CR   FR   PTC   RTC   IR   NPC   MAI   FTC   CFR   Frequency  of   most  common   breaking  changes  
  13. Is  Anything  Being  Broken?   •  Maven  libraries  depend  on

     ...  each  other   – S  uses  L;     – L  updated  to  L’;   – Will  S  compile  with  L’?   •  Assess  impact  by  step  by  step    injec@on  of   breaking  changes   25   Let’s  do  this  for  48,000  (L,L’)-­‐pairs     where  L  is  used  in  some  S.   Steven  Raemaekers,  Arie  van  Deursen,  Joost  Visser.    InjecJng  Breaking  API  Changes  to  Assess  the   Impact  of  Library  Updates.  TUD-­‐SERG-­‐2014-­‐020.  SubmiHed.  
  14. 26   0   500,000   1,000,000   1,500,000  

    2,000,000   2,500,000   3,000,000   3,500,000   4,000,000   4,500,000   MR   CR   FR   PTC   RTC   IR   NPC   MAI   FTC   CFR   Frequency   #Errors   #  CompilaJon  errors   a6er  injecJng   breaking  changes   The  breaking  API   changes   cause  actual  pain.  
  15. SemanJc  Versioning   Major  .  Minor  .  Patch:   • 

    Major:  incremented  when     incompaJble  API  changes  are  made   •  Minor:  incremented  when  funcJonality  is   added  in  backward  compaJble  manner   •  Patch:  incremented  when     backward-­‐compaJble  bug  fixes  are  made   27   “Based  on  pre-­‐exis@ng  widespread  common  prac@ces”   “SemanJc   Versioning”   semver.org  
  16. Versioning  Schemes  used  in  Maven   28   Steven  Raemaekers,

     Arie   van  Deursen,  Joost  Visser.     SemanJc  Versioning    versus   Breaking  Changes:    A  Study   of  the  Maven  Repository.   SCAM  2014.  
  17. DeprecaJon  to  the  Rescue?   “Depreca@ng  exis@ng  func@onality  is  a

     normal   part  of  soJware  development  and  is  oJen   required”  (semver)     DeprecaJon  in  semanJc  versioning:   1.  Update  documentaJon   2.  Introduce  deprecaJon  in  minor  release   3.  A6er  that  remove  method  in  major  release   31  
  18. DeprecaJon  to  the  Rescue?   DeprecaFon  paGern   Frequency  

    %  Update  pairs     Delete  public  method   (no  deprecaJon)   86449   33.0%   Add  public  deprecaJon   (no  deleJon)   793     0.3%   Delete  deprecated  method   0   0.0%   32  
  19. DeprecaJon  to  the  Rescue?   DeprecaFon  paGern   Frequency  

    %  Update  pairs     Delete  public  method   (no  deprecaJon)   86449   33.0%   Add  public  deprecaJon   (no  deleJon)   793     0.3%   Delete  deprecated  method   0   0.0%   33  
  20. Update  Lags   34   L1 uses Jan 1 May

    1 Feb 1 Mar 1 Apr 1 S1 L2 S2 S3 next ver. L3 Aug 1 S3-L Update lag patch major minor One minor release lagging
  21. Update  Lags  in  Maven   CorrelaJon  with  number  of  breaking

     changes?   Hardly:  0.14  for  minor  releases   0.02  and  0.08  for  patch  and  major  releases     35   Number  of  major  /  minor  /  patch  versions  lagging  behind   Breaking  changes  do  not  explain  update  lags  
  22. Disclosure  Triggers  AVacks   Before  we  knew  it:  An  

    empirical  study  of  zero-­‐ day  aVacks  in  the  real   world.  Leyla  Bilge  and   Tudor  Dimitras,    CCS  2012   37  
  23. Digital  ID  for  The  Dutch   •  Ruby  on  Rails

     <  3.2.11:   CVE-­‐2013-­‐0155/0156   –  SQL  InjecJon,  code   execuJon,  bypass   authenJcaJon,  denial  of   service   –  Fix  involved  a  regression   •  2.0.x  affected,  not  fixed   –  Must  upgrade  to  2.3   •  9M  Dutch  ciJzens     •  Tax  payments   •  DigiD  immediately   upgraded   –  Site  was  taken  off  line   for  9  hours   –  Mostly  for  regression   tes@ng  purposes   38   Arie  van  Deursen,  “Design  for  Upgradability  and  the  Rails  DigiD  Outage”,     hVp://avandeursen.com,  January  2013  
  24. 39   Others  on  CVE-­‐2013-­‐0155/0156:     “A  vulnerability  that

     allows     easy  remote  code  execu@on  /  shell  access     and  a  security  patch  that  breaks  compa@bility.       You  are  doing  great  Rails  team!     This  is  preHy  much  the  death  of  Rails  reputa@on”    
  25. Act  Fast  (Or  Else:  Backdoor)   40   Automated  aHacks

     began  compromising  Drupal  7   websites  that  were  not  patched  or  updated  to   Drupal  7.32  within  hours  of  the  announcement  of   SA-­‐CORE-­‐2014-­‐005  -­‐  Drupal  core  -­‐  SQL  injec@on.       You  should  proceed  under  the  assump@on  that   every  Drupal  7  website  was  compromised  unless   updated  or  patched  before  Oct  15th,  11pm  UTC,   that  is  7  hours  aJer  the  announcement.     Simply  upda@ng  to  Drupal  7.32     will  not  remove  backdoors.  
  26. OWASP  Top  10   1.  InjecJon   2.  Broken  authenJcaJon

      3.  Cross-­‐site  scripJng   4.  Insecure  direct  object   references   5.  Security   misconfiguraJon   6.  SensiJve  data   exposure   7.  Missing  funcJon  level   access  control   8.  Cross  site  request   forgery   9.  Using  components   with  known   vulnerabiliJes   10. Unvalidated  redirects   and  forwards   41   hVps://www.owasp.org/index.php/Top_10_2013-­‐Top_10  
  27. Projects  With  At  Least  One     Vulnerable  Dependency  

    43   20%  of  12,000  open  source  systems   70%  of  75  commercial  systems   Cadariu,  Bouwers,  van  Deursen,  Visser.  Tracking  known  security   vulnerabiliJes  in  third-­‐party  components.  SANER,  2015.  
  28. Upgradeability,     Security,  and  Availability   •  Third  party

     libraries  come  with  vulnerabiliJes   aJer  system-­‐go-­‐live   •  Upon  disclosure,  fast  upgrading  is  required   –  System  is  either  exposed  to  security  risk;   –  Or  not  available.   •  “Upgrading  debt”  may  come  with  high  interest   45  
  29. Technical  Debt  at  Scale:   Apocalypse  Now?      

      En@re  engineering  organiza@ons     can  be  brought  to  a  stand-­‐s@ll     under  the  debt  load     of  an  unconsolidated  implementa@on   46   Ward  Cunningham,  OOPSLA  1992  
  30. 48   •  200  interviews  conducted   •  100  published

     on  Guardian  blog   •  All  interviews  anonymous   •  Organized  into  book  a6erwards  
  31.              Too  Big  To  Fail?

         Or  Too  Big  to  Exist?     “Employees  at  the  big  banks  themselves  do  not   believe  their  top  people  know  what's  going  on;   the  big  banks  have  simply  become     too  complex  and  too  big  to  manage.”     “This  system  is  highly  dysfunc@onal,  deeply   entrenched,  and  enormously  abusive,  both  to  its   own  workers  and  the  society  it  operates  in.”     49   hVp://www.theguardian.com/commenJsfree/joris-­‐luyendijk-­‐banking-­‐blog/2013/jun/19/banking-­‐britain-­‐beyond-­‐control  
  32. “Most  Terrifying”   We  rely  upon  self-­‐declara@on,  upon  what  is

     presented  to   us  by  a  bank's  internal  management.       But  oJen  they  don't  know  what's  going  on,  because   banks  today  are  so  vast  and  hugely  complex.     The  real  threat  is  not     a  bank's  management  hiding  things  from  us:     it's  the  management  not  knowing  themselves     what  the  risks  are,  either  because  nobody  realizes  it     or  because  some  people  are  keeping  it  from  their  bosses.   50   hVp://www.theguardian.com/commenJsfree/joris-­‐luyendijk-­‐banking-­‐blog/2013/oct/01/10-­‐best-­‐quotes-­‐financial-­‐insiders-­‐banking-­‐blog  
  33. Technical  Debt  at  the  Bank   "I  am  sure  your

     readers  would  be  shocked  if  they   realized  just  how  crap  IT  has  been  organized  in   many  banks  as  well  as  corpora@ons  and   government  ministries.”   “Nobody  has  a  complete     and  in-­‐depth  overview     any  longer.”   51   hVp://www.theguardian.com/commenJsfree/joris-­‐luyendijk-­‐banking-­‐blog/2012/may/30/former-­‐it-­‐salesman-­‐voices-­‐of-­‐finance  
  34. Some@mes  we  get  a  glimpse,  when  a  company  is  

    unavailable  for  days  due  to  'computer  problems'.       Have  you  no@ced  these  cases  always  take  longer   than  expected?     This  is  not  because  repairs  take  long.       Finding  out  what  the  problem  is  in  the  first  place   –  'root  cause  analysis'  –  that's  nearly  always  the   most  @me-­‐consuming.     52  
  35. Joris  Luyendijk  /  Ron  Rosenbaum:   Journalists  are  Freudians  

    Belief  that:   •  What  maVers  most  is  hidden  and  obscured   •  InvesJgaJve  research  /  therapy  is  needed  to   idenJfy  underlying  phenomena   •  Revealing  shocking     outcomes  will     lead  to  recovery   54  
  36. Conclusions  I:  On  Metaphors   Metaphor  is  one  of  our

     most  important  tools     for  trying  to  comprehend  par@ally     what  cannot  be  comprehended  totally   George  Lakoff  and  Mark  Johnson,     Metaphors  we  Live  By,  1980     55   hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor  
  37. What’s  Your  Refactoring  Metaphor?   Health   •  Update  hygiene

      •  Infected  code   •  Design  in  good  shape   •  Goto  considered   harmful   •  The  design  needs  to   recover  from  some   intense  hacking   Religion   •  I  am  guilty  of  bypassing   the  layering   •  I  confess  violaJng  the   invariant   •  I  have  sinned  against   the  SOLID  principles   •  Please  forgive  me  for   this  singleton   56  
  38. Conclusions  II:   Technical  Debt  is  a  Security  Concern  

    •  Zero-­‐day  vulnerability  &  high  technical  debt?   – Longer  fix  Jme   – Lower  availability,  or  longer  exposed  to  risk   •  Security  risks  may  be  more  compelling  to   management  than  principals  or  interest  rates   57   What  are  the  security  implica@ons  of   your  technical  debt  research?  
  39. Conclusions  III:  On  Learning       I  am  in

     favor  of  wri@ng  code  to  reflect     your  current  understanding  of  a  problem     even  if  that  understanding  is  par@al   58   hVp://c2.com/cgi/wiki?WardExplainsDebtMetaphor  
  40. Debt  is  RelaJve   •  The  refactoring  effort  needed  to

     add  a  feature   (resolve  an  issue)  non  invasively   – Debt  depends  on  features  and  issues  to  solve   •  Systems  are  used  and  society  progresses   – New  libraries  and  versions  come  available   – Actual  usage  affects  our  understanding   59   Can  we  rephrase  Technical  Debt  as  a   knowledge  problem?  
  41. Conclusions  IV:   How  Technical  is  Technical  Debt?   • 

    Technical  debt  is  a  people  problem   – Managing  it  requires  people  management.   •  Calls  for  “Debt  Confessions”:   – How  do  best  projects  deal  with  debt?   – How  do  people  live  with  this  huge  pile  of  debt?   – Mix  qualitaJve  research  and  benchmarks   60   Michaela  Greiler,  Arie  van  Deursen,  Margaret-­‐Anne  Storey.     Test  Confessions:  A  Study  of  TesJng  PracJces  for  Plug-­‐In  Systems.  ICSE,  2012  
  42. Lehman  Versus  Lehman:   Dealing  with  Debt   Arie  van

     Deursen   Del6  University  of  Technology   @avandeursen   61   Keynote  Presented  at  MTD  2015,   The  7th  IEEE  InternaJonal  Workshop  on     Managing  Technical  Debt.   October  2,  2015,  Bremen,  Germany   flickr:  kfcatles