Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A Tester's Perspective on (Open)SSL Vulnerabilities

A Tester's Perspective on (Open)SSL Vulnerabilities

Presentation at the Alten Seminar on New Trends in Testing, Nieuwegein, The Netherlands, June 24, 2014.

http://www.alten.nl/nl/alten/actueel/evenementen/?event=seminar-new-trends-in-testing-24-6

Partially based on my blog post on gotofail: http://avandeursen.com/2014/02/22/gotofail-security/

Arie van Deursen

June 24, 2014
Tweet

More Decks by Arie van Deursen

Other Decks in Programming

Transcript

  1. A  Tester’s  Perspec,ve  on   (Open)SSL  Vulnerabili,es   Arie  van

     Deursen  /  @avandeursen   Del?  University  of  Technology   Alten  Seminar  on  New  Trends  in  Tes,ng.   Nieuwegein,  June  24,  2014.   1  
  2. The  TU  Del?   So?ware  Engineering  Research  Group   Educa&on

      •  Programming,     so?ware  engineering   •  MSc,  BSc  projects   Research   •  So?ware  architecture   •  So?ware  tes,ng   •  Repository  mining   •  Collabora,on   •  Services   •  Model-­‐driven  engineering   •  End-­‐user  programming   2  
  3. TU  Del?  Tes,ng  Research   JavaScript-­‐enabled     Web  Crawling

      Integra,on  tes,ng  in  Plug-­‐in   Architectures   Tes,ng  in  collabora,ve   so?ware  development   3  
  4. Teaching   So?ware     Tes,ng   •  Java  

    •  Eclipse  (IntelliJ)   •  Git   •  JUnit   •  EclEmma   •  Maven   •  DevHub  CI   •  Mockito   4   github.com/SERG-­‐Del?/jpacman-­‐framework  
  5. Cogni,ve  Bias   “System  1”:  Fast,   ins,nc,ve,  emo,onal.  

      "System  2”:  Slow,   delibera,ve,  logical.     System  2  requires  effort.     It’s  happy  to  let  System  1   do  the  work   5  
  6. The  WYSIATI  Cogni,ve  Bias   •  “What  You  See  is

     All  There  Is”   –  If  you  like  the  evidence  you  see,  there  is  lidle  need  to   look  any  further.   •  WYSIATI  in  so?ware  tes,ng:   –  “It  works  on  my  machine”   –  “All  100  tests  pass,  we  can  ship”   •  Good  testers  understand  what  has  not  been  done   6  
  7. Adacks  happen!   Before  and  a?er  disclosure   Before  we

     knew  it:  An   empirical  study  of  zero-­‐ day  adacks  in  the  real   world.  Leyla  Bilge  and   Tudor  Dimitras,    CCS  2012   10  
  8. Unreachable  Code  Analysis   •  Compiler  can  spot  unreachable  code

      •  Undecidable  in  general,  so  no  guarantees   •  False  alarms  when  analysis  is  too  strict   Unreachable!   15  
  9. Test  Coverage?   •  Monitoring  overall  percentages  of  (statement)  

    coverage  unlikely  to  reveal  the  problem  (99%?)   •  Cri,cal  assessment  of  coverage  as  part  of   review  process  a  beder  op,on.   Unreachable!   16  
  10. Excep,on  Handling  in  C?   •  Returning  error  code  idiom

     in  C?   •  Very  defect  prone!   •  Study  at  ASML:     2  defects  per  1000  lines  of  code   (Brun,nk  et  al,  ICSE  2006)   17  
  11. Heartbleed   •  Discovered  April  7,  2014   •  OpenSSL

     leaks  passwords   •  Used  in  2/3d  of  all  web   servers  in  the  world   •  Error  in  “heartbeat”   extension  of  SSL.   •  Introduced  January  2012.   18  
  12. Code  and  Test?   Common  so?ware   development  approach:  

    •  Write  some  code   •  Write  some  (unit)  tests   •  Check  your  coverage   •  Improve  your  code   •  Un,l  coverage  is  OK   •  This  approach  will  not   find  Heartbleed   •  Heartbleed  is  due  to  an   omission.   •  Code-­‐centered  tes,ng   will  not  spot  it   •  Tes=ng  must  be   requirements-­‐based.   N.B.:  Proper  test-­‐driven  development  is  requirements  based   27  
  13. Experience-­‐Based  Tes,ng   •  Security  101:   – Inputs  should  never

     be  trusted   – Buffers  /  bound  checking  is  always  tricky   •  White  box:   – See  bound  manipula,ons   – Think  of  what  might  go  wrong   •  Black  box:   – Independent  inputs?     – Vary  message  and  length  independently   28  
  14. Change  Cipher  Spec  Messages   •  Ordering  of  messages  maders

      •  Sending  ChangeCipherSpec   message  too  fast  may  confuse  the   server   •  Fault  present  since  1998   •  Update  of  2012  made  MITM   adack  easier.   30  
  15. Protocol-­‐Based  Tes,ng   •  CCS  bug  requires  model-­‐based  tes,ng  

    •  Derive  state  machine  for  protocol   •  Hand-­‐write  state-­‐based  test  suite   –  Dedicated  oracles   •  Generate  (random)  event  sequences   –  Property  checking  as  oracle   31  
  16. Summary   •  #gotofail,  Heartbleed  and  CCS  are   very

     different.   •  Finding  such  faults  requires  rich   palede  of  tes,ng  techniques   •  “What  you  see  is  all  there  is”   – Need  to  think  about  the  test  cases     that  were  not  applied.   32