A Tester's Perspective on (Open)SSL Vulnerabilities

Presentation at the Alten Seminar on New Trends in Testing, Nieuwegein, The Netherlands, June 24, 2014.

http://www.alten.nl/nl/alten/actueel/evenementen/?event=seminar-new-trends-in-testing-24-6

Partially based on my blog post on gotofail: http://avandeursen.com/2014/02/22/gotofail-security/

Transcript

1. A  Tester’s  Perspec,ve  on   (Open)SSL  Vulnerabili,es   Arie  van  Deursen  /  @avandeursen   Del?  University  of  Technology   Alten  Seminar  on  New  Trends  in  Tes,ng.   Nieuwegein,  June  24,  2014.   1  
2. The  TU  Del?   So?ware  Engineering  Research  Group   Educa&on   •  Programming,     so?ware  engineering   •  MSc,  BSc  projects   Research   •  So?ware  architecture   •  So?ware  tes,ng   •  Repository  mining   •  Collabora,on   •  Services   •  Model-­‐driven  engineering   •  End-­‐user  programming   2  
3. TU  Del?  Tes,ng  Research   JavaScript-­‐enabled     Web  Crawling   Integra,on  tes,ng  in  Plug-­‐in   Architectures   Tes,ng  in  collabora,ve   so?ware  development   3  
4. Teaching   So?ware     Tes,ng   •  Java   •  Eclipse  (IntelliJ)   •  Git   •  JUnit   •  EclEmma   •  Maven   •  DevHub  CI   •  Mockito   4   github.com/SERG-­‐Del?/jpacman-­‐framework  
5. Cogni,ve  Bias   “System  1”:  Fast,   ins,nc,ve,  emo,onal.     "System  2”:  Slow,   delibera,ve,  logical.     System  2  requires  effort.     It’s  happy  to  let  System  1   do  the  work   5  
6. The  WYSIATI  Cogni,ve  Bias   •  “What  You  See  is  All  There  Is”   –  If  you  like  the  evidence  you  see,  there  is  lidle  need  to   look  any  further.   •  WYSIATI  in  so?ware  tes,ng:   –  “It  works  on  my  machine”   –  “All  100  tests  pass,  we  can  ship”   •  Good  testers  understand  what  has  not  been  done   6  
7. Secure  Socket  Layer  (SSL)  /     Transport  Layer  Security  (TLS)   7  
8. SSL/TLS  Handshake  Protocol   8  
10. Adacks  happen!   Before  and  a?er  disclosure   Before  we  knew  it:  An   empirical  study  of  zero-­‐ day  adacks  in  the  real   world.  Leyla  Bilge  and   Tudor  Dimitras,    CCS  2012   10  
13. hdp://avandeursen.com/2014/02/22/gotofail-­‐security/   Fault  in  Apple’s  Secure  Socket  Layer  code   13  
14. Formakng  is  a  Security  Concern   14  
15. Unreachable  Code  Analysis   •  Compiler  can  spot  unreachable  code   •  Undecidable  in  general,  so  no  guarantees   •  False  alarms  when  analysis  is  too  strict   Unreachable!   15  
16. Test  Coverage?   •  Monitoring  overall  percentages  of  (statement)   coverage  unlikely  to  reveal  the  problem  (99%?)   •  Cri,cal  assessment  of  coverage  as  part  of   review  process  a  beder  op,on.   Unreachable!   16  
17. Excep,on  Handling  in  C?   •  Returning  error  code  idiom  in  C?   •  Very  defect  prone!   •  Study  at  ASML:     2  defects  per  1000  lines  of  code   (Brun,nk  et  al,  ICSE  2006)   17  
18. Heartbleed   •  Discovered  April  7,  2014   •  OpenSSL  leaks  passwords   •  Used  in  2/3d  of  all  web   servers  in  the  world   •  Error  in  “heartbeat”   extension  of  SSL.   •  Introduced  January  2012.   18  
21. •  No  need  for     Man-­‐in-­‐the-­‐Middle   •  Anyone  can  just   read  …  passwords   21  
23. 23   Specs  for  the   TLS  Heartbeat   extension  
25. The  Heartbleed  Fix   25   Too  short   Too  long  
26. Traceability?   Specifica,on   Code   Only  the  fix  has  traceability!   26  
27. Code  and  Test?   Common  so?ware   development  approach:   •  Write  some  code   •  Write  some  (unit)  tests   •  Check  your  coverage   •  Improve  your  code   •  Un,l  coverage  is  OK   •  This  approach  will  not   find  Heartbleed   •  Heartbleed  is  due  to  an   omission.   •  Code-­‐centered  tes,ng   will  not  spot  it   •  Tes=ng  must  be   requirements-­‐based.   N.B.:  Proper  test-­‐driven  development  is  requirements  based   27  
28. Experience-­‐Based  Tes,ng   •  Security  101:   – Inputs  should  never  be  trusted   – Buffers  /  bound  checking  is  always  tricky   •  White  box:   – See  bound  manipula,ons   – Think  of  what  might  go  wrong   •  Black  box:   – Independent  inputs?     – Vary  message  and  length  independently   28  
29. Bonus  Track:  CCS   29  
30. Change  Cipher  Spec  Messages   •  Ordering  of  messages  maders   •  Sending  ChangeCipherSpec   message  too  fast  may  confuse  the   server   •  Fault  present  since  1998   •  Update  of  2012  made  MITM   adack  easier.   30  
31. Protocol-­‐Based  Tes,ng   •  CCS  bug  requires  model-­‐based  tes,ng   •  Derive  state  machine  for  protocol   •  Hand-­‐write  state-­‐based  test  suite   –  Dedicated  oracles   •  Generate  (random)  event  sequences   –  Property  checking  as  oracle   31  
32. Summary   •  #gotofail,  Heartbleed  and  CCS  are   very  different.   •  Finding  such  faults  requires  rich   palede  of  tes,ng  techniques   •  “What  you  see  is  all  there  is”   – Need  to  think  about  the  test  cases     that  were  not  applied.   32