The investigation of Web Application Vulnerabilities in Japan #appsecapac2014

Transcript

1. The investigation of Web Application Vulnerabilities 
 in Japan
 ೔ຊʹ͓͚ΔWebΞϓϦέʔγϣϯ੬ऑੑͷ࣮ଶௐࠪ 
 ߴڮ ߃थ Koki Takahashi kouki@xss.jp 1
2. About  Me • Koki Takahashi • Keio University
 Senior, Faculty of Environment and Information Studies • Graduated “Security Camp 2012” Web-sec Class
 
 a speaker of Security Camp caravan in Morioka • Interested in XSS Vulnerability 2
3. • This research is composed of 4 type approach • Vulnerability checking for server manager, ordinary user,security researcher • Vulnerability reporting and managing optimization / Vulnerability survey Overview 3
4. • Vulnerability detection and reporting • XSS inspection service for administrators • XSS inspection and reporting extension • XSS reporting database and lifecycle • Vulnerability Information Lifecycle Management • Vulnerability survey in Japan Agenda 4
5. work1 Vulnerability detection and reporting 5
6. XSS inspection service for administrators 
 (2011) Vulnerability checking for server manager upload the token to server check if a target server has valid token get page content search a form and submit tags to them check if a sent tags are escaped check if a link is target detect links Access to the service and get token ! Yellow: User Green:Scanner ! ! ! ! ! ! ! ! Output result (email / web ) • Users can scan their websites by getting the token from our service and uploading it on their websites • Only 3 steps for this inspection Koki Nakayasu, Tomonori Yamamoto, Yuki Uehara, Keiji Takeda Proposal and implementation of Web application for Cross-site Scripting Inspection (CSS 2011) 6
7. XSS inspection service for administrators 
 (2011) Vulnerability checking for ordinary user • XSS inspection extension 
 for ordinary users • Users can easily recognise whether 
 pages which they access are safe or not • The extension sends the url to
 the inspection server • The inspection server checks 
 the website • The inspection server saves the result
 to the db for reporting inspection server DB extension Web site Google chrome 2:Send url of browsing 3:Receive scan result Get the response Send parameter 
 to target save result get result User 1:Browse 7
8. The Information Security Early Warning Partnership • In 2004, Vulnerability handling standard published 
 by METI(Ministry of Economy, Trade and Industry) • IPA, JP-CERT/CC manage vulnerability information of software and web sites • Reporter can send vulnerability information anonymously • Prevention of disclosure of vulnerability information • Publish statistics of vulnerability report 8
9. XSS reporting database and lifecycle 
 (2011-) Vulnerability checking for security researcher • In Japan, IPA(information promotion agency)
 collects vulnerability reports • IPA sends these reports to the administrators
 and encourages them to fix • In my research, I reported many
 vulnerabilities on websites • 2011 July - September, we reported
 167 of 198 vulnerabilities • “I need a new XSS reporting system“ 16% 84% me other ιϑτ΢ΣΞ౳ͷ੬ऑੑؔ࿈৘ใʹؔ͢Δಧग़ঢ়گ [2011೥ୈ3࢛൒ظʢ7݄ʙ9݄ʣ] 9
10. • Users can scan, manage, and send vulnerability information in the Web site • Usage scenario • Users put url to the scan server • The scan server sends parameters to the url and check their escape status.
 The result is automatically saved to the DB as a report • These reports are checked weekly by a professor • Users can send reports by a click 
 This system handled
 1500 vulnerability reports
 in 3 years XSS reporting database and lifecycle 
 (2011-) Vulnerability checking for security researcher 10
11. work2 Vulnerability Information Lifecycle Management 11
12. • The system optimizes XSS reporting • Using this vulnerability management system to Vulnerability coordinating 
 
 association • An optimization of reporting and fixing after finding web site vulnerability • Define web vulnerability format based on http request and response 
 
 information XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Koki Takahashi Vulnerability Information Lifecycle Management, a Proposal. (Bachelor thesis, Keio University, 2014) Abstract 12
13. • Proposal Format • File outline…Vulnerability type based on CWE, abstract etc.. • Vulnerability researcher info…name, address etc.. • Website administrator info…name, address etc.. • Page / Web form information • Format can include information of more than one discover XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Approach 13
14. • The format has page forms, request data, response data • Page / Form…Define the form which has a problem • Request… Define the data sent in scan • Response…Define the data received in scan XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Approach 14
15. • Scan from the Web Interface • specify the target by url parameter • reflection XSSɺHTTP header injection • CMS Version checking • Check Word Press/Movable Type version from meta tag • Show the result • Send result data automatically to the vulnerability DB XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Approach 15
16. • View and Edit, Output of vulnerability information • Edit function of the each elements • DB can be imported from API and Web form • Generate the exploit code XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Approach 16
17. • The DB can generate PoC of XSS • Reporter download a PoC from the DB • Reporters send this PoC to the Administrator • The PoC has two frames
 • 1: Users can send parameters same as scan? • 2: Users can check responses in scan • PoC was generated by DB XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation Approach 17
18. XSS reporting database and lifecycle 
 (2011-) Vulnerability reporting and managing optimisation • Report XSS with PoC to Administrator • 36 XSS which reported through IPA • report with PoC which be generated by DB automatically • 30 web sites were fixed • Average fix days is 55 days • 80% of fixation is in 90 days with PoC total fixed in 90 days 80% 66% Evaluation 18
19. work3 The survey of Web Application Vulnerabilities 19
20. The survey of Web Application Vulnerabilities Vulnerability survey • Research about escape leaking which causes Simple Reflection XSS • Purpose / Question • What percentage of web sites are vulnerable? • What kind of web site are vulnerable? ! • Result • 9762 forms have escape leaking (6.52%) • It’s no difference between using https or not , using frame option or not • Block of top 1000 domains is low rate, but tendency can’t be seen 20
21. Subject of survey Vulnerability survey • Top 16000 “.jp” domains of access ranking (2013/10/11) • access ranking
 from alexa ( http:// www.alexa.com ) • Index page of these domains and pages which was linked index page • 871339 pages • 149777 forms ~~.jp
 ~~.com ~~.net ~~.li
 —.jp list of alexa ~~.jp
 —.jp pick up Access top page —.jp
 —.jp/search.php … domain list page list /index.html /search.php
 /about.html
 /news.php
 ….. pick up
 anchors Add these pages to list 21
22. Survey process Vulnerability survey • Access to each page of page list • Send parameters include “<”, “>” to form • Save result, http request, and http response to mongoDB • Result • 9762 forms have escape leaking (6.52%) —.jp
 —.jp/search.php … page list mongoDB scanner Send parameters and Check Response Output json includes
 scan result and request, response 22
23. Preliminary survey 23
24. Preliminary survey(headers) Vulnerability survey • Subject • jp domain: 1000domains • pickup 1000domains 
 from 16000domains randomly • Collected header information • Server: 303 types • X-Powered-By: 124 types subject domain subject page subject form escape leaking 1000 47221 9333 610
 (6.54% of forms) server header x-powered- by header all 303 124 XSSed 59 45 24
25. Preliminary survey(Server headers) Vulnerability survey • Escape leaking on IIS is a low ratio • Escape leaking on nginx is a high rate than other server • but number of forms is small • On Apache is low ratio • On Apache with version number is high rate kind forms escape leaking percentag e Apache 6363 406 6.38% Apache(1.x) 301 35 11.63% Apache(2.x) 1451 163 11.23% IIS 814 44 5.41% IIS 6 343 18 5.25% IIS 7 409 24 5.86% nginx 368 25 11.74% 25
26. Preliminary survey(X-Powered-By headers) Vulnerability survey • ASP.net is low ratio • validateRequest function • PHP • PHP is high rate in any version • PHP/5.x is newer than 4,
 but PHP/5.x is higher than 4 kind forms escape leaking percenta ge ASP.NET 617 37 6.00% PHP 2231 267 11.97% PHP/4.x 146 15 10.27% PHP/5.x 2231 267 12.09% Phusion Passenger 50 1 2.00% Servlet 41 1 2.70% 26
27. Main survey(URL information) 27
28. Main survey(access frequency) Vulnerability survey • 9762 forms have escape leaking (6.52%) • divide 16000 domains each 1000 domains 
 by ranking • Block of top 1000 domains is 2.778% • other blocks are 4~11% • tendency can’t be seen • Famous websites tend to escape parameters 0 3 6 9 12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 5.095 7.511 4.598 7.455 3.695 5.176 3.723 7.689 6.316 4.19 4.963 5.522 5.952 8.897 11.163 2.778 Escape leaking(%) 28
29. Main survey(organizational type)
 Vulnerability survey • co.jp , gr.jp are high rate • Web sites by company have more form than other kind • Inquiry form, Shopping cart etc.. • go.jp (govermental websites) is safely escaping queries in general • Likewise, ac.jp(academic websites) is also safely • In this investigation, subject was limited kind forms escape leaking percentage ac.jp 1998 64 3.20% ad.jp 45 4 8.89% co.jp 36922 3353 9.08% ed.jp 329 6 1.82% go.jp 649 10 1.52% gr.jp 266 46 17.29% lg.jp 106 1 0.94% ne.jp 4216 181 4.29% or.jp 2498 154 6.16% jp 74174 3572 4.82% 29
30. Main survey(file name extension)
 Vulnerability survey • Unescaped output of asp is high rate
 But aspx is very low rate • aspx : ASP.NET • ASP.NET has requestValidation function which blocks request like HTML tags • Many pages of php and cgi 
 are unescaped kind forms escape leaking percenta ge .asp 965 113 11.71% .aspx 2660 2 0.08% .html 9842 382 3.88% .cgi 3861 265 6.86% .php 9809 966 9.85% 30
31. Main survey(HTTP Header) 31
32. Header information
 Vulnerability survey • Subject • jp domain: 16000domains • pick up from access ranking • Collected header information • Server: 2546 types
 (Preliminary: 303types)
 Some server uses server header
 with session id • X-Powered-By: 311 types
 (Preliminary: 124types) subject domain subject page subject form escape leaking 16000 871339 149777 9762
 (6.52% of forms) server header x-powered- by header all 2546 311 XSSed 395 142 32
33. Server headers / Server types
 Vulnerability survey • Escape leaking on IIS is a low ratio • Escape leaking on nginx is a very low rate • in preliminary investigation,
 nginx sample was very few • On Apache is low ratio • On Apache 1.x is high rate kind forms escape leaking percentag e Apache 106781 7594 7.09% Apache(1.x) 8377 1950 23.27% Apache(2.x) 31998 2414 7.54% IIS 10906 264 5.41% IIS 6 6595 144 2.18% IIS 7 5135 85 1.66% nginx 7518 198 2.63% 33
34. X-Powered-By headers / Engine
 Vulnerability survey • ASP.net is low ratio • ASP.NET has requestValidation function which blocks request 
 like HTML tags • Servlet is very low ratio • J2EE Filter function • PHP • PHP is high rate in any version kind forms escape leaking percenta ge ASP.NET 10774 175 1.62% PHP 37600 3222 8.57% PHP/4.x 4542 350 7.71% PHP/5.x 33040 2872 8.70% Phusion Passenger 571 14 2.45% Servlet 2511 18 0.72% 34
35. useful HTTP Header
 Vulnerability survey • Useful HTTP Headers for security • Strict-Transport-Security • X-Frame-Options / Frame-Options • X-XSS-Protection • X-Content-Type-Options • Content-Security-Policy
 X-Content-Security-Policy
 X-WebKit-CSP ! • Show trends of these headers in use List of useful HTTP headers (OWASP) 35
36. Strict-Transport-Security
 Vulnerability survey • Strict-Transport-Security • Web page using this header : 95 page • Escape leaking : 0 • SSL • action url using https
 6.5% (1124 of 17051) • action url using http
 6.3% (7418 of 117678) 0 6 12 18 24 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 3 3 9 2 1 8 1 3 10 29 1 5 2 3 14 1 Strict-Transport-Security(page) 36
37. FrameOption
 Vulnerability survey • X-Frame-Options / Frame-Options • Web page using this header : 1444 page • X-frame-options “deny” 151 • X-frame-options “same origin” 1294 • Frame-options : 0 page • 5.6% of page using this has escape leaking 37
38. X-XSS-Protection
 Vulnerability survey • X-XSS-Protection • Web page using this header : 871 • 0.5% of page using this has escape leaking • Unfortunately, 3 pages have XSS 
 and “X-XSS-Protection 0” header • A page has XSS, but using this header to enable protection 0 40 80 120 160 200 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 18 51 94 89 5 10 138 182 22 124 11 23 21 26 26 31 X-XSS-Protection(page) 38
39. X-Content-Type-Options
 Vulnerability survey • X-Content-Type-Options • Web page using this header : 1558 • 0.5% of page using this has escape leaking 0 80 160 240 320 400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 59 65 80 116 9 133 144 222 47 148 15 44 64 46 318 48 X-Content-Type-Options(page) 39
40. Content security policy
 Vulnerability survey • X-Content-Security-Policy
 X-WebKit-CSP
 Content-Security-Policy • Only a page uses 
 X-Content-Security-Policy • In this investigation, there is hardly any website using CSP • It is to be desired that research, tool, and lecture for using CSP increase 40
41. Content security policy
 Vulnerability survey • X-Content-Security-Policy
 X-WebKit-CSP
 Content-Security-Policy • Only a page uses 
 X-Content-Security-Policy • In this investigation, there is hardly any website using CSP • It is to be desired that research, tool, and lecture for using CSP increase 41
42. Conclusion
 The survey of Web Application Vulnerabilities Vulnerability survey • 6.52% web pages have escape leaking • regardless of access frequency • Block of top 1000 domains is low rate, but tendency can’t be seen • countermeasures for XSS by frame work, language only works • In this investigation, many XSS blocked by ASP.NET function • Others are more vulnerable than that • Web sites made by corporation are more vulnerable • co.jp domain: 9.08% • These web site have many forms 42
43. Conclusion
 The survey of Web Application Vulnerabilities Vulnerability survey • It’s no difference between using https or not , using frame option or not • http : 6.3% https : 6.5% • Strict-Transport-Security header is used in some sites, these sites are safe • X-Frame-Options header is used in about 10% of form
 But 5.6% of these form have escape leaking • Web site using countermeasure header for XSS are more secure than others • In web page which has X-XSS-Protection / X-Content-Type-Options header,
 only 0.5% has escape leaking • There is hardly any website using CSP • Only a form using CSP • It is to be desired that research, tool and lecture for using CSP increase 43