Security monitoring - with open source penetration testing tools

Talk given at Velocity Europe. This is an extended version of my previous security monitoring talk given at Monitorama. This talk contains even more examples and a new section on security monitoring as part of the development process.

Transcript

1. Security Monitoring! With Open Source ! Penetration Testing! Tools Gareth Rushgrove
2. Who (Who is this person?)
3. @garethr
4. UK Government Digital Service
6. How did it come to this, that the government has one of the most exciting start-ups in the UK?!
8. The Problem (Why talk about security monitoring)
9. The continuous delivery argument Gareth Rushgrove
10. How often do you change your applications? Gareth Rushgrove
11. Gareth Rushgrove How often do you conduct penetration tests?
12. The security is part of quality assurance argument Gareth Rushgrove
13. Testing used to be manual, slow and expensive Gareth Rushgrove
14. Testing is now automated, fast and done on every commit Gareth Rushgrove
15. Security testing is still mainly manual, slow and expensive Gareth Rushgrove
16. This presentation (What to expect from this talk)
17. Reactive security monitoring Gareth Rushgrove 1
18. Monitoring security in development Gareth Rushgrove 2
19. Proactive security testing Gareth Rushgrove 3
20. Gareth Rushgrove 4 What you can do
21. Gareth Rushgrove WARNING. Code in this presentation works but isn’t always pretty
22. Reactive monitoring (Watch for weirdness)
23. rkhunter Gareth Rushgrove
24. Gareth Rushgrove
25. rkhunter \ --check \ --no-mail-on-warning \ --skip-keypress Gareth Rushgrove
26. Tuxtendo Rootkit [ Not found ] URK Rootkit [ Not found ] Vampire Rootkit [ Not found ] VcKit Rootkit [ Not found ] Volc Rootkit [ Found ] Xzibit Rootkit [ Not found ] X-Org SunOS Rootkit [ Not found ] zaRwT.KiT Rootkit [ Not found ] ZK Rootkit [ Not found ] Performing additional rootkit checks Suckit Rookit additional checks [ OK ] Checking for possible rootkit files and directories [ None found ] Checking for possible rootkit strings [ None found ] Gareth Rushgrove
28. Gareth Rushgrove
29. rkhunter \ --check \ --nocolors \ --no-mail-on-warning \ --skip-keypress \ --no-summary | rkhunter-librato.py Gareth Rushgrove
30. Gareth Rushgrove
31. Gareth Rushgrove
32. def test_beastkit_not_installed(): assert (metric("beastkit_rootkit") == 0) Gareth Rushgrove
33. >> nosetests -v rkhunter-librato-test.py rkhunter-libratoo-test.test_beastkit_not_installed ... ok --------------------------------------------------------- Ran 1 test in 1.585s OK Gareth Rushgrove
34. Nginx Naxsi Gareth Rushgrove
35. Web Application Firewall Gareth Rushgrove
36. Gareth Rushgrove
37. SecRulesEnabled; DeniedUrl "/RequestDenied"; CheckRule "$SQL >= 8" BLOCK; CheckRule "$RFI >= 8" BLOCK; CheckRule "$TRAVERSAL >= 4" BLOCK; CheckRule "$EVADE >= 4" BLOCK; CheckRule "$XSS >= 8" BLOCK; Gareth Rushgrove
38. 2013/09/18 08:59:57 [error] 891#0: *6 NAXSI_FMT: ip=192.168.50.20&server=victim&uri=/pictures/ search.php&total_processed=14&total_blocked=7&zo ne0=ARGS&id0=1007&var_name0=query, client: 192.168.50.20, server: localhost, request: "GET /pictures/search.php?query=--%3E+ %3Csome_dangerous_input_a1056fd2f0ffbb7f18fec9bd 33257e12ab5e0494b33011967bcbcbc5699408eb%2F%3E+ %3C%21-- HTTP/1.1", host: "victim" ! Gareth Rushgrove
39. id0=1007 Gareth Rushgrove
40. SQL Injection Gareth Rushgrove
41. Gareth Rushgrove
42. grok { type => "nginx_error" match => ["message", " ip=%{IP:client_ip}& server=%{IP:server_ip}& uri=%{PATH:uri}& total_processed=%{NUMBER:total_processed}& total_blocked=%{NUMBER:total_blocked}& zone0=%{WORD:zone}& id0=%{NUMBER:id}"] } Gareth Rushgrove
43. Gareth Rushgrove
44. Fail2Ban Gareth Rushgrove
45. Gareth Rushgrove
46. [ssh] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 3 [ssh-ddos] enabled = true port = ssh filter = sshd-ddos logpath = /var/log/auth.log maxretry = 6 Gareth Rushgrove
47. [ssh-ddos] Ban 192.168.50.20 Gareth Rushgrove
48. [nginx-naxsi] enabled = true port = http,https filter = nginx-naxsi logpath = /var/log/nginx/*error.log maxretry = 2 Gareth Rushgrove
49. grok { type => "naxsi_fail2ban" match => ["message", " WARNING \[nginx-naxsi\] %{WORD:action} %{IP:ip}" ] } Gareth Rushgrove
50. Auditd Gareth Rushgrove
51. Auditd in less than 2 minutes. Maybe. Gareth Rushgrove
52. -a exit,always -S mkdir Gareth Rushgrove
53. type=CWD msg=audit(1379493067.779:57): cwd="/tmp" type=PATH msg=audit(1379493067.779:57): item=0 name="vagrant-puppet" inode=20 dev=fc:00 mode=041777 ouid=0 ogid=0 rdev=00:00 type=SYSCALL msg=audit(1379493067.779:58): arch=c000003e syscall=83 success=yes exit=0 a0=7fff172d0e5e a1=1ed a2=1ed a3=7fff172cf910 items=2 ppid=1239 pid=1241 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=21 comm="mkdir" exe="/bin/mkdir" key=(null) type=CWD msg=audit(1379493067.779:58): cwd="/tmp/ vagrant-puppet" Gareth Rushgrove
54. cwd="/tmp" Gareth Rushgrove
55. syscall=83 Gareth Rushgrove
56. sys_symlink Gareth Rushgrove
57. Gareth Rushgrove
58. comm="mkdir" Gareth Rushgrove
59. cwd="/tmp/vagrant-puppet" Gareth Rushgrove
60. aureport or ausearch and something Gareth Rushgrove
61. Security in development (Monitor development too)
62. Monitor security in your pipeline Gareth Rushgrove
63. Virus scan your artefacts Gareth Rushgrove
64. Gareth Rushgrove ClamAV + Jenkins
65. Gareth Rushgrove
66. Gareth Rushgrove Get data to Graphite
67. Static analysis Gareth Rushgrove
68. Brakeman for Ruby on Rails Gareth Rushgrove
69. Gareth Rushgrove
70. Gareth Rushgrove
71. Vulnerabilities in dependencies Gareth Rushgrove
72. Gareth Rushgrove OWASP dependency check
73. Gareth Rushgrove Ruby security advisories
74. Gareth Rushgrove Bundler audit
75. > bundler-audit ! Name: actionpack Version: 3.2.10 Advisory: OSVDB-91452 Criticality: Medium URL: http://www.osvdb.org/show/osvdb/91452 Title: XSS vulnerability in sanitize_css in Action Pack Solution: upgrade to ~> 2.3.18, ~> 3.1.12, >= 3.2.13 Gareth Rushgrove
76. Gareth Rushgrove
77. it "should have no vulnerable gems" do @issues.should have(0).items end Gareth Rushgrove
78. rspec audit-rspec.rb -f d my application dependencies should have no vulnerable gems (FAILED - 1) should have a safe version of ruby on rails ! Finished in 0.03949 seconds 2 examples, 1 failure Gareth Rushgrove
79. Aside: penetration testing tools (State of open source)
80. Skipfish, nikto, w3af, garmr, sslyze, owasp zap, arachni, sqlmap, sslscan, TLSSLed, slowhttptest, DIRB, SQLiBF Gareth Rushgrove
81. BackTrack Gareth Rushgrove
82. The problem with distributing software as a Linux distribution Gareth Rushgrove
83. Configuration management + Vagrant Gareth Rushgrove
84. Penetration testing tools
85. Vulnerable web apps
86. Source code
87. Puppet module
88. Proactive monitoring (Attack yourself)
89. Gareth Rushgrove
90. nmap monitorama.eu Gareth Rushgrove
91. Starting Nmap 5.21 ( http://nmap.org ) at 2013-09-18 15:09 BST Nmap scan report for monitorama.eu (141.101.116.49) Host is up (0.17s latency). Hostname monitorama.eu resolves to 2 IPs. Only scanned 141.101.116.49 Not shown: 998 filtered ports PORT STATE SERVICE 80/tcp open http 8080/tcp open http-proxy Nmap done: 1 IP address (1 host up) scanned in 24.18 seconds Gareth Rushgrove
92. Gareth Rushgrove
93. it "should have one port open" do @open_ports.should have(1).items end Gareth Rushgrove
94. it "should have port 80 open" do @open_ports.should contain(80) end Gareth Rushgrove
95. Gareth Rushgrove
96. 1) the monitorama.eu website should have one port open Failure/Error: @open_ports.should have(1).items expected 1 items, got 12 # ./nmap-rspec.rb:24:in `block (2 levels) in <top (required)>' ! Finished in 2.47 seconds 2 examples, 1 failure Gareth Rushgrove
97. Arachni Gareth Rushgrove
98. Gareth Rushgrove
99. Web application security scanner Gareth Rushgrove
100. arachni http://victim --modules=xss Gareth Rushgrove
101. + +[+] 2 issues were detected. + +[+] [1] Trusted -- Cross-Site Scripting (XSS) +[~] ~~~~~~~~~~~~~~~~~~~~ +[~] ID Hash: +[~] Severity: High +[~] URL: http://victim/pictures/search.php +[~] Element: form +[~] Method: GET +[~] Tags: xss, regexp, injection, script +[~] Variable: query +[~] Description: +[~] Client-side code (like JavaScript) can be injected + into the web application which is then returned to + the user's browser. This can lead to a compromise + of the client's system or serve as a pivoting + point for other attacks. + Gareth Rushgrove
102. Gareth Rushgrove
103. OWASP ZAP Gareth Rushgrove OWASP ZAP
104. Gareth Rushgrove
105. Spider an entire site Gareth Rushgrove
106. Record session via HTTP proxy Gareth Rushgrove
107. ./zap.sh -daemon Gareth Rushgrove
108. Gareth Rushgrove Python API
109. Gareth Rushgrove
110. zap = ZAP() zap.openurl(TARGET) zap.start_spider(TARGET) zap.start_scan(TARGET) Gareth Rushgrove
111. http://victim/pictures/search.php +------+----------------------------------+ | Risk | Description | +------+----------------------------------+ | High | Cross Site Scripting (Reflected) | +------+----------------------------------+ http://victim/css/ +--------+--------------------+ | Risk | Description | +--------+--------------------+ | Medium | Directory browsing | +--------+--------------------+ http://victim/users/login.php +---------------+---------------------------------------+ | Risk | Description | +---------------+---------------------------------------+ | Informational | X-Frame-Options header not set | | Low | Cookie set without HttpOnly flag | | Low | Password Autocomplete in browser | | Low | X-Content-Type-Options header missing | | Medium | Application Error disclosure | +---------------+---------------------------------------+ Gareth Rushgrove
112. Gauntlt Gareth Rushgrove
113. Gareth Rushgrove
114. Cucumber + security tool integrations Gareth Rushgrove
115. Officially supports! curl, nmap, sslyze, sqlmap, garmr Gareth Rushgrove
116. Gareth Rushgrove
117. $ gauntlt methods.attack Gareth Rushgrove
118. Gareth Rushgrove
119. Support in master dirb, arachni Gareth Rushgrove
120. Gareth Rushgrove
121. $ gauntlt xss.attack Gareth Rushgrove
122. Conclusions (You convinced me, now what?)
123. Gareth Rushgrove Use security monitoring to build and maintain checklists 1
124. Gareth Rushgrove Use penetration tests to discover how attackers work 2
125. Get security (monitoring) into your development pipeline Gareth Rushgrove 3
126. Gareth Rushgrove Help with packaging and configuration management 4
127. Gareth Rushgrove 5 Help integrate security tools with monitoring systems
128. Gareth Rushgrove Get security together with developers and operations 6
129. Questions? (And thanks for listening)