Security monitoring - Penetration testing meets monitoring

Presented at Monitorama EU 2013

How often do you have a full penetration test done on your application? How often do you deploy changes to your application? This talk is for everyone who worries about the difference between the answers to those two questions.

Penetration testing and other forms of security testing are often a mystery to developers and operations people alike, a specialist skill available only on the largest projects. With lots of good open source penetration testing tools available it doesn't have to be that way, especially if we can turn some of those tools into things we run constantly from our monitoring system.

This talk will:

* Identify security relevant metrics from a few tools available in a typical web stack
* Suggest a few more useful system tools which provide both protection and metrics
* Highlight a number of open source penetration testing tools, and show a few of them in use
* Find out which of those tools lend themselves to automation
* Bring it all together in a modern monitoring system

After the talk the audience will hopefully be:

* Scared of putting things on the internet
* Wanting to install a few simple tools that provide some protection
* Aware of a number of penetration testing tools they can use with their monitoring systems

Transcript

1. Security
   Monitoring
   Penetration testing
   meet monitoring
   Gareth Rushgrove
   

   View Slide

2. Who
   (Who is this person?)
   

   View Slide

3. @garethr
   

   View Slide

4. UK Government
   Digital Service
   

   View Slide

5. View Slide

6. How did it come to this, that the government
   has one of the most exciting start-ups in the
   UK?!
   

   View Slide

7. View Slide

8. View Slide

9. Last code
   I wrote
   

   View Slide

10. View Slide

11. The Problem
    (Why talk about security
    monitoring)
    

    View Slide

12. The continuous
    delivery argument
    Gareth Rushgrove
    

    View Slide

13. How often do you
    change your
    applications?
    Gareth Rushgrove
    

    View Slide

14. Gareth Rushgrove
    How often do you
    conduct penetration
    tests?
    

    View Slide

15. The security is part
    of quality assurance
    argument
    Gareth Rushgrove
    

    View Slide

16. Testing used to be
    manual, slow and
    expensive
    Gareth Rushgrove
    

    View Slide

17. Testing is now
    automated, fast and
    done on every commit
    Gareth Rushgrove
    

    View Slide

18. Security testing is
    still mainly manual,
    slow and expensive
    Gareth Rushgrove
    

    View Slide

19. This presentation
    (What to expect from this talk)
    

    View Slide

20. Reactive security
    monitoring
    Gareth Rushgrove
    1
    

    View Slide

21. Proactive security
    testing
    Gareth Rushgrove
    2
    

    View Slide

22. Gareth Rushgrove
    3
    Community efforts
    

    View Slide

23. Reactive
    monitoring
    (Mitigate attacks)
    

    View Slide

24. rkhunter
    Gareth Rushgrove
    

    View Slide

25. Gareth Rushgrove
    

    View Slide

26. rkhunter \
    --check \
    --no-mail-on-warning \
    --skip-keypress
    Gareth Rushgrove
    

    View Slide

27. Tuxtendo Rootkit [ Not found ]
    URK Rootkit [ Not found ]
    Vampire Rootkit [ Not found ]
    VcKit Rootkit [ Not found ]
    Volc Rootkit [ Found ]
    Xzibit Rootkit [ Not found ]
    X-Org SunOS Rootkit [ Not found ]
    zaRwT.KiT Rootkit [ Not found ]
    ZK Rootkit [ Not found ]
    Performing additional rootkit checks
    Suckit Rookit additional checks [ OK ]
    Checking for possible rootkit files and directories [ None found ]
    Checking for possible rootkit strings [ None found ]
    Gareth Rushgrove
    

    View Slide

28. View Slide

29. Gareth Rushgrove
    

    View Slide

30. rkhunter \
    --check \
    --nocolors \
    --no-mail-on-warning \
    --skip-keypress \
    --no-summary | rkhunter-librato.py
    Gareth Rushgrove
    

    View Slide

31. Gareth Rushgrove
    

    View Slide

32. Gareth Rushgrove
    

    View Slide

33. def test_beastkit_not_installed():
    assert (metric("beastkit_rootkit") == 0)
    Gareth Rushgrove
    

    View Slide

34. >> nosetests -v rkhunter-librato-test.py
    rkhunter-libratoo-test.test_beastkit_not_installed ... ok
    ---------------------------------------------------------
    Ran 1 test in 1.585s
    OK
    Gareth Rushgrove
    

    View Slide

35. Nginx Naxsi
    Gareth Rushgrove
    

    View Slide

36. Web Application
    Firewall
    Gareth Rushgrove
    

    View Slide

37. Gareth Rushgrove
    

    View Slide

38. SecRulesEnabled;
    DeniedUrl "/RequestDenied";
    CheckRule "$SQL >= 8" BLOCK;
    CheckRule "$RFI >= 8" BLOCK;
    CheckRule "$TRAVERSAL >= 4" BLOCK;
    CheckRule "$EVADE >= 4" BLOCK;
    CheckRule "$XSS >= 8" BLOCK;
    Gareth Rushgrove
    

    View Slide

39. 2013/09/18 08:59:57 [error] 891#0: *6 NAXSI_FMT:
    ip=192.168.50.20&server=victim&uri=/pictures/
    search.php&total_processed=14&total_blocked=7&zo
    ne0=ARGS&id0=1007&var_name0=query, client:
    192.168.50.20, server: localhost, request:
    "GET /pictures/search.php?query=--%3E+
    %3Csome_dangerous_input_a1056fd2f0ffbb7f18fec9bd
    33257e12ab5e0494b33011967bcbcbc5699408eb%2F%3E+
    %3C%21-- HTTP/1.1", host: "victim"
    Gareth Rushgrove
    

    View Slide

40. id0=1007
    Gareth Rushgrove
    

    View Slide

41. SQL Injection
    Gareth Rushgrove
    

    View Slide

42. Gareth Rushgrove
    

    View Slide

43. grok {
    type => "nginx_error"
    match => ["message", "
    ip=%{IP:client_ip}&
    server=%{IP:server_ip}&
    uri=%{PATH:uri}&
    total_processed=%{NUMBER:total_processed}&
    total_blocked=%{NUMBER:total_blocked}&
    zone0=%{WORD:zone}&
    id0=%{NUMBER:id}"]
    }
    Gareth Rushgrove
    

    View Slide

44. Gareth Rushgrove
    

    View Slide

45. Fail2Ban
    Gareth Rushgrove
    

    View Slide

46. Gareth Rushgrove
    

    View Slide

47. [ssh]
    enabled = true
    port = ssh
    filter = sshd
    logpath = /var/log/auth.log
    maxretry = 3
    [ssh-ddos]
    enabled = true
    port = ssh
    filter = sshd-ddos
    logpath = /var/log/auth.log
    maxretry = 6
    Gareth Rushgrove
    

    View Slide

48. [ssh-ddos] Ban 192.168.50.20
    Gareth Rushgrove
    

    View Slide

49. [nginx-naxsi]
    enabled = true
    port = http,https
    filter = nginx-naxsi
    logpath = /var/log/nginx/*error.log
    maxretry = 2
    Gareth Rushgrove
    

    View Slide

50. grok {
    type => "naxsi_fail2ban"
    match => ["message", "
    WARNING \[nginx-naxsi\]
    %{WORD:action} %{IP:ip}"
    ]
    }
    Gareth Rushgrove
    

    View Slide

51. Auditd
    Gareth Rushgrove
    

    View Slide

52. Auditd in less than 2
    minutes. Maybe.
    Gareth Rushgrove
    

    View Slide

53. -a exit,always -S mkdir
    Gareth Rushgrove
    

    View Slide

54. type=CWD msg=audit(1379493067.779:57): cwd="/tmp"
    type=PATH msg=audit(1379493067.779:57): item=0
    name="vagrant-puppet" inode=20 dev=fc:00 mode=041777
    ouid=0 ogid=0 rdev=00:00
    type=SYSCALL msg=audit(1379493067.779:58):
    arch=c000003e syscall=83 success=yes exit=0
    a0=7fff172d0e5e a1=1ed a2=1ed a3=7fff172cf910 items=2
    ppid=1239 pid=1241 auid=0 uid=0 gid=0 euid=0 suid=0
    fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=21
    comm="mkdir" exe="/bin/mkdir" key=(null)
    type=CWD msg=audit(1379493067.779:58): cwd="/tmp/
    vagrant-puppet"
    Gareth Rushgrove
    

    View Slide

55. cwd="/tmp"
    Gareth Rushgrove
    

    View Slide

56. syscall=83
    Gareth Rushgrove
    

    View Slide

57. sys_symlink
    Gareth Rushgrove
    

    View Slide

58. Gareth Rushgrove
    

    View Slide

59. comm="mkdir"
    Gareth Rushgrove
    

    View Slide

60. cwd="/tmp/vagrant-puppet"
    Gareth Rushgrove
    

    View Slide

61. Aside:
    penetration
    testing tools
    (State of open source)
    

    View Slide

62. Skipfish, nikto, w3af,
    garmr, sslyze, owasp
    zap, arachni, sqlmap,
    sslscan, TLSSLed,
    slowhttptest, DIRB,
    SQLiBF
    Gareth Rushgrove
    

    View Slide

63. BackTrack
    Gareth Rushgrove
    

    View Slide

64. The problem with
    distributing software
    as a Linux distribution
    Gareth Rushgrove
    

    View Slide

65. Configuration
    management +
    Vagrant
    Gareth Rushgrove
    

    View Slide

66. Penetration
    testing tools
    

    View Slide

67. Vulnerable
    web apps
    

    View Slide

68. Source
    code
    

    View Slide

69. Puppet
    module
    

    View Slide

70. Proactive
    monitoring
    (Attack yourself)
    

    View Slide

71. Gareth Rushgrove
    

    View Slide

72. nmap monitorama.eu
    Gareth Rushgrove
    

    View Slide

73. Starting Nmap 5.21 ( http://nmap.org ) at
    2013-09-18 15:09 BST
    Nmap scan report for monitorama.eu
    (141.101.116.49)
    Host is up (0.17s latency).
    Hostname monitorama.eu resolves to 2 IPs. Only
    scanned 141.101.116.49
    Not shown: 998 filtered ports
    PORT STATE SERVICE
    80/tcp open http
    8080/tcp open http-proxy
    Nmap done: 1 IP address (1 host up) scanned in
    24.18 seconds
    Gareth Rushgrove
    

    View Slide

74. Gareth Rushgrove
    

    View Slide

75. it "should have one port open" do
    @host.open_ports.should have(1).items
    end
    Gareth Rushgrove
    

    View Slide

76. Gareth Rushgrove
    

    View Slide

77. 1) the monitorama.eu website should have one port open
    Failure/Error: @host.open_ports.should have(1).items
    expected 1 items, got 2
    Finished in 8.99 seconds
    1 example, 1 failure
    Gareth Rushgrove
    

    View Slide

78. Arachni
    Gareth Rushgrove
    

    View Slide

79. Gareth Rushgrove
    

    View Slide

80. Web application
    security scanner
    Gareth Rushgrove
    

    View Slide

81. arachni http://monitorama.eu --modules=xss
    Gareth Rushgrove
    

    View Slide

82. +
    +[+] 2 issues were detected.
    +
    +[+] [1] Trusted -- Cross-Site Scripting (XSS)
    +[~] ~~~~~~~~~~~~~~~~~~~~
    +[~] ID Hash:
    +[~] Severity: High
    +[~] URL: http://victim/pictures/search.php
    +[~] Element: form
    +[~] Method: GET
    +[~] Tags: xss, regexp, injection, script
    +[~] Variable: query
    +[~] Description:
    +[~] Client-side code (like JavaScript) can be injected
    + into the web application which is then returned to
    + the user's browser. This can lead to a compromise
    + of the client's system or serve as a pivoting
    + point for other attacks.
    +
    Gareth Rushgrove
    

    View Slide

83. Gauntlt
    Gareth Rushgrove
    

    View Slide

84. Gareth Rushgrove
    

    View Slide

85. Cucumber + security
    tool integrations
    Gareth Rushgrove
    

    View Slide

86. Officially supports
    curl, nmap, sslyze,
    sqlmap, garmr
    Gareth Rushgrove
    

    View Slide

87. Gareth Rushgrove
    

    View Slide

88. $ gauntlt methods.attack
    Gareth Rushgrove
    

    View Slide

89. Gareth Rushgrove
    

    View Slide

90. Support in master
    dirb, arachni
    Gareth Rushgrove
    

    View Slide

91. Gareth Rushgrove
    

    View Slide

92. $ gauntlt xss.attack
    Gareth Rushgrove
    

    View Slide

93. Conclusions
    (You convinced me, now what?)
    

    View Slide

94. Gareth Rushgrove
    Use penetration tests
    to discover how
    attackers work
    1
    

    View Slide

95. Gareth Rushgrove
    Use security
    monitoring to build and
    maintain checklists
    2
    

    View Slide

96. Share common
    configuration patterns
    Gareth Rushgrove
    3
    

    View Slide

97. Gareth Rushgrove
    Help with packaging
    and configuration
    management
    4
    

    View Slide

98. Gareth Rushgrove
    5
    Help integrate
    security tools with
    monitoring systems
    

    View Slide

99. Gareth Rushgrove
    Get security together
    with developers and
    operations
    6
    

    View Slide

100. Questions?
     (And thanks for listening)
     

     View Slide