Antonio Bonuccelli

Antonio Bonuccelli

Using ELK to visualise security data: IPTables and Kippo SSH honeypot

098332e9d988080a9057816f84d668f7?s=128

Elasticsearch Inc

November 20, 2014
Tweet

Transcript

  1. Elasticsearch Using ELK to visualize security data: IPTables and Kippo

    SSH honeypot
  2. WHOAMI • Antonio Bonuccelli • Support Engineer • Recently joined

    Elasticsearch family @nellicus
  3. In this presentation • Identify raw data sources of interest

    • Extract relevant information • Visualize security analytics • Demo cloud VM
  4. In this presentation Or more simply, go from here

  5. In this presentation To here

  6. In this presentation To get a better picture of the

    bad guys
  7. In this presentation To get a better picture of the

    bad guys
  8. In this presentation To get a better picture of the

    bad guys
  9. In this presentation To get a better picture of the

    bad guys
  10. Raw data sources

  11. Raw data sources - what • Who is attempting connection

  12. Raw data sources - what • Who is attempting connection

    • What services are they probing
  13. Raw data sources - what • Who is attempting connection

    • What services are they probing • What credentials they use
  14. Raw data sources - what • Who is attempting connection

    • What services are they probing • What credentials they use • Where are they from
  15. Raw data sources - who IPTables - Definition “iptables is

    a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall and the chains and rules it stores.”
  16. Raw data sources - who IPTables – References http://www.netfilter.org/projects/iptables/index.html http://www.netfilter.org/about.html#coreteam

  17. Raw data sources - who IPTables – References http://www.netfilter.org/projects/iptables/index.html http://www.netfilter.org/about.html#coreteam

  18. Raw data sources - how IPTables Logging only denied connections

  19. Raw data sources - how IPTables Logging only denied connections

    # Whitelist everything we want to allow # e.g. Don't lock yourself out -A INPUT -p tcp --dport 12345 -j ACCEPT -A INPUT -p tcp --dport ..... -j ACCEPT
  20. Raw data sources - how IPTables Logging only denied connections

    # Whitelist everything we want to allow # e.g. Don't lock yourself out -A INPUT -p tcp --dport 12345 -j ACCEPT -A INPUT -p tcp --dport ..... -j ACCEPT # Log iptables denied calls -A INPUT -j LOG --log-prefix "iptables denied:"
  21. Raw data sources - how IPTables Logging only denied connections

    # Whitelist everything we want to allow # e.g. Don't lock yourself out -A INPUT -p tcp --dport 12345 -j ACCEPT -A INPUT -p tcp --dport ..... -j ACCEPT # Log iptables denied calls -A INPUT -j LOG --log-prefix "iptables denied:" # Drop all other INPUT and FORWARD # Explicitly allowed policy earlier -A INPUT -j DROP -A FORWARD -j DROP COMMIT
  22. Raw data sources - how IPTables Logging only denied connections

    tony$ telnet host123.virtual-machines.test 1000 Trying 123.123.123.123... ^C
  23. Raw data sources - how IPTables Logging only denied connections

    tony$ telnet host123.virtual-machines.test 1000 Trying 123.123.123.123... ^C
  24. Raw data sources - how IPTables Logging only denied connections

    tony$ telnet host123.virtual-machines.test 1000 Trying 123.123.123.123... ^C Nov 18 13:45:27 li717-137 kernel: iptables denied: IN=eth0 OUT= MAC=f2:3c:91:73:6c:71:84:78:ac:5a:1a:41:08:00 SRC=77.231.116.118 DST=123.123.123.123 LEN=64 TOS=0x00 PREC=0x00 TTL=48 ID=51711 DF PROTO=TCP SPT=50196 DPT=1000 WINDOW=65535 RES=0x00 SYN URGP=0
  25. Raw data sources - how IPTables Logging only denied connections

    tony$ telnet host123.virtual-machines.test 1000 Trying 123.123.123.123... ^C Nov 18 13:45:27 li717-137 kernel: iptables denied: IN=eth0 OUT= MAC=f2:3c:91:73:6c:71:84:78:ac:5a:1a:41:08:00 SRC=77.231.116.118 DST=123.123.123.123 LEN=64 TOS=0x00 PREC=0x00 TTL=48 ID=51711 DF PROTO=TCP SPT=50196 DPT=1000 WINDOW=65535 RES=0x00 SYN URGP=0
  26. Raw data sources - who Kippo SSH HoneyPot “KippoSSH is

    a medium interaction honeypot capable of recording plenty of information about the attacker, including interactive TTY sessions recordings”
  27. Raw data sources - who Kippo SSH HoneyPot – References

    - https://github.com/desaster/kippo
  28. Raw data sources - who Kippo SSH HoneyPot – References

    - https://github.com/desaster/kippo
  29. Raw data sources Kippo SSH HoneyPot – References - https://github.com/desaster/kippo

  30. Raw data sources - how Kippo SSH HoneyPot tony$ egrep

    '^[^#]' /opt/kippo/kippo- master/kippo.cfg [honeypot] ssh_addr = 0.0.0.0 ssh_port = 2222 hostname = miami-32 log_path = log download_path = dl download_limit_size = 10485760 contents_path = honeyfs filesystem_file = fs.pickle
  31. Raw data sources - how Kippo SSH HoneyPot data_path =

    data txtcmds_path = txtcmds public_key = public.key private_key = private.key ssh_version_string = SSH-2.0-OpenSSH_5.1p1 Debian- 5 interact_enabled = false interact_port = 5123
  32. Raw data sources - how Kippo SSH HoneyPot badguy$ ssh

    root@host123.virtual-machines.test Password: (123456)
  33. Raw data sources - how Kippo SSH HoneyPot badguy$ ssh

    root@host123.virtual-machines.test Password: (123456)
  34. Raw data sources - how Kippo SSH HoneyPot badguy$ ssh

    root@host123.virtual-machines.test Password: (123456) 2014-11-16 10:44:53+0000 [SSHService ssh- userauth on HoneyPotTransport,4461,77.231.116.118] login attempt [root/123456] failed
  35. Raw data sources - how Kippo SSH HoneyPot badguy$ ssh

    root@host123.virtual-machines.test Password: (123456) 2014-11-16 10:44:53+0000 [SSHService ssh- userauth on HoneyPotTransport,4461,77.231.116.118] login attempt [root/123456] failed
  36. Raw data sources - how Kippo SSH HoneyPot • Only

    root can bind to port <1024 • We don't want to run Kippo as root • iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 22 -j REDIRECT --to-port 2222
  37. Event collection, processing, enrichment: Logstash

  38. Logstash • INPUT

  39. Logstash • INPUT • FILTER

  40. Logstash • INPUT • FILTER • OUTPUT

  41. Logstash • INPUT input { }

  42. Logstash • INPUT input { file { } file {

    } }
  43. Logstash • INPUT input { file { type => "iptables"

    path => "/var/log/kern.log" } file { type => "honey-kippo" path => "/var/log/kippo/kippo.log" } }
  44. Logstash • INPUT input { file { type => "iptables"

    path => "/var/log/kern.log" } file { type => "honey-kippo" path => "/var/log/kippo/kippo.log" } }
  45. Logstash • FILTER filter { if [type] == "iptables" {

    } continues..
  46. Logstash • FILTER Filter { if [type] == "iptables" {

    grok { match => [ "message","%{IPTABLES_DENIED}"] } } continues..
  47. Logstash • FILTER Filter { if [type] == "iptables" {

    grok { match => [ "message","%{IPTABLES_DENIED}"] } date { match => [ "timestamp", "MMM dd HH:mm:ss"] timezone => "Europe/London" } } continues..
  48. Logstash • FILTER Filter { if [type] == "iptables" {

    grok { match => [ "message","%{IPTABLES_DENIED}"] } date { match => [ "timestamp", "MMM dd HH:mm:ss"] timezone => "Europe/London" } } continues..
  49. Logstash GROK • Parse arbitrary data and structure it

  50. Logstash GROK • Parse arbitrary data and structure it •

    Build regex patterns and reuse them
  51. Logstash GROK • Parse arbitrary data and structure it •

    Build regex patterns and reuse them • Ships with 120 patterns and counting
  52. Logstash GROK • Parse arbitrary data and structure it •

    Build regex patterns and reuse them • Ships with 120 patterns and counting • Allows to define custom patterns
  53. Logstash GROK IPTABLES_DENIED % {SYSLOGTIMESTAMP:timestamp} % {HOSTNAME:_host} kernel: iptables denied:

    IN=(? <in>eth0) OUT= MAC=(?<mac_addr>\S+) SRC=% {IP:src_ip} DST=%{IP:dst_ip} LEN=\d+ TOS=0x\d+ PREC=0x\d+ TTL=\d+ ID=\d+(?:\sDF)? PROTO=(? <proto>\S+) SPT=(?<src_port>\d+) DPT=(? <dst_port>\d+)(?:\sWINDOW=\d+)?(?:\sRES=0x\d+)? (?:\s[ACKSYNFIRT]{3})+(?:\sURGP=\d)?
  54. Logstash GROK IPTABLES_DENIED % {SYSLOGTIMESTAMP:timestamp} % {HOSTNAME:_host} kernel: iptables denied:

    IN=(? <in>eth0) OUT= MAC=(?<mac_addr>\S+) SRC=% {IP:src_ip} DST=%{IP:dst_ip} LEN=\d+ TOS=0x\d+ PREC=0x\d+ TTL=\d+ ID=\d+(?:\sDF)? PROTO=(? <proto>\S+) SPT=(?<src_port>\d+) DPT=(? <dst_port>\d+)(?:\sWINDOW=\d+)?(?:\sRES=0x\d+)? (?:\s[ACKSYNFIRT]{3})+(?:\sURGP=\d)?
  55. Logstash GROK IPTABLES_DENIED % {SYSLOGTIMESTAMP:timestamp} % {HOSTNAME:_host} kernel: iptables denied:

    IN=(? <in>eth0) OUT= MAC=(?<mac_addr>\S+) SRC=% {IP:src_ip} DST=%{IP:dst_ip} LEN=\d+ TOS=0x\d+ PREC=0x\d+ TTL=\d+ ID=\d+(?:\sDF)? PROTO=(? <proto>\S+) SPT=(?<src_port>\d+) DPT=(? <dst_port>\d+)(?:\sWINDOW=\d+)?(?:\sRES=0x\d+)? (?:\s[ACKSYNFIRT]{3})+(?:\sURGP=\d)?
  56. Logstash GROK Before Nov 16 12:12:44 li717-137 kernel: iptables denied:

    IN=eth0 OUT= MAC=f2:3c:91:73:6c:71:84:78:ac:5a:1a:41:08:00 SRC=77.231.116.118 DST=123.123.123.123 LEN=60 TOS=0x00 PREC=0x00 TTL=48 ID=57130 DF PROTO=TCP SPT=49222 DPT=39000 WINDOW=29200 RES=0x00 SYN URGP=0
  57. Logstash GROK After ... "timestamp" => "Nov 16 12:12:44", "_host"

    => "host123", "in" => "eth0", "src_ip" => "77.231.116.118", "dst_ip" => "123.123.123.123", "proto" => "TCP", "src_port" => "49222", "dst_port" => "39000", ...
  58. Logstash • FILTER Filter { if [type] == "iptables" {

    grok { match => [ "message","%{IPTABLES_DENIED}"] } date { match => [ "timestamp", "MMM dd HH:mm:ss"] timezone => "Europe/London" } } continues..
  59. Logstash • FILTER Continues.. else if [type] == "honey-kippo" {

    //Do likewise for kippo //match timestamp //grok it } }//FILTER ENDS
  60. Logstash • FILTER Continues.. else if [type] == "honey-kippo" {

    //Do likewise for kippo //match timestamp //grok it } geoip { source => "src_ip" } }//FILTER ENDS
  61. Logstash GEOIP "geoip" => { "ip" => "222.186.21.48", "country_code2" =>

    "CN", "country_code3" => "CHN", "country_name" => "China", "continent_code" => "AS", "region_name" => "04", “city_name" => "Nanjing", "latitude" => 32.0617, "longitude" => 118.77780000000001, "timezone" => "Asia/Shanghai", "real_region_name" => "Jiangsu", "location" => [ [0] 118.77780000000001, [1] 32.0617 ] }
  62. Logstash GEOIP "geoip" => { "ip" => "222.186.21.48", "country_code2" =>

    "CN", "country_code3" => "CHN", "country_name" => "China", "continent_code" => "AS", "region_name" => "04", “city_name" => "Nanjing", "latitude" => 32.0617, "longitude" => 118.77780000000001, "timezone" => "Asia/Shanghai", "real_region_name" => "Jiangsu", "location" => [ [0] 118.77780000000001, [1] 32.0617 ] }
  63. Logstash • OUTPUT Output { ...

  64. Logstash • OUTPUT Output { if "_grokparsefailure" not in [tags]

    { } ...
  65. Logstash • OUTPUT Output { if "_grokparsefailure" not in [tags]

    { if [type] == "iptables" { } } ...
  66. Logstash • OUTPUT Output { if "_grokparsefailure" not in [tags]

    { if [type] == "iptables" { elasticsearch { protocol => "http" host => "127.0.0.1" index => "logstash-os-%{+YYYY.MM.dd}" index_type => "firewall" } } } ...
  67. Logstash • OUTPUT output { if "_grokparsefailure" not in [tags]

    { if [type] == "iptables" { elasticsearch { protocol => "http" host => "127.0.0.1" index => "logstash-os-%{+YYYY.MM.dd}" index_type => "firewall" } } } ...
  68. Logstash • OUTPUT ... if [type] == "honey-kippo" { elasticsearch

    { protocol => "http" host => "127.0.0.1" index => "logstash-honey-%{+YYYY.MM.dd}" index_type => "honey" } } ...
  69. Logstash • OUTPUT Output { if "_grokparsefailure" not in [tags]

    { if ... if ... } stdout { codec => rubydebug } }//OUTPUT ENDS
  70. Elasticsearch

  71. Elasticsearch • Distributed JSON document store • Full-text search engine

    • Scalable • Multi-tenant • Resilient • Lucene for the masses • Schema free • Ridiculously fast • Hides complexity from user
  72. Elasticsearch • Distributed JSON document store • Full-text search engine

    • Scalable • Multi-tenant • Resilient • Lucene for the masses • Schema free • Ridicously fast • Hides complexity from user
  73. Elasticsearch • More on ES is outside the scope of

    this presentations • For our purposes we will have a 2 nodes cluster up and running with
  74. Elasticsearch $ wget https://download.elasticsearch.org/elasticsearch /elasticsearch/elasticsearch-1.4.0.tar.gz

  75. Elasticsearch $ wget https://download.elasticsearch.org/elasticsearch /elasticsearch/elasticsearch-1.4.0.tar.gz $ mkdir node1 $ mkdir

    node2
  76. Elasticsearch $ wget https://download.elasticsearch.org/elasticsearch /elasticsearch/elasticsearch-1.4.0.tar.gz $ mkdir node1 $ mkdir

    node2 $ ./node1/elasticsearch-1.4.0/bin/elasticsearch --node.name=node1 --cluster.name=mycluster -d $ ./node2/elasticsearch-1.4.0/bin/elasticsearch --node.name=node2 --cluster.name=mycluster -d
  77. Elasticsearch $ wget https://download.elasticsearch.org/elasticsearch /elasticsearch/elasticsearch-1.4.0.tar.gz $ mkdir node1 $ mkdir

    node2 $ ./node1/elasticsearch-1.4.0/bin/elasticsearch --node.name=node1 --cluster.name=mycluster -d $ ./node2/elasticsearch-1.4.0/bin/elasticsearch --node.name=node2 --cluster.name=mycluster -d
  78. Elasticsearch $ curl -XGET localhost:9200/_cluster/health?pretty { "cluster_name" : "mycluster", "status"

    : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 0, "active_shards" : 0, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0 }
  79. Elasticsearch • This is ok for test/dev (and this demo)

    • Much more that can be configured • You will need to have a good reason to change defaults settings
  80. Kibana (4 beta v2)

  81. Kibana • No coding required

  82. Kibana • No coding required • Create amazing dashboards

  83. Kibana • No coding required • Create amazing dashboards •

    Gain visual insight from your data
  84. Kibana Beta 4 – What's new • Java backend process

  85. Kibana Beta 4 – What's new • Java backend process

    • Support for aggregations
  86. Kibana Beta 4 – What's new • Java backend process

    • Support for aggregations • Accepts raw JSON
  87. Kibana Beta 4 – What's new • Java backend process

    • Support for aggregations • Accepts raw JSON • New amazing visualizations
  88. Kibana Beta 4 – What's new • Java backend process

    • Support for aggregations • Accepts raw JSON • New amazing visualizations • More
  89. Kibana Configuration • No need! Defaults are fine (in this

    case) • Just untar and launch (./kibana) • Stores its information into ES index .kibana
  90. Kibana Configuration • No need! Defaults are fine (in this

    case) • Stores its information into ES index .kibana
  91. Questions?

  92. Thank you!