$30 off During Our Annual Pro Sale. View Details »

Threat Detection Using Time Series Analysis of Darknet Probes and OSSEC Reports

Clay Wells
April 20, 2016

Threat Detection Using Time Series Analysis of Darknet Probes and OSSEC Reports

Clay Wells

April 20, 2016
Tweet

More Decks by Clay Wells

Other Decks in Technology

Transcript

  1. Threat Detection
    Clay Wells ([email protected]) Threat Detection April 19, 2016 1 / 83

    View Slide

  2. Threat Detection Using Time Series Analysis and
    Summary Statistics of Darknet Probes and OSSEC
    Reports
    Clay Wells
    School of Arts and Sciences
    University of Pennsylvania
    [email protected]
    April 19, 2016

    View Slide

  3. About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 1 / 83

    View Slide

  4. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 2 / 83

    View Slide

  5. About - Me
    • ≈ 9 years at UPenn
    • ≈11 months in current position
    • ≈ 8 years department of biostatistics & epidemiology
    • 10 years at UF (programmer, sysadmin)
    • Red Hat 4.0 Colgate, based on 2.0.18 kernel (not RHEL 4)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 3 / 83

    View Slide

  6. About - UPenn, By The Numbers
    Clay Wells ([email protected]) Threat Detection April 19, 2016 4 / 83

    View Slide

  7. About - UPenn, By The Numbers
    Clay Wells ([email protected]) Threat Detection April 19, 2016 5 / 83

    View Slide

  8. About - UPenn, By The Numbers
    Clay Wells ([email protected]) Threat Detection April 19, 2016 6 / 83

    View Slide

  9. About - UPenn, By The Numbers
    Clay Wells ([email protected]) Threat Detection April 19, 2016 7 / 83

    View Slide

  10. About - Inspiration
    • Lots of data from a various sources
    Darknet sensor ⇒ OSSEC-HIDS ⇒ HECTOR
    • What can we learn from our data?
    • How can we use our data for threat detection?
    • We need to do something, need to start somewhere!
    Clay Wells ([email protected]) Threat Detection April 19, 2016 8 / 83

    View Slide

  11. About - Inspiration, Tons of Data
    Report ’OSSEC Report: Successful Auths’ completed.
    ------------------------------------------------
    ->Processed alerts: 687351
    ->Post-filtering alerts: 4085
    ->First alert: 2016 Jan 25 00:00:01
    ->Last alert: 2016 Jan 25 23:59:39
    Clay Wells ([email protected]) Threat Detection April 19, 2016 9 / 83

    View Slide

  12. About - Inspiration, Tons of Data
    Top entries for ’Source ip’:
    ------------------------------------------------
    12X.XX.XXX.XX |860
    10.0.X.XX |663
    10.0.X.XX |413
    12X.XX.XXX.X8 |288
    13X.XX.XXX.X3 |52
    12X.XX.XXX.XX |51
    98.115.235.181 |24
    76.99.36.224 |13
    12X.XX.XXX.XXX |11
    13X.XX.XX.XXX |10
    Clay Wells ([email protected]) Threat Detection April 19, 2016 10 / 83

    View Slide

  13. About - Inspiration, Tons of Data
    Top entries for ’Username’:
    ------------------------------------------------
    root |1278
    nag |518
    nagmon |384
    mavvel |62
    m_nagios |51
    nag_mon |49
    Clay Wells ([email protected]) Threat Detection April 19, 2016 11 / 83

    View Slide

  14. About - Inspiration, Tons of Data
    Top entries for ’Location’:
    ------------------------------------------------
    (host1.xxx.X.edu) 128.XX.XXX.XX-.. |1119
    (host2.xxx.X.edu) 128.XX.XXX.XX->/va.. |585
    (host3.xxx.X.edu) 128.XX.XXX.XX->/var/log/..|406
    (host4.xxx.X.edu) 128.XX.XX.XX->/var/l.. |343
    (host4.xxx.X.edu) 128.XX.XX.XX->/var/l.. |343
    (host5.xxx.X.edu) 128.XX.XXX.XXX->/va.. |278
    (host6.xxx.X.edu) 128.XX.XXX.XXX->/var/l.. |188
    (host6.xxx.X.edu) 128.XX.XXX.XXX->/var/l.. |188
    (host7.xxx.X.edu) 128.XX.XXX.XXX->/var/.. |148
    (host7.xxx.X.edu) 128.XX.XXX.XXX>/var/.. |148
    Clay Wells ([email protected]) Threat Detection April 19, 2016 12 / 83

    View Slide

  15. About - Inspiration
    Clay Wells ([email protected]) Threat Detection April 19, 2016 13 / 83

    View Slide

  16. About - Inspiration
    Clay Wells ([email protected]) Threat Detection April 19, 2016 14 / 83

    View Slide

  17. About - Inspiration
    Clay Wells ([email protected]) Threat Detection April 19, 2016 15 / 83

    View Slide

  18. About - Inspiration, Spaghetti Plot
    http://www.nicebread.de/visually-weighted-regression-in-r-a-la-solomon-hsiang/
    Clay Wells ([email protected]) Threat Detection April 19, 2016 16 / 83

    View Slide

  19. About - Inspiration, Watercolor Plot
    http://www.nicebread.de/visually-weighted-regression-in-r-a-la-solomon-hsiang/
    Clay Wells ([email protected]) Threat Detection April 19, 2016 17 / 83

    View Slide

  20. Thank you, Internet
    { data, dropped packets, threats, blackhole,
    ip header, blurry hands, internet, ... }
    for all the lovely images
    Clay Wells ([email protected]) Threat Detection April 19, 2016 18 / 83

    View Slide

  21. About - Future Work
    Green star, lower right corner
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 19 / 83

    View Slide

  22. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 20 / 83

    View Slide

  23. Setting The Stage - Evidence-Based Decision Making
    Good displays of data help to reveal knowledge relevant to understanding
    mechanism, process and dynamics, cause and effect.
    - Edward Tufte
    Visual Explanations: Images and Quantities, Evidence and Narrative.
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 21 / 83

    View Slide

  24. Definitions - Threats
    Clay Wells ([email protected]) Threat Detection April 19, 2016 22 / 83

    View Slide

  25. Setting the Stage - Threats
    Source: European Union Agency for Network and Information Security - Threat Landscape 2015
    Clay Wells ([email protected]) Threat Detection April 19, 2016 23 / 83

    View Slide

  26. Definitions - Data Sources
    • Darknet sensor ⇐ Blackhole, dead address
    • Dropped packets ⇐ Unsolicited traffic sent to our sensor
    • OSSEC ⇐ Host-based Intrusion Detection System
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 24 / 83

    View Slide

  27. Definitions - Summary Statistics
    • Quartile
    • Interquartile Range (IQR)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 25 / 83

    View Slide

  28. Summary Statistics - Example
    Clay Wells ([email protected]) Threat Detection April 19, 2016 26 / 83

    View Slide

  29. Software Used
    • iptables
    • OSSEC
    • HECTOR (SEIM - Justin Klein Keane)
    • Python
    • RStudio IDE for R - Why not Python?
    Clay Wells ([email protected]) Threat Detection April 19, 2016 27 / 83

    View Slide

  30. Data Sources
    • Darknet sensor ⇒ OSSEC ⇒ HECTOR
    • Fantastic foundation
    Clay Wells ([email protected]) Threat Detection April 19, 2016 28 / 83

    View Slide

  31. Data Sources - Darknet Probe Reports
    Port Number Hit Count
    -------------------------
    25/tcp 644
    0/udp 128
    23/tcp 64
    6000/tcp 27
    21320/tcp 21
    3389/tcp 20
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 29 / 83

    View Slide

  32. Data Sources - Analyzing OSSEC Alerts
    Typical OSSEC alert
    OSSEC HIDS Notification.
    2015 Nov 16 07:16:16
    Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure
    Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)."
    Portion of the log(s):
    Nov 16 07:16:15 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN
    OSSEC HIDS Notification.
    2015 Nov 16 07:16:18
    Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure
    Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)."
    Portion of the log(s):
    Nov 16 07:16:17 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN
    Clay Wells ([email protected]) Threat Detection April 19, 2016 30 / 83

    View Slide

  33. Setting the Stage - HECTOR
    • Interally developed SEIM
    • Justin Klein Keane
    • https://github.com/madirish/hector
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 31 / 83

    View Slide

  34. Setting the Stage - HECTOR
    • https://github.com/madirish/hector
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 32 / 83

    View Slide

  35. Setting the Stage - Start Asking Questions
    • Visibility into active attacks/threats?
    • What’s being targeted?
    • What can/should we measure?
    • What more can we learn?
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 33 / 83

    View Slide

  36. Setting the Stage - OSSEC Reports, Digging Deeper
    Created OSSEC reports (daily, intra-day) by group, location, all locations
    • Attack alerts
    • Drupal attacks
    • Successful logins
    Clay Wells ([email protected]) Threat Detection April 19, 2016 34 / 83

    View Slide

  37. Goals - Insights, Questions, Clarity
    We’re not setting out to find answers.
    • What type of packets are sensors receiving?
    • What new data might we be interested in capturing?
    • Establish and track baselines over time?
    Clay Wells ([email protected]) Threat Detection April 19, 2016 35 / 83

    View Slide

  38. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 36 / 83

    View Slide

  39. Overview of Our Approach - Data Mining
    • Data
    • Create scripts to generate datasets (Target Data)
    • Clean/prepare data (Preprocessed Data)
    • Create smaller datasets (Transformed Data)
    • Create visualizations (Patterns)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 37 / 83

    View Slide

  40. Darknet Setup - Log Dropped Packets
    iptables -A INPUT -d 128.XXX.XXX.XXX/32 -m state \
    --state NEW -m comment --comment "Log dropped packets" \
    -j LOG --log-prefix "iptables "
    --log-ip-options --log-tcp-options
    Clay Wells ([email protected]) Threat Detection April 19, 2016 38 / 83

    View Slide

  41. Darknet Setup - Log Dropped Packets
    Clay Wells ([email protected]) Threat Detection April 19, 2016 39 / 83

    View Slide

  42. Darknet Setup - Logged to Syslog
    Dropped packet entry in /var/log/message
    Apr 3 14:14:02 host01 kernel: [1052872.333883] iptables
    IN=eth0 OUT= MAC=b8:xx SRC=46.161.40.120 DST=xxx.xx.xx.xx
    LEN=40 TOS=0x08 PREC=0x40 TTL=237 ID=52036 PROTO=TCP
    SPT=41427 DPT=3389 WINDOW=1024 RES=0x00 SYN URGP=0
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 40 / 83

    View Slide

  43. Darknet Setup - Custom OSSEC rule

    4100
    iptables

    Darknet sensor detection for HECTOR.


    Clay Wells ([email protected]) Threat Detection April 19, 2016 41 / 83

    View Slide

  44. Darknet Setup - Search OSSEC rule ID
    $ sudo find /var/ossec/rules -type f -exec grep -H 4100 {} \;
    /../rules/local_rules.xml:
    /../rules/local_rules.xml: 4100
    /../rules/racoon_rules.xml:
    /../rules/racoon_rules.xml: 14100
    /../rules/firewall_rules.xml:
    /../rules/firewall_rules.xml: 4100
    Clay Wells ([email protected]) Threat Detection April 19, 2016 42 / 83

    View Slide

  45. Darknet Setup - OSSEC firewall rule


    firewall
    Firewall rules grouped.

    Clay Wells ([email protected]) Threat Detection April 19, 2016 43 / 83

    View Slide

  46. Darknet Setup - Darknet SQL Table
    -- Darknet sensor table
    CREATE TABLE IF NOT EXISTS ‘darknet‘ (
    ‘id‘ INT NOT NULL AUTO_INCREMENT,
    ‘src_ip‘ INT UNSIGNED NOT NULL,
    ‘dst_ip‘ INT UNSIGNED NOT NULL,
    ‘src_port‘ INT UNSIGNED NOT NULL,
    ‘dst_port‘ INT UNSIGNED NOT NULL,
    ‘proto‘ ENUM(’tcp’,’udp’,’icmp’),
    ‘country_code‘ VARCHAR(2),
    ‘received_at‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    PRIMARY KEY (‘id‘),
    INDEX USING HASH (‘src_ip‘)
    ) ENGINE = INNODB;
    Clay Wells ([email protected]) Threat Detection April 19, 2016 44 / 83

    View Slide

  47. OSSEC - Overview
    Host based Intrusion Detection System
    • Integrated log analysis
    • Windows registry monitoring
    • Rootkit detection
    • Active response
    Clay Wells ([email protected]) Threat Detection April 19, 2016 45 / 83

    View Slide

  48. OSSEC - Rules

    104250
    admin/
    Drupal access denied to admin screen.

    drupal


    104260

    Multiple Drupal access denied to admin
    screens.
    drupal

    Clay Wells ([email protected]) Threat Detection April 19, 2016 46 / 83

    View Slide

  49. OSSEC - Reports
    cat /var/ossec/logs/alerts/alerts.log | \
    /var/ossec/bin/ossec-reportd -n ’Drupal Alert Report’ \
    -f group drupal -r location rule -r rule location
    Clay Wells ([email protected]) Threat Detection April 19, 2016 47 / 83

    View Slide

  50. OSSEC - Report Results
    Report ’Drupal Alert Report’ completed.
    ------------------------------------------------
    ->Processed alerts: 818949
    ->Post-filtering alerts: 114023
    ->First alert: 2016 Apr 03 00:00:10
    ->Last alert: 2016 Apr 03 16:08:19
    Top entries for ’Source ip’:
    ------------------------------------------------
    10.0.0.4 |64074
    127.0.0.1 |8851
    157.55.39.224 |996
    207.46.13.160 |867
    68.180.228.230 |814
    Clay Wells ([email protected]) Threat Detection April 19, 2016 48 / 83

    View Slide

  51. OSSEC - Report Results
    Top entries for ’Level’:
    ------------------------------------------------
    Severity 3 |110525
    Severity 6 |3279
    Severity 10 |114
    Severity 11 |92
    Severity 12 |8
    Severity 9 |5
    Top entries for ’Group’:
    ------------------------------------------------
    drupal |114023
    Clay Wells ([email protected]) Threat Detection April 19, 2016 49 / 83

    View Slide

  52. OSSEC - Report Results
    (neutron.x.edu) 128.x.x.x->/var/log/.. |63586
    rule: ’104250’
    rule: ’104220’
    rule: ’104260’
    rule: ’104262’
    104220 - Drupal failed login! |1322
    location: ’(neutron.x.edu) 128.x.x.x->/var/log/messages’
    location: ’(quasar.x.edu) 128.x.x.x->/var/log/messages’
    location: ’(ah-web1) 54.x.x.x->/var/log/messages’
    location: ’(ah-web2) 54.x.x.x->/var/log/messages’
    104262 - Multiple Drupal access denied to ad.. |8
    location: ’(quasar.x.edu) 128.x.x.x->/var/log/messages’
    location: ’(neutron.x.edu) 128.x.x.x->/var/log/messages’
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 50 / 83

    View Slide

  53. OSSEC - Darknet Report Results
    Top entries for ’Location’:
    ------------------------------------------------
    (host02) 16x.x.x.x->/var/log/messages |802 |
    (host01) 16x.x.x.x->/var/log/messages |516 |
    (host03) 128.x.x.x->/var/log/messages |454 |
    Top entries for ’Rule’:
    ------------------------------------------------
    200001 - Darknet sensor detection for HECTOR. |1772 |
    Clay Wells ([email protected]) Threat Detection April 19, 2016 51 / 83

    View Slide

  54. OSSEC - Darknet Report Results
    Related entries for ’Source ip’:
    ------------------------------------------------
    128.x.x.x |122 |
    location: ’(host02) 16x.x.x.7->/var/log/messages’
    location: ’(host01) 16x.x.x.11->/var/log/messages’
    58.213.133.130 |84 |
    location: ’(host03) 128.x.x.x->/var/log/messages’
    80.82.78.38 |25 |
    location: ’(host02) 16x.x.x.7->/var/log/messages’
    location: ’(host01) 16x.x.x.11->/var/log/messages’
    location: ’(host03) 128.x.x.x->/var/log/messages’
    Clay Wells ([email protected]) Threat Detection April 19, 2016 52 / 83

    View Slide

  55. OSSEC - Alerts SQL Table
    -- OSSEC alerts from clients
    CREATE TABLE IF NOT EXISTS ‘ossec_alert‘ (
    ‘alert_id‘ INT NOT NULL AUTO_INCREMENT,
    ‘alert_date‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    ‘host_id‘ INT NOT NULL,
    ‘alert_log‘ VARCHAR(255) DEFAULT NULL,
    ‘rule_id‘ INT NOT NULL,
    ‘rule_src_ip‘ VARCHAR(15) DEFAULT NULL,
    ‘rule_src_ip_numeric‘ INT UNSIGNED,
    ‘rule_user‘ VARCHAR(20) DEFAULT NULL,
    ‘rule_log‘ TEXT DEFAULT NULL,
    ‘alert_ossec_id‘ VARCHAR(50) NOT NULL,
    PRIMARY KEY (‘alert_id‘),
    INDEX (‘host_id‘), INDEX (‘rule_id‘),
    INDEX USING HASH (‘rule_src_ip_numeric‘),
    INDEX USING HASH (‘rule_id‘),
    INDEX USING HASH (‘host_id‘),
    INDEX USING BTREE (‘alert_date‘)
    ) ENGINE = INNODB;
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 53 / 83

    View Slide

  56. Data We’re Looking At
    • Darknet Probes, 2012-2016 (2014)
    • Attack severity levels,
    • Attack count over 14 days, intraday
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 54 / 83

    View Slide

  57. Pause for Questions
    Questions?
    Clay Wells ([email protected]) Threat Detection April 19, 2016 55 / 83

    View Slide

  58. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 56 / 83

    View Slide

  59. Data Preparation - Darknet Data
    • Query the DB
    • Clean/prepare the data for R
    Clay Wells ([email protected]) Threat Detection April 19, 2016 57 / 83

    View Slide

  60. Data Preparation - Darknet Query Script
    report-darknet.py ⇒ data-darknet-all.csv
    2016-04-03 21:01:39, 2222, tcp, 2222/tcp, 52.37.175.240, US
    2016-04-03 21:01:45, 2222, tcp, 2222/tcp, 52.37.175.240, US
    2016-04-03 21:04:27, 2222, tcp, 2222/tcp, 85.25.200.140, DE
    2016-04-03 21:09:23, 2222, tcp, 2222/tcp, 59.45.79.103, CN
    2016-04-03 21:09:51, 2222, tcp, 2222/tcp, 59.45.79.103, CN
    2016-04-03 21:18:39, 2222, tcp, 2222/tcp, 182.126.161.124, CN
    Clay Wells ([email protected]) Threat Detection April 19, 2016 58 / 83

    View Slide

  61. Data Preparation - OSSEC Report Data
    • Parse OSSEC reports (archived logs)
    • Clean/prepare the data
    (*)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 59 / 83

    View Slide

  62. Data Preparation - OSSEC Report Data
    get-severity.sh ⇒ severity-2015-mon−d.csv
    Top entries for ’Level’:
    ------------------------------------------------
    Severity 6 |718
    Severity 10 |14
    Severity 8 |4
    Severity 12 |1
    Clay Wells ([email protected]) Threat Detection April 19, 2016 60 / 83

    View Slide

  63. Data Preparation - OSSEC Report Results
    get-severity.sh ⇒ severity-2015-mon−d.csv
    2015-Oct-30, Severity 6, 147
    2015-Oct-30, Severity 10, 12
    2015-Oct-30, Severity 8, 2
    2015-Oct-30, Severity 12, 1
    2015-Oct-30, Severity 3, 1
    Clay Wells ([email protected]) Threat Detection April 19, 2016 61 / 83

    View Slide

  64. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 62 / 83

    View Slide

  65. Using R - Dataframes and Summary Statistics
    • Dataframe for each port
    • Dataframe by year
    • Frequency dataframe by month
    • Compute summary statistics
    Clay Wells ([email protected]) Threat Detection April 19, 2016 63 / 83

    View Slide

  66. Using R - Importing Data, dataframes
    data.darknet.all <- read.csv("data-darknet-all-clean.csv")
    data.Timestamp <- as.POSIXlt(data.darknet.all$Timestamp)
    data.darknet.all.0 <-
    data.darknet.all[data.darknet.all$Port %in% c(0),]
    data.darknet.0.2014 <-
    data.darknet.all.0[data.darknet.all.0$Year==2014,]
    port0.2014 <- rowsum(data.darknet.0.2014$Count,
    group=data.darknet.0.2014$Month)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 64 / 83

    View Slide

  67. R Functions, Summary Statistics
    port0.2014.median <- median(port0.2014)
    port0.2014.avg <- mean(port0.2014)
    port0.2014.top <- quantile(port0.2014, 3/4)
    port0.2014.low <- quantile(port0.2014, 1/4)
    port0.2014.max <- max(port0.2014)
    Clay Wells ([email protected]) Threat Detection April 19, 2016 65 / 83

    View Slide

  68. Using R - Darknet Visualizations
    Clay Wells ([email protected]) Threat Detection April 19, 2016 66 / 83

    View Slide

  69. Using R - Darknet Visualizations, Outliers
    Clay Wells ([email protected]) Threat Detection April 19, 2016 67 / 83

    View Slide

  70. Using R - Darknet Visualizations, Modified
    Clay Wells ([email protected]) Threat Detection April 19, 2016 68 / 83

    View Slide

  71. Using R - Darknet Visualizations
    Clay Wells ([email protected]) Threat Detection April 19, 2016 69 / 83

    View Slide

  72. Using R - Darknet Visualizations
    port0.all dataframe
    201205 10, 201309 780, 201310 1025, 201311 914
    201312 815, 201401 457, 201402 800, 201403 1050
    201404 1075, 201405 670, 201406 678, 201407 951
    201408 1076, 201409 1551, 201410 1979, 201411 1478
    201412 2387, 201501 2340, 201502 2512, 201503 1205
    201509 2086, 201510 2104, 201511 3129, 201512 2518
    201601 1480, 201602 6832
    Clay Wells ([email protected]) Threat Detection April 19, 2016 70 / 83

    View Slide

  73. R Plotting Functions
    plot(port0.all, type=’b’, xlab=’Month’,
    ylab=’Freq’, ylim=c(10,port0.all.max),
    main=’Port 0/UDP Probes, 2012-2016’)
    abline(h=port0.all.median, v=NULL, col="blue", lty="dashed")
    abline(h=port0.all.avg, v=NULL, col="orange", lty="dashed")
    abline(h=port0.all.top, v=NULL, col="grey75", lty="dotted")
    abline(h=port0.all.low, v=NULL, col="grey75", lty="dotted")
    Clay Wells ([email protected]) Threat Detection April 19, 2016 71 / 83

    View Slide

  74. Using R - Darknet Visualizations
    Clay Wells ([email protected]) Threat Detection April 19, 2016 72 / 83

    View Slide

  75. Using R - Darknet Visualizations
    Clay Wells ([email protected]) Threat Detection April 19, 2016 73 / 83

    View Slide

  76. 2014, Heartbleed
    Clay Wells ([email protected]) Threat Detection April 19, 2016 74 / 83

    View Slide

  77. 2014 Port 443 Probes - Events?
    Clay Wells ([email protected]) Threat Detection April 19, 2016 75 / 83

    View Slide

  78. 2014, POODLE
    http://arstechnica.com/security/2014/10/ssl-broken-again-in-poodle-attack/
    Clay Wells ([email protected]) Threat Detection April 19, 2016 76 / 83

    View Slide

  79. Data Visualization - Attack Alerts
    Clay Wells ([email protected]) Threat Detection April 19, 2016 77 / 83

    View Slide

  80. Using R - Demo
    RScript Demo
    Clay Wells ([email protected]) Threat Detection April 19, 2016 78 / 83

    View Slide

  81. Overview
    About
    Setting The Stage
    Definitions
    Software Used
    Data Sources
    Our Approach
    Overview of Our Approach
    Darknet Setup
    OSSEC
    Data Preparation
    Darknet Data
    OSSEC Report Data
    Using R
    Summary
    Clay Wells ([email protected]) Threat Detection April 19, 2016 79 / 83

    View Slide

  82. Summary
    • Dig into existing data, start asking questions
    • Start creating simple visualizations
    • Start compiling summary statistics
    • Continue adding logs, logs, and more logs!
    Clay Wells ([email protected]) Threat Detection April 19, 2016 80 / 83

    View Slide

  83. Resources
    • iptables - http://netfilter.org/projects/iptables/
    • OSSEC - https://ossec.github.io/
    • HECTOR - https://github.com/madirish/hector
    • R - https://www.r-project.org/
    • RStudio - urlhttps://www.rstudio.com/
    • Lattice: Multivariate Data Visualization with R by Sarkar, Deepayan
    Clay Wells ([email protected]) Threat Detection April 19, 2016 81 / 83

    View Slide

  84. Statistics & Data Visualization, Learning More
    Books
    • Statistics, 4th Edition by Freedman, Pisani, Purves
    • The Visual Display of Quantitative Information by Edward Tufte
    Data Visualization Video Playlist, Ted Talks
    • https://www.youtube.com/playlist?\list=
    PL31HXzEkTvxDFiKSZa-Ylw33bQ_NeHyGk
    Clay Wells ([email protected]) Threat Detection April 19, 2016 82 / 83

    View Slide

  85. Thank You!
    { You, EDUCAUSE, Christine Brisson, Warren Petrofsky,
    Justin Klein Keane, InfoSec Reading Group,
    UPenn ISC Security Team, John Mulhern III, ... }
    Questions?
    Clay Wells ([email protected]) Threat Detection April 19, 2016 83 / 83

    View Slide