Slide 1
Slide 1 text
Threat Detection
Clay Wells ([email protected]) Threat Detection April 19, 2016 1 / 83
Slide 2
Slide 2 text
Threat Detection Using Time Series Analysis and
Summary Statistics of Darknet Probes and OSSEC
Reports
Clay Wells
School of Arts and Sciences
University of Pennsylvania
[email protected]
April 19, 2016
Slide 3
Slide 3 text
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 1 / 83
Slide 4
Slide 4 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 2 / 83
Slide 5
Slide 5 text
About - Me
• ≈ 9 years at UPenn
• ≈11 months in current position
• ≈ 8 years department of biostatistics & epidemiology
• 10 years at UF (programmer, sysadmin)
• Red Hat 4.0 Colgate, based on 2.0.18 kernel (not RHEL 4)
Clay Wells ([email protected]) Threat Detection April 19, 2016 3 / 83
Slide 6
Slide 6 text
About - UPenn, By The Numbers
Clay Wells ([email protected]) Threat Detection April 19, 2016 4 / 83
Slide 7
Slide 7 text
About - UPenn, By The Numbers
Clay Wells ([email protected]) Threat Detection April 19, 2016 5 / 83
Slide 8
Slide 8 text
About - UPenn, By The Numbers
Clay Wells ([email protected]) Threat Detection April 19, 2016 6 / 83
Slide 9
Slide 9 text
About - UPenn, By The Numbers
Clay Wells ([email protected]) Threat Detection April 19, 2016 7 / 83
Slide 10
Slide 10 text
About - Inspiration
• Lots of data from a various sources
Darknet sensor ⇒ OSSEC-HIDS ⇒ HECTOR
• What can we learn from our data?
• How can we use our data for threat detection?
• We need to do something, need to start somewhere!
Clay Wells ([email protected]) Threat Detection April 19, 2016 8 / 83
Slide 11
Slide 11 text
About - Inspiration, Tons of Data
Report ’OSSEC Report: Successful Auths’ completed.
------------------------------------------------
->Processed alerts: 687351
->Post-filtering alerts: 4085
->First alert: 2016 Jan 25 00:00:01
->Last alert: 2016 Jan 25 23:59:39
Clay Wells ([email protected]) Threat Detection April 19, 2016 9 / 83
Slide 12
Slide 12 text
About - Inspiration, Tons of Data
Top entries for ’Source ip’:
------------------------------------------------
12X.XX.XXX.XX |860
10.0.X.XX |663
10.0.X.XX |413
12X.XX.XXX.X8 |288
13X.XX.XXX.X3 |52
12X.XX.XXX.XX |51
98.115.235.181 |24
76.99.36.224 |13
12X.XX.XXX.XXX |11
13X.XX.XX.XXX |10
Clay Wells ([email protected]) Threat Detection April 19, 2016 10 / 83
Slide 13
Slide 13 text
About - Inspiration, Tons of Data
Top entries for ’Username’:
------------------------------------------------
root |1278
nag |518
nagmon |384
mavvel |62
m_nagios |51
nag_mon |49
Clay Wells ([email protected]) Threat Detection April 19, 2016 11 / 83
Slide 14
Slide 14 text
About - Inspiration, Tons of Data
Top entries for ’Location’:
------------------------------------------------
(host1.xxx.X.edu) 128.XX.XXX.XX-.. |1119
(host2.xxx.X.edu) 128.XX.XXX.XX->/va.. |585
(host3.xxx.X.edu) 128.XX.XXX.XX->/var/log/..|406
(host4.xxx.X.edu) 128.XX.XX.XX->/var/l.. |343
(host4.xxx.X.edu) 128.XX.XX.XX->/var/l.. |343
(host5.xxx.X.edu) 128.XX.XXX.XXX->/va.. |278
(host6.xxx.X.edu) 128.XX.XXX.XXX->/var/l.. |188
(host6.xxx.X.edu) 128.XX.XXX.XXX->/var/l.. |188
(host7.xxx.X.edu) 128.XX.XXX.XXX->/var/.. |148
(host7.xxx.X.edu) 128.XX.XXX.XXX>/var/.. |148
Clay Wells ([email protected]) Threat Detection April 19, 2016 12 / 83
Slide 15
Slide 15 text
About - Inspiration
Clay Wells ([email protected]) Threat Detection April 19, 2016 13 / 83
Slide 16
Slide 16 text
About - Inspiration
Clay Wells ([email protected]) Threat Detection April 19, 2016 14 / 83
Slide 17
Slide 17 text
About - Inspiration
Clay Wells ([email protected]) Threat Detection April 19, 2016 15 / 83
Slide 18
Slide 18 text
About - Inspiration, Spaghetti Plot
http://www.nicebread.de/visually-weighted-regression-in-r-a-la-solomon-hsiang/
Clay Wells ([email protected]) Threat Detection April 19, 2016 16 / 83
Slide 19
Slide 19 text
About - Inspiration, Watercolor Plot
http://www.nicebread.de/visually-weighted-regression-in-r-a-la-solomon-hsiang/
Clay Wells ([email protected]) Threat Detection April 19, 2016 17 / 83
Slide 20
Slide 20 text
Thank you, Internet
{ data, dropped packets, threats, blackhole,
ip header, blurry hands, internet, ... }
for all the lovely images
Clay Wells ([email protected]) Threat Detection April 19, 2016 18 / 83
Slide 21
Slide 21 text
About - Future Work
Green star, lower right corner
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 19 / 83
Slide 22
Slide 22 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 20 / 83
Slide 23
Slide 23 text
Setting The Stage - Evidence-Based Decision Making
Good displays of data help to reveal knowledge relevant to understanding
mechanism, process and dynamics, cause and effect.
- Edward Tufte
Visual Explanations: Images and Quantities, Evidence and Narrative.
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 21 / 83
Slide 24
Slide 24 text
Definitions - Threats
Clay Wells ([email protected]) Threat Detection April 19, 2016 22 / 83
Slide 25
Slide 25 text
Setting the Stage - Threats
Source: European Union Agency for Network and Information Security - Threat Landscape 2015
Clay Wells ([email protected]) Threat Detection April 19, 2016 23 / 83
Slide 26
Slide 26 text
Definitions - Data Sources
• Darknet sensor ⇐ Blackhole, dead address
• Dropped packets ⇐ Unsolicited traffic sent to our sensor
• OSSEC ⇐ Host-based Intrusion Detection System
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 24 / 83
Slide 27
Slide 27 text
Definitions - Summary Statistics
• Quartile
• Interquartile Range (IQR)
Clay Wells ([email protected]) Threat Detection April 19, 2016 25 / 83
Slide 28
Slide 28 text
Summary Statistics - Example
Clay Wells ([email protected]) Threat Detection April 19, 2016 26 / 83
Slide 29
Slide 29 text
Software Used
• iptables
• OSSEC
• HECTOR (SEIM - Justin Klein Keane)
• Python
• RStudio IDE for R - Why not Python?
Clay Wells ([email protected]) Threat Detection April 19, 2016 27 / 83
Slide 30
Slide 30 text
Data Sources
• Darknet sensor ⇒ OSSEC ⇒ HECTOR
• Fantastic foundation
Clay Wells ([email protected]) Threat Detection April 19, 2016 28 / 83
Slide 31
Slide 31 text
Data Sources - Darknet Probe Reports
Port Number Hit Count
-------------------------
25/tcp 644
0/udp 128
23/tcp 64
6000/tcp 27
21320/tcp 21
3389/tcp 20
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 29 / 83
Slide 32
Slide 32 text
Data Sources - Analyzing OSSEC Alerts
Typical OSSEC alert
OSSEC HIDS Notification.
2015 Nov 16 07:16:16
Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure
Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)."
Portion of the log(s):
Nov 16 07:16:15 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN
OSSEC HIDS Notification.
2015 Nov 16 07:16:18
Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure
Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)."
Portion of the log(s):
Nov 16 07:16:17 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN
Clay Wells ([email protected]) Threat Detection April 19, 2016 30 / 83
Slide 33
Slide 33 text
Setting the Stage - HECTOR
• Interally developed SEIM
• Justin Klein Keane
• https://github.com/madirish/hector
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 31 / 83
Slide 34
Slide 34 text
Setting the Stage - HECTOR
• https://github.com/madirish/hector
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 32 / 83
Slide 35
Slide 35 text
Setting the Stage - Start Asking Questions
• Visibility into active attacks/threats?
• What’s being targeted?
• What can/should we measure?
• What more can we learn?
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 33 / 83
Slide 36
Slide 36 text
Setting the Stage - OSSEC Reports, Digging Deeper
Created OSSEC reports (daily, intra-day) by group, location, all locations
• Attack alerts
• Drupal attacks
• Successful logins
Clay Wells ([email protected]) Threat Detection April 19, 2016 34 / 83
Slide 37
Slide 37 text
Goals - Insights, Questions, Clarity
We’re not setting out to find answers.
• What type of packets are sensors receiving?
• What new data might we be interested in capturing?
• Establish and track baselines over time?
Clay Wells ([email protected]) Threat Detection April 19, 2016 35 / 83
Slide 38
Slide 38 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 36 / 83
Slide 39
Slide 39 text
Overview of Our Approach - Data Mining
• Data
• Create scripts to generate datasets (Target Data)
• Clean/prepare data (Preprocessed Data)
• Create smaller datasets (Transformed Data)
• Create visualizations (Patterns)
Clay Wells ([email protected]) Threat Detection April 19, 2016 37 / 83
Slide 40
Slide 40 text
Darknet Setup - Log Dropped Packets
iptables -A INPUT -d 128.XXX.XXX.XXX/32 -m state \
--state NEW -m comment --comment "Log dropped packets" \
-j LOG --log-prefix "iptables "
--log-ip-options --log-tcp-options
Clay Wells ([email protected]) Threat Detection April 19, 2016 38 / 83
Slide 41
Slide 41 text
Darknet Setup - Log Dropped Packets
Clay Wells ([email protected]) Threat Detection April 19, 2016 39 / 83
Slide 42
Slide 42 text
Darknet Setup - Logged to Syslog
Dropped packet entry in /var/log/message
Apr 3 14:14:02 host01 kernel: [1052872.333883] iptables
IN=eth0 OUT= MAC=b8:xx SRC=46.161.40.120 DST=xxx.xx.xx.xx
LEN=40 TOS=0x08 PREC=0x40 TTL=237 ID=52036 PROTO=TCP
SPT=41427 DPT=3389 WINDOW=1024 RES=0x00 SYN URGP=0
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 40 / 83
Slide 43
Slide 43 text
Darknet Setup - Custom OSSEC rule
4100
iptables
Darknet sensor detection for HECTOR.
Clay Wells ([email protected]) Threat Detection April 19, 2016 41 / 83
Slide 44
Slide 44 text
Darknet Setup - Search OSSEC rule ID
$ sudo find /var/ossec/rules -type f -exec grep -H 4100 {} \;
/../rules/local_rules.xml:
/../rules/local_rules.xml: 4100
/../rules/racoon_rules.xml:
/../rules/racoon_rules.xml: 14100
/../rules/firewall_rules.xml:
/../rules/firewall_rules.xml: 4100
Clay Wells ([email protected]) Threat Detection April 19, 2016 42 / 83
Slide 45
Slide 45 text
Darknet Setup - OSSEC firewall rule
firewall
Firewall rules grouped.
Clay Wells ([email protected]) Threat Detection April 19, 2016 43 / 83
Slide 46
Slide 46 text
Darknet Setup - Darknet SQL Table
-- Darknet sensor table
CREATE TABLE IF NOT EXISTS ‘darknet‘ (
‘id‘ INT NOT NULL AUTO_INCREMENT,
‘src_ip‘ INT UNSIGNED NOT NULL,
‘dst_ip‘ INT UNSIGNED NOT NULL,
‘src_port‘ INT UNSIGNED NOT NULL,
‘dst_port‘ INT UNSIGNED NOT NULL,
‘proto‘ ENUM(’tcp’,’udp’,’icmp’),
‘country_code‘ VARCHAR(2),
‘received_at‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (‘id‘),
INDEX USING HASH (‘src_ip‘)
) ENGINE = INNODB;
Clay Wells ([email protected]) Threat Detection April 19, 2016 44 / 83
Slide 47
Slide 47 text
OSSEC - Overview
Host based Intrusion Detection System
• Integrated log analysis
• Windows registry monitoring
• Rootkit detection
• Active response
Clay Wells ([email protected]) Threat Detection April 19, 2016 45 / 83
Slide 48
Slide 48 text
OSSEC - Rules
104250
admin/
Drupal access denied to admin screen.
drupal
104260
Multiple Drupal access denied to admin
screens.
drupal
Clay Wells ([email protected]) Threat Detection April 19, 2016 46 / 83
Slide 49
Slide 49 text
OSSEC - Reports
cat /var/ossec/logs/alerts/alerts.log | \
/var/ossec/bin/ossec-reportd -n ’Drupal Alert Report’ \
-f group drupal -r location rule -r rule location
Clay Wells ([email protected]) Threat Detection April 19, 2016 47 / 83
Slide 50
Slide 50 text
OSSEC - Report Results
Report ’Drupal Alert Report’ completed.
------------------------------------------------
->Processed alerts: 818949
->Post-filtering alerts: 114023
->First alert: 2016 Apr 03 00:00:10
->Last alert: 2016 Apr 03 16:08:19
Top entries for ’Source ip’:
------------------------------------------------
10.0.0.4 |64074
127.0.0.1 |8851
157.55.39.224 |996
207.46.13.160 |867
68.180.228.230 |814
Clay Wells ([email protected]) Threat Detection April 19, 2016 48 / 83
Slide 51
Slide 51 text
OSSEC - Report Results
Top entries for ’Level’:
------------------------------------------------
Severity 3 |110525
Severity 6 |3279
Severity 10 |114
Severity 11 |92
Severity 12 |8
Severity 9 |5
Top entries for ’Group’:
------------------------------------------------
drupal |114023
Clay Wells ([email protected]) Threat Detection April 19, 2016 49 / 83
Slide 52
Slide 52 text
OSSEC - Report Results
(neutron.x.edu) 128.x.x.x->/var/log/.. |63586
rule: ’104250’
rule: ’104220’
rule: ’104260’
rule: ’104262’
104220 - Drupal failed login! |1322
location: ’(neutron.x.edu) 128.x.x.x->/var/log/messages’
location: ’(quasar.x.edu) 128.x.x.x->/var/log/messages’
location: ’(ah-web1) 54.x.x.x->/var/log/messages’
location: ’(ah-web2) 54.x.x.x->/var/log/messages’
104262 - Multiple Drupal access denied to ad.. |8
location: ’(quasar.x.edu) 128.x.x.x->/var/log/messages’
location: ’(neutron.x.edu) 128.x.x.x->/var/log/messages’
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 50 / 83
Slide 53
Slide 53 text
OSSEC - Darknet Report Results
Top entries for ’Location’:
------------------------------------------------
(host02) 16x.x.x.x->/var/log/messages |802 |
(host01) 16x.x.x.x->/var/log/messages |516 |
(host03) 128.x.x.x->/var/log/messages |454 |
Top entries for ’Rule’:
------------------------------------------------
200001 - Darknet sensor detection for HECTOR. |1772 |
Clay Wells ([email protected]) Threat Detection April 19, 2016 51 / 83
Slide 54
Slide 54 text
OSSEC - Darknet Report Results
Related entries for ’Source ip’:
------------------------------------------------
128.x.x.x |122 |
location: ’(host02) 16x.x.x.7->/var/log/messages’
location: ’(host01) 16x.x.x.11->/var/log/messages’
58.213.133.130 |84 |
location: ’(host03) 128.x.x.x->/var/log/messages’
80.82.78.38 |25 |
location: ’(host02) 16x.x.x.7->/var/log/messages’
location: ’(host01) 16x.x.x.11->/var/log/messages’
location: ’(host03) 128.x.x.x->/var/log/messages’
Clay Wells ([email protected]) Threat Detection April 19, 2016 52 / 83
Slide 55
Slide 55 text
OSSEC - Alerts SQL Table
-- OSSEC alerts from clients
CREATE TABLE IF NOT EXISTS ‘ossec_alert‘ (
‘alert_id‘ INT NOT NULL AUTO_INCREMENT,
‘alert_date‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
‘host_id‘ INT NOT NULL,
‘alert_log‘ VARCHAR(255) DEFAULT NULL,
‘rule_id‘ INT NOT NULL,
‘rule_src_ip‘ VARCHAR(15) DEFAULT NULL,
‘rule_src_ip_numeric‘ INT UNSIGNED,
‘rule_user‘ VARCHAR(20) DEFAULT NULL,
‘rule_log‘ TEXT DEFAULT NULL,
‘alert_ossec_id‘ VARCHAR(50) NOT NULL,
PRIMARY KEY (‘alert_id‘),
INDEX (‘host_id‘), INDEX (‘rule_id‘),
INDEX USING HASH (‘rule_src_ip_numeric‘),
INDEX USING HASH (‘rule_id‘),
INDEX USING HASH (‘host_id‘),
INDEX USING BTREE (‘alert_date‘)
) ENGINE = INNODB;
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 53 / 83
Slide 56
Slide 56 text
Data We’re Looking At
• Darknet Probes, 2012-2016 (2014)
• Attack severity levels,
• Attack count over 14 days, intraday
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 54 / 83
Slide 57
Slide 57 text
Pause for Questions
Questions?
Clay Wells ([email protected]) Threat Detection April 19, 2016 55 / 83
Slide 58
Slide 58 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 56 / 83
Slide 59
Slide 59 text
Data Preparation - Darknet Data
• Query the DB
• Clean/prepare the data for R
Clay Wells ([email protected]) Threat Detection April 19, 2016 57 / 83
Slide 60
Slide 60 text
Data Preparation - Darknet Query Script
report-darknet.py ⇒ data-darknet-all.csv
2016-04-03 21:01:39, 2222, tcp, 2222/tcp, 52.37.175.240, US
2016-04-03 21:01:45, 2222, tcp, 2222/tcp, 52.37.175.240, US
2016-04-03 21:04:27, 2222, tcp, 2222/tcp, 85.25.200.140, DE
2016-04-03 21:09:23, 2222, tcp, 2222/tcp, 59.45.79.103, CN
2016-04-03 21:09:51, 2222, tcp, 2222/tcp, 59.45.79.103, CN
2016-04-03 21:18:39, 2222, tcp, 2222/tcp, 182.126.161.124, CN
Clay Wells ([email protected]) Threat Detection April 19, 2016 58 / 83
Slide 61
Slide 61 text
Data Preparation - OSSEC Report Data
• Parse OSSEC reports (archived logs)
• Clean/prepare the data
(*)
Clay Wells ([email protected]) Threat Detection April 19, 2016 59 / 83
Slide 62
Slide 62 text
Data Preparation - OSSEC Report Data
get-severity.sh ⇒ severity-2015-mon−d.csv
Top entries for ’Level’:
------------------------------------------------
Severity 6 |718
Severity 10 |14
Severity 8 |4
Severity 12 |1
Clay Wells ([email protected]) Threat Detection April 19, 2016 60 / 83
Slide 63
Slide 63 text
Data Preparation - OSSEC Report Results
get-severity.sh ⇒ severity-2015-mon−d.csv
2015-Oct-30, Severity 6, 147
2015-Oct-30, Severity 10, 12
2015-Oct-30, Severity 8, 2
2015-Oct-30, Severity 12, 1
2015-Oct-30, Severity 3, 1
Clay Wells ([email protected]) Threat Detection April 19, 2016 61 / 83
Slide 64
Slide 64 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 62 / 83
Slide 65
Slide 65 text
Using R - Dataframes and Summary Statistics
• Dataframe for each port
• Dataframe by year
• Frequency dataframe by month
• Compute summary statistics
Clay Wells ([email protected]) Threat Detection April 19, 2016 63 / 83
Slide 66
Slide 66 text
Using R - Importing Data, dataframes
data.darknet.all <- read.csv("data-darknet-all-clean.csv")
data.Timestamp <- as.POSIXlt(data.darknet.all$Timestamp)
data.darknet.all.0 <-
data.darknet.all[data.darknet.all$Port %in% c(0),]
data.darknet.0.2014 <-
data.darknet.all.0[data.darknet.all.0$Year==2014,]
port0.2014 <- rowsum(data.darknet.0.2014$Count,
group=data.darknet.0.2014$Month)
Clay Wells ([email protected]) Threat Detection April 19, 2016 64 / 83
Slide 67
Slide 67 text
R Functions, Summary Statistics
port0.2014.median <- median(port0.2014)
port0.2014.avg <- mean(port0.2014)
port0.2014.top <- quantile(port0.2014, 3/4)
port0.2014.low <- quantile(port0.2014, 1/4)
port0.2014.max <- max(port0.2014)
Clay Wells ([email protected]) Threat Detection April 19, 2016 65 / 83
Slide 68
Slide 68 text
Using R - Darknet Visualizations
Clay Wells ([email protected]) Threat Detection April 19, 2016 66 / 83
Slide 69
Slide 69 text
Using R - Darknet Visualizations, Outliers
Clay Wells ([email protected]) Threat Detection April 19, 2016 67 / 83
Slide 70
Slide 70 text
Using R - Darknet Visualizations, Modified
Clay Wells ([email protected]) Threat Detection April 19, 2016 68 / 83
Slide 71
Slide 71 text
Using R - Darknet Visualizations
Clay Wells ([email protected]) Threat Detection April 19, 2016 69 / 83
Slide 72
Slide 72 text
Using R - Darknet Visualizations
port0.all dataframe
201205 10, 201309 780, 201310 1025, 201311 914
201312 815, 201401 457, 201402 800, 201403 1050
201404 1075, 201405 670, 201406 678, 201407 951
201408 1076, 201409 1551, 201410 1979, 201411 1478
201412 2387, 201501 2340, 201502 2512, 201503 1205
201509 2086, 201510 2104, 201511 3129, 201512 2518
201601 1480, 201602 6832
Clay Wells ([email protected]) Threat Detection April 19, 2016 70 / 83
Slide 73
Slide 73 text
R Plotting Functions
plot(port0.all, type=’b’, xlab=’Month’,
ylab=’Freq’, ylim=c(10,port0.all.max),
main=’Port 0/UDP Probes, 2012-2016’)
abline(h=port0.all.median, v=NULL, col="blue", lty="dashed")
abline(h=port0.all.avg, v=NULL, col="orange", lty="dashed")
abline(h=port0.all.top, v=NULL, col="grey75", lty="dotted")
abline(h=port0.all.low, v=NULL, col="grey75", lty="dotted")
Clay Wells ([email protected]) Threat Detection April 19, 2016 71 / 83
Slide 74
Slide 74 text
Using R - Darknet Visualizations
Clay Wells ([email protected]) Threat Detection April 19, 2016 72 / 83
Slide 75
Slide 75 text
Using R - Darknet Visualizations
Clay Wells ([email protected]) Threat Detection April 19, 2016 73 / 83
Slide 76
Slide 76 text
2014, Heartbleed
Clay Wells ([email protected]) Threat Detection April 19, 2016 74 / 83
Slide 77
Slide 77 text
2014 Port 443 Probes - Events?
Clay Wells ([email protected]) Threat Detection April 19, 2016 75 / 83
Slide 78
Slide 78 text
2014, POODLE
http://arstechnica.com/security/2014/10/ssl-broken-again-in-poodle-attack/
Clay Wells ([email protected]) Threat Detection April 19, 2016 76 / 83
Slide 79
Slide 79 text
Data Visualization - Attack Alerts
Clay Wells ([email protected]) Threat Detection April 19, 2016 77 / 83
Slide 80
Slide 80 text
Using R - Demo
RScript Demo
Clay Wells ([email protected]) Threat Detection April 19, 2016 78 / 83
Slide 81
Slide 81 text
Overview
About
Setting The Stage
Definitions
Software Used
Data Sources
Our Approach
Overview of Our Approach
Darknet Setup
OSSEC
Data Preparation
Darknet Data
OSSEC Report Data
Using R
Summary
Clay Wells ([email protected]) Threat Detection April 19, 2016 79 / 83
Slide 82
Slide 82 text
Summary
• Dig into existing data, start asking questions
• Start creating simple visualizations
• Start compiling summary statistics
• Continue adding logs, logs, and more logs!
Clay Wells ([email protected]) Threat Detection April 19, 2016 80 / 83
Slide 83
Slide 83 text
Resources
• iptables - http://netfilter.org/projects/iptables/
• OSSEC - https://ossec.github.io/
• HECTOR - https://github.com/madirish/hector
• R - https://www.r-project.org/
• RStudio - urlhttps://www.rstudio.com/
• Lattice: Multivariate Data Visualization with R by Sarkar, Deepayan
Clay Wells ([email protected]) Threat Detection April 19, 2016 81 / 83
Slide 84
Slide 84 text
Statistics & Data Visualization, Learning More
Books
• Statistics, 4th Edition by Freedman, Pisani, Purves
• The Visual Display of Quantitative Information by Edward Tufte
Data Visualization Video Playlist, Ted Talks
• https://www.youtube.com/playlist?\list=
PL31HXzEkTvxDFiKSZa-Ylw33bQ_NeHyGk
Clay Wells ([email protected]) Threat Detection April 19, 2016 82 / 83
Slide 85
Slide 85 text
Thank You!
{ You, EDUCAUSE, Christine Brisson, Warren Petrofsky,
Justin Klein Keane, InfoSec Reading Group,
UPenn ISC Security Team, John Mulhern III, ... }
Questions?
Clay Wells ([email protected]) Threat Detection April 19, 2016 83 / 83