Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 1 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 2 / 83
≈11 months in current position • ≈ 8 years department of biostatistics & epidemiology • 10 years at UF (programmer, sysadmin) • Red Hat 4.0 Colgate, based on 2.0.18 kernel (not RHEL 4) Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 3 / 83
sources Darknet sensor ⇒ OSSEC-HIDS ⇒ HECTOR • What can we learn from our data? • How can we use our data for threat detection? • We need to do something, need to start somewhere! Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 8 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 20 / 83
data help to reveal knowledge relevant to understanding mechanism, process and dynamics, cause and effect. - Edward Tufte Visual Explanations: Images and Quantities, Evidence and Narrative. (*) Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 21 / 83
HIDS Notification. 2015 Nov 16 07:16:16 Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)." Portion of the log(s): Nov 16 07:16:15 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN OSSEC HIDS Notification. 2015 Nov 16 07:16:18 Received From: (xxxx.upenn.edu) 128.xx.xx.xx -> /var/log/secure Rule: 5701 fired (level 8) -> "Possible attack on the ssh server (or version gathering)." Portion of the log(s): Nov 16 07:16:17 writing sshd[8284]: Bad protocol version identification ’SSH-2.0’ from UNKNOWN Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 30 / 83
active attacks/threats? • What’s being targeted? • What can/should we measure? • What more can we learn? (*) Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 33 / 83
find answers. • What type of packets are sensors receiving? • What new data might we be interested in capturing? • Establish and track baselines over time? Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 35 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 36 / 83
CREATE TABLE IF NOT EXISTS ‘darknet‘ ( ‘id‘ INT NOT NULL AUTO_INCREMENT, ‘src_ip‘ INT UNSIGNED NOT NULL, ‘dst_ip‘ INT UNSIGNED NOT NULL, ‘src_port‘ INT UNSIGNED NOT NULL, ‘dst_port‘ INT UNSIGNED NOT NULL, ‘proto‘ ENUM(’tcp’,’udp’,’icmp’), ‘country_code‘ VARCHAR(2), ‘received_at‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (‘id‘), INDEX USING HASH (‘src_ip‘) ) ENGINE = INNODB; Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 44 / 83
CREATE TABLE IF NOT EXISTS ‘ossec_alert‘ ( ‘alert_id‘ INT NOT NULL AUTO_INCREMENT, ‘alert_date‘ TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, ‘host_id‘ INT NOT NULL, ‘alert_log‘ VARCHAR(255) DEFAULT NULL, ‘rule_id‘ INT NOT NULL, ‘rule_src_ip‘ VARCHAR(15) DEFAULT NULL, ‘rule_src_ip_numeric‘ INT UNSIGNED, ‘rule_user‘ VARCHAR(20) DEFAULT NULL, ‘rule_log‘ TEXT DEFAULT NULL, ‘alert_ossec_id‘ VARCHAR(50) NOT NULL, PRIMARY KEY (‘alert_id‘), INDEX (‘host_id‘), INDEX (‘rule_id‘), INDEX USING HASH (‘rule_src_ip_numeric‘), INDEX USING HASH (‘rule_id‘), INDEX USING HASH (‘host_id‘), INDEX USING BTREE (‘alert_date‘) ) ENGINE = INNODB; (*) Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 53 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 56 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 62 / 83
each port • Dataframe by year • Frequency dataframe by month • Compute summary statistics Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 63 / 83
Our Approach Overview of Our Approach Darknet Setup OSSEC Data Preparation Darknet Data OSSEC Report Data Using R Summary Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 79 / 83
Edition by Freedman, Pisani, Purves • The Visual Display of Quantitative Information by Edward Tufte Data Visualization Video Playlist, Ted Talks • https://www.youtube.com/playlist?\list= PL31HXzEkTvxDFiKSZa-Ylw33bQ_NeHyGk Clay Wells (clayw@sas.upenn.edu) Threat Detection April 19, 2016 82 / 83
clayball
soracom
tomuro
c4yokohama
ufoo68
oracle4engineer
tomohide_tanaka
olegkovalov
noriyukitakei
mashabow
shnjtk
finengine
kloudleinc
samanthasiow
orderedlist
geeforr
schacon
zenorocha
destraynor
rasmusluckow
marcelosomers
pauljervisheath
ufuk
moore
malarkey
