squert - an open source UI for NSM data

Slide 1

Slide 1 text

squert - an open source web interface for NSM data paul halliday | NSCC | SOC, Augusta 2014

Slide 2

Slide 2 text

project history types of data the interface I am going to talk about

Slide 3

Slide 3 text

“…I was trying to lookup squert at work but the search was blocked by our web proxy” “…while I was researching information for this post I very quickly realized that safe search is a requirement!” Got any freaky new genres in the pipe Ron?

Slide 4

Slide 4 text

sguil (circa 2003) event driven distributed scales well < tcl/tk >

Slide 5

Slide 5 text

my job

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

21 locations 13 campuses 2 data centers ERP, finance links everywhere

Slide 9

Slide 9 text

perception =

Slide 10

Slide 10 text

turn this into this

Slide 11

Slide 11 text

step 1: build step 2: deploy and configure step 3: open the flood gates tedious frustrating overwhelming

Slide 12

Slide 12 text

“Written By Analysts, For Analysts”

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

problem wrong audience timestamps (yeah you UTC!) lack of summary information no visuals/helpers

Slide 15

Slide 15 text

solution

Slide 16

Slide 16 text

version 0.1.0 < php >

Slide 17

Slide 17 text

version 0.6.0 ip2c.tcl – afrinic|apnic|arin|lacnic -> to MySQL

Slide 18

Slide 18 text

enter NSM in seconds even Mom can do it! then in 2009 obscurity can be good no more hiding but

Slide 19

Slide 19 text

version 0.9.0 STOP! time to regroup

Slide 20

Slide 20 text

problems slow… long load times static content no plan lots of duplication generally inefficient

Slide 21

Slide 21 text

architecture (original) client server we good?

Slide 22

Slide 22 text

< js >

Slide 23

Slide 23 text

{ “id”: 1, “signature”:”bad guys”, “src_ip”:”65.55.58.201”, “dst_ip”:”10.0.0.1” } architecture (now) client server JSON we still good?

Slide 24

Slide 24 text

version 1.0 JavaScript a plan! survey says: “we hate it!”

Slide 25

Slide 25 text

Edward R. Tufte: books on displaying information data first don't layer decorations 1 + 1 = 3 graphical excellence

Slide 26

Slide 26 text

Graphical Excellence (or how not to invade Russia) - size of army - 2D location - direction of travel - location on certain dates - temp. on certain dates

Slide 27

Slide 27 text

Graphical Excellence as explained by Tufte a well designed presentation of interesting data that which gives the viewer the greatest number of ideas in the shortest amount of time complex ideas communicated with clarity this is where I want squert to be

Slide 28

Slide 28 text

version 1.4 < js >

Slide 29

Slide 29 text

so what can it do?

Slide 30

Slide 30 text

suricata ids bro network security monitor bro agent for sguil PCAP (selective) windows eventlogs/app logs barracuda spam firewall the data

Slide 31

Slide 31 text

MySQL ElasticSearch Bro Suricata Disk PCAP Sguil Intrusion Detection System (IDS). Uses Signatures to detect bad stuf Network Security Monitor. Comprehensive log collector Alerts squert Client requests Disk Logstash Windows Servers Spam Firewall Peak: 2500 entries / second (notice log only) Sguil agent syslog syslog syslog Logs Logs Logs Alerts Sguil agent Sguil agent Client requests

Slide 32

Slide 32 text

the interface

Slide 33

Slide 33 text

content links feature icons toggles summary click history

Slide 34

Slide 34 text

grouping - ON allows us to produce stats for each signature

Slide 35

Slide 35 text

grouping - OFF provides a time-line of events in context

Slide 36

Slide 36 text

the event frequency and intensity

Slide 37

Slide 37 text

event expansion payload transcript

Slide 38

Slide 38 text

event expansion - payload *

Slide 39

Slide 39 text

Slide 40

Slide 40 text

event expansion - transcript summary P0f output

Slide 41

Slide 41 text

event expansion - transcript

Slide 42

Slide 42 text

event expansion – external source

Slide 43

Slide 43 text

Slide 44

Slide 44 text

event expansion – external source

Slide 45

Slide 45 text

event expansion – external source

Slide 46

Slide 46 text

event categorization

Slide 47

Slide 47 text

option 1 – class only function keys: F1…F8

Slide 48

Slide 48 text

option 2 – class & comment

Slide 49

Slide 49 text

auto categorization

Slide 50

Slide 50 text

auto categorization

Slide 51

Slide 51 text

filters and URLs

Slide 52

Slide 52 text

filters – explicit or shell

Slide 53

Slide 53 text

explicit

Slide 54

Slide 54 text

shell

Slide 55

Slide 55 text

seems complicated.. No. This was complicated

Slide 56

Slide 56 text

URLs

Slide 57

Slide 57 text

URLs

Slide 58

Slide 58 text

content tabs

Slide 59

Slide 59 text

summary

Slide 60

Slide 60 text

summary

Slide 61

Slide 61 text

views

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

Squert: http://www.squertproject.org Sguil: http://sguil.net Secuity Onion: http://blog.securityonion.net Suricata: http://suricata-ids.org Bro: http://www.bro.org Me: int13h - GitHub @01110000