Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Building Security Data Lake
Search
Richard Fan
December 13, 2023
Technology
0
16
Building Security Data Lake
vBrownBag podcast
Building Security Data Lake
https://youtu.be/6qQ7_asdI4I?si=CSsn0jz2vo00Y02Q
Richard Fan
December 13, 2023
Tweet
Share
More Decks by Richard Fan
See All by Richard Fan
JAWS Pankration 2024 - Achieve software supply chain security using AWS Nitro Enclaves and GitHub Actions
richardfan1126
0
43
Preserving privacy on data collaboration with AWS Clean Rooms
richardfan1126
0
40
Achieve software supply chain security using AWS Nitro Enclaves and GitHub Actions
richardfan1126
0
130
When Data Collaboration Meets Privacy: Privacy-enhancing Technologies on AWS
richardfan1126
0
53
AWS Security Hub Central Configuration - An Easy way to monitor your Organization security posture
richardfan1126
0
59
Create your first AWS Nitro Enclaves application
richardfan1126
0
51
Other Decks in Technology
See All in Technology
Railsの話をしよう
yahonda
0
160
Contract One Engineering Unit 紹介資料
sansan33
PRO
0
8.9k
Sansan Engineering Unit 紹介資料
sansan33
PRO
1
3k
RDS の負荷が高い場合に AWS で取りうる具体策 N 連発/a-series-of-specific-countermeasures-available-on-aws-when-rds-is-under-high-load
emiki
7
4.2k
Introduction to Bill One Development Engineer
sansan33
PRO
0
300
OpenTelemetry が拡げる Gemini CLI の可観測性
phaya72
2
420
難しいセキュリティ用語をわかりやすくしてみた
yuta3110
0
330
それでも私が品質保証プロセスを作り続ける理由 #テストラジオ / Why I still continue to create QA process
pineapplecandy
0
140
OAuthからOIDCへ ― 認可の仕組みが認証に拡張されるまで
yamatai1212
0
140
私のMCPの使い方
tsubakimoto_s
0
100
いまからでも遅くない!SSL/TLS証明書超入門(It's not too late to start! SSL/TLS Certificates: The Absolute Beginner's Guide)
norimuraz
0
280
エンタメとAIのための3Dパラレルワールド構築(GPU UNITE 2025 特別講演)
pfn
PRO
0
550
Featured
See All Featured
Faster Mobile Websites
deanohume
310
31k
Site-Speed That Sticks
csswizardry
13
910
Measuring & Analyzing Core Web Vitals
bluesmoon
9
620
Learning to Love Humans: Emotional Interface Design
aarron
274
41k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
35
6.1k
Statistics for Hackers
jakevdp
799
220k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
Building Adaptive Systems
keathley
44
2.8k
A better future with KSS
kneath
239
18k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.6k
Facilitating Awesome Meetings
lara
56
6.6k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Transcript
Building Security Data Lake Richard Fan March 29, 2023
EXPRESSVPN Richard Fan Security Engineer from ExpressVPN A Builder and
Tech advocate AWS Community Builder • https://dev.to/richardfan1126 • https://medium.com/@richardfan1126 • https://github.com/richardfan1126 Who am I?
Security Data
EXPRESSVPN What is security data? Security Data
EXPRESSVPN Why do we need Security logs? • Detect threat
• Incident response • Compliance • Vulnerability management Security Data
EXPRESSVPN Security Data Storing (or Not storing) locally • No
correlation • Difficult to track • Time consuming during incident response Send to SIEM • Centralized • Analytics / threat detection • Strong query capability Capturing security logs
EXPRESSVPN The growing amount/complexity of security logs • Shift-left •
Adoption of cloud • 2 common approaches ◦ Drop less-important events ◦ Scale-up SIEM and send all events to it Security Data
EXPRESSVPN SIEM is not an ultimate solution • Too expensive
• Short retention period • Difficult to integrate with other data processor Security Data
Data Lake
EXPRESSVPN Data Lake comes in Data Lake • Store data
in large scale • Centralize data repository • Turn raw data into useful data • NOT a data archive • NOT a database (Security) Data Lake Security Data Lake • Threat detection • Event context • Real-time alert
EXPRESSVPN How to start? • Identify all your data sources
• Identify ingestion methods • Evaluate your situation ◦ Engineering ◦ Threat hunting ◦ SIEM options • Decide where SIEM fits in Data Lake
EXPRESSVPN Connector split Source split Data Lake SIEM in Security
data lake
EXPRESSVPN Data lake to SIEM SIEM to Data lake Data
Lake SIEM in Security data lake
Threat hunting
EXPRESSVPN Threat hunting life cycle in data lake Threat hunting
EXPRESSVPN Detection as Code • Better documentation • Code repository
/ Code Review (GitOps) • Common language • Vendor agnostic Threat hunting
EXPRESSVPN Detection as Code - Sigma Threat hunting
EXPRESSVPN Detection as Code - Sigma Threat hunting Splunk Elasticsearch
Our Story
EXPRESSVPN How do we build security data lake? Technology •
Ingestion • Storage ◦ S3 • Analytics • Detection-as-code ◦ Sigma Our story
EXPRESSVPN How do we build security data lake? Company •
SOC team • IT team • Security Engineering team • Cross-team collaboration • Security knowledge Our story
EXPRESSVPN Takeaway • Evaluate your current state • Start small
• Estimate cost • Embrace IaC / DaC • Don’t forget about people Our story
Thank you