Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Building Security Data Lake
Search
Richard Fan
December 13, 2023
Technology
0
16
Building Security Data Lake
vBrownBag podcast
Building Security Data Lake
https://youtu.be/6qQ7_asdI4I?si=CSsn0jz2vo00Y02Q
Richard Fan
December 13, 2023
Tweet
Share
More Decks by Richard Fan
See All by Richard Fan
JAWS Pankration 2024 - Achieve software supply chain security using AWS Nitro Enclaves and GitHub Actions
richardfan1126
0
43
Preserving privacy on data collaboration with AWS Clean Rooms
richardfan1126
0
40
Achieve software supply chain security using AWS Nitro Enclaves and GitHub Actions
richardfan1126
0
130
When Data Collaboration Meets Privacy: Privacy-enhancing Technologies on AWS
richardfan1126
0
53
AWS Security Hub Central Configuration - An Easy way to monitor your Organization security posture
richardfan1126
0
59
Create your first AWS Nitro Enclaves application
richardfan1126
0
51
Other Decks in Technology
See All in Technology
AI時代におけるデータの重要性 ~データマネジメントの第一歩~
ryoichi_ota
0
700
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
12
81k
React19.2のuseEffectEventを追う
maguroalternative
2
510
dbtとBigQuery MLで実現する リクルートの営業支援基盤のモデル開発と保守運用
recruitengineers
PRO
3
130
AIエージェント入門 〜基礎からMCP・A2Aまで〜
shukob
0
110
AWS Top Engineer、浮いてませんか? / As an AWS Top Engineer, Are You Out of Place?
yuj1osm
2
220
やる気のない自分との向き合い方/How to Deal with Your Unmotivated Self
sanogemaru
1
520
研究開発部メンバーの働き⽅ / Sansan R&D Profile
sansan33
PRO
3
20k
Copilot Studio ハンズオン - 生成オーケストレーションモード
tomoyasasakimskk
0
130
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
sansan33
PRO
0
2.8k
AI時代こそ求められる設計力- AWSクラウドデザインパターン3選で信頼性と拡張性を高める-
kenichirokimura
3
350
Performance Insights 廃止から Database Insights 利用へ/transition-from-performance-insights-to-database-insights
emiki
0
320
Featured
See All Featured
Mobile First: as difficult as doing things right
swwweet
225
10k
4 Signs Your Business is Dying
shpigford
185
22k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
BBQ
matthewcrist
89
9.8k
Rails Girls Zürich Keynote
gr2m
95
14k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Fantastic passwords and where to find them - at NoRuKo
philnash
52
3.4k
Art, The Web, and Tiny UX
lynnandtonic
303
21k
Principles of Awesome APIs and How to Build Them.
keavy
127
17k
Statistics for Hackers
jakevdp
799
220k
How to train your dragon (web standard)
notwaldorf
97
6.3k
The Power of CSS Pseudo Elements
geoffreycrofte
79
6k
Transcript
Building Security Data Lake Richard Fan March 29, 2023
EXPRESSVPN Richard Fan Security Engineer from ExpressVPN A Builder and
Tech advocate AWS Community Builder • https://dev.to/richardfan1126 • https://medium.com/@richardfan1126 • https://github.com/richardfan1126 Who am I?
Security Data
EXPRESSVPN What is security data? Security Data
EXPRESSVPN Why do we need Security logs? • Detect threat
• Incident response • Compliance • Vulnerability management Security Data
EXPRESSVPN Security Data Storing (or Not storing) locally • No
correlation • Difficult to track • Time consuming during incident response Send to SIEM • Centralized • Analytics / threat detection • Strong query capability Capturing security logs
EXPRESSVPN The growing amount/complexity of security logs • Shift-left •
Adoption of cloud • 2 common approaches ◦ Drop less-important events ◦ Scale-up SIEM and send all events to it Security Data
EXPRESSVPN SIEM is not an ultimate solution • Too expensive
• Short retention period • Difficult to integrate with other data processor Security Data
Data Lake
EXPRESSVPN Data Lake comes in Data Lake • Store data
in large scale • Centralize data repository • Turn raw data into useful data • NOT a data archive • NOT a database (Security) Data Lake Security Data Lake • Threat detection • Event context • Real-time alert
EXPRESSVPN How to start? • Identify all your data sources
• Identify ingestion methods • Evaluate your situation ◦ Engineering ◦ Threat hunting ◦ SIEM options • Decide where SIEM fits in Data Lake
EXPRESSVPN Connector split Source split Data Lake SIEM in Security
data lake
EXPRESSVPN Data lake to SIEM SIEM to Data lake Data
Lake SIEM in Security data lake
Threat hunting
EXPRESSVPN Threat hunting life cycle in data lake Threat hunting
EXPRESSVPN Detection as Code • Better documentation • Code repository
/ Code Review (GitOps) • Common language • Vendor agnostic Threat hunting
EXPRESSVPN Detection as Code - Sigma Threat hunting
EXPRESSVPN Detection as Code - Sigma Threat hunting Splunk Elasticsearch
Our Story
EXPRESSVPN How do we build security data lake? Technology •
Ingestion • Storage ◦ S3 • Analytics • Detection-as-code ◦ Sigma Our story
EXPRESSVPN How do we build security data lake? Company •
SOC team • IT team • Security Engineering team • Cross-team collaboration • Security knowledge Our story
EXPRESSVPN Takeaway • Evaluate your current state • Start small
• Estimate cost • Embrace IaC / DaC • Don’t forget about people Our story
Thank you