Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Stripe CTF 3.0 - Follow Up
Search
Bogdan Gaza
July 01, 2014
Programming
1
120
Stripe CTF 3.0 - Follow Up
From BigData Meetup #4 - Timisoara, June 2014
Bogdan Gaza
July 01, 2014
Tweet
Share
More Decks by Bogdan Gaza
See All by Bogdan Gaza
Continuous delivery pipelines using Python and Docker
hurrycane
1
490
RailsGirls Timisoara - Web Applications
hurrycane
2
63
Real time webapps using python
hurrycane
3
170
Other Decks in Programming
See All in Programming
CI改善もDatadogとともに
taumu
0
200
責務と認知負荷を整える! 抽象レベルを意識した関心の分離
yahiru
8
1.3k
一休.com のログイン体験を支える技術 〜Web Components x Vue.js 活用事例と最適化について〜
atsumim
0
930
Serverless Rust: Your Low-Risk Entry Point to Rust in Production (and the benefits are huge)
lmammino
1
150
「個人開発マネタイズ大全」が教えてくれたこと
bani24884
1
180
Ça bouge du côté des animations CSS !
goetter
2
150
機能が複雑化しても 頼りになる FactoryBotの話
tamikof
0
110
Grafana Loki によるサーバログのコスト削減
mot_techtalk
1
150
Formの複雑さに立ち向かう
bmthd
1
930
How mixi2 Uses TiDB for SNS Scalability and Performance
kanmo
40
16k
pylint custom ruleで始めるレビュー自動化
shogoujiie
0
150
SwiftUI Viewの責務分離
elmetal
PRO
2
270
Featured
See All Featured
It's Worth the Effort
3n
184
28k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
A designer walks into a library…
pauljervisheath
205
24k
GitHub's CSS Performance
jonrohan
1030
460k
Become a Pro
speakerdeck
PRO
26
5.2k
Git: the NoSQL Database
bkeepers
PRO
427
65k
Mobile First: as difficult as doing things right
swwweet
223
9.4k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3k
Code Reviewing Like a Champion
maltzj
521
39k
The Pragmatic Product Professional
lauravandoore
32
6.4k
Embracing the Ebb and Flow
colly
84
4.6k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Transcript
A distributed systems challenge: Stripe CTF Bogdan Gâza Big Data
#5 Bogdan Gâza
Big Data #5 Bogdan Gâza $whoami
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza Level 0 The mysterious program
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza Different hash: open addressing v
hash(V) hash(X) x
Big Data #5 Bogdan Gâza Bloom filters N hash functions
k1 k2 k3 kn probabilistic fals positives / no fals negatives
Big Data #5 Bogdan Gâza Level 1 Gitcoins
Big Data #5 Bogdan Gâza Level 1 Gitcoins
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza SHA1 - ridiculous parallel bash:
400 Hash/s stripe go miners: 1.9 MHash/s gpu: 1-2 GHash/s
Big Data #5 Bogdan Gâza Level 2 DDos Defense
Big Data #5 Bogdan Gâza Proxy Node Node Node
Big Data #5 Bogdan Gâza Load balancing algorithm 4 req
/ ip request requests < 25 ms apart Solution
Big Data #5 Bogdan Gâza Level 3 Instant code search
Big Data #5 Bogdan Gâza LB Indexer Indexer Indexer 4
minutes to index 4 nodes with 500MB of RAM Scala Latency based scoring To pass the level < 0.15s / query Around 100M words Arbitrary substring
Big Data #5 Bogdan Gâza Twitter Stack: finagle / twitter
server Problem
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza 1st approach: inverted index vs
substring search Solution Trie vs substring search marisa / patricia Trie / radix tree / suffix tree vs substring search DAWG vs substring search Sharding
Big Data #5 Bogdan Gâza Level 4 SQLCluster
Big Data #5 Bogdan Gâza SQLite SQLite SQLite SQLite SQLite
SQLite Unreliable network ! octopus octopus simulates: netsplit / lagsplit / SPOF
Big Data #5 Bogdan Gâza Consensus Reliability in the presence
of faulty processes examples: who can commit to the DB who is the leader state machine replication
Big Data #5 Bogdan Gâza ZAB - High performance broadcast
in primary backup systems - 2011 RAFT - Understandable consensus algorithm - 2013 Paxos - Part-time parliament - Laslie Lamport ‘90 Consensus
Big Data #5 Bogdan Gâza ZAB - High performance broadcast
in primary backup systems - 2011 RAFT - Understandable consensus algorithm - 2013 Paxos - Part-time parliament - Laslie Lamport ‘90 Consensus
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza
Big Data #5 Bogdan Gâza Thanks!