Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Jan Stępień - Tracking those who Track
Search
MunichDataGeeks
July 02, 2013
Technology
1
190
Jan Stępień - Tracking those who Track
Talk by Jan Stępień at the firsta Munich DataGeeks Meetup
Data: 02.07.2013
MunichDataGeeks
July 02, 2013
Tweet
Share
More Decks by MunichDataGeeks
See All by MunichDataGeeks
Florian Haselbeck- Advancing Synthetic Protein Design with Large Language Models
munichdata
0
19
Tobias Ladner- Formal Verification of Neural Networks in Safety-Critical Environments
munichdata
0
15
Uladzislau Sazanovich - JetBrains AI: Deep Dive
munichdata
0
35
Jan Hauffa - A Case Study on Retrieval-Augmented Generation for Document Q&A: Experiences and Future Perspectives
munichdata
0
44
Thomas Schmidt - Revolutionizing SQL Data Model Testing: Introducing SQL-Mock by DeepL
munichdata
0
14
Maximilian Duesberg - The Data is Clear - But Humans are not
munichdata
0
55
Dr.Christoph Mittendorf-Beyond Bard and Transformers: Unconventional ML Use Cases
munichdata
0
77
Heidi Seibold - Are (data) scientists bad at science?
munichdata
0
80
Roland Rodde- Vegetation management for powerlines with remote sensing data
munichdata
0
83
Other Decks in Technology
See All in Technology
依存ライブラリはどこに?
takesection
0
110
今さら聞けない!? AWSの生成AIサービス Amazon Bedrock入門!
minorun365
PRO
11
2.6k
技術広報として2023年度に頑張ったこと / What we did well in FY2023 as a DevRel
pauli
5
490
ハイパフォーマンスな組織をつくるための開発生産性の考え方 / developer-productivity-high-performer-link-and-motivation
lmi
3
260
Vos logs méritent mieux que la config par défaut
lyrixx
2
350
プッシュ型子育てサービスを、先行プロジェクト実施自治体において開始します
govtechtokyo
0
270
業務で使えるかもしれない…!?GitHub Actions の Tips 集 / CI/CD Test Night #7
ponkio_o
PRO
24
7.1k
マイクロサービス環境におけるDB戦略 in DMMプラットフォーム
pospome
11
3k
暗黙知を集積するプラットフォーム : 「健常者エミュレータ事例集」の取り組み
sora32127
1
160
やっていきテスト
k6s4i53rx
0
160
8週連続ウェビナー_イチから学ぶFivetran
cmsuzu
0
170
10分でわかるfreeeのQA
freee
0
260
Featured
See All Featured
WebSockets: Embracing the real-time Web
robhawkes
59
6.9k
Become a Pro
speakerdeck
PRO
8
4.4k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
The World Runs on Bad Software
bkeepers
PRO
60
6.6k
Designing Experiences People Love
moore
135
23k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
8
8.2k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
1
1.2k
Six Lessons from altMBA
skipperchong
19
2.9k
KATA
mclloyd
14
11k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
272
12k
Debugging Ruby Performance
tmm1
68
11k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
124
32k
Transcript
Tracking those who track us Jan Stępień
My name is Jan Stępień and I come from Warsaw
Data analysis is not just big data
Data analysis is fun
It all started with ads tracking “like” buttons other irrelevant
things
1. Use an adblock plugin 2. Block all network communication
to unwelcome domains
My machine website.com ads.website.com
My machine website.com ads.website.com
Let’s capture all those requests!
03.2012 – 06.2013 106 414 requests 322 distinct days approx.
330 requests per day
SQLite3 + Incanter + R + Weka
http_if_none_match http_referer http_accept_encoding http_accept http_cookie http_connection http_host http_user_agent http_version path_info
http_accept_charset http_accept_language http_cache_control http_if_modified_since request_method request_path request_uri query_string remote_host remote_addr script_name server_name server_port server_protocol http_dnt timestamp
timestamp
03 04 05 06 07 08 09 10 11 12
01 02 03 04 05 06 15k 10k 5k 0
00 01 02 03 04 05 06 07 08 09
10 11 12 13 14 15 16 17 18 19 20 21 22 23 100 0 200 300 400 500
8k 6k 4k 2k 0 Mo Tu We Th Fr
Sa Su
http_host
www.google-analytics.com 36197 static.adzerk.net 13983 edge.quantserve.com 11659 www.facebook.com 9641 ad.doubleclick.net 3822
pagead2.googlesyndication.com 3764 s.youtube.com 2173 b.scorecardresearch.com 1974 pubads.g.doubleclick.net 1465 googleads.g.doubleclick.net 1231
48.9% of requests sent to domains owned by Google
http_referer
22902 distinct referrers 4692 distinct domains
Let’s try to combine this dataset with something else
Weather influence?
ogimet.com Humidity, min/max/avg temperature, cloud coverage, visibility, rain/snow, wind speed/direction,
etc.
No correlations!
Tags at stackoverflow.com
http://stackoverflow.com/questions/123/title
data.stackexchange.com
Thanks, wordle.net!
Can be my WWW traffic grouped into clusters?
1. Group requests into 15 minute intervals 2. Count domains
per interval
5008 intervals Each described by over 4500 values
1. Select request from popular domains 2. Group requests into
15 minute intervals 3. Count domains per interval
5008 intervals Each described by 95 values Only 2% of
cells with non-zero values
Principal Component Analysis 95 domains → 16 descriptors
X-means K-means based clustering algorithm
cluster 0 1268 cluster 1 702 cluster 2 651 cluster
3 2387 What is the meaning behind these clusters?
3 stackoverflow.com
2 reddit.com redditmedia.com bbc.co.uk
1 linkedin.com dictionary.reference.com meetup.com
0 rubyonrails.pl developer.android.com tex.stackexchange.com amazon.com youtube.com
How accurate is this clustering? Let’s build a classifier on
the original data
0 1 2 3 ← classified as 1188 29 11
40 cluster 0 47 654 1 0 cluster 1 10 1 622 18 cluster 2 50 0 18 2319 cluster 3 cluster 0: rubyonrails.pl developer.android.com amazon.com youtube.com cluster 1: linkedin.com dictionary.reference.com meetup.com cluster 2: reddit.com redditmedia.com bbc.co.uk cluster 3: stackoverflow.com
Let’s wrap up
Data analysis is not just big data
Data analysis is fun
Thank you very much The picture of Warsaw is ©
Dennis Jarvis 2009