Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web scraping for data scientists

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Irio Musskopf Irio Musskopf
May 24, 2016
Programming
0
81

Web scraping for data scientists

Avatar for Irio Musskopf

Irio Musskopf

May 24, 2016
Tweet

More Decks by Irio Musskopf

See All by Irio Musskopf
Using Machine Learning and Open Data to Report 216 Brazilian Congresspeople for Corruption
Avatar for Irio Musskopf irio
0
360
Por que functional programming é mais rápido?
Avatar for Irio Musskopf irio
0
69
No país das maravilhas
Avatar for Irio Musskopf irio
0
51
Desenvolvendo o mínimo com Ruby on Rails
Avatar for Irio Musskopf irio
0
140
Implementando pagamentos usando Moip
Avatar for Irio Musskopf irio
0
92
vim 101
Avatar for Irio Musskopf irio
1
220

Other Decks in Programming

See All in Programming
副作用をどこに置くか問題：オブジェクト指向で整理する設計判断ツリー
Avatar for Koya Masuda koxya
1
610
FOSDEM 2026: STUNMESH-go: Building P2P WireGuard Mesh Without Self-Hosted Infrastructure
Avatar for Date Huang tjjh89017
0
160
Grafana：建立系統全知視角的捷徑
Avatar for Blueswen blueswen
0
330
humanlayerのブログから学ぶ、良いCLAUDE.mdの書き方
Avatar for Yuto tsukamoto1783
0
190
AIによる開発の民主化を支える コンテキスト管理のこれまでとこれから
Avatar for Mulyu mulyu
3
240
ぼくの開発環境2026
Avatar for yuzneri yuzneri
0
210
Fragmented Architectures
Avatar for Denys Poltorak denyspoltorak
0
150
組織で育むオブザーバビリティ
Avatar for ryotaro kobayashi ryota_hnk
0
170
MUSUBIXとは
Avatar for Hisaho nahisaho
0
130
疑似コードによるプロンプト記述、どのくらい正確に実行される？
Avatar for kokuyouwind kokuyouwind
0
380
【卒業研究】会話ログ分析によるユーザーごとの関心に応じた話題提案手法
Avatar for MomokoShirakawa momok47
0
200
登壇資料を作る時に意識していること #登壇資料_findy
Avatar for konifar konifar
4
1.1k

Featured

See All Featured
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
410
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
240
WENDY [Excerpt]
Avatar for Tessa Abrams tessaabrams
9
36k
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
Avatar for Szymon szymonslowik
1
910
We Analyzed 250 Million AI Search Results: Here's What I Found
Avatar for Josh Blyskal joshbly
1
700
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
359
30k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
698
190k
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
630
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
2.1k
StorybookのUI Testing Handbookを読んだ
Avatar for Shingo Yamazaki zakiyama
31
6.6k
Intergalactic Javascript Robots from Outer Space
Avatar for Vicent Martí tanoku
273
27k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
141
34k

Transcript

  1. Web scraping Irio Musskopf Data Science Retreat for data scientists

  2. Finding data Not always easy

  3. 1. Downloadable dataset

  4. 2.APIs

  5. 3. Scraping

  6. 4.Talk with other companies

  7. 4.Produce yourself

  8. Doesn’t matter how complex the system is. It is possible.

  9. Doesn’t matter how complex the system is. It is possible.

    Unless there’s a captcha.
  10. None

  11. DEMO

  12. Selectors Limitations User agents Proxies

  13. Irio Musskopf [email protected] Thanks