Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web scraping for data scientists

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Irio Musskopf Irio Musskopf
May 24, 2016
Programming
0
81

Web scraping for data scientists

Avatar for Irio Musskopf

Irio Musskopf

May 24, 2016
Tweet

More Decks by Irio Musskopf

See All by Irio Musskopf
Using Machine Learning and Open Data to Report 216 Brazilian Congresspeople for Corruption
Avatar for Irio Musskopf irio
0
360
Por que functional programming é mais rápido?
Avatar for Irio Musskopf irio
0
69
No país das maravilhas
Avatar for Irio Musskopf irio
0
51
Desenvolvendo o mínimo com Ruby on Rails
Avatar for Irio Musskopf irio
0
140
Implementando pagamentos usando Moip
Avatar for Irio Musskopf irio
0
92
vim 101
Avatar for Irio Musskopf irio
1
220

Other Decks in Programming

See All in Programming
コマンドとリード間の連携に対する脅威分析フレームワーク
Avatar for Aja9tochin pandayumi
1
450
登壇資料を作る時に意識していること #登壇資料_findy
Avatar for konifar konifar
4
1.1k
15年続くIoTサービスの SREエンジニアが挑む 分散トレーシング導入
Avatar for Melonps melonps
2
200
Smart Handoff/Pickup ガイド - Claude Code セッション管理
Avatar for rasshii yukiigarashi
0
130
AWS re:Invent 2025参加 直前 Seattle-Tacoma Airport(SEA)におけるハードウェア紛失インシデントLT
Avatar for tetutetu214 tetutetu214
2
110
AI Agent Tool のためのバックエンドアーキテクチャを考える #encraft
Avatar for izumin5210 izumin5210
6
1.8k
フロントエンド開発の勘所 -複数事業を経験して見えた判断軸の違い-
Avatar for Yoichiro Kubo heimusu
7
2.8k
HTTPプロトコル正しく理解していますか？ 〜かわいい猫と共に学ぼう。ฅ^•ω•^ฅ ﾆｬ〜
Avatar for ♛Heitor Hirose hekuchan
2
690
AIと一緒にレガシーに向き合ってみた
Avatar for nyafunta9858 nyafunta9858
0
230
[KNOTS 2026登壇資料]AIで拡張‧交差する プロダクト開発のプロセス および携わるメンバーの役割
Avatar for hisatake hisatake
0
280
それ、本当に安全？ ファイルアップロードで見落としがちなセキュリティリスクと対策
Avatar for ikechi penpeen
7
3.9k
Lambda のコードストレージ容量に気をつけましょう
Avatar for tatsuki-yamada tattwan718
0
120

Featured

See All Featured
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
430
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
333
24k
Performance Is Good for Brains [We Love Speed 2024]
Avatar for Tammy Everts tammyeverts
12
1.4k
Sam Torres - BigQuery for SEOs
Avatar for Tech SEO Connect techseoconnect
PRO
0
180
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
Avatar for Aleyda Solis aleyda
1
1.9k
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
8.6k
Visualization
Avatar for Eitan Lees eitanlees
150
17k
Paper Plane (Part 1)
Avatar for Katie Cordina Lindsey katiecoart
PRO
0
4.1k
Collaborative Software Design: How to facilitate domain modelling decisions
Avatar for Kenny Baas-Schwegler baasie
0
140
Money Talks: Using Revenue to Get Sh*t Done
Avatar for Nikki Halliwell nikkihalliwell
0
150
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
200
Building an army of robots
Avatar for Kyle Neath kneath
306
46k

Transcript

  1. Web scraping Irio Musskopf Data Science Retreat for data scientists

  2. Finding data Not always easy

  3. 1. Downloadable dataset

  4. 2.APIs

  5. 3. Scraping

  6. 4.Talk with other companies

  7. 4.Produce yourself

  8. Doesn’t matter how complex the system is. It is possible.

  9. Doesn’t matter how complex the system is. It is possible.

    Unless there’s a captcha.
  10. None

  11. DEMO

  12. Selectors Limitations User agents Proxies

  13. Irio Musskopf [email protected] Thanks