Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web scraping for data scientists

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Irio Musskopf Irio Musskopf
May 24, 2016
Programming
90
0

Web scraping for data scientists

Avatar for Irio Musskopf

Irio Musskopf

May 24, 2016

More Decks by Irio Musskopf

See All by Irio Musskopf
Using Machine Learning and Open Data to Report 216 Brazilian Congresspeople for Corruption
Avatar for Irio Musskopf irio
0
390
Por que functional programming é mais rápido?
Avatar for Irio Musskopf irio
0
80
No país das maravilhas
Avatar for Irio Musskopf irio
0
60
Desenvolvendo o mínimo com Ruby on Rails
Avatar for Irio Musskopf irio
0
140
Implementando pagamentos usando Moip
Avatar for Irio Musskopf irio
0
95
vim 101
Avatar for Irio Musskopf irio
1
230

Other Decks in Programming

See All in Programming
吝嗇家のためのAI活用 / AI development for miser - ChatGPT + Issue Driven Development
Avatar for philomagi tooppoo
0
180
Vite+ Unified Toolchain for the Web
Avatar for Naoki Haba naokihaba
0
690
ローカルLLMでどこまでコードが書けるか -縮小版 /　How much code can be written on a local LLM Shortened
Avatar for Naoki Kishida kishida
2
180
これからAgentCoreを触る方へ トレンドはGatewayです
Avatar for Har1101 har1101
6
480
Strategic Design in the Frontend: Moduliths & Micro Frontends @DDDEurope
Avatar for Manfred Steyer manfredsteyer
PRO
0
140
Snowflake Summitでの新機能 CoCo / CoWork / snowflake-summit-2026-overall-what-new-coco
Avatar for Tatsuhiro tatsuhiro
1
220
決定論的オーケストレーションの設計と実装 / Design and Implementation of Deterministic Orchestration
Avatar for nrs nrslib
4
1.6k
壊れたパーサから始める関数型設計と構成的なパーサ #fp_matsuri
Avatar for ライガー raiga0310
2
170
Even G2とAWSで推しのエージェントを召喚しよう！
Avatar for Har1101 har1101
1
140
エージェンティックRAGにAWSで入門しよう！
Avatar for Har1101 har1101
9
1.9k
コーディングルールの鮮度を保ちたい for SRE NEXT 2026 / keep-fresh-go-internal-conventions-sre-next-2026
Avatar for NAGATA Hiroaki handlename
0
120
Skillsは効率化、Agentsは"自分の拡張"——Builder時代のエージェント編成（CC Night 2026）
Avatar for Hiroki Uemura wemra
1
200

Featured

See All Featured
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
Avatar for r-kagaya rkaga
62
44k
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.6k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
220k
End of SEO as We Know It (SMX Advanced Version)
Avatar for Michael King ipullrank
3
4.3k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.9k
Done Done
Avatar for chrislema chrislema
186
16k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
9.9k
Tips & Tricks on How to Get Your First Job In Tech
Avatar for Honza Javorek honzajavorek
1
550
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
The Curse of the Amulet
Avatar for Matthew Lei leimatthew05
2
13k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
340
From π to Pie charts
Avatar for Rasagy Sharma rasagy
0
230

Transcript

  1. Web scraping Irio Musskopf Data Science Retreat for data scientists

  2. Finding data Not always easy

  3. 1. Downloadable dataset

  4. 2.APIs

  5. 3. Scraping

  6. 4.Talk with other companies

  7. 4.Produce yourself

  8. Doesn’t matter how complex the system is. It is possible.

  9. Doesn’t matter how complex the system is. It is possible.

    Unless there’s a captcha.
  10. None

  11. DEMO

  12. Selectors Limitations User agents Proxies

  13. Irio Musskopf [email protected] Thanks