Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to collect large scale data using Javascript

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Leonardo Rifeli Leonardo Rifeli
June 02, 2022
Programming
0
59

How to collect large scale data using Javascript

Avatar for Leonardo Rifeli

Leonardo Rifeli

June 02, 2022
Tweet

More Decks by Leonardo Rifeli

See All by Leonardo Rifeli
Acate: Processamento distribuído - Como processamos milhões de dados diariamente
Avatar for Leonardo Rifeli leonardorifeli
0
31
Reviewr Data Consolidation Case
Avatar for Leonardo Rifeli leonardorifeli
0
120
Distributed processing: How we process millions of data daily with EMR
Avatar for Leonardo Rifeli leonardorifeli
0
50
Building Crawlers with serverless
Avatar for Leonardo Rifeli leonardorifeli
0
78

Other Decks in Programming

See All in Programming
【卒業研究】会話ログ分析によるユーザーごとの関心に応じた話題提案手法
Avatar for MomokoShirakawa momok47
0
190
CSC307 Lecture 06
Avatar for Javier Gonzalez-Sanchez javiergs
PRO
0
680
そのAIレビュー、レビューしてますか？ / Are you reviewing those AI reviews?
Avatar for r-kagaya rkaga
6
4.5k
AIによる開発の民主化を支える コンテキスト管理のこれまでとこれから
Avatar for Mulyu mulyu
3
190
AIによるイベントストーミング図からのコード生成 / AI-powered code generation from Event Storming diagrams
Avatar for nrs nrslib
2
1.9k
今こそ知るべき耐量子計算機暗号(PQC)入門 / PQC: What You Need to Know Now
Avatar for mackey0225 mackey0225
3
370
QAフローを最適化し、品質水準を満たしながらリリースまでの期間を最短化する #RSGT2026
Avatar for shibayu36 shibayu36
2
4.3k
AI によるインシデント初動調査の自動化を行う AI インシデントコマンダーを作った話
Avatar for azukiazusa azukiazusa1
1
710
IFSによる形状設計／デモシーンの魅力 @ 慶應大学SFC
Avatar for がむ gam0022
1
300
2026年 エンジニアリング自己学習法
Avatar for yumechi(Motoki Hirao) yumechi
0
130
フロントエンド開発の勘所 -複数事業を経験して見えた判断軸の違い-
Avatar for Yoichiro Kubo heimusu
7
2.8k
Fragmented Architectures
Avatar for Denys Poltorak denyspoltorak
0
150

Featured

See All Featured
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
650
Designing Powerful Visuals for Engaging Learning
Avatar for Mike Taylor tmiket
0
230
So, you think you're a good person
Avatar for Per Axbom axbom
PRO
2
1.9k
Darren the Foodie - Storyboard
Avatar for Kevinho.art khoart
PRO
2
2.4k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4.2k
The browser strikes back
Avatar for Jono Alderson jonoalderson
0
360
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
410
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
89
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
160
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
38
3k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
1
120

Transcript

  1. How to collect large scale data using Javascript seo local

    | reviews | pesquisas
  2. None

  3. Agora a experiência é o novo marketing

  4. Somos a Harmo, a plataforma de marketing de experiência mais

    completa do Brasil.

  5. SEO Local A única plataforma 3 x 1 do Brasil

    Faça a gestão da presença digital da sua rede de lojas e seja encontrado no topo do ranking das pesquisas de forma 100% orgânica. 1 2 3

  6. Reviews A única plataforma 3 x 1 do Brasil Colete,

    analise e responda todos os reviews dos seus clientes, conquiste a confiança do consumidor e seja a marca escolhida. 1 2 3

  7. A única plataforma 3 x 1 do Brasil Pesquisas multimétricas

    para medir a experiência do cliente durante toda a jornada. Identifique promotores e ative o programa de indicação de reviews. Pesquisas 1 2 3

  8. Harmo, uma poderosa máquina de geração de ROI. Escute, interaja,

    analise e atue focado nos anseios dos clientes, durante toda a jornada, transformando os seus clientes no principal canal de aquisição de novos clientes.

  9. Grandes marcas atestam a qualidade da nossa plataforma e metodologia

    com foco em resultados

  10. NUMBERS Establishments +30k Reviews +15kk Integrations +54k Emails +6,6kk SMS

    +250k Answer of Review +1kk

  11. ▷ Distributed Process ▷ Scrapping vs Crawlers ▷ Some Concepts

    ▷ Why Javascript? ▷ Architecture for Scale ▷ Lessons Learning ▷ Example ▷ Conclusion Topics

  12. Distributed Process

  13. None

  14. Scraping vs Crawlers

  15. None

  16. Collector Concepts

  17. Be "Browserless"

  18. Recursion is your friends

  19. Single Responsability

  20. Normalize Data (input & output)

  21. Code reuse with packages

  22. Collector !== Processor

  23. Why Javascript?

  24. Use native streams

  25. Dynamic typing

  26. Do more with less

  27. Most used in the world

  28. Architecture for Scale

  29. None
  30. None
  31. None

  32. Lessons Learning

  33. Use code-base version alert

  34. Code reuse with packages

  35. Create E2E tests from the begin

  36. Be "Browserless"

  37. Use Puppeteer *reduce images

  38. None
  39. None

  40. Use Promise.all

  41. None

  42. Use monorepos

  43. Otherwise it will be chaos

  44. None
  45. None

  46. ▷ Web Scraping vs Web Crawling: The Differences ▷ HOW

    TO RUN ASYNC JAVASCRIPT FUNCTIONS IN SEQUENCE OR PARALLEL Links

  47. Collector Example

  48. None
  49. None

  50. Leonardo Rifeli | CTO [email protected] harmo.me seo local | reviews

    | pesquisas