Building Crawlers with serverless

How do we building crawlers/scrapings with serverless architecture.

Event: https://www.meetup.com/pt-BR/Serverless-Floripa/events/253774158/

Transcript

1. Crawlers with Serverless.
3. Topics ▷ About Serverless ▷ About FaaS ▷ Reviewr ▷ Our Vision ▷ Lessons Learned ▷ Conclusion
4. About Serverless
6. Examples ▷ Auth0 ▷ Azure Functions ▷ AWS API Gateway ▷ AWS Lambda ▷ ... *github.com/serverless/examples
7. About FaaS
8. Servers are fully-abstracted
9. Uncoupled responsibilities
10. Different languages
11. Auto scaling
12. Event driven
13. Pay only for what you use
16. Reviewr
19. inserir no contexto de reviews Google, Facebook, iFood, Trip e Booking
35. PLANILHA
36. PLANILHA
37. PLANILHA
52. Our Vision
53. Our Vision - Concepts Collector Processor Dataviz
54. Collector Concepts
55. Be "Browserless"
56. Single Responsability
57. Execution Control
58. Recursion Is Your Friend
59. Normalize It
60. Collect != Process
61. Processor Concepts
62. Orchestration flow
63. Data Validation
64. Parallel Process
65. Language By Your Need
66. Secondary Services
67. Process != Data Mining
68. Data Storage
69. Lessons Learned
70. Costs Predictability
71. Costs Predictability - Lambda Example *serverlesscalc.com
72. Mindset Disruption
73. Mindset Disruption
74. Versioning & CI
75. Versioning & CI
76. Versioning & CI
77. Versioning & CI
78. Monitoring & Alerts
79. Monitoring & Alerts
80. Monitoring & Alerts
81. Monitoring & Alerts
82. Secondary Services
83. Secondary Services
84. Secondary Services - Current
85. AWS Step Functions
86. AWS Step Functions
87. AWS Step Functions
88. Cold Start
89. Cold Start
90. Do More With Less
91. Do More With Less
92. Do More With Less - Team CTO & Partner Marcelo Andrade Head of Engineering & Partner Leonardo Rifeli Fullstack developer Raphael Ornellas
93. Conclusion
94. THANKS! engineering.reviewr.tech marcelo.andrade@reviewr.me leonardo.rifeli@reviewr.me