https://speakerdeck.com/lilith 2026-09-09 06:09:34 -0400 Maintaining 100+ Ruby scrapers for financial websites — the hardest part isn't writing the spider, it's knowing which one broke today. This talk covers real-world lessons from running 101 spiders in production: Concern-based modular architecture, four classic failure patterns (anti-scraping, virtual scrolling, dynamic PDF generation, SPA migration), monitoring with Bugsnag, and using AI to decide whether to fix or delete a broken spider. Maintaining 100+ Ruby scrapers for financial websites — the hardest part isn't writing the spider, it's knowing which one broke today. This talk covers real-world lessons from running 101 spiders in production: Concern-based modular architecture, four classic failure patterns (anti-scraping, virtual scrolling, dynamic PDF generation, SPA migration), monitoring with Bugsnag, and using AI to decide whether to fix or delete a broken spider. Wed, 09 Sep 2026 00:00:00 -0400 https://speakerdeck.com/lilith/your-spider-broke-again-maintaining-100-plus-ruby-scrapers-in-production https://speakerdeck.com/lilith/your-spider-broke-again-maintaining-100-plus-ruby-scrapers-in-production