Firecrawl Scraping Fix & Enhancements

I am building a business/ a site that tracks upcoming events and activities within a specific geography and also a relevant business directory for those geographies. My first priority is to get Firecrawl collecting complete, reliable data again. Right now the crawler skips so many sites from my supplied list and, even where it runs, the fields returned are patchy. I’m looking for someone who can step into the existing stack, trace why those targeted websites are being missed, and restructure the extraction logic so every record comes back clean and consistent. You will receive: • A spreadsheet of all sites that should be scraped What I need you to do: 1. Diagnose and repair the Firecrawl setup so every site in the list is reached and parsed correctly, handling pagination, lazy-loaded content, or anti-bot blocks as needed. 2. Normalise the output so each listing contains the same fields and is ready for downstream use. (Those fields include Event Name, Age Range (including Adults Only), Date, Time, Address, Location (New York City or Nassau County or Suffolk County). 3. Set up a repeatable schedule to scrape at 2 am ET everyday (cron, GitHub Actions, or similar) with basic logging so I can spot failures fast. 4. Make sure old events are archived. Once the crawler is rock-solid, I’d like to move on to revenue-driving upgrades: tiered pricing packages, separate dashboards for customers vs. businesses, and friction-free login / signup. We have some of this already done but we probably need to make improvements. If you’re comfortable switching hats from data wrangler to full-stack developer (PHP / Laravel preferred but open to Node or Django), there’s ongoing work here. Acceptance Criteria for the scraping milestone: • 100 % of the provided site list is hit on every run • No critical fields empty in the exported data • Error rate under 2 % over three consecutive scheduled runs If this sounds like your wheelhouse, tell me how you’d tackle the crawler fixes first, mention any similar projects, and include your estimated timeline. The website is [login to view URL] and you will see that some of this is already built.

ID Projek: 40264629