Google's index as a scraping filter

While building BuildEstate I needed listings from three property portals that all block scrapers on their search pages. The trick: use Firecrawl's search with a site: query like site:99acres.com "2BHK Powai ready". Google's index does the filtering, returns individual listing URLs, and those pages have far weaker rate limiting than search results. Scrape them in parallel and deduplicate.