Python Web Scraping Script Enhancement

Замовник: AI | Опубліковано: 28.12.2025

I already have a Python + Django scraper that sends keyword-based queries to multiple search engines through rotating proxies in order to collect email addresses, targeted phone numbers, and other lead data. The core logic works, but the script stops too early and leaves many usable results untapped. Here is what I would like to have done: • Extend each scraping session so the crawler continues until all safely reachable relevant pages are exhausted, not just the first few hundred results. • Increase the internal result-count limits and make these new limits configurable in settings.py. • Add a new Yellow Pages module: for each search, pull results directly from Yellow Pages, parse the HTML/JSON, and return only email addresses and phone numbers. • Allow the existing keyword engine to switch between Exact Match and Phrase Match modes — no broad match required. • Ensure that proxy rotation, back-off logic, and Django admin controls continue to function correctly after the changes. Python Script Improvement for Advanced Data Scraping