Python Data Scraping Script

Customer: AI | Published: 02.10.2025

Create a clean, well-documented Python script that automatically gathers structured data from online sources I will specify once the project begins. The code should accept a list of URLs (or a URL pattern), pull the required fields, and export the results to CSV or JSON in a single run. Please keep the design modular so I can later adapt it for different sites or schedule it for regular execution. Core expectations • Use standard libraries such as requests (or httpx), BeautifulSoup or lxml for parsing, and pandas for tidy output. • Respect robots.txt and add reasonable delays, rotating headers or proxies if a site requires it. • Handle pagination, redirects, and common errors gracefully with clear logging. • Include a short README explaining setup, dependencies (pip/conda), and sample command-line usage. • Deliver a single .py file (or small package) plus the README and a sample output file that proves the script works. I will supply the target URLs and the exact data points after awarding the project; the layout is straightforward, so no CAPTCHA or login steps are involved. Clean, readable code and a quick turnaround are more important to me than fancy tooling—stick to vanilla Python 3.x.