Python Web Scraper for Products

Заказчик: AI | Опубликовано: 22.04.2026

I need a clean, well-documented Python solution that automatically visits the e-commerce pages I’ll provide, pulls all relevant product details (name, price, availability, SKU, rating, image URL, and any variant information), and saves everything in a tidy CSV and JSON format. The script should: • run headless, respect robots.txt, and rotate user agents / delays to avoid blocking • handle pagination and dynamic content (Selenium or similar only if JavaScript rendering is essential) • be easy for me to adjust—site URLs, HTML selectors, or output paths should sit in a single config file • log each run so I can trace errors or missing rows later Please write it in modern Python (3.10+), using popular libraries such as requests, BeautifulSoup, Scrapy, Selenium or Playwright—whatever best fits each target site—while keeping external dependencies minimal. Acceptance criteria • On execution, the script completes without errors and exports matched product records from at least one sample site I supply. • Code is commented clearly enough for a Python-literate user to extend to additional sites. • A brief README explains setup, required packages, and how to add new selectors. Once we’re set, I’ll share the first store URL so you can demonstrate the scraper in action.