Scrapy-Powered JSON Scraping API

Замовник: AI | Опубліковано: 08.01.2026

I’m building a small, always-on micro-service that lets me send any of our target URLs, fetch the page once, and receive everything back as structured JSON. Every target page follows the same HTML pattern, so the heavy lifting is limited to getting the data out cleanly and exposing it through an HTTP endpoint. Core flow • A caller posts a URL to the endpoint. • Your Scrapy spider fetches it, detects an expiry date on the page, and formats that date exactly in the custom string style I’ll provide. • If no date is found, the JSON should simply flag `"status": "expired"`. • The full page content comes back as JSON so other systems can parse it later. I’d also like the page title included and am happy if you drop in any other lightweight metadata that might be handy down the road. Technical notes – Scrapy is mandatory; no need for Selenium or BeautifulSoup. – The service must run in a persistent, serverless-friendly environment like Vercel (I’m open to FastAPI, Flask, or similar wrappers for the HTTP layer—your call). – Please keep the code clean, with clear config for things such as request headers or rate throttling. Expected deliverables 1. Scrapy project with the spider and API wrapper. 2. JSON response schema (brief README is fine). 3. Deployment guide or script for Vercel so I can push with one command. 4. A quick demo against the sample URLs I’ll DM. If this sounds straightforward, let’s get it running!