1,000 products from 50 categories → one structured Excel file
A polite Python scraper walks every category and page of a product catalogue, cleans the fields and delivers a ready-to-filter spreadsheet with a summary and a request log. Re-run it any time to refresh the data.
Sample of the output
First rows of the “Books” sheet — every value typed correctly (numbers are numbers, ratings 1–5).
| Title | Category | Price | Rating | Stock |
|---|---|---|---|---|
| It's Only the Himalayas | Travel | £45.17 | ★★ | In stock |
| Full Moon over Noah’s Ark: An Odyssey to Mount Ararat and Beyond | Travel | £49.43 | ★★★★ | In stock |
| See America: A Celebration of Our National Parks & Treasured Sites | Travel | £48.87 | ★★★ | In stock |
| Vagabonding: An Uncommon Guide to the Art of Long-Term World Travel | Travel | £36.94 | ★★ | In stock |
| Under the Tuscan Sun | Travel | £37.33 | ★★★ | In stock |
| A Summer In Europe | Travel | £44.34 | ★★ | In stock |
Biggest categories
Products and average price — from the “By category” sheet.
Ratings
Distribution across all products.
How the scraper behaves
- Respects the site: checks robots.txt first and waits 0.7 s between requests, so the server never feels it.
- Survives hiccups: retries with growing pauses on timeouts and 429/5xx errors.
- Caches every page: a second run does not touch the site and works offline.
- Logs everything: each URL with status and timing goes to the “Request log” sheet.
- Clean types: prices as numbers, star ratings as 1–5, absolute links.
- Flags odd data instead of hiding it: the source files 219 books under placeholder categories (“Default”, “Add a comment”) — they are kept and marked for review.
Before we start on your site
- We agree the fields, the scope and the update frequency.
- I check the site’s terms and robots.txt; logins, paywalls and personal data are out of scope.
- You get a small sample first, then the full run.
What you receive
An Excel file with filters and frozen headers, plus the script to refresh it.
1,000 rows: title, category, price, rating, availability, link and a review note where the source looks odd.
Count, average/min/max price and rating per category, with a native Excel chart.
Every request with status and timing — proof of a clean, polite run.
Source: books.toscrape.com — a public sandbox built specifically for scraping practice. Real client sites are scraped only within their terms.