[Blog](https://scrapfly.io/blog) / [data-parsing](https://scrapfly.io/blog/tag/data-parsing) / [How to Scrape idealo: Product Offers, Prices, and Price History](https://scrapfly.io/blog/posts/how-to-scrape-idealo) # How to Scrape idealo: Product Offers, Prices, and Price History by [Mayada Shaaban](https://scrapfly.io/blog/author/mayada-shaaban-90143e67) Oct 16, 2026 24 min read [\#data-parsing](https://scrapfly.io/blog/tag/data-parsing) [\#hidden-api](https://scrapfly.io/blog/tag/hidden-api) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo&text=How%20to%20Scrape%20idealo%3A%20Product%20Offers%2C%20Prices%2C%20and%20Price%20History "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-idealo) **Web Scraping API**Scrape any website with anti-bot bypass, proxy rotation, and JS rendering. [ Learn More ](https://scrapfly.io/products/web-scraping-api) [ Docs ](https://scrapfly.io/docs/scrape-api/getting-started) On a single idealo product page, one item can carry hundreds of live merchant offers and a price chart. The page also runs Akamai and gives you no clean key to join those offers to your catalog. This guide walks the full extraction with the code from Scrapfly's maintained idealo scraper. You pull product ids from search and brand listings, load every merchant offer on a product page, and call the price chart's JSON endpoint. [How to Track Competitor Prices Using Web ScrapingIn this web scraping guide, we'll explain how to create a tool for tracking competitor prices using Python. It will scrape specific products from different providers, compare their prices and generate insights.](https://scrapfly.io/blog/posts/how-to-track-competitor-pricing-using-web-scraping) ## Key Takeaways - **One product, many offers:** one product page holds the whole merchant price spread. - **Offers live in DOM rows:** each `productOffers-listItem` holds shop, price, and delivery. - **Load every offer first:** a JS loop clicks "load more" until the button disappears. - **Metadata is in JSON-LD:** name, brand, rating, and the low/high price aggregate. - **Price history is an endpoint:** `/price-chart/.../history` returns a daily JSON series. - **No reliable EAN key:** barcode search hits only some products, so fall back to title and cache the id. - **idealo runs Akamai:** you need anti-bot handling plus a German vantage point. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## Why scrape idealo? idealo is the dominant price-comparison engine in the German-speaking market. Scraping one retailer gives you one price. Scraping one idealo product page gives you every merchant offer for that product, each with its own price, shop rating, and delivery window. idealo's offer data feeds four jobs: - **Cross-merchant price monitoring:** track the lowest and highest offer for a product from one source. - **[Minimum advertised price monitoring](https://scrapfly.io/blog/posts/how-to-build-minimum-advertised-price-monitoring-tool):** watch for merchants selling below MAP across the DACH region. - **Price-history and trend analysis:** read seasonality and discount patterns from the chart series. - **[Market and assortment research](https://scrapfly.io/blog/posts/observing-ecommerce-market-trends-with-web-scraping):** see which merchants stock a product and at what coverage. This guide targets idealo.de, idealo's flagship site. idealo also runs five other live country sites in Austria, France, Italy, Spain, and the UK (idealo.at, .fr, .it, .es, .co.uk). idealo.at uses the same `/preisvergleich/OffersOfProduct/` URLs as idealo.de. The other four use their own paths, such as `/prix/` on idealo.fr and `/compare/` on idealo.co.uk, so the URL builders need changes there. Every one of those jobs starts with getting past idealo.de's Akamai edge, which answers a plain HTTP client with a 403. ## How do you set up an idealo scraping project? You need a Python project with the Scrapfly SDK, set up to request idealo from a German IP. idealo's Akamai protection means a plain `requests` or `httpx` call is unreliable, so the SDK handles fetching and unblocking from the first request. You need two packages. The Scrapfly SDK's `all` extra fetches pages and bundles [parsel](https://scrapfly.io/blog/posts/guide-to-html-parsing-with-parsel-python) for parsing, and `loguru` handles logging. bash ```bash pip install "scrapfly-sdk[all]" loguru export SCRAPFLY_KEY="YOUR_SCRAPFLY_KEY" ``` The scraper starts with one async client, a shared config, and a retry wrapper. Scrapfly ships SDKs for Python, TypeScript, Go, and Rust, so the same approach works outside Python. python ```python import os import re import json import asyncio import uuid from urllib.parse import urljoin, urlencode from typing import List, Literal, Optional, TypedDict from loguru import logger as log from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse SCRAPFLY = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"]) BASE_CONFIG = { "asp": True, "country": "DE", "render_js": True, "proxy_pool": "public_residential_pool", } BASE_URL = "https://www.idealo.de" MAX_RETRIES = 3 RETRY_DELAY = 5 async def scrape_with_retry(config: ScrapeConfig, _retries: int = 0) -> ScrapeApiResponse: """scrape a config and retry on any failure - idealo blocks requests intermittently""" try: return await SCRAPFLY.async_scrape(config) except Exception as e: if _retries < MAX_RETRIES: log.debug(f"retrying failed request to {config.url}: {e}") await asyncio.sleep(RETRY_DELAY) return await scrape_with_retry(config, _retries=_retries + 1) raise Exception(f"unable to scrape {config.url}, max retries exceeded") from e ``` The `asp` flag turns on Unblocker (previously ASP), the anti-bot layer of [Scrapfly](https://scrapfly.io/products/web-scraping-api), and `country="DE"` makes listings resolve to the German market. The retry wrapper covers the intermittent blocks idealo still throws, with a five-second pause between attempts. The parsers below return `TypedDict` result types such as `IdealoOffer` and `IdealoProduct`. They are plain field lists, so copy them from the full scraper in the repository or swap each one for `dict`. idealo prints prices in German format, such as `1.082,00 €`. Two small helpers turn that text into numbers for every parser that follows. python ```python def _parse_price(text: Optional[str]) -> Optional[float]: if not text: return None cleaned = text.replace("\xa0", " ").replace("€", "").replace(" ", "") match = re.search(r"[\d.]+,\d+|\d+", cleaned) if not match: return None try: return float(match.group(0).replace(".", "").replace(",", ".")) except ValueError: return None def _parse_int(text: Optional[str]) -> Optional[int]: if not text: return None match = re.search(r"[\d.]+", text.replace("\xa0", " ").replace(" ", "")) if not match: return None try: return int(match.group(0).replace(".", "")) except ValueError: return None ``` The dot is a thousands separator and the comma is the decimal mark, so `_parse_price` swaps them before converting. With a working fetch and number parsing in place, the first real target is the listing pages that hand you product ids. ## How do you scrape idealo search and manufacturer results? Search and manufacturer pages list products with their idealo ids and URLs. Every later step needs those ids, so listings come first. The two listing entry points on idealo.de share one result layout: - **Search:** `MainSearchProductCategory.html?q=` for keyword results. - **Manufacturer:** `Hersteller/.html` for a brand's catalog (Nike is `1274`). Every result is a card matching `[class*="sr-resultList__item"]`. The parser below reads one card into a dict with the product id, name, URL, image, lowest price, and offer count. python ```python def _parse_listing_item(card) -> IdealoListingItem: href = card.css('a[href*="OffersOfProduct"]::attr(href)').get() name = card.css('[class*="sr-productSummary__title"]::text').get() shop_name = card.css('[class*="sr-singleOffer__shopName"] span::text').get() wishlist_raw = card.css("[data-wishlist-heart]::attr(data-wishlist-heart)").get() wishlist = json.loads(wishlist_raw) if wishlist_raw else {} product_id = wishlist.get("id") if not product_id and href: match = re.search(r"/OffersOfProduct/(\d+)", href) product_id = match.group(1) if match else None return IdealoListingItem( product_id=product_id, name=name.strip() if name else None, url=urljoin(BASE_URL, href) if href else None, image=card.css('[class*="sr-resultItemTile__image"]::attr(src)').get(), price=_parse_price("".join(card.css('[class*="sr-detailedPriceInfo__price"] ::text').getall())), currency="EUR", offer_count=_parse_int(card.css('[class*="sr-detailedPriceInfo__offerCount"]::text').get()), shop_name=shop_name.strip() if shop_name else None, ) ``` The product id comes from the card's `data-wishlist-heart` JSON first, with the `/OffersOfProduct/` href as a fallback. Not every card is a product. Some are a single shop's offer with no product page, so `url` is `None`, `shop_name` holds the shop, and the id is a hash. ### How do you paginate idealo search results? Search pagination reads the highest page number from the pagination bar, then builds offset URLs for the remaining pages and fetches them concurrently. python ```python SEARCH_PAGE_SIZE = 15 def _parse_total_pages(sel) -> int: pages = [] pagination = sel.css('[class*="sr-pagination"]') for value in pagination.css("[aria-label]::attr(aria-label)").getall(): if value.strip().isdigit(): pages.append(int(value.strip())) for value in pagination.css('[class*="sr-pageElement"]::text, [class*="sr-textElement"]::text').getall(): if value.strip().isdigit(): pages.append(int(value.strip())) return max(pages) if pages else 1 def parse_search(response: ScrapeApiResponse) -> dict: """parse search listings from idealo.de""" sel = response.selector results = [_parse_listing_item(card) for card in sel.css('[class*="sr-resultList__item"]')] return {"results": results, "total_pages": _parse_total_pages(sel)} async def scrape_search(query: str, max_pages: int = 3) -> List[IdealoListingItem]: """scrape search listings from idealo.de""" params = urlencode({"q": query}) first_url = f"{BASE_URL}/preisvergleich/MainSearchProductCategory.html?{params}" log.info(f"scraping the first search page {first_url}") first_page = await scrape_with_retry( ScrapeConfig(first_url, wait_for_selector='[class*="sr-resultList"]', **BASE_CONFIG) ) data = parse_search(first_page) results = data["results"] total_pages = min(data["total_pages"], max_pages) log.info(f"found {data['total_pages']} search pages, scraping {total_pages}") if total_pages > 1: other_pages = [] for page in range(2, total_pages + 1): offset = (page - 1) * SEARCH_PAGE_SIZE url = f"{BASE_URL}/preisvergleich/MainSearchProductCategory/100I16-{offset}.html?{params}" other_pages.append(ScrapeConfig(url, wait_for_selector='[class*="sr-resultList"]', **BASE_CONFIG)) responses = await asyncio.gather( *[scrape_with_retry(config) for config in other_pages], return_exceptions=True ) for response in responses: if isinstance(response, Exception): log.error(f"failed to scrape search page: {response}") continue results.extend(parse_search(response)["results"]) log.success(f"scraped {len(results)} search results") return results ``` `wait_for_selector` holds the browser until the result list renders, and `max_pages` caps how many pages you pay for. A failed page logs an error instead of aborting the whole run. A search for `sonnenfinsternis brillen` (solar eclipse glasses) across three pages returns product rows like these: json ```json [ { "product_id": "210983733", "name": "Buki Solar-Finsternis-Kit 4 Brillen & 2 Smartphone-Linsen, Filter EN ISO 12312-2 (ECL01)", "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/210983733_-solar-finsternis-kit-4-brillen-2-smartphone-linsen-filter-en-iso-12312-2-ecl01-buki.html", "image": "https://cdn.idealo.com/folder/Product/210983/7/210983733/s1_produktbild_mittelgross/buki-solar-finsternis-kit-4-brillen-2-smartphone-linsen-filter-en-iso-12312-2-ecl01.jpg", "price": 12.88, "currency": "EUR", "offer_count": 4, "shop_name": null }, { "product_id": "202160262", "name": "Baader Planetarium Solar Viewer AstroSolar", "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/202160262_-solar-viewer-astrosolar-baader-planetarium.html", "image": "https://cdn.idealo.com/folder/Product/202160/2/202160262/s1_produktbild_mittelgross/baader-planetarium-solar-viewer-astrosolar.jpg", "price": 5.0, "currency": "EUR", "offer_count": 3, "shop_name": null } ] ``` The search returned 155 results, and only 6 were product rows. The other 149 were single-shop offers with no product page, so filter on `url` before you queue product scrapes. ### How do you scrape an idealo manufacturer page? A manufacturer page uses the same result cards, plus a brand name, a meta description, and a total result count in the header. python ```python async def scrape_manufacturer(url: str) -> IdealoManufacturer: """scrape a manufacturer listing page from idealo.de""" log.info(f"scraping manufacturer page {url}") response = await scrape_with_retry( ScrapeConfig(url, wait_for_selector='[class*="sr-resultList"]', **BASE_CONFIG) ) manufacturer = parse_manufacturer(response) log.success(f"scraped manufacturer {manufacturer['name']} with {len(manufacturer['products'])} products") return manufacturer def parse_manufacturer(response: ScrapeApiResponse) -> IdealoManufacturer: """parse manufacturer listing data from idealo.de""" sel = response.selector name = sel.css("h1::text").get() return IdealoManufacturer( name=name.strip() if name else None, description=sel.css('meta[name="description"]::attr(content)').get(), url=response.context["url"], result_count=_parse_int(sel.css('[class*="sr-resultTitle__resultCount"]::text').get()), products=[_parse_listing_item(card) for card in sel.css('[class*="sr-resultList__item"]')], ) ``` Running it against the Nike page at `/preisvergleich/Hersteller/1274.html` returns the brand header plus the first page of products: json ```json { "name": "Nike", "description": "Vergleiche und kaufe günstig Artikel von Nike online bei idealo.de. Günstig ✓ neueste Trends ✓ große Auswahl ✓", "url": "https://www.idealo.de/preisvergleich/Hersteller/1274.html", "result_count": 118141, "products": [ { "product_id": "1370489", "name": "Nike Air Force 1 '07", "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/1370489_-air-force-1-07-nike.html", "image": "https://cdn.idealo.com/folder/Product/1370/4/1370489/s1_produktbild_mittelgross/nike-air-force-1-07.jpg", "price": 69.95, "currency": "EUR", "offer_count": 2178, "shop_name": null } ] } ``` The scraper reads one manufacturer page, which held 36 of Nike's 118,141 results. `result_count` tells you how deep the catalog goes before you decide whether to page further. Each of those product ids leads to a product page with the full offer list. ## How do you scrape an idealo product page and its merchant offers? The product page at `OffersOfProduct/{id}.html` carries the product metadata and the list of live merchant offers. One pass over the page extracts both. The first page load shows only the opening block of offers, and the rest sit behind a "load more" button. The scraper runs this JavaScript in the rendered page, clicking the button every 500 milliseconds until it disappears: python ```python LOAD_MORE_JS = """ const clickUntilGone = async () => { while (true) { const btn = document.querySelector('button.productOffers-listLoadMore'); if (!btn) break; btn.click(); await new Promise(r => setTimeout(r, 500)); } }; await clickUntilGone(); """ ``` `LOAD_MORE_JS` needs a real browser to click in, which is why `BASE_CONFIG` sets `render_js=True`. The script runs before Scrapfly returns the HTML. Each merchant offer is an `li.productOffers-listItem` DOM row. The row's `data-mtrx-click` attribute holds the shop name and price as JSON, and CSS classes carry the rest. Here is what each offer field is for: | Field | What it maps to | |---|---| | `shop_name` | The shop behind the offer, such as `eBay` or `smartphoneonly.de` | | `merchant_name` | The marketplace seller, when the shop is a marketplace like eBay | | `price` | The offer price in EUR, from `data-mtrx-click` or the price text | | `delivery_info` | The delivery window idealo shows, such as `bis Do. 20.08.` | | `shop_rating` / `shop_rating_count` | The shop's idealo rating and how many ratings it has | | `shop_url` | The shop's rating page on idealo | | `url` | idealo's redirect link to the offer on the shop's site | The offer parser walks every row and skips any row without a shop name or offer link. python ```python def _parse_offers(sel) -> List[IdealoOffer]: offers = [] for li in sel.css("li.productOffers-listItem"): mtrx_raw = li.attrib.get("data-mtrx-click") mtrx = json.loads(mtrx_raw) if mtrx_raw else {} mtrx_products = mtrx.get("products") or [{}] shop_name_raw = li.css(".productOffers-listItemOfferShopV2LogoLink::attr(data-shop-name)").get() shop_name = mtrx.get("shop_name") or ( shop_name_raw.split(" - Shop aus")[0].strip() if shop_name_raw else None ) url = li.css("a.productOffers-listItemOfferCtaLeadout::attr(href)").get() or li.css( "a.productOffers-listItemTitle::attr(href)" ).get() if not shop_name or not url: continue price = mtrx_products[0].get("price") if price is None: price = _parse_price(li.css(".productOffers-listItemOfferPrice::text").get()) delivery = li.css(".productOffers-listItemOfferDeliveryStatusDatesRange::text").get() merchant = li.css(".productOffers-listItemOfferShopV2MarketPlaceMerchantName a::text").get() rating_count = li.css(".productOffers-listItemOfferShopV2NORatings--numberOfRatings::text").get() offers.append( IdealoOffer( shop_name=shop_name, shop_url=li.css(".productOffers-listItemOfferShopV2StarsLink::attr(href)").get(), shop_rating=li.css(".productOffers-listItemOfferShopV2Stars b::text").get(), shop_rating_count=_parse_int(rating_count), price=price, currency="EUR", delivery_info=delivery.strip() if delivery else None, merchant_name=merchant.strip() if merchant else None, url=urljoin(BASE_URL, url), ) ) return offers ``` Shop name and price come from the `data-mtrx-click` JSON first, with the logo attribute and price text as fallbacks. The delivery and rating fields read class names only, so check those selectors first when they start returning `None`. The product-level fields come from a JSON-LD `Product` block in `script[data-testid="Product"]`. It gives you the name, brand, rating, currency, and an `AggregateOffer` with the offer count and the low and high price across all merchants. python ```python def parse_product(response: ScrapeApiResponse) -> IdealoProduct: """parse product data from idealo.de product pages""" sel = response.selector raw = sel.css('script[data-testid="Product"]::text').get() data = json.loads(raw) if raw else {} match = re.search(r"/OffersOfProduct/(\d+)", response.context["url"]) images = data.get("image") or [] rating = data.get("aggregateRating") or {} offer_summary = data.get("offers") or {} return IdealoProduct( product_id=match.group(1) if match else data.get("sku"), name=data.get("name"), brand=(data.get("brand") or {}).get("name"), url=data.get("url") or response.context["url"], image=images[0] if images else None, description=data.get("description"), rating=rating.get("ratingValue"), rating_count=rating.get("ratingCount"), low_price=offer_summary.get("lowPrice"), high_price=offer_summary.get("highPrice"), currency=offer_summary.get("priceCurrency"), offer_count=offer_summary.get("offerCount"), offers=_parse_offers(sel), ) async def scrape_products(urls: List[str]) -> List[IdealoProduct]: """scrape product pages from idealo.de""" products = [] to_scrape = [ScrapeConfig(url, js=LOAD_MORE_JS, **BASE_CONFIG) for url in urls] responses = await asyncio.gather( *[scrape_with_retry(config) for config in to_scrape], return_exceptions=True ) for response in responses: if isinstance(response, Exception): log.error(f"failed to scrape product: {response}") continue products.append(parse_product(response)) log.success(f"scraped {len(products)} products") return products ``` Brand falls back to `None` because idealo does not attach one to every product. `scrape_products` passes `LOAD_MORE_JS` through the `js` parameter and scrapes every URL concurrently. Running it against the iPhone 17 Pro product page returns the product fields plus one entry per merchant (offers trimmed to two): json ```json { "product_id": "207644441", "name": "Apple iPhone 17 Pro", "brand": "Apple", "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/207644441_-iphone-17-pro-apple.html", "image": "https://cdn.idealo.com/folder/Product/207644/4/207644441/s1_produktbild_gross/apple-iphone-17-pro.jpg", "description": "Bereits ab 1.082,00 € ✓ Große Shopvielfalt ✓ Testberichte & Meinungen ✓ | Jetzt Apple iPhone 17 Pro günstig kaufen bei idealo.de", "rating": 4.550000190734863, "rating_count": 151, "low_price": 1082.0, "high_price": 2455.19, "currency": "EUR", "offer_count": 395, "offers": [ { "shop_name": "eBay", "shop_url": "/relocator/relocate?sid=9701&type=rating&userid=helmer-store", "shop_rating": "4,9", "shop_rating_count": 272, "price": 1082.0, "currency": "EUR", "delivery_info": "bis Do. 20.08.", "merchant_name": "helmer-store", "url": "https://www.idealo.de/relocator/relocate?categoryId=19116&offerKey=93303bba37b25505f532f2c2dc957464&pos=1&price=1082.00&productid=207644441&sid=9701&type=offer" }, { "shop_name": "smartphoneonly.de", "shop_url": "https://www.idealo.de/preisvergleich/Shop/302353.html", "shop_rating": "4,5", "shop_rating_count": 1435, "price": 1095.0, "currency": "EUR", "delivery_info": "bis Sa. 15.08.", "merchant_name": null, "url": "https://www.idealo.de/relocator/relocate?categoryId=19116&offerKey=b2881f343ac616f6f02475707e158966&pos=2&price=1095.00&productid=207644441&sid=302353&type=offer" } ] } ``` With the load-more loop in place, that run parsed all 395 offers the JSON-LD aggregate reports. `shop_rating` stays a German-format string like `4,9`, so convert it with `_parse_price` if you sort on it. JSON-LD is one kind of hidden web data. The guide below covers the other kinds of embedded data that pages ship on first load. [How to Scrape Hidden Web DataThe visible HTML doesn't always represent the whole dataset available on the page. In this article, we'll be taking a look at scraping of hidden web data. What is it and how can we scrape it using Python?](https://scrapfly.io/blog/posts/how-to-scrape-hidden-web-data) ### How do you match an idealo product to your EAN or GTIN? idealo has no reliable EAN or GTIN lookup. Product pages do not print the barcode, and search matches a barcode only some of the time. A search for `4006381333931` returned the STABILO point 88 fineliner it belongs to as the first product, while `5449000000996` returned only unrelated shop offers. That leaves catalog owners without a dependable key to join idealo's offers against their own SKUs. The workable approach is to search by barcode first and fall back to the product title or model. Match the candidates to your catalog, then store idealo's product id against your SKU. After that first match, later runs skip the search and go straight to the product page. A small wrapper around `scrape_search` does the matching. It keeps only product rows, since single-shop offers have no product page, and returns `None` when idealo does not list the item. python ```python async def find_product(query: str): results = await scrape_search(query, max_pages=1) products = [r for r in results if r["url"] and "OffersOfProduct" in r["url"]] return products[0] if products else None # cache product_id against your SKU match = asyncio.run(find_product("Apple iPhone 17 Pro")) print(match) ``` The resolver returns the first product candidate, whose URL you then pass to `scrape_products`: json ```json { "product_id": "207644441", "name": "Apple iPhone 17 Pro", "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/207644441_-iphone-17-pro-apple.html", "image": "https://cdn.idealo.com/folder/Product/207644/4/207644441/s1_produktbild_mittelgross/apple-iphone-17-pro.jpg", "price": 1125.0, "currency": "EUR", "offer_count": 467, "shop_name": null } ``` The offer count and price here come from a live run on 2026-09-29, about seven weeks after the product sample above. They drift that fast, which is why you cache the id and not the price. The same product page also carries a price chart, and its data comes from a separate endpoint. Scrapfly #### Extract structured data automatically? Scrapfly's Extraction API uses AI to turn any webpage into structured data — no selectors needed. [Try Free →](https://scrapfly.io/register)## How do you scrape idealo price history? idealo shows a price chart on each product page, and a JSON endpoint serves the chart's underlying series. One API call returns the full daily history for a period, so you do not have to build it from your own snapshots. The endpoint is `/price-chart/sites/{siteId}/products/{productId}/history?period={period}`. Site id `1` is idealo.de, and `period` accepts `1M`, `3M`, `6M`, or `1Y`. The scraper makes two requests that share one Scrapfly session. It loads the product page first and waits for the chart, then calls the endpoint with a `referer` header pointing at that page. python ```python PeriodType = Literal["1Y", "3M", "6M", "1M"] def _parse_history_price(value) -> Optional[float]: try: return float(value) / 100 except (TypeError, ValueError): return None async def scrape_price_history(id: str, period: PeriodType, site_id=1) -> IdealoPriceHistory: """scrape price history from idealo.de""" url = f"{BASE_URL}/preisvergleich/OffersOfProduct/{id}.html" log.info(f"scraping price history page {url}") session = f"idealo-price-history-{uuid.uuid4()}" page = await scrape_with_retry( ScrapeConfig(url, wait_for_selector='[class*="price-chart"]', session=session, **BASE_CONFIG) ) history_url = f"{BASE_URL}/price-chart/sites/{site_id}/products/{id}/history?period={period}" log.info(f"scraping price history API {history_url}") response = await scrape_with_retry( ScrapeConfig( history_url, session=session, headers={"accept": "application/json", "referer": url}, **{**BASE_CONFIG, "render_js": False}, ) ) price_history = parse_price_history(response, id, period) log.success(f"scraped price history {price_history['product_id']} with {len(price_history['points'])} price points") return price_history ``` The session keeps the same cookies and proxy IP across both calls, so the API request looks like it came from the chart widget. The second call turns `render_js` off, because the endpoint returns plain JSON. The parser converts every price from cents to euros and keeps idealo's own summary statistics alongside the points: python ```python def parse_price_history(response: ScrapeApiResponse, product_id: str, period: PeriodType) -> IdealoPriceHistory: """parse price history data from idealo.de price-chart API""" data = json.loads(response.content) stats = data.get("statistics") or {} points = [ IdealoPriceHistoryPoint(date=point.get("x"), price=_parse_history_price(point.get("y"))) for point in data.get("data") or [] ] return IdealoPriceHistory( product_id=product_id, period=period, currency="EUR", start_date=data.get("startDate"), avg_price=_parse_history_price(stats.get("avgPrice")), lowest_price=_parse_history_price(stats.get("lowestPrice")), lowest_price_days=stats.get("lowestPriceDays"), highest_price=_parse_history_price(stats.get("highestPrice")), highest_price_days=stats.get("highestPriceDays"), points=points, ) ``` A three-month pull for the iPhone 17 (`207643424`) returns the statistics plus a daily series (points trimmed to two): json ```json { "product_id": "207643424", "period": "3M", "currency": "EUR", "start_date": "2026-05-13", "avg_price": 743.24, "lowest_price": 698.0, "lowest_price_days": 90, "highest_price": 789.9, "highest_price_days": 0, "points": [ {"date": "2026-05-13", "price": 748.9}, {"date": "2026-05-14", "price": 729.0} ] } ``` The three-month series has 85 daily points, ending at 789.9 EUR on 2026-08-13. Say you need finer-grained history than the endpoint exposes, or history for fields it does not track. Then capture point-in-time snapshots on a schedule and accumulate your own series. The price tracker guide covers the scheduling and storage build in full. This guide stops at extracting what idealo already exposes. ## How do you bypass idealo blocking with Scrapfly? idealo runs Akamai bot protection to guard its aggregated data, and listings resolve to the visitor's location. Reliable extraction needs anti-bot handling plus a German vantage point. Three things go wrong without both: - **Wrong geography returns wrong data:** requests from outside Germany get partial or different listings, so you must appear German. - **Bot defenses block naive scrapers:** plain `requests` or `httpx` calls get a 403 from idealo's Akamai edge (`server: AkamaiGHost`) before any page loads. - **Offers and listings need a browser:** the load-more clicks and the result list only exist after JavaScript runs. The Scrapfly Web Scraping API covers all three through the four settings in the scraper's `BASE_CONFIG`: python ```python BASE_CONFIG = { "asp": True, # bypass Akamai bot protection "country": "DE", # German exit IP for correct listings "render_js": True, # real browser for load-more and results "proxy_pool": "public_residential_pool", # residential IPs, not datacenter } ``` `asp=True` turns on the [Web Unblocker](https://scrapfly.io/products/unblocker), which detects the anti-bot vendor and manages proxies, fingerprints, and challenges. Python SDK 0.12.0 and later also accept `unblocker=True` for the same setting. The residential pool puts each request on a home connection in Germany, which idealo trusts more than a datacenter range. Scrapfly reports a 98% success rate on the hardest public pages and bypasses 25+ anti-bot systems, Akamai, Cloudflare, and DataDome among them. Requests route through proxies across 190+ countries. A do-it-yourself setup needs German residential proxies, a browser for the load-more clicks, and retries for the intermittent blocks. That holds up for small runs, and the upkeep grows each time idealo changes its defenses. ## Powering idealo Scraping With Scrapfly ScrapFly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) is a single HTTP endpoint for collecting web data at scale, with a **98% success rate** on the hardest pages and **proxies in 190+ countries**. - [Web Unblocker API](https://scrapfly.io/products/unblocker) (previously ASP) - bypasses 25+ anti-bot systems, including Cloudflare, DataDome, PerimeterX, and Akamai. - [Smart proxy rotation](https://scrapfly.io/docs/scrape-api/proxy) - residential and datacenter pools with country-level geo-targeting. - [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - render SPAs and dynamic pages through real cloud browsers. - [Browser automation scenarios](https://scrapfly.io/docs/scrape-api/javascript-scenario) - scroll, click, fill forms, and wait for elements without managing a browser fleet. - [Format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) - return pages as HTML, JSON, clean text, or LLM ready Markdown. - [Session management](https://scrapfly.io/docs/scrape-api/session) - keep cookies, headers, and IPs consistent across multi step flows. - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache) - cache successful responses to cut cost on repeat scraping jobs. - [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript), [Scrapy](https://scrapfly.io/docs/sdk/scrapy), and [no-code integrations](https://scrapfly.io/docs/integration/getting-started) including Make, n8n, Zapier, LangChain, and LlamaIndex. ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. [Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started) ## FAQ Is there an official idealo API for offer data?Not for reading public offer data. idealo runs merchant integrations for retailers feeding in listings. There is no public API to read a product's live offers, so the data has to come from the product pages. Can I look up idealo offers by EAN or GTIN?Not reliably. idealo does not print EANs on its pages, and its search matches a barcode for some products and not others. Try the barcode as the search query, fall back to product title or model, then cache idealo's product id against your SKU. Does idealo still operate outside Germany?Yes. idealo.de is the flagship, and each idealo.de product page also lists that product's prices on idealo.at, idealo.fr, idealo.it, idealo.es, and idealo.co.uk. idealo.at shares idealo.de's URL paths, and the other four use their own, so adapt the URL builders before pointing the scraper at them. Do I need German proxies to scrape idealo.de?Effectively yes. Listings resolve to the visitor's location, so set the request country to `DE`, as the scraper's `BASE_CONFIG` does. Can I scrape idealo's price history in one request?One API call returns the whole series for a period, from one month up to a year. The scraper loads the product page first in the same session, so the full flow is two requests. Is it legal to scrape idealo?Scraping publicly available data is generally permissible, but rules vary by where you are and how you use it. Review idealo's terms and the legal note below, and avoid collecting personal data. ## Summary idealo gives you every merchant offer and a price chart for a product on one page, and the scraper reaches it in three steps. Search and manufacturer pages supply the product ids. On the product page, a load-more loop exposes every merchant offer, and JSON-LD holds the metadata. The price history comes from a session-backed endpoint. idealo hides EANs and matches them in search only sometimes, so you resolve a product once by barcode or title and cache its id. At scale, the flow needs anti-bot handling for Akamai and a German vantage point. The Scrapfly Web Scraping API handles both in a single request. From here you can feed the offer prices into a monitoring pipeline, or adapt the URL builders for idealo's other five markets. The full scraper lives in the [scrapfly-scrapers repository](https://github.com/scrapfly/scrapfly-scrapers/tree/main/idealo-scraper). The `idealo-scraper` folder also holds the result types, tests, and sample output. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents - [Key Takeaways](#key-takeaways) - [Why scrape idealo?](#why-scrape-idealo) - [How do you set up an idealo scraping project?](#how-do-you-set-up-an-idealo-scraping-project) - [How do you scrape idealo search and manufacturer results?](#how-do-you-scrape-idealo-search-and-manufacturer-results) - [How do you paginate idealo search results?](#how-do-you-paginate-idealo-search-results) - [How do you scrape an idealo manufacturer page?](#how-do-you-scrape-an-idealo-manufacturer-page) - [How do you scrape an idealo product page and its merchant offers?](#how-do-you-scrape-an-idealo-product-page-and-its-merchant-offers) - [How do you match an idealo product to your EAN or GTIN?](#how-do-you-match-an-idealo-product-to-your-ean-or-gtin) - [How do you scrape idealo price history?](#how-do-you-scrape-idealo-price-history) - [How do you bypass idealo blocking with Scrapfly?](#how-do-you-bypass-idealo-blocking-with-scrapfly) - [Powering idealo Scraping With Scrapfly](#powering-idealo-scraping-with-scrapfly) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ python crawling ### Guide to List Crawling: Everything You Need to Know Complete list crawling tutorial assess site defenses, bypass anti-bot systems, choose tools (Beautiful Soup, Playwright,... ](https://scrapfly.io/blog/posts/guide-to-list-crawling) [ python ecommerce ### How to Build a Grocery Price Comparison Tool with Python Build a grocery price comparison tool that scrapes Walmart, Instacart, and Kroger prices by zip code, matches products, ... ](https://scrapfly.io/blog/posts/how-to-build-a-grocery-price-comparison-tool-with-python) [ python scrapeguide ### How to Scrape YellowPages.com in 2026 Tutorial on how to scrape yellowpages.com business and review data using Python. How to avoid blocking to scrape data at... ](https://scrapfly.io/blog/posts/how-to-scrape-yellowpages) Extract structured data with AI, **1,000 free credits** [Start Free](https://scrapfly.io/register)