[Blog](https://scrapfly.io/blog) / [hidden-api](https://scrapfly.io/blog/tag/hidden-api) / [How to Scrape Target.com Product and Pricing Data](https://scrapfly.io/blog/posts/how-to-scrape-target-com) # How to Scrape Target.com Product and Pricing Data by [Hisham Medhat](https://scrapfly.io/blog/author/hisham) Aug 24, 2026 20 min read [\#hidden-api](https://scrapfly.io/blog/tag/hidden-api) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com&text=How%20to%20Scrape%20Target.com%20Product%20and%20Pricing%20Data "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-target-com) **Web Scraping API**Scrape any website with anti-bot bypass, proxy rotation, and JS rendering. [ Learn More ](https://scrapfly.io/products/web-scraping-api) [ Docs ](https://scrapfly.io/docs/scrape-api/getting-started) Point Python's `requests` at a Target product page, parse for the price, and you get `None`. The price isn't in the HTML; it lives in the Redsky API. Target's bot defense decides whether you may ask, and it judges your client, not your pace. One [r/webscraping developer](https://www.reddit.com/r/webscraping/comments/tb08l4/targets_redsky_api_no_longer_allowing_scraping/) hit that wall on Redsky in 2022, IP-blocked within seconds at one request a second. This guide covers the whole path: the Redsky endpoints, tcin and store\_id, location-keyed prices, and the bypass. [How to Scrape Walmart.com Product Data (2026 Update)Tutorial on how to scrape walmart.com product and review data using Python. How to avoid blocking to web scrape data at scale and other tips.](https://scrapfly.io/blog/posts/how-to-scrape-walmartcom) [**Latest Target.com Scraper Code**github.com/scrapfly/scrapfly-scrapers/tree/main/target-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/target-scraper) ## Key Takeaways - **The HTML has no price.** The page shell includes the product title, while price, rating, and availability arrive through Redsky-backed XHRs. - **Let the page fetch it.** Render the product page, read its `deferred_enrichment` XHRs. - **Credentials come from the site.** A homepage warm-up hands you the `key` and `visitor_id`. - **Prices are store-keyed.** Store-keyed items return their own price per store\_id. - **Fingerprint, not rate.** Target runs HUMAN (PerimeterX) bot defense. - **Skip the legacy endpoint.** The old v2/pdp/tcin path now returns HTTP 410 Gone. - **Scrapfly handles the blocking.** Point the Web Scraping API at Redsky and parse. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## What Product and Pricing Data Can You Scrape From Target.com? You can extract the product title, current and regular price, ratings, review counts, the variant list, and store-level price and stock. The cleanest source is Target's internal Redsky API; the page HTML holds only a fraction. Three approaches come up, and they are not equal: - **`requests` plus [How to Parse Web Data with Python and Beautifulsoup](https://scrapfly.io/blog/posts/web-scraping-with-python-beautifulsoup)**: fast, and the shell carries the title, but price and availability are injected later, so they come back empty. - **Capturing the page's own XHR calls**: you load the product page in a browser and read the JSON it fetches for itself. - **Calling Redsky directly**: the fastest path once you hold a valid `key` and `visitor_id`, and the way to price one tcin across many stores. Side by side: | Approach | JavaScript | Speed | Data captured | Anti-bot exposure | |---|---|---|---|---| | `requests` + BeautifulSoup | Not run | Fast | Title and other static fields; no price, rating, or availability | Low | | Captured XHR on the PDP | Run in a browser | Slow | Price, stock, variants, ratings | Handled by the bypass | | Direct Redsky calls | Not needed | Fast | Price, stock, stores | High, needs a bypass | The two JSON paths work together. You capture the product page once to get the fields and the credentials, then hit Redsky directly for the store-by-store numbers. Most of what a pricing scraper wants is not in the static HTML at all. The product page ships an H1 title and a `data-test="price-module-placeholder"` div where the price will go; the number arrives later over XHR. Price and availability also change by store, which is the other reason a single HTML scrape falls short. Grocery and other store-keyed items carry a separate price per location; nationally priced items do not. For background on the Python tooling here, see our [Everything to Know to Start Web Scraping in Python Today](https://scrapfly.io/blog/posts/everything-to-know-about-web-scraping-python). The [How to Scrape Amazon.com Product Data and Reviews](https://scrapfly.io/blog/posts/how-to-scrape-amazon) covers the same hidden-API pattern on another retailer. None of that helps if a naive scraper can't reach the API at all, which is the bot defense's job. ## Why Does Scraping Target.com Get Your Scraper Blocked? Target's storefront runs HUMAN Security (formerly PerimeterX). Its `humanSensor` script loads from `client.px-cloud.net` and fingerprints your client, and the Redsky API adds a check of its own on top. Request pacing does not help, because the decision turns on what your client looks like, not on how many requests it sent. Slowing down cannot fix a fingerprint. The failure shows up in a few distinct ways: - **JavaScript-rendered prices return `None`.** A plain `requests.get()` gets a React shell. Price, ratings, and availability are injected later, so your price selector returns `None`. - **Your client signature is checked before the API answers.** TLS and header fingerprints mark a default `httpx` or `requests` call as automated, so the Redsky endpoints refuse it even while the storefront HTML still loads. - **A client-side sensor gates the storefront.** The `humanSensor` script builds a token from browser signals. A client that never runs that JavaScript has no token to present. - **Datacenter IPs get negative trust.** Bot-defense vendors score datacenter ranges as likely bots, so the same scraper behaves differently from a residential IP. [How TLS Fingerprint is Used to Block Web Scrapers?TLS fingeprinting is a popular way to identify web scrapers that not many developers are aware of. What is it and how can we fortify our scrapers to avoid being detected?](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls) You can see the wall directly. A plain `httpx` call to Redsky's `pdp_client_v1` product endpoint, with no anti-bot handling, gets refused at the door: python```python import httpx url = "https://redsky.target.com/redsky_aggregations/v1/web/pdp_client_v1" params = {"key": "9f36aeafbe60771e321a7cc95a78140772ab3e96", "tcin": "14777576", "store_id": "3991", "pricing_store_id": "3991", "channel": "WEB"} print(httpx.get(url, params=params, timeout=30).status_code) ``` text```text 403 ``` The request shape is correct, and the endpoint still refuses it. The fix is a fingerprint-aware transport, which the scale section covers. [How to Bypass PerimeterX when Web Scraping in 2026In this article we'll take a look at a popular anti scraping service PerimeterX. How does it detect web scrapers and bots and what can we do to prevent our scrapers from being detected?](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping) ## How Do You Find Target's Redsky Product API? Open a product page with DevTools on the Network tab, filtered to Fetch/XHR. The page fires a `deferred_enrichment/modules` call for itself, plus `redsky.target.com` calls for pricing and stores. Those responses carry the price and product data. Learning to find them yourself keeps the method working as Target shifts URLs around. This is the standard [How to Scrape Hidden APIs](https://scrapfly.io/blog/posts/how-to-scrape-hidden-apis) discovery flow, and [What are devtools and how they're used in web scraping?](https://scrapfly.io/blog/answers/browser-developer-tools-in-web-scraping) are all you need for it. ### Reading the tcin From a Target Product URL The tcin is Target's product identifier and the value the Redsky API is keyed on. It sits in the product URL right after `/-/A-`: text```text https://www.target.com/p/apple-airpods-pro-3/-/A-85978609 tcin = 85978609 ``` Target also calls this the Item Number in its seller tools. Do not validate on length. Our committed search results carry eight-digit tcins like `95288385` next to ten-digit ones like `1011111213`, so match everything after `/-/A-` with `\d+`. ### Locating the Redsky Endpoints and Credentials in DevTools With the Network tab open and the Fetch/XHR filter on, reload the product page and watch the Redsky calls. Three of them carry the data this guide uses: - `deferred_enrichment/modules` returns the product detail the page renders, split into named modules. - `product_summary_with_fulfillment_v1` takes one or more tcins plus a store\_id and returns price, stock, and fulfillment. - `store_location_v1` returns details for one store\_id, and it also fires on the homepage. Every one of those calls carries a `key` and a `visitor_id` in the query string. The key is a public web key Target ships to every browser, so there is no signup, and the `visitor_id` is generated per session. Both values sit in the URLs of the calls the site makes for itself. That is why the scraper loads target.com first and reads them off the captured `store_location_v1` request: python```python from urllib.parse import parse_qs, urlparse def extract_redsky_credentials(response): """read the redsky key and visitor_id from the store_location_v1 XHR on the homepage""" for xhr in response.scrape_result["browser_data"]["xhr_call"]: if "store_location_v1" not in xhr["url"]: continue params = parse_qs(urlparse(xhr["url"]).query) key, visitor_id = params.get("key", [None])[0], params.get("visitor_id", [None])[0] if key and visitor_id: return {"key": key, "visitor_id": visitor_id} raise ValueError("missing store_location_v1 XHR with key and visitor_id") ``` python```python {'key': '9f36aeafbe60771e321a7cc95a78140772ab3e96', 'visitor_id': '019FAF20EDCD0200BC058B6F546DD3EE'} ``` Harvesting both from a live page beats hardcoding them, because Target can rotate the key and rejects requests whose `visitor_id` does not match the session. Many older guides and AI answers still reference the legacy `redsky.target.com/v2/pdp/tcin/...` endpoint. That path is retired and returns HTTP 410 Gone, so use the current `redsky_aggregations/v1/web/...` shape. With the tcin, the endpoints, and the credentials in hand, you can read the product JSON. ## How to Scrape Target Product and Pricing Data Load the product page with JavaScript rendering, wait for the `deferred_enrichment` XHR calls, and read the modules they return. Each module is a named block of the same JSON the page renders. Two modules hold what a pricing scraper needs. `ProductDetailWebDatasourceWithStore` carries the `price` object, and `ProductDetailWebDatasourceFulfillmentAndVariations` carries stock and the variant list. Ratings come from a third module, `ProductDetailReviewsAndQuestions`. Install the packages first: bash```bash pip install scrapfly-sdk httpx parsel ``` Every request in this guide shares one config. The `asp` flag handles the bot defense, and the US residential pool keeps store geo consistent: python```python import os from scrapfly import ScrapeConfig, ScrapflyClient SCRAPFLY = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"]) BASE_CONFIG = { "asp": True, # bypass Target's HUMAN (PerimeterX) defense "country": "US", "proxy_pool": "public_residential_pool", } async def scrape_product(url: str) -> dict: response = await SCRAPFLY.async_scrape(ScrapeConfig( url, **BASE_CONFIG, render_js=True, wait_for_selector="xhr:deferred_enrichment/modules", rendering_wait=5000, )) return parse_product(response) ``` The `wait_for_selector="xhr:deferred_enrichment/modules"` line is what makes this work. Scrapfly holds the render until that request fires, then hands you the captured calls in `browser_data`. Parsing walks those captured calls, keeps the module payloads, and flattens the fields: python```python import json def extract_deferred_modules(response) -> dict: modules = {} for xhr in response.scrape_result["browser_data"]["xhr_call"]: if "deferred_enrichment/modules" not in xhr["url"]: continue body = (xhr.get("response") or {}).get("body") if not body: continue for module in json.loads(body).get("modules", []): modules[module["module_type"]] = module.get("module_data", {}) return modules def parse_product(response) -> dict: modules = extract_deferred_modules(response) store = modules["ProductDetailWebDatasourceWithStore"]["data"]["product"] ratings = modules.get("ProductDetailReviewsAndQuestions", {}).get("ratings_and_reviews", {}) price = store.get("price") or {} return { "tcin": store["tcin"], "title": response.selector.css('h1[data-test="product-title"]::text').get(), "price": { "current_retail": price.get("current_retail"), "reg_retail": price.get("reg_retail"), "formatted_current_price": price.get("formatted_current_price"), "location_id": price.get("location_id"), }, "rating": ratings.get("average"), "review_count": ratings.get("count"), } # in a notebook, or wrap in asyncio.run() in a script print(await scrape_product("https://www.target.com/p/-/A-14777576")) ``` The point-in-time response below is keyed to price location 2429. A later run or another store can return a different price, rating, and review count: json```json { "tcin": "14777576", "title": "King Arthur Flour Unbleached Bread Flour - 5lbs", "price": { "current_retail": 6.89, "reg_retail": 6.89, "formatted_current_price": "$6.89", "location_id": 2429 }, "rating": 4.9, "review_count": 4169 } ``` Products sold in sizes or colors behave differently. The parent `price` block only shows a display value, and the real numbers sit on each child tcin under `children`: python```python def parse_variants(response) -> list: modules = extract_deferred_modules(response) store = modules["ProductDetailWebDatasourceWithStore"]["data"]["product"] return [{"tcin": child["tcin"], "price": (child.get("price") or {}).get("formatted_current_price")} for child in store.get("children", [])] ``` python```python # 33 children on this tank top, first four shown [{'tcin': '95205382', 'price': '$10.50'}, {'tcin': '95205383', 'price': '$10.50'}, {'tcin': '95205384', 'price': '$10.50'}, {'tcin': '95205385', 'price': '$10.50'}] ``` The full scraper in our target-scraper repository merges both, so each variant arrives with its own price and stock flag. Its shape differs from the simplified version above; it returns `variants` rather than a single top-level `price`. For working with these responses, see our notes on [Ultimate Guide to JSON Parsing in Python](https://scrapfly.io/blog/posts/how-to-use-python-to-parse-json). Notice the flour came back keyed to a store nobody asked for, and which store you get follows the IP the request leaves from. That is Target's location pricing leaking through. Scrapfly #### Scale your web scraping effortlessly Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data. [Try Free →](https://scrapfly.io/register)## How Does Target's Location-Aware Pricing Work When Scraping by store\_id? Target prices and availability are tied to a specific store, so the Redsky `price` block is keyed by store\_id. The response also carries `price.location_id`, which names the store a price came from. Pass a different store\_id and you get a different price and stock for the same tcin. There is a wrinkle, though. Nationally priced items, like electronics and apparel, fall back to Target's online pricing location (`3991`) whichever store you pass. You cannot pass `3991` yourself; Redsky rejects it as a store\_id. Store-keyed items behave differently. The `location_id` echoes your store while the price moves with it. Rendering a product page for every store would be slow, so pass the store\_id straight to `product_summary_with_fulfillment_v1` instead. That call needs the credentials from the homepage warm-up, kept in one Scrapfly session: python```python from urllib.parse import urlencode from uuid import uuid4 REDSKY = "https://redsky.target.com/redsky_aggregations/v1/web" async def redsky_session(): """warm up a session on target.com and harvest the redsky credentials""" session = str(uuid4()).replace("-", "") warm_up = await SCRAPFLY.async_scrape(ScrapeConfig( "https://www.target.com/", session=session, render_js=True, wait_for_selector="xhr:store_location_v1", rendering_wait=5000, **BASE_CONFIG, )) return session, extract_redsky_credentials(warm_up) async def scrape_availability(tcins: list, store_id: str, zip_code: str) -> dict: session, credentials = await redsky_session() url = f"{REDSKY}/product_summary_with_fulfillment_v1?" + urlencode({ **credentials, "tcins": ",".join(tcins), "store_id": store_id, "pricing_store_id": store_id, "required_store_id": store_id, "scheduled_delivery_store_id": store_id, "zip": zip_code, "channel": "WEB", "page": f"/p/A-{tcins[0]}", }) response = await SCRAPFLY.async_scrape(ScrapeConfig(url, session=session, **BASE_CONFIG)) summaries = json.loads(response.content)["data"]["product_summaries"] if not summaries: raise ValueError(f"redsky returned no summary for {tcins} at store {store_id}") return {"price": summaries[0]["price"], "sold_out": summaries[0]["fulfillment"]["sold_out"]} ``` The `session` parameter pins the cookies and IP that the warm-up earned, so Redsky treats the second call as the same visitor. Run the same flour tcin against two stores: python```python # in a notebook, or wrap in asyncio.run() in a script for store_id, zip_code in [("3412", "10001"), ("2093", "77002")]: record = await scrape_availability(["14777576"], store_id, zip_code) print(store_id, zip_code, record["price"]["formatted_current_price"], "location_id=", record["price"]["location_id"]) ``` text```text 3412 10001 $7.69 location_id= 3412 2093 77002 $6.89 location_id= 2093 ``` For a store-keyed item the `location_id` tracks the store\_id you pass, and the Houston price runs 80 cents below New York. Target itself states that [pricing and availability may vary by location](https://www.target.com/help/articles/product-support-services/product-availability). To compare a product across regions, iterate one tcin over several store\_ids. [How to Track Competitor Prices Using Web ScrapingIn this web scraping guide, we'll explain how to create a tool for tracking competitor prices using Python. It will scrape specific products from different providers, compare their prices and generate insights.](https://scrapfly.io/blog/posts/how-to-track-competitor-pricing-using-web-scraping) To feed that loop, you first need tcins and store\_ids in bulk. ## How Do You Find Target tcins and store\_ids at Scale? Collect tcins from rendered search pages, and collect store\_ids from Target's store sitemap. Every pricing call needs one of each, so gather them in bulk before you price anything. ### Collecting tcins From Target Search and Category Pages The search results page renders client-side, and the cards load as you scroll. Render the page, scroll it, then read the tcin out of each card link: python```python import re async def scrape_search(keyword: str) -> list: response = await SCRAPFLY.async_scrape(ScrapeConfig( "https://www.target.com/s?" + urlencode({"searchTerm": keyword}), render_js=True, auto_scroll=True, rendering_wait=5000, **BASE_CONFIG, )) products = [] for card in response.selector.css('[data-test="@web/site-top-of-funnel/ProductCardWrapper"]'): link = card.css('a[href*="/A-"]::attr(href)').get() or "" if match := re.search(r"/A-(\d+)", link): products.append({ "tcin": match.group(1), "title": card.css('[data-test="@web/ProductCard/title"] ::text').get(), "brand": card.css('[data-test="@web/ProductCard/ProductCardBrandAndRibbonMessage/brand"] ::text').get(), "price": card.css('[data-test="current-price"] span ::text').get(), }) return products ``` python```python [{'tcin': '91670102', 'title': 'Hamilton Beach The Scoop Single-Serve Coffee Maker 47621: White, 14 oz Capacity, Uses Pods & Ground Coffee, Dishwasher-Safe', 'brand': 'Hamilton Beach', 'price': '$39.99'}, {'tcin': '94139349', 'title': 'Cuisinart 14 Cup Programmable Drip Coffee Maker Stainless Steel', 'brand': 'Cuisinart', 'price': '$119.99'}] ``` Target mounts the cards lazily, so `auto_scroll` is the safe default when a render comes back with fewer cards than the page shows. Page deeper by adding the `Nao` parameter, which moves the result offset in steps of 24. ### Getting store\_ids From Target's Store Sitemap Target publishes every store URL in a sitemap at a `.xml.gz` path, and each URL ends in the numeric store\_id. Despite the extension it currently serves plain XML, so check the gzip magic bytes. One fetch gives you the whole chain: python```python import gzip from parsel import Selector async def scrape_store_sitemap(url: str) -> list: response = await SCRAPFLY.async_scrape(ScrapeConfig(url, **BASE_CONFIG)) content = response.scrape_result["content"] raw = content.read() if hasattr(content, "read") else content.encode("latin1") if raw.startswith(b"\x1f\x8b"): raw = gzip.decompress(raw) locations = [] for loc in Selector(text=raw.decode("utf-8")).xpath("//url/loc/text()").getall(): slug, store_id = loc.strip("/").split("/")[-2:] locations.append({"url": loc, "slug": slug, "store_id": store_id}) return locations # in a notebook, or wrap in asyncio.run() in a script print(len(await scrape_store_sitemap("https://www.target.com/sl/sitemap_0001.xml.gz"))) ``` text```text 2020 ``` That single file lists 2,020 stores with their IDs. To turn an ID into an address, pass it to `store_location_v1` with the same warm-up credentials the pricing call uses: python```python async def scrape_store(store_id: str) -> dict: session, credentials = await redsky_session() url = f"{REDSKY}/store_location_v1?" + urlencode({ **credentials, "store_id": store_id, "channel": "WEB", "page": "/c/root", }) response = await SCRAPFLY.async_scrape(ScrapeConfig(url, session=session, **BASE_CONFIG)) return json.loads(response.content)["data"]["store"] ``` python```python # trimmed; the response also carries geofence coordinates and store capabilities {'store_id': '3412', 'location_name': 'Chelsea 23rd and 8th', 'status': 'Open', 'mailing_address': {'address_line1': '258 8th Ave', 'city': 'New York', 'region': 'NY', 'postal_code': '10011-1619'}, 'main_voice_phone_number': '646-720-0024'} ``` Feed each store\_id into the pricing loop to build a price-by-location view of any product. Running thousands of these calls is where the bot defense stops being a footnote. ## How to Scrape Target at Scale Without Getting Blocked The blocker at scale is Target's HUMAN (PerimeterX) bot defense. Route every Target request through a scraping API that handles client fingerprinting, residential IPs, and the sensor challenge for you. With Scrapfly's [Anti-Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection), you set `asp=True` and the same Redsky request that returned 403 returns clean JSON. The bot defense is what makes or breaks a Target scraper. Scrapfly's [PerimeterX bypass](https://scrapfly.io/bypass/perimeterx) page reports a self-tracked 95% success rate on that vendor, and the same `asp` flag covers the others. Three settings carry this scraper. `asp=True` clears the bot defense, the US residential pool keeps store pricing sane, and `session` carries the warm-up cookies into the Redsky calls. Our maintained Target.com scraper packages all of that, including product pages, search, availability by store, and the store locator: python```python import asyncio import target # from scrapfly-scrapers/target-scraper async def main(): product = await target.scrape_product( "https://www.target.com/p/-/A-14777576" ) availability = await target.scrape_availability( tcins=["14777576"], store_id="2093", zip_code="77002", ) print(product["title"], availability["14777576"]["price"]["formatted_current_price"]) asyncio.run(main()) ``` text```text King Arthur Flour Unbleached Bread Flour - 5lbs $6.89 ``` The repository also ships a test suite and sample output for every path. Read those files to learn the JSON shape before you write any parsing code. Scrapfly ships SDKs for Python, TypeScript, Go, and Rust plus a Scrapy extension, so this approach works whatever your stack. You only pay for the requests that land. ScrapFly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) collects public web data through one HTTP endpoint, with a **98% success rate on the hardest public pages** and residential and datacenter coverage in **190+ countries**. - [Anti-Scraping Protection bypass](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - enables Scrapfly's managed anti-bot adaptation for supported public targets. - [Smart proxy rotation](https://scrapfly.io/docs/scrape-api/proxy) - residential and datacenter pools with country and ASN level geo-targeting. - [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - render SPAs and JavaScript-heavy pages through real cloud browsers. - [Browser automation scenarios](https://scrapfly.io/docs/scrape-api/javascript-scenario) - scroll, click, fill forms, and wait for elements without managing a browser fleet. - [Format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) - return pages as HTML, JSON, clean text, or LLM ready Markdown. - [Session management](https://scrapfly.io/docs/scrape-api/session) - keep cookies, headers, and IPs consistent across multi step flows. - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache) - cache successful responses to cut cost on repeat scraping jobs. - [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript), [Rust](https://scrapfly.io/docs/sdk/rust), [Scrapy](https://scrapfly.io/docs/sdk/scrapy), and [no-code integrations](https://scrapfly.io/docs/integration/getting-started) including [Make](https://scrapfly.io/integration/make), [n8n](https://scrapfly.io/integration/n8n), [Zapier](https://scrapfly.io/integration/zapier), [LangChain](https://scrapfly.io/integration/langchain), and [LlamaIndex](https://scrapfly.io/integration/llamaindex). ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. [Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started) ## FAQ Is scraping Target's Redsky API legal?Scraping publicly accessible product and pricing data is generally permissible, but it is governed by Target's Terms of Service and applicable laws. Avoid collecting personal data or logging in, and treat this as general information rather than legal advice. Do I need a Target API key to use Redsky?No, Target ships a public web key to every browser and pairs it with a per-session `visitor_id`. Read both off a live page instead of hardcoding them. Why did my old Target Redsky scraper stop working?Target retired the legacy `v2/pdp/tcin` endpoint that older guides cite, and it now returns HTTP 410 Gone. The Redsky endpoints also refuse fingerprinted clients, so a plain `httpx` call gets a 403 on request one. Do I need Playwright or Selenium to scrape Target?You need a rendered page to capture the product XHRs, but Scrapfly can run that render for you instead of a local [Web Scraping with Playwright and Python](https://scrapfly.io/blog/posts/web-scraping-with-playwright-and-python) fleet. The pricing and store calls need no browser themselves, though the warm-up that harvests their credentials does. Can I scrape Target product ratings and reviews too?Yes. Ratings and review counts arrive in the `ProductDetailReviewsAndQuestions` module on the product page; full review text comes from a separate endpoint that is out of scope here. ## Summary Target's product and pricing data is cleanest as JSON. Render the product page to capture its `deferred_enrichment` modules, then call Redsky directly with the key and `visitor_id` the site handed you. Prices are keyed to a store, and the response `location_id` tells you which one. Iterating one tcin across store\_ids is how you compare a product between regions. The parsing is the easy part. Target's HUMAN (PerimeterX) bot defense stops most Target scrapers. The Redsky endpoints refuse a fingerprinted client with a 403, however slowly you request. A fingerprint-aware client gets you started. Routing the Redsky calls through a scraping API that handles the bot defense keeps collection working once you scale. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents- [Key Takeaways](#key-takeaways) - [What Product and Pricing Data Can You Scrape From Target.com?](#what-product-and-pricing-data-can-you-scrape-from-target-com) - [Why Does Scraping Target.com Get Your Scraper Blocked?](#why-does-scraping-target-com-get-your-scraper-blocked) - [How Do You Find Target's Redsky Product API?](#how-do-you-find-target-s-redsky-product-api) - [Reading the tcin From a Target Product URL](#reading-the-tcin-from-a-target-product-url) - [Locating the Redsky Endpoints and Credentials in DevTools](#locating-the-redsky-endpoints-and-credentials-in-devtools) - [How to Scrape Target Product and Pricing Data](#how-to-scrape-target-product-and-pricing-data) - [How Does Target's Location-Aware Pricing Work When Scraping by store\_id?](#how-does-target-s-location-aware-pricing-work-when-scraping-by-store-id) - [How Do You Find Target tcins and store\_ids at Scale?](#how-do-you-find-target-tcins-and-store-ids-at-scale) - [Collecting tcins From Target Search and Category Pages](#collecting-tcins-from-target-search-and-category-pages) - [Getting store\_ids From Target's Store Sitemap](#getting-store-ids-from-target-s-store-sitemap) - [How to Scrape Target at Scale Without Getting Blocked](#how-to-scrape-target-at-scale-without-getting-blocked) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ blocking ### How to Bypass PerimeterX when Web Scraping in 2026 In this article we'll take a look at a popular anti scraping service PerimeterX. How does it detect web scrapers and bot... ](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping) [ python scrapeguide ### How to Scrape Lowe's Product, Price, Search, and Store Location Data Scrape Lowe's product and pricing data from embedded page state, discover products and stores, and handle Akamai with Sc... ](https://scrapfly.io/blog/posts/how-to-scrape-lowe) [ curl ### How to Use cURL GET Requests Here's everything you need to know about cURL GET requests and some common pitfalls you should avoid. ](https://scrapfly.io/blog/posts/how-to-use-curl-get-requests) Scale your web scraping effortlessly, **1,000 free credits** [Start Free](https://scrapfly.io/register)