[Blog](https://scrapfly.io/blog) / [blocking](https://scrapfly.io/blog/tag/blocking) / [How to Scrape Skyscanner Flight Prices in Python (2026)](https://scrapfly.io/blog/posts/how-to-scrape-skyscanner) # How to Scrape Skyscanner Flight Prices in Python (2026) by [Mayada Shaaban](https://scrapfly.io/blog/author/mayada-shaaban-90143e67) Aug 14, 2026 24 min read [\#blocking](https://scrapfly.io/blog/tag/blocking) [\#headless-browser](https://scrapfly.io/blog/tag/headless-browser) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner&text=How%20to%20Scrape%20Skyscanner%20Flight%20Prices%20in%20Python%20%282026%29 "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-skyscanner) **Web Scraping API**Scrape any website with anti-bot bypass, proxy rotation, and JS rendering. [ Learn More ](https://scrapfly.io/products/web-scraping-api) [ Docs ](https://scrapfly.io/docs/scrape-api/getting-started) Skyscanner compares fares from hundreds of airlines and travel agencies, which makes it a rich source of flight pricing data. But the prices you see never arrive in the first HTML response. Skyscanner runs an async search in the browser, polls for results, and only then paints the fare cards. A plain HTTP request returns an empty shell or a loading screen. This guide shows how to scrape Skyscanner flights in Python, start to finish. You'll build a Skyscanner scraper that constructs deep-link URLs and renders the search with the Scrapfly Web Scraping API. It waits until the results settle, then reads the itinerary JSON the page fetches for itself. JFK to LHR one-way is the running demonstration, and `run.py` in the repo runs the same route. [**Latest Skyscanner Scraper Code**github.com/scrapfly/scrapfly-scrapers/skyscanner-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/skyscanner-scraper) ## Key Takeaways The essentials before you write any code: - Skyscanner loads flights via JavaScript, so render the page and wait for them. - Build deep-link URLs instead of driving the search form. - Skip the HTML entirely; capture the `web-unified-search` XHR the page calls. - The JSON holds every itinerary, so lazy-loaded cards never limit your results. - Wait for the "results sorted by" label; it means the search finished polling. - Round trips work: put both dates in the URL path. CI only covers the one-way run. - For scale, the Scrapfly Web Scraping API handles rendering, captcha, and proxies. With those essentials in mind, let's start with why Skyscanner is worth scraping in the first place. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## Why Scrape Skyscanner Flight Data? Scrape Skyscanner when you need aggregated flight-search data across many airlines and online travel agencies for a specific route, date, market, and cabin class. Skyscanner is a metasearch aggregator, not an airline. Skyscanner compares providers instead of selling one carrier's inventory, so a single page gives you a cross-provider price view. A Skyscanner flight data scraper lets you scrape Skyscanner flight prices across every provider on that page. That breadth supports a handful of concrete projects: - Fare comparison across multiple airlines and travel agencies for the same route. - Running a Skyscanner price scraper that tracks the cheapest itineraries on high-value routes over time. - Travel-app enrichment with price, schedule, stops, and provider metadata. - Market and currency comparison, since the same route can show different providers or prices by location. - Competitive intelligence for airlines, agencies, and travel data teams. Skyscanner isn't your only option, so pick the source that matches the question. Skyscanner gives you cross-provider aggregation and comparison. The same approach can scrape flight aggregator pages and scrape airline metasearch results. Airline-direct sites work better when you need a single carrier's owned fare classes, which is what [scraping Air France](https://scrapfly.io/blog/posts/how-to-scrape-air-france-flights) covers, and [Google Flights](https://scrapfly.io/blog/posts/how-to-scrape-google-flights) is the other metasearch angle. Official or partner APIs are preferable where their coverage, rights, and fields fit your project. Keep your collection responsible. Scrape public result pages, respect rate limits and site terms, and stay away from account, checkout, and payment flows. With the reasons clear, let's look at what a single result page gives you. ## What Skyscanner Flight Data Can You Scrape? A Skyscanner itinerary exposes the price, marketing carrier, both times, duration, and stop count. It also carries the operating airline where it differs from the marketing one, plus a count of how many providers sell that itinerary. The table below lists the fields the scraper targets and what each one means: | Field | Sample | Notes | |---|---|---| | origin | `JFK` | Uppercase IATA code from your search input. | | destination | `LHR` | Uppercase IATA code from your search input. | | departure\_date | `2026-08-12` | The date you searched, in `YYYY-MM-DD`. | | trip\_type | `one_way` / `round_trip` | Derived from the leg count in the response. | | market\_country | `US` | Needed for geo comparisons. | | currency | `USD` | Hardcoded label, not read from the response. Check it against the `price` symbol. | | carrier | `British Airways` | Marketing carrier on the first leg. | | operated\_by | `American Airlines` | Only set when it differs from the carrier. | | departure\_time | `9:40 PM` | Formatted from the ISO timestamp. | | arrival\_time | `9:50 AM` | Formatted from the ISO timestamp; the date is dropped, so next-day arrivals look same-day. | | duration | `7h 10` | Built from the raw minute count. | | stops | `Direct` / `1 stop` | Formatted from the stop count. | | price | `$893` | Skyscanner's pre-formatted itinerary price. | | provider\_deal\_count | `8` | How many providers sell this itinerary. | | captured\_at | ISO timestamp | Needed for monitoring over time. | After parsing, each itinerary becomes a flat record. Here's one from the committed JFK to LHR run in the repo's `results/` directory: json```json { "origin": "JFK", "destination": "LHR", "departure_date": "2026-08-12", "trip_type": "round_trip", "market_country": "US", "currency": "USD", "carrier": "British Airways", "operated_by": "American Airlines", "departure_time": "7:15 PM", "arrival_time": "7:20 AM", "duration": "7h 05", "stops": "Direct", "price": "$893", "provider_deal_count": 8, "captured_at": "2026-08-12T17:49:02.214192+00:00", "return_date": "2026-08-19", "return_carrier": "British Airways", "return_operated_by": null, "return_departure_time": "5:25 PM", "return_arrival_time": "8:20 PM", "return_duration": "7h 55", "return_stops": "Direct" } ``` The itinerary price is the cheapest "from" price across providers. The per-provider breakdown behind an "8 deals from" label lives in the same JSON under `pricingOptions`. You can go deeper than the deal count if you need to. Next, let's reach a result page. ## How Skyscanner Flight Result URLs Work Skyscanner exposes constructable result URLs, so you don't have to fill in the search form at all. The observed one-way format follows a fixed pattern: text```text https://www.skyscanner.com/transport/flights////?adults=1&cabinclass=economy&rtn=0 ``` Filling in the JFK to LHR route for August 15, 2026 gives the URL the scraper requests: text```text https://www.skyscanner.com/transport/flights/JFK/LHR/260815/?adults=1&cabinclass=economy&rtn=0 ``` Each part of that URL maps to one search input: | Input | Sample | URL placement | |---|---|---| | Origin | JFK | `/flights/JFK/...` | | Destination | LHR | `/flights/.../LHR/...` | | Departure date | 2026-08-15 | `260815` | | Adults | 1 | `adults=1` | | Cabin | Economy | `cabinclass=economy` | | Trip type | One-way | `rtn=0` | Origin and destination are IATA codes. The date uses a `YYMMDD` format. The `rtn=0` flag marks a one-way search, and `rtn=1` marks a round trip. Direct URLs beat form-driving for three practical reasons. You skip fragile autocomplete and date-picker selectors. You can batch routes and dates with a loop. And you can reproduce any request from a repo without recording a browser session. A direct URL still isn't a static page, though. Skyscanner renders and polls results client-side, so the scraper has to run JavaScript and wait for prices. The URL is also only part of the search context. Proxy country, site TLD, and Skyscanner's market settings can change the provider set and the prices you get back. Before we send a request, let's set up the project. ## Project Setup This Skyscanner scraper needs Python 3.10 or newer, a Scrapfly API key, and two packages. The scrapfly-sdk handles requests, rendering, and anti-bot bypass. Loguru prints readable progress lines as the scrape runs. The flight data arrives as JSON, so there is no HTML parser in this list. Install both with pip: bash```bash pip install scrapfly-sdk loguru ``` This pulls both libraries and their dependencies into your environment, giving you the Scrapfly client and the logger in one step. Store your API key in an environment variable so it stays out of the code: bash```bash export SCRAPFLY_KEY="YOUR_SCRAPFLY_KEY" ``` This exposes the key to your Python script for the current terminal session. You never have to hardcode it in the source. Then create the client and a shared config once at the top of your script: python```python import json import os from datetime import datetime, timezone from typing import Dict, List, Optional, TypedDict from loguru import logger as log from scrapfly import ScrapeConfig, ScrapflyClient SCRAPFLY = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"]) BASE_CONFIG = { "asp": True, "proxy_pool": "public_residential_pool", "render_js": True, "rendering_wait": 5000, "country": "US", } ``` This builds one reusable client that reads your key from the environment and authenticates every request. `BASE_CONFIG` collects the settings every Skyscanner scrape needs, so each call spreads it in rather than repeating them. Finally, declare the two shapes the scraper returns. One record per itinerary, wrapped in one object per search. Both use `total=False` because the `return_*` keys only exist on round trips: python```python class FlightResult(TypedDict, total=False): origin: str destination: str departure_date: str trip_type: str market_country: str currency: str carrier: Optional[str] operated_by: Optional[str] departure_time: Optional[str] arrival_time: Optional[str] duration: Optional[str] stops: Optional[str] price: Optional[str] provider_deal_count: Optional[int] captured_at: str return_date: str return_carrier: Optional[str] return_operated_by: Optional[str] return_departure_time: Optional[str] return_arrival_time: Optional[str] return_duration: Optional[str] return_stops: Optional[str] class FlightSearch(TypedDict, total=False): url: str origin: str destination: str departure_date: str return_date: str trip_type: str market_country: str flight_count: int flights: List[FlightResult] ``` `TypedDict` documents the shape without changing it. The output stays a plain dict, so it serializes to JSON with no conversion step. The [Web Scraping API getting started](https://scrapfly.io/docs/scrape-api/getting-started) guide covers the parameters you'll use next. With the client ready, let's request a results page. ## Scrape a Skyscanner Flight Results Page The Skyscanner results URL loads a real page, but flights appear only after JavaScript runs. The scraper must render the page, clear the anti-bot checks, and wait for the search to settle. We'll build that in four steps. ### Build a Skyscanner Search URL Start with two small helpers that turn route inputs into a deep-link URL. One converts your date to Skyscanner's path format, the other assembles the URL: python```python def to_yymmdd(departure_date: str) -> str: """Convert YYYY-MM-DD to Skyscanner's YYMMDD path segment.""" return datetime.strptime(departure_date, "%Y-%m-%d").strftime("%y%m%d") def build_url( origin: str, destination: str, yymmdd: str, adults: int = 1, cabin_class: str = "economy", rtn: int = 0, return_yymmdd: Optional[str] = None, ) -> str: """Build a Skyscanner flight-search URL; round trips need both dates in the path.""" date_path = f"{yymmdd}/" if rtn and return_yymmdd: date_path += f"{return_yymmdd}/" return ( f"https://www.skyscanner.com/transport/flights/" f"{origin.upper()}/{destination.upper()}/{date_path}" f"?adults={adults}&cabinclass={cabin_class}&rtn={rtn}" ) ``` Calling `build_url("JFK", "LHR", to_yymmdd("2026-08-15"))` returns the JFK to LHR URL from the previous section. A round trip adds a second segment: `rtn=1` plus `return_yymmdd` produces `.../JFK/LHR/260812/260819/?...&rtn=1`. Keeping the two jobs separate means you can feed the builder any route, date, cabin, or trip type. Now let's hand that URL to Scrapfly. ### Configure Scrapfly for JavaScript Rendering and ASP The `BASE_CONFIG` from setup carries most of the request. `render_js=True` runs the client-side search, `asp=True` clears the PerimeterX challenge, and `country="US"` with the residential proxy pool routes through the market we tested. Two more parameters go on the call itself: python```python RESULTS_SORTED_SELECTOR = "//span[contains(text(),'results sorted by')]" response = await SCRAPFLY.async_scrape( ScrapeConfig( url, **BASE_CONFIG, rendering_wait=5000, wait_for_selector=RESULTS_SORTED_SELECTOR, ) ) ``` `wait_for_selector` holds the render until the results settle. `rendering_wait=5000` adds five seconds on top, which gives Skyscanner's polling time to deliver a complete result set rather than a first partial batch. ### Wait for the Search to Settle The wait selector is the most important choice here. Skyscanner paints ticket-shaped shimmer placeholders while searching, so a selector that matches a card can fire before a single real fare exists. Don't rely on network-idle either, because Skyscanner keeps polling in the background long after the first results show. ### Capture the Flight Search XHR Skyscanner's page doesn't hold the flight data in its HTML. It fetches the data from an internal API and paints the cards from the response, so that API call is what you want. Scrapfly records every background request the browser makes and returns them under `browser_data.xhr_call`, so you can read the same JSON the page read. The search endpoint is easy to spot by path: python```python SEARCH_XHR_PATH = "/g/radar/api/v2/web-unified-search/" def _extract_xhr_results(xhr_calls: List[Dict]) -> List[Dict]: """Extract itinerary results from the latest web-unified-search XHR call.""" search_calls = [c for c in xhr_calls if SEARCH_XHR_PATH in c["url"]] if not search_calls: raise RuntimeError( "Skyscanner XHR search call not found, try increasing rendering_wait" ) data = json.loads(search_calls[-1]["response"]["body"]) status = data.get("context", {}).get("status", "unknown") log.info( f"using XHR response (status={status!r}, {len(search_calls)} calls captured)" ) if "itineraries" not in data: raise RuntimeError( f"'itineraries' key missing in XHR response (status={status!r})" ) return data["itineraries"]["results"] ``` Skyscanner polls that endpoint repeatedly as the search fills in, so take the last call rather than the first. Earlier ones hold partial results. The guards raise instead of returning an empty list, because a missing XHR call and a search with no flights need different fixes. Reading the JSON gives you typed values instead of scraped strings, and it avoids the CSS classes Skyscanner regenerates on every deploy. That is why there is no HTML parsing anywhere in this scraper. [How to Scrape Hidden APIsIn this tutorial we'll be taking a look at scraping hidden APIs which are becoming more and more common in modern dynamic websites - what's the best way to scrape them?](https://scrapfly.io/blog/posts/how-to-scrape-hidden-apis) Scrapfly #### Need to bypass anti-bot protection? Scrapfly's Anti-Scraping Protection handles Cloudflare, DataDome, and more — automatically. [Try Free →](https://scrapfly.io/register)## Parse Skyscanner Flight Data The search response holds a list of itineraries, each with legs, carriers, and pricing already structured. Parsing is a matter of reshaping that into flat records. ### Extract Itineraries from the Search JSON The scraper labels the trip type from the leg count: one leg is `one_way`, two is `round_trip`. Both are real: the committed round-trip run returns 1,471 two-leg itineraries. Inside a leg you get ISO timestamps, a duration in minutes, a stop count, and marketing and operating carrier lists. A few small formatters turn those raw values into readable fields: python```python def _format_time(iso_dt: str) -> str: dt = datetime.fromisoformat(iso_dt) return dt.strftime("%I:%M %p").lstrip("0") def _format_duration(minutes: int) -> str: return f"{minutes // 60}h {minutes % 60:02d}" def _format_stops(count: int) -> str: if count == 0: return "Direct" if count == 1: return "1 stop" return f"{count} stops" def _carrier_name(carriers: List[Dict]) -> Optional[str]: return carriers[0]["name"] if carriers else None ``` Keep these as separate functions so that when Skyscanner changes a field, you fix one formatter instead of picking apart a long parsing expression. ### Normalize Prices, Times, Stops, and Carriers Now assemble one flat record per itinerary, attaching the route context you already know: python```python def _parse_leg(leg: Dict) -> Dict: carrier = _carrier_name(leg["carriers"]["marketing"]) operator = _carrier_name(leg["carriers"].get("operating", [])) return { "carrier": carrier, "operated_by": operator if operator and operator != carrier else None, "departure_time": _format_time(leg["departure"]), "arrival_time": _format_time(leg["arrival"]), "duration": _format_duration(leg["durationInMinutes"]), "stops": _format_stops(leg["stopCount"]), } def _parse_flight( item: Dict, origin: str, destination: str, departure_date: str, return_date: Optional[str], market_country: str, captured_at: str, ) -> Optional[FlightResult]: legs = item["legs"] outbound = _parse_leg(legs[0]) trip_type = "round_trip" if len(legs) == 2 else "one_way" flight = FlightResult( origin=origin.upper(), destination=destination.upper(), departure_date=departure_date, trip_type=trip_type, market_country=market_country, currency="USD", carrier=outbound["carrier"], operated_by=outbound["operated_by"], departure_time=outbound["departure_time"], arrival_time=outbound["arrival_time"], duration=outbound["duration"], stops=outbound["stops"], price=item["price"]["formatted"], provider_deal_count=len(item["pricingOptions"]), captured_at=captured_at, ) if trip_type == "round_trip": inbound = _parse_leg(legs[1]) flight["return_date"] = return_date flight["return_carrier"] = inbound["carrier"] flight["return_operated_by"] = inbound["operated_by"] flight["return_departure_time"] = inbound["departure_time"] flight["return_arrival_time"] = inbound["arrival_time"] flight["return_duration"] = inbound["duration"] flight["return_stops"] = inbound["stops"] return flight ``` `_parse_leg` runs once for the outbound leg and again for the inbound one on a round trip, which is why the `return_*` fields mirror the outbound names. `operated_by` stays `None` when the operating airline matches the marketing one, so the field only carries information when a codeshare differs. Note the `.get("operating", [])`: Skyscanner omits that key on itineraries with no separate operator. Index it directly and those itineraries raise `KeyError`, which the loop below swallows, so they vanish from your results without an error. Then loop over every itinerary, stamping one shared capture timestamp across the batch: python```python def parse_flights_from_xhr( xhr_results: List[Dict], origin: str, destination: str, departure_date: str, return_date: Optional[str] = None, market_country: str = "US", ) -> List[FlightResult]: """Parse flight itineraries from the web-unified-search XHR JSON response.""" captured_at = datetime.now(timezone.utc).isoformat() flights: List[FlightResult] = [] for item in xhr_results: try: flights.append( _parse_flight( item, origin, destination, departure_date, return_date, market_country, captured_at ) ) except (KeyError, IndexError, ValueError): continue log.success(f"parsed {len(flights)} flights from {origin}->{destination}") return flights ``` The `try/except` matters here. Skyscanner mixes in occasional itineraries with a missing carrier list or an unusual leg shape. One malformed entry shouldn't discard a search that returned hundreds of good ones. Store context on every record: market country, currency, source URL, captured timestamp, trip type, and route. Key each itinerary by route, date, carrier, and departure and arrival times to de-duplicate across runs. ### Run the Scraper End to End The driver wires it together. It builds the URL, renders the search, pulls the XHR results, parses them, and wraps everything in a result object with its own search context: python```python async def scrape_flights( origin: str, destination: str, departure_date: str, adults: int = 1, cabin_class: str = "economy", return_date: Optional[str] = None, market_country: str = "US", ) -> FlightSearch: """Scrape Skyscanner one-way or round-trip flight results via XHR interception.""" rtn = 1 if return_date else 0 url = build_url( origin, destination, to_yymmdd(departure_date), adults, cabin_class, rtn=rtn, return_yymmdd=to_yymmdd(return_date) if return_date else None, ) trip_desc = f"{departure_date} -> {return_date}" if return_date else departure_date log.info(f"scraping skyscanner {origin}->{destination} on {trip_desc}") response = await SCRAPFLY.async_scrape( ScrapeConfig( url, **BASE_CONFIG, wait_for_selector=RESULTS_SORTED_SELECTOR, ) ) xhr_results = _extract_xhr_results( response.scrape_result["browser_data"]["xhr_call"] ) flights = parse_flights_from_xhr( xhr_results, origin, destination, departure_date, return_date, market_country ) return FlightSearch( url=url, origin=origin.upper(), destination=destination.upper(), departure_date=departure_date, trip_type="round_trip" if rtn else "one_way", market_country=market_country, flight_count=len(flights), flights=flights, ) ``` Call it with a route and a date. Adding `return_date` makes it a round trip: python```python import asyncio flights = asyncio.run( scrape_flights("JFK", "LHR", "2026-08-12", return_date="2026-08-19") ) print(f"{flights['flight_count']} itineraries") print(flights["flights"][0]) ``` The committed round-trip run of the JFK to LHR search returns the full result set, each itinerary a flat record carrying both legs: text```text 1471 itineraries {'origin': 'JFK', 'destination': 'LHR', 'departure_date': '2026-08-12', 'trip_type': 'round_trip', 'market_country': 'US', 'currency': 'USD', 'carrier': 'British Airways', 'operated_by': 'American Airlines', 'departure_time': '7:15 PM', 'arrival_time': '7:20 AM', 'duration': '7h 05', 'stops': 'Direct', 'price': '$893', 'provider_deal_count': 8, 'captured_at': '2026-08-12T17:49:02.214192+00:00', 'return_date': '2026-08-19', 'return_carrier': 'British Airways', 'return_operated_by': None, 'return_departure_time': '5:25 PM', 'return_arrival_time': '8:20 PM', 'return_duration': '7h 55', 'return_stops': 'Direct'} ``` Skyscanner lazy-loads its cards as you scroll. An HTML scraper would have returned only the top handful and needed a scroll loop to go deeper. The XHR response carries the whole result page the search built, so one request replaces the scroll loop. When this stops working, the cause is usually blocking, which is next. ## Handle Skyscanner Blocking and Geo-Targeted Prices Skyscanner runs enough protection that a failed scrape can look like a successful HTTP response. Treat the final URL as part of your success criteria: a PerimeterX captcha page returns HTTP 200. This section covers the failure modes you'll hit. ### Detect PerimeterX Captcha Redirects The challenge you'll see most is a PerimeterX redirect to `/sttc/px/captcha-v2/`. These responses return HTTP 200, so the status code alone tells you nothing. Read `response.scrape_result["url"]` instead; a captcha page never made the search XHR call, so the scraper raises rather than returning empty. This symptom table maps each failure to a response: | Symptom | Likely cause | Response | |---|---|---| | Final URL contains `/sttc/px/captcha` | PerimeterX challenge | Retry or fall back to a Cloud Browser | | `XHR search call not found` | Render ended before the search fired | Raise `rendering_wait` above 5000 | | `'itineraries' key missing` | Search returned but found nothing | Check the route, date, and cabin are valid | | Fewer flights than the page shows | Polling still incomplete | Raise `rendering_wait`; the last call wins | | Wrong currency or provider mix | Market and proxy mismatch | Align country, locale, and currency | Retry with care. With `asp=True`, Scrapfly already rotates fingerprints and IPs, so a second attempt often clears a transient challenge. But if a market or trip type returns captchas every time, fall back instead of hammering the site. [How to Bypass PerimeterX when Web Scraping in 2026In this article we'll take a look at a popular anti scraping service PerimeterX. How does it detect web scrapers and bots and what can we do to prevent our scrapers from being detected?](https://scrapfly.io/blog/posts/how-to-bypass-perimeterx-human-anti-scraping) ### Align Proxy Country, Market, Currency, and Locale Skyscanner prices and provider sets are market-sensitive, so the proxy country shapes the data you collect. Use `country="US"` in this guide because we validated it end to end. If you test another market, validate it on its own and record the market, country, and currency on every result. The same JFK to LHR route can show a different provider mix and price abroad. The parser hardcodes `currency="USD"` while `price` keeps whatever formatting Skyscanner returned. The field is a fixed label, never read from the response. In the current committed runs the two agree, because both resolved a US market and every price string is dollar-formatted. Change `country` and nothing in the parser notices. Derive the currency from the price symbol, or set it from the market you actually resolved, and never trust the hardcoded field on its own. Watch the TLD too. Skyscanner may redirect between `.com`, `.net`, and local market variants. `FlightSearch` stores the URL you requested; store `response.scrape_result["url"]` alongside it if you need the market to be unambiguous after a redirect. You can also scrape [in another language or currency](https://scrapfly.io/blog/posts/how-to-scrape-in-another-language-or-currency) to shift the displayed prices. When the one-request pattern can't hold a market, a Cloud Browser can step in. ### When to Use Cloud Browser Instead A Cloud Browser is the fallback when the single-request Web Scraping API pattern isn't enough. It runs a full stealth browser you can control, which suits flows that need interaction rather than one render. Reach for it in these cases: - Multi-city itineraries, which the deep-link URL format doesn't cover. - Markets where the Web Scraping API repeatedly hits a challenge. - Flows that need clicks, like filtering or booking hand-offs. - Debugging with live browser state and screenshots. Keep the Cloud Browser as a targeted fallback, not the default. The [Cloud Browser approach](https://scrapfly.io/blog/posts/web-scraping-with-cloud-browsers) is heavier per request, so use it only where interaction earns its cost. ## Scrape Skyscanner with Scrapfly ScrapFly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) is a single HTTP endpoint for collecting web data at scale, with a **99.99% success rate** across **190+ countries**. - [Anti-Scraping Protection bypass](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - automatically defeats Cloudflare, DataDome, PerimeterX, Akamai, and 17 other anti-bot vendors. - [Smart proxy rotation](https://scrapfly.io/docs/scrape-api/proxy) - residential and datacenter pools with country and ASN level geo-targeting. - [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - render SPAs and dynamic pages through real cloud browsers. - Browser automation scenarios - scroll, click, fill forms, and wait for elements without managing a browser fleet. - [Format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) - return pages as HTML, JSON, clean text, or LLM ready Markdown. - [Session management](https://scrapfly.io/docs/scrape-api/session) - keep cookies, headers, and IPs consistent across multi step flows. - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache) - cache successful responses to cut cost on repeat scraping jobs. - [Python SDK](https://scrapfly.io/docs/sdk/python) - native client for requests, rendering, and ASP. - [TypeScript SDK](https://scrapfly.io/docs/sdk/typescript) - the same API for Node and browser projects. - [Scrapy integration](https://scrapfly.io/docs/sdk/scrapy) - drop-in support for existing Scrapy spiders. - [No-code integrations](https://scrapfly.io/docs/integration/getting-started) - Make, n8n, Zapier, LangChain, and LlamaIndex. For this scraper, the API runs Skyscanner's client-side search and gets past the PerimeterX challenge with `asp=True`. It routes through US residential IPs with one `country` value. It also records the background XHR calls so you can read the flight JSON, all from a single call. ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. [Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started) ## FAQ Is there a Skyscanner API for flight prices?Skyscanner has partner and affiliate API paths for approved use cases, but coverage, fields, and freshness may not fit every project. A Skyscanner Web Scraping API like Scrapfly is a fallback when you need the data shown on the site. Can I scrape Skyscanner with BeautifulSoup only?No. Plain `requests` plus BeautifulSoup will miss the fares, because the flights are JavaScript-rendered and the page is anti-bot protected. This scraper skips HTML parsing altogether and reads Skyscanner's own search API response instead. Why capture the XHR instead of parsing the cards?The JSON holds every itinerary with typed fields. The DOM holds only the cards rendered so far, dressed in class names that change on every deploy. Reading the XHR means no scroll loop and no selector maintenance. Can I scrape round-trip Skyscanner results?Yes. Pass `return_date` and the builder adds a second date segment to the URL path, which is what Skyscanner needs. Each itinerary then carries two legs, and the parser fills the `return_*` fields from the inbound one. The committed round-trip run holds 1,471 two-leg itineraries. One caveat: the repository's test matrix only exercises the one-way path, so the return leg has no CI coverage. Is scraping Skyscanner legal?Scraping public data is generally legal, but it depends on where you are, the site's terms, and how you use the data. Scrape only public result pages, avoid account and checkout flows, and review Skyscanner's terms before collecting at scale. Does this extract every Skyscanner result?It returns far more than the cards on screen, in one request and with no scroll loop. Whether that is literally every itinerary Skyscanner found is not something the response tells you, so treat the count as "the full XHR page", not a guarantee. If a run returns fewer flights than the page advertises, the search was still polling, so raise `rendering_wait`. With the common questions settled, here's the whole workflow pulled together. ## Summary Scraping Skyscanner comes down to a repeatable sequence. You choose a route, date, cabin, and trip type, then construct a deep-link URL instead of driving the search form. You fetch that URL with the Scrapfly Web Scraping API and render it. Wait for the "results sorted by" label so the search finishes polling before the render ends. Instead of parsing cards, you read the `web-unified-search` XHR response the page fetched for itself, and reshape its itineraries into flat records. That is what keeps the scraper short. Keep the scope honest. This path covers the US market only, and other markets need their own testing since the proxy country shapes the provider mix and prices. Multi-city searches need a Cloud Browser. Within that scope, your Skyscanner scraper turns a single rendered request into the complete structured result set for a search. From there you can scrape Skyscanner flight prices on a schedule and monitor them over time. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents- [Key Takeaways](#key-takeaways) - [Why Scrape Skyscanner Flight Data?](#why-scrape-skyscanner-flight-data) - [What Skyscanner Flight Data Can You Scrape?](#what-skyscanner-flight-data-can-you-scrape) - [How Skyscanner Flight Result URLs Work](#how-skyscanner-flight-result-urls-work) - [Project Setup](#project-setup) - [Scrape a Skyscanner Flight Results Page](#scrape-a-skyscanner-flight-results-page) - [Build a Skyscanner Search URL](#build-a-skyscanner-search-url) - [Configure Scrapfly for JavaScript Rendering and ASP](#configure-scrapfly-for-javascript-rendering-and-asp) - [Wait for the Search to Settle](#wait-for-the-search-to-settle) - [Capture the Flight Search XHR](#capture-the-flight-search-xhr) - [Parse Skyscanner Flight Data](#parse-skyscanner-flight-data) - [Extract Itineraries from the Search JSON](#extract-itineraries-from-the-search-json) - [Normalize Prices, Times, Stops, and Carriers](#normalize-prices-times-stops-and-carriers) - [Run the Scraper End to End](#run-the-scraper-end-to-end) - [Handle Skyscanner Blocking and Geo-Targeted Prices](#handle-skyscanner-blocking-and-geo-targeted-prices) - [Detect PerimeterX Captcha Redirects](#detect-perimeterx-captcha-redirects) - [Align Proxy Country, Market, Currency, and Locale](#align-proxy-country-market-currency-and-locale) - [When to Use Cloud Browser Instead](#when-to-use-cloud-browser-instead) - [Scrape Skyscanner with Scrapfly](#scrape-skyscanner-with-scrapfly) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ python scrapeguide ### How to Scrape Google Flights Data in 2026 Learn how to scrape Google Flights with Python and Scrapfly: extract prices, airlines, routes, stops, CO2 emissions, and... ](https://scrapfly.io/blog/posts/how-to-scrape-google-flights) [ python playwright ### How to Scrape Emirates Flights with Python (2026) Learn how to scrape Emirates flight search results with Python in 2026, including fares, fare brands, routes, schedules,... ](https://scrapfly.io/blog/posts/how-to-scrape-emirates-flights) [ python scrapeguide ### How to Scrape Google Search Results in 2026 In this scrape guide we'll be taking a look at how to scrape Google Search - the biggest index of public web. We'll cov... ](https://scrapfly.io/blog/posts/how-to-scrape-google) Bypass anti-bot protection automatically, **1,000 free credits** [Start Free](https://scrapfly.io/register)