[Blog](https://scrapfly.io/blog) / [blocking](https://scrapfly.io/blog/tag/blocking) / [How to Scrape LoopNet Commercial Real Estate Listings](https://scrapfly.io/blog/posts/how-to-scrape-loopnet) # How to Scrape LoopNet Commercial Real Estate Listings by [Mayada Shaaban](https://scrapfly.io/blog/author/mayada-shaaban-90143e67) Oct 27, 2026 25 min read [\#blocking](https://scrapfly.io/blog/tag/blocking) [\#data-parsing](https://scrapfly.io/blog/tag/data-parsing) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet&text=How%20to%20Scrape%20LoopNet%20Commercial%20Real%20Estate%20Listings "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-loopnet) LoopNet is mostly known as the place where brokers advertise commercial properties, though for web scrapers it's also a big public dataset - offices, retail spaces, warehouses and land, usually with price, size, address and broker contact attached. Data like this is great for commercial real estate market research and lead generation! So, how can we collect it? LoopNet is also protected by Akamai, so if we request a search page with a regular HTTP client we'll just get a 403 response with an "Access Denied" page. To get around this we'll be using Python with ScrapFly SDK and its Unblocker (previously ASP) feature. Since all of the listing data is rendered on the server, we won't even need a headless browser! We'll start by taking a look at how to scrape LoopNet's search pages and their pagination, and then we'll scrape the listing pages themselves. Along the way we'll also see why the JSON-LD data embedded in search pages isn't enough and why we parse the HTML instead. Let's dive in! [How to Scrape Real Estate Property Data using PythonIntroduction to scraping real estate property data. What is it, why and how to scrape it? We'll also list dozens of popular scraping targets and common challenges.](https://scrapfly.io/blog/posts/how-to-scrape-real-estate-property-data-using-python) [**Latest LoopNet Scraper Code**github.com/scrapfly/scrapfly-scrapers/loopnet-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/loopnet-scraper) ## Key Takeaways - LoopNet is protected by Akamai, however ScrapFly's Unblocker (previously ASP) (`unblocker=True`) with `country="US"` is enough to get through - no headless browser needed. - search pages have JSON-LD data too, however some listings are missing from it so we scrape the placard HTML instead - and pagination is just the page number added to the URL (e.g. `/2/`). - auction placards are the odd ones out as they have no price on them (all 40 we scraped had `price: null`), meaning for the current bid we need to go to the auction listing page. - listing pages are much easier as most of the data is in the `RealEstateListing` JSON-LD object (the `additionalProperty` array goes into our `details` dictionary), though its `@type` can be either a list or a string so we check if `RealEstateListing` is in it rather than comparing directly. - LoopNet doesn't return a 404 for listings that were removed - we get a 200 and a redirect to `/listingnotfound//` instead, which is easy to spot in the final URL. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## What Commercial Real Estate Data Can LoopNet Expose? If we open up a LoopNet search page, like the one for commercial properties for sale in Savannah, GA, we can see that every result is displayed as a small card (LoopNet calls them placards). Each one shows the listing's name, address, price and a couple of building facts, which is plenty if we just want an overview of the market. Auction placards are a bit different though, as they don't display a price at all. Clicking on any of these placards takes us to the listing page, which holds everything else: the full property info, a photo gallery and the broker's contact. Most of this is also available as JSON-LD data in the page source - so if we need more than the basics, scraping listing pages is the way to go. For example, here's what our scraper got from the land parcel at 1410 Dean Forest Rd in Savannah, GA: json ```json { "id": "41166496", "price": null, "property_type": "Land", "details": { "Sale Type": "Investment or Owner User", "No. Lots": "1", "Property Type": "Land", "Property Subtype": "Commercial", "Proposed Use": "Commercial", "Total Lot Size": "3.00 AC", "Walk Score": "10" } } ``` Compare that to the retail building at 2312 Abercorn St in Savannah, which had 15 of these properties (cap rate, building class, zoning etc.) and 45 images. That's why our scraper keeps them in an open `details` dictionary rather than a fixed set of columns. [How to Scrape Zillow Real Estate Property Data in PythonTutorial on how to scrape Zillow.com sale and rent property data, using Python and how to avoid blocking to scrape at scale.](https://scrapfly.io/blog/posts/how-to-scrape-zillow) Next, let's take a look at how LoopNet delivers all of this data, as the placards and the JSON-LD don't even agree on how many listings a search page has! ## How Does LoopNet Deliver Search and Listing Data? If we take a look at the page source of a LoopNet search page, we'll find each listing twice: once as an `article.placard-card` HTML element and once more in a JSON-LD [structured data](https://scrapfly.io/blog/posts/how-to-scrape-hidden-web-data) script, which websites use to describe their content to search engines. Listing pages, on the other hand, don't have any placards and keep the whole listing in a single `RealEstateListing` JSON-LD object instead. ### Why Search Parsing Reads Placards Instead of JSON-LD JSON-LD is usually the easiest data to scrape, so why not use it for search pages too? To find out, we counted both on a nationwide auction search on October 23, 2026 and got 40 placards but only 33 JSON-LD records: text ```text auction search: https://www.loopnet.com/search/commercial-real-estate/usa/auctions/ article.placard-card elements: 40 schema-jsonld itemListElement: 33 in placards, missing from JSON-LD: 7 in JSON-LD, missing from placards: 0 missed by the schema block: /Listing/123-Unit-South-Dallas-SFR-Portfolio/42187391/ /Listing/160-Acres-of-AG-Zoned-Land/39908785/ /Listing/CBD-Mixed-Use-MF-Conversion-Potential/42102165/ /Listing/Debt-Free-Office-Condos-Columbia-MD/41564695/ /Listing/REO-Medical-Office-w-Revenue-Upside/42279884/ /Listing/Student-Housing-Near-Eastern-New-Mexico/41445211/ /Listing/Vacant-Professional-Office-Campus-50K-SF/41902550/ ``` As you can see, every listing from the JSON-LD data also has a placard, but 7 of the placards aren't in the JSON-LD at all - and they're real listings too! What these 7 have in common is that their titles are marketing headlines like "Student Housing Near Eastern New Mexico" rather than street addresses. That's why our scraper goes with the placards - this way we get all 40 listings, not just 33! On top of that, each placard has a few `data-automation-*` attributes with extra data we won't find in the JSON-LD. One of these is the listing type, which tells us whether a listing is a sublease or an auction for example. ### How Does LoopNet Listing JSON-LD Use `RealEstateListing`? Listing pages have two JSON-LD scripts - a `BreadcrumbList` for the page navigation and the listing data itself. The tricky part here is the listing's `@type` field, which isn't always the same. For example, our retail listing returned `["RealEstateListing", "Product"]` while the land parcel returned just `"RealEstateListing"` as a string. In other words, if we were to check `@type == "RealEstateListing"`, our scraper would find the land parcel and silently skip the retail listing! To avoid this, we turn `@type` into a list first and then check whether `RealEstateListing` is in it. Since both page types are plain HTML, we won't need much to scrape them - let's set up our project. ## Project Setup In this tutorial we'll be using Python with a few community packages: - scrapfly-sdk - Scrapfly's Python SDK which will let us fetch LoopNet pages past Akamai. - [parsel](https://scrapfly.io/blog/posts/guide-to-html-parsing-with-parsel-python) - HTML parsing library which will help us parse the placards and JSON-LD scripts. It already comes with the SDK's `all` extra, so there's no need to install it separately. - loguru - a pretty logging library that'll help us keep track of what's going on. These packages can be installed via `pip` command: bash ```bash pip install "scrapfly-sdk[all]>=0.12" loguru ``` Later on we'll also be writing a few tests for our scraper, which need four more packages: bash ```bash pip install pytest pytest-asyncio pytest-rerunfailures cerberus ``` Alternatively, if you'd rather skip ahead and run the finished scraper, you can grab it from our scrapers repository and install it with [Poetry](https://python-poetry.org/docs/#installation) (Python 3.10 or newer is required): bash ```bash git clone https://github.com/scrapfly/scrapfly-scrapers.git cd scrapfly-scrapers/loopnet-scraper poetry install export SCRAPFLY_KEY="your key from https://scrapfly.io/dashboard" poetry run python run.py ``` With everything installed, let's start by taking a look at how to fetch LoopNet pages. ## How Do You Fetch LoopNet with Scrapfly ScrapeConfig? LoopNet is very protective of its listing data. If we try to request its search pages with a regular HTTP client we'll be greeted with a 403 response and an "Access Denied" page from Akamai - an anti-scraping service that analyzes the scraper's IP address, HTTP connection and javascript environment. Instead of fortifying all of these ourselves, let's take advantage of ScrapFly's web scraping API, which offers several powerful features: - Unblocker - ScrapFly automatically detects and bypasses anti-scraping services like Akamai. - Proxies from many countries - which helps to access geographically locked content. - Javascript Rendering - using a pool of real browsers, we can render dynamic javascript pages. For our LoopNet scraper we'll only need the first two, which can be enabled via `unblocker=True` (the newer name of the SDK's `asp` option) and `country="US"`: python ```python import os from scrapfly import ScrapeConfig, ScrapflyClient SCRAPFLY = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"]) BASE_CONFIG = { "unblocker": True, # ^ enable the Unblocker (previously Anti Scraping Protection) "country": "US", # ^ use proxies from the US } result = SCRAPFLY.scrape( ScrapeConfig( "https://www.loopnet.com/search/commercial-real-estate/savannah-ga/for-sale/", **BASE_CONFIG, ) ) print(result.upstream_status_code, len(result.content)) ``` Here, `upstream_status_code` is the status code we got from LoopNet itself, not from ScrapFly's API. To make sure this config works for the whole website, we used it to fetch a few search and listing pages and counted the placards and JSON-LD blocks we found on each one: text ```text savannah for-sale upstream=200 placards=40 ld_blocks=1 savannah for-sale page 2 upstream=200 placards=40 ld_blocks=1 savannah for-lease upstream=200 placards=40 ld_blocks=1 usa auctions upstream=200 placards=40 ld_blocks=1 listing 41217887 upstream=200 placards=0 ld_blocks=2 listing 41166496 upstream=200 placards=0 ld_blocks=2 same URL with a plain client, no ASP: curl_status: 403 server: AkamaiGHost body: Access Denied ``` All of them worked, while a plain curl request to the same search page got blocked even with a common browser user agent. We'll be using this `BASE_CONFIG` for every page in this tutorial. [Akamai Bot Manager: Understanding \_abck Cookies and Sensor DataDeep technical guide to Akamai Bot Manager detection layers, \_abck and ak\_bmsc cookies, sensor data mechanics, JavaScript challenges, and practical scraping workflows using Scrapfly Unblocker (previously ASP).](https://scrapfly.io/blog/posts/akamai-bot-manager-understanding-abck-cookies-and-sensor-data) ### Why LoopNet Doesn't Need JavaScript Rendering Note that we're not enabling `render_js` in our config, as LoopNet's listing data comes with the initial HTML. We did try it on a listing page, and all it did was make the HTML bigger (374 KB instead of 224 KB) - the JSON-LD data inside was exactly the same. So, let's move on to scraping LoopNet's search pages! ## How Do You Parse LoopNet Search Results with Python? Let's start our scraper by taking a look at LoopNet's search pages. If we open one up and inspect it with Chrome Dev tools (accessed via F12 key), we can see that every listing preview is an `article.placard-card` element. To parse these, all we need is a few CSS selectors and the placard's `data-automation-*` attributes: python ```python from typing import List from scrapfly import ScrapeApiResponse def parse_search(response: ScrapeApiResponse) -> List[dict]: sel = response.selector results = [] # every listing is an article.placard-card with the listing id in data-lk and the # location and type details in its data-automation-* attributes for card in sel.css("article.placard-card[data-lk]"): name = card.css(".building-name::text").get() street = card.css(".placard-address::text").get() or name # the card details are rows of spans (type and price, then size and lots or year); # the price is the span that starts with "$" (the others say e.g. "Price Upon Request") data_points = [dp.strip() for dp in card.css(".placard-data .data-row span:not(.data-sep)::text").getall()] data_points = [dp for dp in data_points if dp] price = next((dp for dp in data_points if dp.startswith("$")), None) results.append({ "id": card.attrib.get("data-lk"), "name": name.strip() if name else None, "url": card.css("a.placard-link::attr(href)").get(""), "address": street.strip() if street else None, "city": card.attrib.get("data-automation-city") or None, "state": card.attrib.get("data-automation-state") or None, "zip_code": card.attrib.get("data-automation-postal-code") or None, "price": price, "property_type": card.attrib.get("data-automation-property-type") or None, "listing_type": card.attrib.get("data-automation-listing-type") or None, "image": card.css("img.embla__slide__img::attr(src)").get(), "data_points": data_points, }) return results ``` Above, we define our `parse_search` function which given a search page response will go through every placard and extract listing info such as name, address, price and property type. One thing to note here is that LoopNet puts the listing's name in `.building-name` and the address in `.placard-address` - so when a card has no address, we use the name instead. If we run our parser on the Savannah for-sale search we'll get 40 listings (only four of them without a price) that look like this: json ```json { "id": "30627850", "name": "20ac Downtown Savannah, Zoned Mixed-Use", "url": "https://www.loopnet.com/Listing/925-1303-E-President-St-Savannah-GA/30627850/", "address": "925-1303 E President St, Savannah, GA 31404", "city": "Savannah", "state": "GA", "zip_code": "31404", "price": null, "property_type": "Land", "listing_type": "LandForSale", "image": "https://images.loopnet.co.uk/i2/VsTEfNLuV9BjR9o7SljYFXocOyjITR4oS2D4Y6yY5ug/115/image.jpg", "data_points": [ "Commercial Land", "Price Upon Request", "19.77 AC", "1 Lot" ] } ``` That's a good start, though two of these fields need a closer look - the listing type and the price. ### How Do You Read LoopNet Listing Types? If we take a look at the `listing_type` field, we can see that it's not just a simple sale, lease or auction label. In the three searches we fetched on October 23, 2026 we found 12 different values: text ```text for-sale search: LandForSale, PropertyForSale, ConsolidatedCondosForSale for-lease search: PropertyDirectSpaceForLease, CommercialSublease, ShoppingCenterDirectSpaceForLease, LandDirectForLease auctions search: PropertyForAuction, LandForAuction, CondoForAuction, CommercialPortfolioForAuction, PropertyLoanForAuction ``` As you can see, the end of each value tells us what the listing is for. However, we can't just look for `ForLease` either, as `CommercialSublease` doesn't contain it. So instead of keeping a list of exact values, we match on the suffix (e.g. `Lease`). ### How Do You Handle LoopNet Sale, Lease, and Auction Prices? The price field is a bit trickier as LoopNet formats it differently for each listing type and auction placards don't include it at all: text ```text sale placard: "$5,800,000" formatted total null "Price Upon Request" lease placard: "$10.00 SF/YR" rate, per square foot per year auction placard: null 40 of 40 auction records ``` So, we keep `price` as a raw string together with the `listing_type` which tells us how to read it. For auctions though, the placard data points have no `$` value so to get the bid we need to scrape the auction listing page itself, where it's stored in the JSON-LD `offers` object rather than the `additionalProperty` array we used for the other details: text ```text auction listing 42186416 (85 Harristown Rd, Glen Rock, NJ), offers object: price: "$2,900,000" priceCurrency: "USD" validThrough: 2026-10-28T16:00:00+00:00 additionalProperty: Current Bid "--", Bid Increment "$500,000", Reserve Status "Reserve Not Met" top-level additionalProperty: ['Floor Size', 'Occupancy Rate', 'Amenities', 'Demographics'] ``` Here, the `details` dictionary ends up with only a few building facts while the bid info is in `offers`. When we checked it on October 23, 2026 this listing had no bids placed yet so `Current Bid` is just `--` and `price` is the $2,900,000 starting bid displayed on the page, though the current bid will go up with each new bid until the auction closes at the `validThrough` date. Unfortunately, `offers` isn't consistent across listings either - the retail listing has it as a list with one dictionary in it while the land parcel has no `offers` field at all. So, to parse the bid info we need to handle all three of these cases: python ```python offers = listing_ld.get("offers") or {} # retail listings wrap offers in a list if isinstance(offers, list): offers = offers[0] bid_info = {} for prop in offers.get("additionalProperty") or []: bid_info[prop.get("name")] = prop.get("value") current_bid = bid_info.get("Current Bid") # "--" until someone bids ``` Note that we have to check the type with `isinstance` here, as calling `offers[0]` on a dictionary like in our auction example would raise `KeyError: 0`. Now we have a working scraper for a single search page - next, let's take a look at how to get the rest of the results through pagination! Scrapfly #### Need to bypass anti-bot protection? Scrapfly's Unblocker handles Cloudflare, DataDome, and more — automatically. [Try Free →](https://scrapfly.io/register)## How Do You Paginate LoopNet Search Results? To scrape the rest of the search results we need to take a look at LoopNet's pagination. If we click on the second page of results, we can see that LoopNet simply adds the page number to the end of the URL - `/savannah-ga/for-sale/` becomes `/savannah-ga/for-sale/2/`. So, all we need to know is how many pages there are, which we can find in the pager at the bottom of the page: python ```python from loguru import logger as log from scrapfly import ScrapflyScrapeError def _get_total_pages(response: ScrapeApiResponse) -> int: pages = [ int(pg) for pg in response.selector.css( "nav.pagination a.pagination__page::attr(data-page)" ).getall() if pg.isdigit() ] return max(pages) if pages else 1 async def scrape_search(search_url: str, max_pages: int = 3) -> dict: first_page = await SCRAPFLY.async_scrape(ScrapeConfig(search_url, **BASE_CONFIG)) results = parse_search(first_page) total_pages = _get_total_pages(first_page) pages_to_scrape = min(total_pages, max_pages) base_url = search_url.rstrip("/") to_scrape = [ ScrapeConfig(f"{base_url}/{page}/", **BASE_CONFIG) for page in range(2, pages_to_scrape + 1) ] async for result in SCRAPFLY.concurrent_scrape(to_scrape): if isinstance(result, ScrapflyScrapeError): log.error("scrape failed, skipping page") continue results.extend(parse_search(result)) return {"total_pages": total_pages, "data": results} ``` Here, we're using a common paging scraping idiom: we scrape the first page to find the total amount of pages and then collect the remaining pages concurrently. Note that LoopNet's pager doesn't show every page number - for the Savannah for-lease search it links to `2, 3, 4, 5, 9` - though the last page was always there in our tests, so taking the highest number works just fine. To make sure this works, we tried it on three different searches on October 23, 2026 and logged what the pager gave us: text ```text savannah for-sale: pager [2,3,4,5] total_pages=5 savannah for-lease: pager [2,3,4,5,9] total_pages=9 usa auctions: pager [2,3,4] total_pages=4 ``` The for-lease search, for example, jumps from page 5 straight to 9, and `max()` still gave us 9 pages. Then, we compared the number of records on a few of the for-sale pages: text ```text page 1 records: 40 page 2 records: 40 overlap: 0 page 5 records: 39 (last page) page 6 records: 40 resolved URL: .../savannah-ga/for-sale/ ``` Pages 1 and 2 had 40 different listings each, and the last page had 39. Page 6 is where things get weird though - it doesn't exist, but LoopNet doesn't return an error. Instead, we get a 200 response with the first page all over again! That's why we read `total_pages` from the pager first, as a scraper that keeps going until it gets an empty page would never stop. [How to Scrape Infinite Scroll, Load More & Paginated PagesLearn to scrape infinite scroll, load more, and paginated pages in 2026 by reverse-engineering the request first and using a browser only when you must.](https://scrapfly.io/blog/posts/how-to-scrape-infinite-scroll-load-more-and-paginated-pages) A full pager gives you every listing URL in a market. The detail page is where the rest of each property's data lives. ## How Do You Parse a LoopNet RealEstateListing Page? Next, let's scrape the listing pages. If we view the page source of one, we'll find two JSON-LD scripts in it - a `BreadcrumbList` and the `RealEstateListing` we're after. So before parsing anything, we need to pick the right script: python ```python import json import re def _parse_listing_ld(sel) -> dict: for script in sel.css('script[type="application/ld+json"]'): try: data = json.loads(script.css("::text").get() or "{}") except json.JSONDecodeError: continue types = data.get("@type") if isinstance(types, str): types = [types] if isinstance(types, list) and "RealEstateListing" in types: return data return {} ``` This function loads each JSON-LD script into a Python dictionary and returns the one with `RealEstateListing` in its `@type`. If you remember from earlier, our retail listing had `@type` as a list while the land parcel had it as a plain string - that's why we wrap strings into a list before checking. With the JSON-LD data found, we can parse the listing itself: python ```python def parse_listing(response: ScrapeApiResponse) -> dict: sel = response.selector listing_ld = _parse_listing_ld(sel) details = {} for prop in listing_ld.get("additionalProperty", []) or []: name, value = prop.get("name"), prop.get("value") if isinstance(value, list): value = ", ".join(str(v) for v in value) if name and value is not None: details[name] = str(value) address = (listing_ld.get("contentLocation") or {}).get("address", {}) or {} listing_id_match = re.search(r"/(\d+)/?$", response.context["url"]) return { "id": listing_id_match.group(1) if listing_id_match else None, "name": listing_ld.get("name", ""), "url": listing_ld.get("url", response.context["url"]), "description": listing_ld.get("description"), "price": details.get("Price"), "address": address.get("streetAddress"), "city": address.get("addressLocality"), "state": address.get("addressRegion"), "zip_code": address.get("postalCode"), "property_type": details.get("Property Type"), "property_subtype": details.get("Property Subtype"), "details": details, **parse_media_and_broker(sel, listing_ld), } ``` Most of the listing data is in the `additionalProperty` array, so we're turning it into a `details` dictionary where each property name is a key - like `Cap Rate` or `Building Size`. One thing to note here is that we take the `price` from `details` rather than the `offers` object - the land listing didn't have `offers` at all, while both of our listings had `additionalProperty`. Here's a summary of what it got from our retail and land listings: text ```text 2312 Abercorn St (41217887) name: 2312 Abercorn St price: $850,000 type: Retail / Storefront Retail/Residential details: 15 keys, including Cap Rate 5.53%, Gross Leasable Area 2,800 SF, Year Built 1990, Building Class C 1410 Dean Forest Rd (41166496) name: 1410 Dean Forest Rd - Tapestry Park Commerical ... price: None type: Land / Commercial details: 7 keys: Sale Type, No. Lots, Property Type, Property Subtype, Proposed Use, Total Lot Size, Walk Score ``` As you can see, these two only have four `details` keys in common. That's why we didn't go with fixed columns here - every property type comes with its own set of details! ### How Do You Extract LoopNet Broker, Images, and Video? Next up, the broker and the photos, which `parse_listing()` adds to its record through the `parse_media_and_broker()` call in its return value. We can get the broker's name and company from the same JSON-LD data, though the phone number is only shown in the HTML. The same goes for images, which we can find in the listing's photo carousel: python ```python def parse_media_and_broker(sel, listing_ld: dict) -> dict: providers = listing_ld.get("provider") or [] agent = providers[0] if providers else {} broker = None if agent: phone = sel.css("a.number .phone-number::text").get() broker = { "name": agent.get("name"), "company": (agent.get("memberOf") or {}).get("name"), "phone": phone.strip() if phone else None, "profile_url": agent.get("@id"), } images = [] for img_url in sel.css( "section.carousel-wrapper img::attr(src), " "section.carousel-wrapper img::attr(lazy-src)" ).getall(): if img_url and "images1.loopnet.com" in img_url and img_url not in images: images.append(img_url) videos = listing_ld.get("video") or [] return { "broker": broker, "images": images, "video_url": videos[0].get("embedUrl") if videos else None, } ``` Here, note that we're reading both the `src` and `lazy-src` attributes of the carousel images. LoopNet only loads the first image or two right away and lazy-loads the rest - on our retail listing, 1 URL was in `src` and 44 in `lazy-src`, so with `src` alone we'd only get a single photo! We're also only keeping images hosted on `images1.loopnet.com`, as the carousel contains icons and tracking pixels from other hosts too. Here's what we got for both listings: text ```text 41217887 broker: Beth Vantosh, Vantosh Realty Group, +1 912-663-3392 images: 45 video: None 41166496 broker: Sarah Roberts, Roberts Properties, Inc., +1 912-737-4827 images: 18 video: loopnet.com/matterportexterior/?t=... ``` ### How Do You Catch Dead LoopNet Listing URLs? Finally, what happens when a listing gets removed? If we request a listing that doesn't exist, LoopNet doesn't return a 404 like we'd expect. Instead, we get redirected to a "This listing cannot be found." page with a 200 status code, which has no `RealEstateListing` data in it: text ```text requested: /Listing/does-not-exist-xyz/99999999/ upstream: 200 body: 82,047 bytes ld+json blocks: 0 resolved: https://www.loopnet.com/listingnotfound/99999999/ parse_listing() -> name: "" price: None details: {} images: 0 ``` So, to catch these we can check whether `response.context["url"]` contains the `/listingnotfound/` path before saving the record - and an empty `details` dictionary is a good second check. One more thing - LoopNet reuses listing IDs! So the ID we saved for a Savannah listing last month might be a totally different property in another state today, which means saved IDs need to be scraped again before we can trust them. Because of this, testing our scraper takes a slightly different approach - let's take a look at it next. ## How Do You Validate a LoopNet Scraper Against Real Records? To test our scraper, we can't really rely on exact values as LoopNet listings change constantly - the retail listing's $850,000 price could be different next week! So instead, let's validate the structure of our results with `cerberus`: python ```python import pytest from cerberus import Validator import loopnet search_schema = { "id": {"type": "string", "nullable": True}, "name": {"type": "string", "nullable": True}, "url": {"type": "string"}, "address": {"type": "string", "nullable": True}, "city": {"type": "string", "nullable": True}, "state": {"type": "string", "nullable": True}, "zip_code": {"type": "string", "nullable": True}, "image": {"type": "string", "nullable": True}, "price": {"type": "string", "nullable": True}, "property_type": {"type": "string", "nullable": True}, "listing_type": {"type": "string", "nullable": True}, "data_points": {"type": "list", "schema": {"type": "string"}}, } @pytest.mark.asyncio @pytest.mark.flaky(reruns=3, reruns_delay=30) async def test_search_scraping(): search_results = await loopnet.scrape_search( search_url="https://www.loopnet.com/search/commercial-real-estate/savannah-ga/for-sale/", max_pages=1, ) validator = Validator(search_schema, allow_unknown=True) for item in search_results["data"]: assert validator.validate(item), validator.errors assert search_results["total_pages"] >= 1 assert len(search_results["data"]) >= 10 ``` Almost every field is `nullable`, which reflects what the live pages return. For example, auction records come without a price, and we've also seen lease cards without a property type and placards without a ZIP code. text ```text poetry run pytest test.py test.py::test_listing_scraping PASSED test.py::test_search_scraping PASSED 2 passed ``` The `flaky` decorator earns its place here. If LoopNet's listings change in the middle of a test run, the test is simply retried after 30 seconds. Note that there's no Akamai code anywhere in our scraper or tests - ScrapFly takes care of all of it for us. ## How Does Scrapfly Handle LoopNet's Akamai Protection? The Akamai layer that answers a plain request with a 403 is what `unblocker=True` clears, with no header or TLS tuning of your own. ScrapFly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) is a single HTTP endpoint for collecting web data at scale, with a **98% success rate** on the hardest public pages and proxies across **190+ countries**. - [Web Unblocker](https://scrapfly.io/products/unblocker) (previously ASP) - handles 25+ anti-bot systems, including Cloudflare, DataDome, PerimeterX and Akamai. - [Smart proxy rotation](https://scrapfly.io/docs/scrape-api/proxy) - residential and datacenter pools with country and ASN level geo-targeting. - [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - render SPAs and dynamic pages through real cloud browsers. - [Browser automation scenarios](https://scrapfly.io/docs/scrape-api/javascript-scenario) - scroll, click, fill forms, and wait for elements without managing a browser fleet. - [Format conversion](https://scrapfly.io/docs/scrape-api/getting-started#api_param_format) - return pages as HTML, JSON, clean text, or LLM ready Markdown. - [Session management](https://scrapfly.io/docs/scrape-api/session) - keep cookies, headers, and IPs consistent across multi step flows. - [Smart caching](https://scrapfly.io/docs/scrape-api/getting-started#api_param_cache) - cache successful responses to cut cost on repeat scraping jobs. - [Python](https://scrapfly.io/docs/sdk/python), [TypeScript](https://scrapfly.io/docs/sdk/typescript), [Scrapy](https://scrapfly.io/docs/sdk/scrapy), and [no-code integrations](https://scrapfly.io/docs/integration/getting-started) including [Make](https://scrapfly.io/integration/make), [n8n](https://scrapfly.io/integration/n8n), [Zapier](https://scrapfly.io/integration/zapier), [LangChain](https://scrapfly.io/integration/langchain), and [LlamaIndex](https://scrapfly.io/integration/llamaindex). For LoopNet that comes down to a two-key `BASE_CONFIG` in place of your own fingerprinting, browser pool, and residential proxy rotation. ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. [Try Web Scraping API](https://scrapfly.io/docs/scrape-api/getting-started) ## FAQ Does a LoopNet Scraper Need JavaScript Rendering?Not for anything in this tutorial. LoopNet sends its listing data with the initial HTML. For more on when a real browser is needed, see our [scraping with browsers](https://scrapfly.io/blog/posts/scraping-using-browsers) guide. Can the Same LoopNet Parser Handle For-Lease Listings?Usually, yes. LoopNet simply uses the same placards for leases, only with rates like "$10.00 SF/YR" in place of a total price. That being said, we only checked lease search pages, so attention should be paid to the `details` keys of lease listing pages. Does Scraping LoopNet Replace CoStar Subscription Data?No, CoStar keeps its analytics and off-market data for paying subscribers. For more real estate sources, see our [real estate scraping](https://scrapfly.io/blog/posts/how-to-scrape-real-estate-property-data-using-python) guide. ## Summary In this tutorial we built a LoopNet.com scraper which is capable of scraping commercial real estate search results across all of their pages, as well as the listing pages themselves for property details like price, building size, images and broker contacts. We started by parsing the search placards rather than the JSON-LD data, as on the nationwide auction search the JSON-LD only had 33 of the 40 listings. Then, we took a look at the listing pages and their `RealEstateListing` JSON-LD data. Finally, we took a look at what LoopNet does with search pages that don't exist and with removed listings. For this we used Python with ScrapFly's SDK and parsel, and to get past Akamai we used ScrapFly's Unblocker with US proxies - no headless browser needed. For more on ScrapFly see our documentation and try it out for free! Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents - [Key Takeaways](#key-takeaways) - [What Commercial Real Estate Data Can LoopNet Expose?](#what-commercial-real-estate-data-can-loopnet-expose) - [How Does LoopNet Deliver Search and Listing Data?](#how-does-loopnet-deliver-search-and-listing-data) - [Why Search Parsing Reads Placards Instead of JSON-LD](#why-search-parsing-reads-placards-instead-of-json-ld) - [How Does LoopNet Listing JSON-LD Use RealEstateListing?](#how-does-loopnet-listing-json-ld-use-realestatelisting) - [Project Setup](#project-setup) - [How Do You Fetch LoopNet with Scrapfly ScrapeConfig?](#how-do-you-fetch-loopnet-with-scrapfly-scrapeconfig) - [Why LoopNet Doesn't Need JavaScript Rendering](#why-loopnet-doesn-t-need-javascript-rendering) - [How Do You Parse LoopNet Search Results with Python?](#how-do-you-parse-loopnet-search-results-with-python) - [How Do You Read LoopNet Listing Types?](#how-do-you-read-loopnet-listing-types) - [How Do You Handle LoopNet Sale, Lease, and Auction Prices?](#how-do-you-handle-loopnet-sale-lease-and-auction-prices) - [How Do You Paginate LoopNet Search Results?](#how-do-you-paginate-loopnet-search-results) - [How Do You Parse a LoopNet RealEstateListing Page?](#how-do-you-parse-a-loopnet-realestatelisting-page) - [How Do You Extract LoopNet Broker, Images, and Video?](#how-do-you-extract-loopnet-broker-images-and-video) - [How Do You Catch Dead LoopNet Listing URLs?](#how-do-you-catch-dead-loopnet-listing-urls) - [How Do You Validate a LoopNet Scraper Against Real Records?](#how-do-you-validate-a-loopnet-scraper-against-real-records) - [How Does Scrapfly Handle LoopNet's Akamai Protection?](#how-does-scrapfly-handle-loopnet-s-akamai-protection) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ blocking nodejs ### Web Scraping With Node-Unblocker Tutorial on using Node-Unblocker - a nodejs library - to avoid blocking while web scraping and using it to optimize web ... ](https://scrapfly.io/blog/posts/web-scraping-with-node-unblocker) [ python scrapeguide ### How to Scrape Real Estate Property Data using Python Introduction to scraping real estate property data. What is it, why and how to scrape it? We'll also list dozens of popu... ](https://scrapfly.io/blog/posts/how-to-scrape-real-estate-property-data-using-python) [ python scrapeguide ### 6 Best Web Scraping Tools for Real Estate in 2026 Which real estate scraping tool returns the output you are missing? Compare 6 by job in 2026: page HTML, structured JSON... ](https://scrapfly.io/blog/posts/best-web-scraping-tools-for-real-estate) Bypass anti-bot protection automatically, **1,000 free credits** [Start Free](https://scrapfly.io/register)