How to Scrape Immobilienscout24.de Real Estate Data

article feature image

Immobilienscout24.de is a popular website for real estate ads in Germany, featuring millions of property listings across the country. However, due to its high level of protection, scraping immobilienscout24.de can be challenging.

In this article, we'll explain how to scrape and avoid immobilienscout24.de web scraping blocking. We'll also go over a step-by-step guide on how to scrape immobilienscout24.de real estate listings from search and property pages. Let's dive in!

Latest immobilienscout24.de Scraper Code

https://github.com/scrapfly/scrapfly-scrapers/

Why Scrape Immobilienscout24.de?

Manually navigating property listings from immobilienscout24.de can be a time-consuming task. Scraping immobilienscout24.de automates this process, allowing for getting thousands of property listing data in no time.

Web scraping immobilienscout24.de opens the door to a wide range of investing opportunities. Investors and buyers can identify properties with high potential returns by filtering through a wide set of property listings, helping them make strategic investment decisions.

Immobilienscout24.de scraping enables buyers to tailor their property search based on specific criteria by allowing them to look for properties in a particular location, with specific facilities or within a certain price range.

Project Setup

In this guide, we'll be scraping immobilienscout24.de using a few Python libraries:

  • scrapfly-sdk: A Python SDK for ScrapFly web scraping API, which allows for web scraping at scale without getting blocked.
  • parsel: For parsing the HTML using XPath and CSS selectors.

We'll also use asyncio to run our code asynchronously, allowing for increasing our web scraping speed.

Although asyncio is pre-installed in Python, install the other libraries using the following pip command:

pip install scrapfly-sdk parsel

How to Avoid Immobilienscout24.de Scraping Blocking?

Immobilienscout24.de uses anti-scraping technologies that can detect and block bots from accessing the website. For example, let's try to scrape it using a simple Playwright headless browser:

from playwright.sync_api import sync_playwright

with sync_playwright() as playwight:

    # Lanuch a chrome browser
    browser = playwight.chromium.launch(headless=False)
    page = browser.new_page()
    # Go to immobilienscout24.de
    page.goto("https://www.immobilienscout24.de/expose/147036156#/")
    # Take a screenshot
    page.screenshot(path="screenshot.png")

By running this code, we can see that the website detected us as web scrapers, requiring us to solve a CAPTCHA challenge before proceeding to the web page:

web scraping detection on immobilienscout24.de
Web scraping detection on immobilienscout24.de

To bypass immobilienscout24.de web scraping blocking, we'll use ScrapFly. A web scraping API that allows for scraping at scale by providing:

Here is how we can bypass any website blocking using the ScrapFly asp feature with the Python SDK:

from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse

scrapfly = ScrapflyClient(key="Your ScrapFly API key")

api_response: ScrapeApiResponse = scrapfly.scrape(
    ScrapeConfig(
        url="https://www.immobilienscout24.de/expose/147036156#/",
        # Cloud headless browser similar to Playwright
        render_js=True,
        # Bypass anti-scraping protection
        asp=True,
        # Set the geographical location to Germany
        country="DE",
    )
)
# Print the website's status code
print(api_response.upstream_status_code)
"200"

Now that we can scrape immobilienscout24.de without getting blocked using ScrapFly, let's use it to scrape some real estate data!

How to Scrape Immobilienscout24.de Propoerty Pages?

In this section, we'll scrape property pages from immobilienscout24.de. Go to any property page on the website and you will get a page similar to this:

property page on immobilienscout24.de
Property page on immobilienscout24.de

First, we'll start by parsing the HTML using XPath and CSS selectors to get the property data from the web page:

def parse_property_page(response: ScrapeApiResponse):
    """parse property listing data from property pages"""
    
    def strip_text(text):
        """remove extra spaces while handling None values"""
        if text != None:
            text = text.strip()
        return text
    
    selector = response.selector
    property_link = selector.xpath("//link[@rel='canonical']").attrib["href"]
    title = strip_text(selector.xpath("//h1[@id='expose-title']/text()").get())
    description = selector.xpath("//meta[@name='description']").attrib["content"]
    address = selector.xpath("//div[@class='address-block']/div/span[2]/text()").get()
    floors_number = strip_text(selector.xpath("//dd[contains(@class, 'etage')]/text()").get())
    living_space = strip_text(selector.xpath("//dd[contains(@class, 'wohnflaeche')]/text()").get())
    vacant_from = strip_text(selector.xpath("//dd[contains(@class, 'bezugsfrei')]/text()").get())
    number_of_rooms = strip_text(selector.xpath("//dd[contains(@class, 'zimmer')]/text()").get())
    garage = strip_text(selector.xpath("//dd[contains(@class, 'garage-stellplatz')]/text()").get())
    additional_sepcs = []
    for spec in selector.xpath("//div[contains(@class, 'criteriagroup boolean-listing')]//span[contains(@class, 'palm-hide')]"):
        additional_sepcs.append(spec.xpath("./text()").get())
    price_without_heating = strip_text(selector.xpath("//dd[contains(@class, 'kaltmiete')]/text()").get())
    price_per_meter = strip_text(selector.xpath("//dd[contains(@class, 'preism')]/text()").get())
    basic_rent = strip_text(selector.xpath("//div[contains(@class, 'kaltmiete')]/span/text()").get())
    additional_costs = strip_text(selector.xpath("//dd[contains(@class, 'nebenkosten')]/text()").extract()[1].replace("\n", ""))
    # additional_costs = selector.xpath("//dd[contains(@class, 'nebenkosten')]/text()").extract()
    heating_costs = strip_text(selector.xpath("//dd[contains(@class, 'heizkosten')]/text()").extract()[1].replace("\n", ""))
    # heating_costs = selector.xpath("//dd[contains(@class, 'heizkosten')]/text()").extract()
    total_rent = strip_text(selector.xpath("//dd[contains(@class, 'gesamtmiete')]/text()").get())
    deposit = strip_text(selector.xpath("//dd[contains(@class, 'ex-spacelink')]/div/text()").get())
    garage_parking_rent = selector.xpath("//dd[contains(@class, 'garagestellplatz')]/text()").get()
    if garage_parking_rent:
        garage_parking_rent = strip_text(garage_parking_rent)
    construction_year = strip_text(selector.xpath("//dd[contains(@class, 'baujahr')]/text()").get())
    energy_sources = strip_text(selector.xpath("//dd[contains(@class, 'wesentliche-energietraeger')]/text()").get())
    energy_certificate = strip_text(selector.xpath("//dd[@class='is24qa-energieausweis grid-item three-fifths']/text()").get())
    energy_certificate_type = strip_text(selector.xpath("//dd[contains(@class, 'energieausweis')]/text()").get())
    energy_certificate_date = strip_text(selector.xpath("//dd[contains(@class, 'baujahr-laut-energieausweis')]/text()").get())
    final_energy_requirement = strip_text(selector.xpath("//dd[contains(@class, 'endenergiebedarf')]/text()").get())
    property_images = []
    for image in selector.xpath("//div[@class='sp-slides']//div[contains(@class, 'sp-slide')]"):
        try:
            if image.xpath("./img").attrib["data-src"]:
                property_images.append(image.xpath("./img").attrib["data-src"].split("/ORIG")[0])
        except:
            pass
    video_available = bool(selector.xpath("//button[contains(@class, 'gallery-video')]/text()").get())
    internet_speed = selector.xpath("//a[contains(@class, 'mediaavailcheck')]/text()").get()
    internet_available = bool(internet_speed)
    agency_name = selector.xpath("//span[@data-qa='companyName']/text()").get()
    agency_address = ""
    for text in selector.xpath("//ul[li[span[@data-qa='companyName']]]/li[position() >= 3 and position() <= 4]/text()").getall():
        agency_address = agency_address + text

    # return the data into a JSON object
    data = {
        "id": int(property_link.split("/")[-1]),
        "title": title,
        "description": description,
        "address": address,
        "propertyLlink": property_link,
        "propertySepcs": {
            "floorsNumber": floors_number,
            "livingSpace": living_space,
            "livingSpaceUnit": "Square Meter",
            "vacantFrom": vacant_from,
            "numberOfRooms": int(number_of_rooms) if number_of_rooms is not None else None,
            "Garage/parking space": garage,
            "additionalSpecs": additional_sepcs,
            "internetAvailable": internet_available,
        },
        "price": {
                "priceWithoutHeadting": price_without_heating,
                "priceperMeter": price_per_meter,
                "additionalCosts": additional_costs,
                "heatingCosts": heating_costs,
                "totalRent": total_rent,
                "basisRent": basic_rent,
                "deposit": deposit,
                "garage/parkingRent": garage_parking_rent,
                "priceCurrency": price_without_heating.split(" ")[1],    
        },
        "building": {
            "constructionYear": int(construction_year) if construction_year is not None else None,
            "energySources": energy_sources,
            "energyCertificate": energy_certificate,
            "energyCertificateType": energy_certificate_type,
            "energyCertificateDate": int(energy_certificate_date) if energy_certificate_date is not None else None, 
            "finalEnergyRrequirement": final_energy_requirement,
        },
        "attachments": {
            "propertyImages": property_images,
            "videoAvailable": video_available,
        },
        "agencyName": agency_name,
        "agencyAddress": agency_address,
    }
    return data

This looks very complex but all we're doing is requesting the HTML page and then selecting parts of it using XPath selectors bunding all results as a final property
JSON dataset.

Introduction to XPath for HTML Parsing

For more on how HTML is parsed using XPath see our introduction tutorial which includes interactive examples!

Introduction to XPath for HTML Parsing

Next, we'll use this function with the rest of our web scraping logic to scrape property data from each page:

import asyncio
import json
from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse
from typing import Dict, List

scrapfly = ScrapflyClient(key="Your ScrapFly API key")

BASE_CONFIG = {
    # bypass web scraping blocking
    "asp": True,
    # set the proxy country to Germany
    "country": "de"
}

def parse_property_page(response: ScrapeApiResponse):
    # The rest of the parse_property_page function

async def scrape_properties(urls: List[str]) -> List[Dict]:
    """scrape listing data from immoscout24 proeprty pages"""
    # add the property pages in a scraping list
    to_scrape = [ScrapeConfig(url, **BASE_CONFIG) for url in urls]
    properties = []
    # scrape all property pages concurrently
    async for response in scrapfly.concurrent_scrape(to_scrape):
        data = parse_property_page(response)
        properties.append(data)
    print(f"scraped {len(properties)} property listings")
    return properties
Run the code
if __name__ == "__main__":
    proeprties_data = asyncio.run(scrape_properties(
        urls = [
            "https://www.immobilienscout24.de/expose/140046281#/",
            "https://www.immobilienscout24.de/expose/146053176#/",
            "https://www.immobilienscout24.de/expose/129312192#/",
        ]
    ))
    # print the result in JSON format
    print(json.dumps(proeprties_data, indent=2, ensure_ascii=False))

Here, we add all property page URLs to the scrape_properties function as a scraping list and scrape them concurrently. Then, we use the parsing function we created earlier to extract the data from each page response. Finally, we append the data into the properties array.

The result is an array containing the property data of each page:

Sample output
{
  "id": 146053176,
  "title": "GUDRUN12 | MODERNE WOHNKULTUR NEUHAUSEN",
  "description": "Etagenwohnung (Wohnung/Miete): 4 Zimmer - 151,2 qm - Gudrunstrasse 12, 80634 München, Neuhausen bei ImmobilienScout24 (Scout-ID: 146053176)",
  "address": "Neuhausen, 80634 München",
  "propertyLlink": "https://www.immobilienscout24.de/expose/146053176",
  "propertySepcs": {
    "floorsNumber": "5 von 5",
    "livingSpace": "151,19  m²",
    "livingSpaceUnit": "Square Meter",
    "vacantFrom": "ab sofort",
    "numberOfRooms": 4,
    "Garage/parking space": "1 Tiefgaragen-Stellplatz",
    "additionalSpecs": [
      "Balkon/ Terrasse",
      "Keller",
      "Personenaufzug",
      "Gäste-WC",
      "Stufenloser Zugang"
    ],
    "internetAvailable": true
  },
  "price": {
    "priceWithoutHeadting": "3.950 €",
    "priceperMeter": "26,13 €/m²",
    "additionalCosts": "290 €",
    "heatingCosts": "in Nebenkosten enthalten",
    "totalRent": "4.350 €",
    "basisRent": "3.950 €",
    "deposit": "11850",
    "garage/parkingRent": "110 €",
    "priceCurrency": "€"
  },
  "building": {
    "constructionYear": 2023,
    "energySources": "Erdwärme",
    "energyCertificate": "liegt vor",
    "energyCertificateType": "liegt vor",
    "energyCertificateDate": 2023,
    "finalEnergyRrequirement": "20 kWh/(m²*a)"
  },
  "attachments": {
    "propertyImages": [
      "https://pictures.immobilienscout24.de/listings/2d732742-1bb8-4d15-9ed5-30757c43be46-1662427378.jpg",
      "https://pictures.immobilienscout24.de/listings/25064676-22bc-41f2-9e98-0b839c4cf88a-1662427383.jpg",
      "https://pictures.immobilienscout24.de/listings/0a980a95-e35e-4b40-8cfc-f7a7c23f954b-1662427388.jpg",
      "https://pictures.immobilienscout24.de/listings/3cdf6d77-06ae-4368-9537-93287959bf01-1662427395.jpg",
      "https://pictures.immobilienscout24.de/listings/c334ffb0-1543-4eec-9209-a4f01771f78b-1662427403.jpg",
      "https://pictures.immobilienscout24.de/listings/ffb2fa95-9ce6-4f31-bf06-cc6389d0d0f0-1662427412.jpg",
      "https://pictures.immobilienscout24.de/listings/637f4761-a4d3-465d-8357-7a1a5d28c764-1662427419.jpg",
      "https://pictures.immobilienscout24.de/listings/e45960fa-bb1d-498c-979a-dfd2091c5b23-1662427428.jpg",
      "https://pictures.immobilienscout24.de/listings/145b8f2b-0470-488a-9787-07d9ba7d239d-1662427433.jpg",
      "https://pictures.immobilienscout24.de/listings/dc64348f-64ca-45c5-a084-f74a7dcc1d41-1662427443.jpg",
      "https://pictures.immobilienscout24.de/listings/c13ac086-e257-4f79-9c98-692cde4313fa-1662427451.jpg",
      "https://pictures.immobilienscout24.de/listings/40431281-1ab0-466d-95a9-4af8bede7671-1662427462.jpg",
      "https://pictures.immobilienscout24.de/listings/3a53e04f-af66-4e59-9500-38a76b9a2dd0-1662427469.jpg",
      "https://pictures.immobilienscout24.de/listings/010e146e-0cef-49c3-9eaf-83b17d69f98d-1662427478.jpg",
      "https://pictures.immobilienscout24.de/listings/c1efebf5-dfc3-4964-ab98-7fd592a0cb8d-1662427489.jpg",
      "https://pictures.immobilienscout24.de/listings/07d6b19f-0940-4a63-bcda-a7fca9990c18-1662427498.jpg",
      "https://pictures.immobilienscout24.de/listings/9ea76cea-af1d-4721-83a4-8371ee17654d-1662427508.jpg",
      "https://pictures.immobilienscout24.de/listings/e9ea0c7b-fa3d-43aa-aed9-accdfc9d08be-1662427522.jpg",
      "https://pictures.immobilienscout24.de/listings/46d7a604-719a-4fe7-9933-3e307e2f11ad-1662427529.jpg",
      "https://pictures.immobilienscout24.de/listings/3c403945-880a-4752-a413-15b233c06c47-1662427543.jpg",
      "https://pictures.immobilienscout24.de/listings/62b67471-62c0-4920-aa91-79499823eeaa-1662427558.jpg",
      "https://pictures.immobilienscout24.de/listings/ab5b7835-beb2-4d5c-9987-17c616015279-1662427565.jpg",
      "https://pictures.immobilienscout24.de/listings/3a4158d9-e8db-41d3-99f9-2f47ef422ee2-1653199853.png",
      "https://pictures.immobilienscout24.de/listings/3a4158d9-e8db-41d3-99f9-2f47ef422ee2-1653199853.png"
    ],
    "videoAvailable": false
  },
  "agencyName": "PAG Planungsgesellschaft für Anlagen-und Gebäudetechnik mbH",
  "agencyAddress": "Nördliche Münchner Straße 14a82031 Grünwald"
}

Our immobilienscout24.de scraper can scrape property pages. Next, we'll scrape search pages which will help us to find the right properties for our real estate datasets.

How to Scrape Immobilienscout24.de Search Pages?

Before we start scraping immobilienscout24.de search pages, let's take a look at them. Go to any search page and you will get a page similar to this:

search page on immobilienscout24.de
Search page on immobilienscout24.de

Instead of scraping the search pages by parsing the HTML, we'll extract all the data directly in JSON from the API.

How to Scrape Hidden APIs

Learn how to find hidden APIs, how to scrape them, and what are some common challenges people face when developing web scrapers for hidden APIs.

How to Scrape Hidden APIs

First, we need to get the API responsible for fetching the search data. To do that, follow these steps:

  1. Open the browser developer tools by clicking the F12 key.
  2. Head over the Network tab and filter requests by Fetch/XHR.
  3. Click on the next search page button.

By following the above steps, you will see all the background requests sent by the browser to the web server while changing the search page:

background requests on browser developer tools
Background requests on browser developer tools

To identify the request responsible for fetching the search data, click on the XHR request that has the following response data:

Search page data from the API
Search page data from the API

The data in the above request are the same on the web page but before rendering into the HTML. We can also see all the request headers sent along the request by clicking on the Headers tab.

To scrape search data, we'll replicate this request within our scraper by using the headers and URL of the API request:

import asyncio
import json
from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse
from typing import Dict, List

scrapfly = ScrapflyClient(key="Your ScrapFly API key")

BASE_CONFIG = {
    # bypass web scraping blocking
    "asp": True,
    # set the proxy country to Germany
    "country": "de",
    # the API request headers
    "headers": {
        "accept": "application/json"
    }
}

def parse_search_api(response: ScrapeApiResponse):
    """parse JSON data from the search API"""
    # skip invalid API responses
    if response.scrape_result["content_type"].split(";")[0] != "application/json":
        return
    data = json.loads(response.scrape_result['content'])
    max_search_pages = data["searchResponseModel"]["resultlist.resultlist"]["paging"]["numberOfPages"]
    search_data = data["searchResponseModel"]["resultlist.resultlist"]["resultlistEntries"][0]["resultlistEntry"]
    # remove similar property listings from each property data
    for json_object in search_data:
        if "similarObjects" in json_object.keys():
            json_object.pop("similarObjects")
    return {
        "max_search_pages": max_search_pages,
        "search_data": search_data
    }


async def obtain_session(url: str) -> str:
    """create a session to save the cookies and authorize the search API"""
    session_id="immobilienscout24_search_session"
    await scrapfly.async_scrape(ScrapeConfig(
        url, **BASE_CONFIG, render_js=True, session=session_id
    ))
    return session_id


async def scrape_search(url: str, scrape_all_pages: bool, max_scrape_pages: int = 10) -> List[Dict]:
    """scrape property listings from the search API, which follows the same search page URLs"""
    print("warming up a search session")
    session_id = await obtain_session(url=url)
    first_page = await scrapfly.async_scrape(ScrapeConfig(url, **BASE_CONFIG, session=session_id))
    result_data = parse_search_api(first_page)
    search_data = result_data["search_data"]
    max_search_pages = result_data["max_search_pages"]
    if scrape_all_pages == False and max_scrape_pages < max_search_pages:
        max_scrape_pages = max_scrape_pages
    # scrape all available pages in the search if scrape_all_pages = True or max_pages > total_search_pages
    else:
        max_scrape_pages = max_search_pages
    print(f"scraping search {url} pagination ({max_scrape_pages - 1} more pages)")
    # scrape the remaining search pages
    for page in range(2, max_scrape_pages + 1):
        response = await scrapfly.async_scrape(ScrapeConfig(
            first_page.context["url"].split("?pagenumber")[0] + f"?pagenumber={page}", **BASE_CONFIG, session=session_id))
        try:
            data = parse_search_api(response)["search_data"]
            search_data.extend(data)
        except:
            print("invalid search page")
            pass
    print(f"scraped {len(search_data)} proprties from {url}")
    return search_data
Run the code
if __name__ == "__main__":
    search_data = asyncio.run(scrape_search(
        url="https://www.immobilienscout24.de/Suche/de/bayern/muenchen/wohnung-mieten?pagenumber=1",
        scrape_all_pages=False,
        max_scrape_pages=3
    ))
    # print the result in JSON format
    print(json.dumps(search_data, indent=2))

🙋‍ Chances of getting blocked while scraping the search pages are high. Run the ScrapFly code tabs to avoid blocking.

Here, we use the parse_search_api to parse the search API JSON response. Next, we use the scrape_search function to scrape the first search page and get the total number of available search pages. Next, loop page numbers to scrape the desired search pages.

The result is a list containing all the property data found in three search pages:

Sample output
[
  {
    "@id": "147184265",
    "@modification": "2023-11-07T14:41:10.273+01:00",
    "@creation": "2023-09-25T16:07:38.000+02:00",
    "@publishDate": "2023-09-25T16:07:38.000+02:00",
    "realEstateId": 147184265,
    "disabledGrouping": "false",
    "resultlist.realEstate": {
      "@xsi.type": "search:ApartmentRent",
      "@id": "147184265",
      "title": "Need more Room? We have the solution for you, 4 Room Apartment in Haidhausen.",
      "address": {
        "postcode": "81541",
        "city": "München",
        "quarter": "Haidhausen",
        "description": {
          "text": "Obere Au, München"
        }
      },
      "companyWideCustomerId": "001.1216624",
      "floorplan": "false",
      "streamingVideo": "false",
      "listingType": "XL",
      "showcasePlacementColor": "#343434",
      "privateOffer": "false",
      "contactDetails": {
        "salutation": "FEMALE",
        "firstname": "Mary",
        "lastname": "Goos",
        "phoneNumber": "01512 3057194",
        "cellPhoneNumber": "01512 3057194",
        "company": "Mary Goos Immobilien"
      },
      "realtorCompanyName": "Mary Goos Immobilien",
      "realtorLogoForResultList": {
        "@xsi.type": "common:Picture",
        "floorplan": "false",
        "titlePicture": "false",
        "urls": [
          {
            "url": {
              "@scale": "SCALE",
              "@href": "https://pictures.immobilienscout24.de/usercontent/89c490e6-3f9d-4990-8dae-167208cce481.JPG/ORIG/resize/%WIDTH%x%HEIGHT%%3E/quality/50"
            }
          }
        ]
      },
      "galleryAttachments": {
        "attachment": [
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/3c4d44c9-01d2-472f-93dc-44857382c003-1670329995.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/1a8372fc-8971-488f-9965-dce5e8090d19-1670330001.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/6659217d-9576-4c0b-aa7c-0f96b919bf12-1670330007.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/2c96f8d7-1494-44f7-8eb8-4e2e0b60706f-1670330012.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/47d132db-fc39-4aca-95d2-b0521d0d4ddc-1670330016.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/668846c3-fbf2-4dd5-beb4-f26c2e88a0ff-1670330023.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/284993a8-d30c-47ad-938f-58dbd1e0d66e-1670330027.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/1da81680-bc23-42eb-878d-2599fecbcec0-1670330032.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/56ee46a4-cc48-4472-8a42-505b9a93b07f-1670330039.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/49c17d91-19c1-4e7f-a58e-5f63c608c204-1670330044.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/8f83e65a-edf1-419e-8226-8d7acb1e39fa-1670330049.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/504f34bb-24c8-4d14-ab36-6bf5b53b402b-1670330054.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/ce9a3f67-416d-4909-936a-d1f86b1a1a44-1670330058.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/3f2cdb52-5bff-4504-b40f-d929d1a021b6-1670330062.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/54e1513a-40d8-489b-b8ca-c99c4459a0b0-1670330064.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/29d5de41-7808-4616-a897-4c29c718b51f-1670330069.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/e0b57f07-7667-4751-b3d1-3b2dc1aca31e-1670330073.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/c0b5d77f-e690-4f3e-a3c3-b51297b3c92a-1670330077.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/8affb254-3bc2-46fa-b7fb-37502372db02-1670330079.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/a55d3e59-3db8-4c1d-8067-daf7d7281ef0-1670330082.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/e4661a5c-05c6-4533-b194-ba4942493dcb-1670330086.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/72e01904-3069-45ee-a0c3-9634aa30b89a-1670330090.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/78f3ba95-50ca-444d-b859-0f5d0d5f79f4-1670330092.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/35192c38-c046-4d26-af00-0d37103b7d7e-1670330097.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/b10ccd4c-9be5-48d7-a295-3cbfd7def75c-1670330102.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/8a7d80ac-99c2-4e27-a9dd-c5f36fc1f673-1670330106.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/a284a324-023b-4c75-a69a-e21a4d2dd715-1670330111.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/efd0863f-03a0-4f49-9745-29c53ec49798-1670330115.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/bf9dab52-5288-471d-b39f-f8ce727df50a-1670330119.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/50e97288-23b0-4c58-97cb-49f4eae98dc2-1670330123.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/8627321a-3208-4cfb-9af8-b164fcf5ca02-1670330128.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/bf6fb520-57c3-4d61-9995-0e4a7f49e456-1670330132.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          },
          {
            "@xsi.type": "common:Picture",
            "floorplan": "false",
            "titlePicture": "false",
            "urls": [
              {
                "url": {
                  "@scale": "SCALE_AND_CROP",
                  "@href": "https://pictures.immobilienscout24.de/listings/e1585c63-6860-41b3-9078-bfa2bf0fb894-1670330137.jpg/ORIG/legacy_thumbnail/%WIDTH%x%HEIGHT%/format/jpg/quality/50"
                }
              }
            ]
          }
        ]
      },
      "spotlightListing": "false",
      "price": {
        "value": 2700,
        "currency": "EUR",
        "marketingType": "RENT",
        "priceIntervalType": "MONTH"
      },
      "livingSpace": 104,
      "numberOfRooms": 4,
      "energyPerformanceCertificate": "true",
      "energyEfficiencyClass": "A+",
      "builtInKitchen": "false",
      "balcony": "true",
      "garden": "false",
      "calculatedTotalRent": {
        "totalRent": {
          "value": 3130,
          "currency": "EUR",
          "marketingType": "RENT",
          "priceIntervalType": "MONTH"
        },
        "calculationMode": "SUMMARIZED"
      },
      "constructionYear": 2017
    },
    "attributes": [
      {
        "attribute": [
          {
            "label": "Kaltmiete",
            "value": "2.700 €"
          },
          {
            "label": "Wohnfläche",
            "value": "104 m²"
          },
          {
            "label": "Zimmer",
            "value": 4
          }
        ]
      }
    ],
    "realEstateTags": {
      "tag": [
        "Balkon/Terrasse",
        "Keller",
        "Aufzug"
      ]
    },
    "hasNewFlag": "false"
  }
]

Cool! Our immobilienscout24.de scraper can scrape loads of real estate data with a few lines of code.

FAQ

To wrap up this guide on immobilienscout24.de web scraping, let's take a look at some frequently asked questions.

Yes, all the data on Immobilienscout24.de are publicly available. So it's legal to scrape at reasonable scraping rates. However, using personal data commercially (such as private realtors' information) may be difficult due to GDPR rules in the EU countries. Refer to our previous guide on web scraping legality for more information.

Is there a public API for immobilienscout24.de?

At the time of writing this article, there is no public API available for immobilienscout24.de. However, we have seen that we can use the private API Immobilienscout24.de use to get property listings for search data.

Latest immobilienscout24.de Scraper Code
https://github.com/scrapfly/scrapfly-scrapers/

Web Scraping Immobilienscout24.de Summary

Immobilienscout24.de is a popular website for real estate ads in Germany. Which is a highly protected website with the ability to detect and block web scrapers.

In this article, we explained how to bypass Immobilienscout24.de web scraping blocking. We also went through a step-by-step guide on how to scrape immobilienscout24.de with Python to get real estate data from property and search pages.

Related Posts

How to Scrape Reddit Posts, Subreddits and Profiles

In this article, we'll explore how to scrape Reddit. We'll extract various social data types from subreddits, posts, and user pages. All of which through plain HTTP requests without headless browser usage.

How to Scrape With Headless Firefox

Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.

How to Scrape LinkedIn.com Profile, Company, and Job Data

In this scrape guide we'll be taking a look at one of the most popular web scraping targets - LinkedIn.com. We'll be scraping people profiles, company profiles as well as job listings and search.