[Blog](https://scrapfly.io/blog) / [python](https://scrapfly.io/blog/tag/python) / [How to Scrape Domain.com.au Real Estate Property Data](https://scrapfly.io/blog/posts/how-to-scrape-domain-com-au-real-estate-property-data) # How to Scrape Domain.com.au Real Estate Property Data by [Mazen Ramadan](https://scrapfly.io/blog/author/mazen) Apr 18, 2026 12 min read [\#python](https://scrapfly.io/blog/tag/python) [\#real-estate](https://scrapfly.io/blog/tag/real-estate) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data "Share on LinkedIn") Domain.com.au is a popular website for real estate listings in Australia, with thousands of properties available for sale or rent. In this article, we will explain how to scrape real estate data from domain.com.au using only a few lines of Python. We will be extracting property listing data from both property and search pages. Let's get started! ## Key Takeaways Master domain.com.au scraping with advanced Python techniques, real estate data extraction, and property monitoring for comprehensive Australian market analysis. - Reverse engineer Domain.com.au's API endpoints by intercepting browser network requests and analyzing JSON responses - Extract structured property data including prices, locations, and property details from Australian real estate listings - Implement pagination handling and search parameter management for comprehensive property data collection - Configure proxy rotation and fingerprint management to avoid detection and rate limiting - Handle dynamic content loading with proper wait strategies and element visibility checks - Implement data validation and error handling for reliable Australian property information extraction [**View Source Code**github.com/scrapfly/scrapfly-scrapers/tree/main/domaincom-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/domaincom-scraper) **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## Why Scrape Domain.com.au? Scraping domain.com.au opens the door to a broad list of property listing data. Allowing businesses and traders to track not only the market trends but also prices, demand and supply changes over time. Real estate data from scraping domain.com.au can help investors identify underpriced properties or areas with potential growth, leading to better decision-making and higher profit margins. Moreover, manually exploring real estate data from websites can take time and effort. Therefore, domain.com.au web scraping can save a lot of manual effort by retrieving thousands of listings quickly. For more details, refer to our previous article on [web scraping real estate use cases](https://scrapfly.io/use-case/real-estate-web-scraping). ## Project Setup In this guide, we'll be scraping domain.com.au using a few Python packages: - `JMESPath` For refining and parsing JSON datasets. - `scrapfly-sdk` A Python SDK for ScrapFly, a web scraping API that bypasses web scraping blocking. - `asyncio` For running our code asynchronously to increase our web scraping speed. It's worth noting that `asyncio` comes pre-installed in Python. You can install the other packages using the following pip command: shell```shell pip install jmespath scrapfly-sdk ``` ## Scrape Domain.com.au Property Pages Let's start by scraping property pages on domain.com.au. Go to any property page on the website like [this one](https://www.domain.com.au/5401-63-la-trobe-street-melbourne-vic-3000-2018796388) and you will get a page similar to this: Property page on domain.com.auTo scrape this page data, we'll parse data directly from `script` tags in JSON as this page uses hidden web data to store property details. To view this data, open the [browser developer tools](https://scrapfly.io/blog/answers/browser-developer-tools-in-web-scraping) by clicking the `F12` key. Then, find the `script` tag with the id `__NEXT_DATA__`, which contain the property listing data: Property page hidden web data on domain.com.auThis data is the same on the web page but before getting rendered into the HTML, which is often known as hidden web data. [How to Scrape Hidden Web DataThe visible HTML doesn't always represent the whole dataset available on the page. In this article, we'll be taking a look at scraping of hidden web data. What is it and how can we scrape it using Python?](https://scrapfly.io/blog/posts/how-to-scrape-hidden-web-data) To scrape the property pages' data, we'll extract the JSON data inside the `script` tag. We'll also refine it using JMESPath to exclude the unnecessary details: '); const s = document.createElement("script"); s.src = [ "https://gist.github.com/", "scrapfly-dev/", "28a056e52e13d3d7c7f29d30c927346a", ".js" ].join(""); document.write('In the above code, we define three functions, let's break them down: - `parse_property_page()` for parsing the property page data after extracting it from the `script` tag using [Quick Intro to Parsing JSON with JMESPath in Python](https://scrapfly.io/blog/posts/parse-json-jmespath-python). - `parse_hidden_data()` for extracting the hidden data in JSON from the `script` tag. - `scrape_properties()` for scraping the property pages by adding the page URLs to a scraping list and scraping them concurrently. Here is what the result should look like: Sample outputjson```json [ { "listingId": 2018799963, "listingUrl": "https://www.domain.com.au/101-29-31-market-street-melbourne-vic-3000-2018799963", "unitNumber": "101", "streetNumber": "29-31", "street": "Market Street", "suburb": "Melbourne", "postcode": "3000", "createdOn": "2023-09-28T13:15:16.053", "propertyType": "Apartment / Unit / Flat", "beds": 3, "phone": "03 98201111", "agencyName": "Kay & Burton Stonnington", "propertyDeveloperName": "Kay & Burton Stonnington", "agencyProfileUrl": "https://www.domain.com.au/real-estate-agencies/kayburtonstonnington-88/", "propertyDeveloperUrl": "https://www.domain.com.au/real-estate-agencies/kayburtonstonnington-88/", "description": [ "Inspect by Private Appointment", "", "A statement of grandeur from the city's maritime age, this majestic board room apartment is a landmark property in terms of stateliness, luxury and amenity, beautifully positioned in the city's west end on the corner of Flinders Lane.", "", "An award-winning landmark since its inception, the apartment is privately cocooned within the iconic heritage surroundings of the Port Authority Building designed in flamboyant Beaux Arts style by Sydney Smith Ogg & Serpell in 1929. Today a magnificent place of residence, this three bedroom, two bathroom apartment with secure parking for 2.5 cars is one of Melbourne's most noteworthy town residences.", "", "Featured in Vogue Living, the 325 sqm (approx.) interiors have been renovated by SJB Interiors to reflect the richness of the building but also Melbourne in the 21st century offering flexibility, liveability and purity of detail.", "", "Highly decorated, the heritage interiors are backdropped by regal ornate ceilings, handsome blackwood paneling, beautiful Jarrah, Ash and Blackwood parquetry flooring, bronze light fittings, soaring imperial marble fireplace mantels and banks of majestic bronze fanlight windows taking in the surrounding plane tree lined streetscape with private views towards Southbank.", "", "The outstanding craftmanship is testament to the power, wealth and extravagance of the Harbour commissioners presiding over maritime Melbourne, most notably in the board room which is the keystone of this illustrious home. A remarkable space, a flagship for entertaining yet intimate for daily living with the softening inclusions of a marble kitchen, ethanol fireplace and moveable bar balancing the classical architecture. A second marble kitchen adds additional flexibility to the internal footprint that also hosts a refined sitting room and handsome study.", "", "Luxurious natural materials respond elegantly to the buildings original design including the three bedroom accommodation that features two bedrooms with silk faced wardrobes and marble ensuites, one with a freestanding bath. Practical modern appointments include two Gaggenau ovens, central heating and a large storage cage.", "", "Crafted from granite with a grand foyer and internal lift, the exclusive Port Authority Building is situated within walking distance of Melbourne's financial and Law Court districts, a stroll to Southbank restaurants, the Yarra River and Crown Entertainment complex and enjoys transport options in every direction including trams and trains from Flinders Street and Spencer Street stations.", "", "In conjunction with Cushman & Wakefield - Oliver Hay 0419 528 540" ], "loanfinder": { "homeLoansApiUrl": "https://home-loans-api.domainloanfinder.com.au", "configName": "dhl-domain-web", "calculators": [ { "name": "repayments", "propertyPrice": "1080000", "priceSource": "median", "ingressApi": "https://www.domain.com.au/transactions-users-ingress-api/v1/dlf", "usersApi": "https://www.domain.com.au/transactions-users-api/v1/users/loan-tools/repayments", "loanTerm": "30", "formActionUrl": "https://www.domain.com.au/loanfinder/basics/new-home?ltm_source=domain<m_medium=referral<m_campaign=domain_r_calculator_buy-listing_web<m_content=newloan_compare-home-loans", "displayLogo": false, "submitText": "Compare Home Loans", "queryParams": { "ltm_source": "domain", "ltm_medium": "referral", "ltm_campaign": "domain_r_calculator_buy-listing_web", "ltm_content": "newloan_compare-home-loans" }, "experimentName": "buy_listing_page_calculator_repayments", "groupStats": { "listingId": "2018799963", "deviceType": "desktop" }, "hasDeviceSessionId": false }, { "name": "upfront-costs", "propertyPrice": "1080000", "ingressApi": "https://www.domain.com.au/transactions-users-ingress-api/v1/dlf", "usersApi": "https://www.domain.com.au/transactions-users-api/v1/users/loan-tools/ufc", "formActionUrl": "https://www.domain.com.au/loanfinder/basics/new-home?ltm_source=domain<m_medium=referral<m_campaign=domain_ufc_calculator_buy-listing_web<m_content=newloan_check-options", "submitText": "Compare Home Loans", "firstHomeBuyer": "firstHomeBuyer", "queryParams": { "ltm_source": "domain", "ltm_medium": "referral", "ltm_campaign": "domain_ufc_calculator_buy-listing_web", "ltm_content": "newloan_check-options" }, "experimentName": "buy_listing_page_calculator_stamp_duty", "groupStats": { "listingId": "2018799963", "deviceType": "desktop" }, "hasDeviceSessionId": false } ], "productReviewConfig": { "brandId": "ba7f14c4-3f43-3018-b06c-93d6657e7602", "widgetId": "04115d3d-113e-3cb1-8103-ba0bb5e8d765", "widgetUrl": "https://cdn.productreview.com.au/assets/widgets/loader.js" } }, "schools": [ { "id": "", "educationLevel": "secondary", "name": "Hester Hornbrook Academy - City Campus", "distance": 332.25763189076645, "state": "VIC", "postCode": "3000", "year": "", "gender": "", "type": "Private", "address": "Melbourne, VIC 3000", "url": "", "domainSeoUrlSlug": "hester-hornbrook-academy-city-campus-vic-3000-52519" }, ..... ], "suburbInsights": { "beds": 3, "propertyType": "Unit", "suburb": "Melbourne", "suburbProfileUrl": "/suburb-profile/melbourne-vic-3000", "medianPrice": 1080000, "medianRentPrice": 1000, "avgDaysOnMarket": 110, "auctionClearance": 64, "nrSoldThisYear": 102, "entryLevelPrice": 600000, "luxuryLevelPrice": 3065000, "renterPercentage": 0.6970133677881032, "singlePercentage": 0.7628455892048701, "demographics": { "population": 47279, "avgAge": "20 to 39", "owners": 0.3029866322118968, "renters": 0.6970133677881032, "families": 0.2371544107951299, "singles": 0.7628455892048701, "clarification": true }, "salesGrowthList": [ { "medianSoldPrice": 1053000, "annualGrowth": 0, "numberSold": 205, "year": "2018", "daysOnMarket": 108 }, ..... ], "mostRecentSale": null }, "gallery": { "slides": [ { "thumbnail": "https://rimh2.domainstatic.com.au/liH61Q-Xq2U3NVz8_CuS7s4WhQk=/fit-in/144x106/filters:format(jpeg):quality(80):no_upscale()/2018799963_1_1_230928_031516-w7000-h4667", "images": { "original": { "url": "https://rimh2.domainstatic.com.au/P_chzZfBOvp4O1uRKgJ1YaGcVyg=/fit-in/1920x1080/filters:format(jpeg):quality(80):no_upscale()/2018799963_1_1_230928_031516-w7000-h4667", "width": 1620, "height": 1080 }, "tablet": { "url": "https://rimh2.domainstatic.com.au/GZAXgEYb2R39uL25FZBNJwtIvxI=/fit-in/1020x1020/filters:format(jpeg):quality(80):no_upscale()/2018799963_1_1_230928_031516-w7000-h4667", "width": 1020, "height": 680 }, "mobile": { "url": "https://rimh2.domainstatic.com.au/0hmZIfTO914hdD_tdBgAPcAQHew=/fit-in/600x800/filters:format(jpeg):quality(80):no_upscale()/2018799963_1_1_230928_031516-w7000-h4667", "width": 600, "height": 400 } }, "embedUrl": null, "mediaType": "image" }, ..... ], "features": [], "structuredFeatures": [], "faqs": [ { "question": "How many bedrooms does 101/29-31 Market Street, Melbourne have?", "answer": [ { "text": "101/29-31 Market Street, Melbourne is a 3 bedroom apartment." } ] }, ..... ] } ] ``` Our domain.com.au scraper can scrape real estate data from property pages. Let's apply what we learned with search pages so we can discover desired properties for our property scraper. Scrapfly #### Scale your web scraping effortlessly Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data. [Try Free →](https://scrapfly.io/register)## Scrape Domain.com.au Search Pages Just like property pages, we can find the search pages' data under `script` tags: Search pages hidden web data on domain.com.auDomain.com.u search supports pagination by adding a `?page` parameter at the end of the search page URLs: ``` https://www.domain.com.au/rent/melbourne-vic-3000/?page=page_number ``` We'll extract the data directly from the `script` tag in JSON and use the pagination parameter to crawl over search pages: '); const s = document.createElement("script"); s.src = \[ "https://gist.github.com/", "scrapfly-dev/", "c81ee443c0c66fa06b243c26f7581ff0", ".js" \].join(""); document.write('Here, we use the `parse_hidden_data` we defined earlier to extract the data in JSON from the `script` tag. Then, we use the `parse_search_page` function to remove the unnecessary details using JMESPath. Next, we crawl over search pages by scraping the first search page and extracting the number of maximum search pages available. Finally, we add the remaining search pages to a scraping list and scrape them concurrently. The result is a list of property listing data from three search pages, similar to this: Sample outputjson```json [ { "id": 16754956, "listingType": "listing", "listingModel": { "promoType": "premiumplus", "url": "/317-55-queens-road-melbourne-vic-3000-16754956", "images": [ "https://rimh2.domainstatic.com.au/SXfU4fEx8JoWy2uvdFvIaAI-eYk=/660x440/filters:format(jpeg):quality(80)/16754956_1_1_231120_072357-w3000-h2002", "https://rimh2.domainstatic.com.au/NkkQtevg-5Ru_feyL39QmtO1gfA=/660x440/filters:format(jpeg):quality(80)/13013890_2_1_190321_053622-w3000-h2002", "https://rimh2.domainstatic.com.au/BQlwroGo_nBdi_cwTJ9OBDwkSxg=/660x440/filters:format(jpeg):quality(80)/16754956_3_1_231120_072357-w2878-h1921", "https://rimh2.domainstatic.com.au/wF6PD7BvXuR9KiPulNOS9YRFdpg=/660x440/filters:format(jpeg):quality(80)/13013890_3_1_190321_053622-w2878-h1921", "https://rimh2.domainstatic.com.au/kNRsjFXUDI5n_IcNubVGGNxYhXM=/660x440/filters:format(jpeg):quality(80)/13013890_4_1_190321_053650-w3000-h2002", "https://rimh2.domainstatic.com.au/hlbd5TKcUerkqDOJNnbWI0fOLzA=/660x440/filters:format(jpeg):quality(80)/13013890_5_1_190321_053650-w3000-h2002", "https://rimh2.domainstatic.com.au/ucF8iRTfyXnZiNhwznxd7F92VTw=/660x440/filters:format(jpeg):quality(80)/13013890_6_1_190321_053650-w4134-h2756", "https://rimh2.domainstatic.com.au/Yt8IctotQzRMzmBkCARqFl12DfY=/660x440/filters:format(jpeg):quality(80)/12823878_3_1_190103_043521-w440-h330", "https://rimh2.domainstatic.com.au/LQrslg4tyTUafVXHV7I0D2OXHSo=/660x440/filters:format(jpeg):quality(80)/13013890_8_1_190321_053650-w4134-h2756" ], "brandingAppearance": "light", "price": "$620 pw", "hasVideo": false, "branding": { "agencyId": 345, "agents": [ { "agentName": "Simon Koulouris", "agentPhoto": null } ], "agentNames": "Simon Koulouris", "brandLogo": "https://rimh2.domainstatic.com.au/1zCcz7Jq9n-5IG0HlindcwPAGkI=/170x60/filters:format(jpeg):quality(80)/https://images.domain.com.au/img/Agencys/345/logo_345.png?buster=2023-11-23", "skeletonBrandLogo": "https://rimh2.domainstatic.com.au/PeWP7T-vRHfYKqkZ5q_1SG5eVuE=/200x70/filters:format(jpeg):quality(80):no_upscale()/https://images.domain.com.au/img/Agencys/345/logo_345.png?buster=2023-11-23", "brandName": "Dingle Partners", "brandColor": "#fff", "agentPhoto": null, "agentName": "Simon Koulouris" }, "address": { "street": "317/55 Queens Road", "suburb": "MELBOURNE", "state": "VIC", "postcode": "3000", "lat": -37.8478546, "lng": 144.977615 }, "features": { "beds": 2, "baths": 1, "parking": 1, "propertyType": "ApartmentUnitFlat", "propertyTypeFormatted": "Apartment / Unit / Flat", "isRural": false, "landSize": 0, "landUnit": "m\u00b2", "isRetirement": false }, "inspection": { "openTime": "2023-11-25T12:45:00", "closeTime": "2023-11-25T13:00:00" }, "auction": null, "tags": { "tagText": "New", "tagClassName": "is-new" }, "displaySearchPriceRange": null, "enableSingleLineAddress": false } } ] ``` Awesome! We can successfully scrape real estate data from realestate.com.au property and search pages. However, our scraper is very likely to get blocked after sending additional requests. Let's take look at a solution! ## Bypass Domain.com.au Blocking with Scrapfly To scrape domain.com.au without getting blocked, we'll use ScrapFly. We can use the `asp` feature to scrape domain.com.au without getting blocked: python```python import httpx from parsel import Selector response = httpx.get("some domain.com.au url") selector = Selector(response.text) # in ScrapFly SDK becomes from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse scrapfly_client = ScrapflyClient("Your ScrapFly API key") result: ScrapeApiResponse = scrapfly_client.scrape(ScrapeConfig( # some domain.com.au url "https://www.domain.com.au/610-399-bourke-street-melbourne-vic-3000-2018835548", # we can select specific proxy country country="AU", # and enable anti scraping protection bypass asp=True, # allows JavaScript rendering similar to headless browsers render_js=True )) # use the built-in parsel selector selector = result.selector ``` Check out [Scrapfly's web scraping API](https://scrapfly.io/web-scraping-api) for all the details. ## FAQ Is it legal to scrape domain.com.au?Yes, all the data on domain.com.au are publicly available and it's legal to scrape it as long as the scraper is not damaging the website. For more details, refer to our [is web scraping legal](https://scrapfly.io/is-web-scraping-legal) guide. Is there an API for domain.com.au?Currently, there is no public API for domain.com.au. However, domain.com.au scraping is straightforward and you can make an API yourself with Python scraping and Fastapi; for that see [turning your web scrapers into API](https://scrapfly.io/blog/posts/how-to-turn-web-scrapers-into-data-apis). Are there alternatives for domain.com.au?Yes, realestate.com.au is another popular website for real estate ads in Australia. Refer to our [\#realestate](https://scrapfly.io/blog/tags/real-estate/) blog tag for the guide and for more real estate target websites. See our [realestate.com.au scraping guide](https://scrapfly.io/blog/posts/how-to-scrape-realestate-com-au-property-listing-data) for a detailed walkthrough. What parsing method does domain.com.au use for listing data?Domain.com.au embeds property data in `__NEXT_DATA__` script tags as JSON. This is a common pattern in Next.js sites and can be extracted with [JMESPath](https://scrapfly.io/blog/posts/parse-json-jmespath-python) for clean field selection. [**View Source Code**github.com/scrapfly/scrapfly-scrapers/tree/main/domaincom-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/domaincom-scraper) ## Web Scraping Domain.com.au Summary In this guide, we explained how to scrape domain.com.au, a popular Australian website for real estate ads. We went through a step-by-step guide on scraping domain.com.au listings from property and search pages using Python by extracting the data directly in JSON from `script` tags. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. Table of Contents- [Key Takeaways](#key-takeaways) - [Why Scrape Domain.com.au?](#why-scrape-domain-com-au) - [Project Setup](#project-setup) - [Scrape Domain.com.au Property Pages](#scrape-domain-com-au-property-pages) - [Scrape Domain.com.au Search Pages](#scrape-domain-com-au-search-pages) - [Bypass Domain.com.au Blocking with Scrapfly](#bypass-domain-com-au-blocking-with-scrapfly) - [FAQ](#faq) - [Web Scraping Domain.com.au Summary](#web-scraping-domain-com-au-summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Explore this Article with AI [ ChatGPT ](https://chat.openai.com/?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data) [ Gemini ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data) [ Grok ](https://x.com/i/grok?text=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data) [ Perplexity ](https://www.perplexity.ai/search/new?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data) [ Claude ](https://claude.ai/new?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-domain-com-au-real-estate-property-data) ## Related Articles [ python api ### How to Scrape Hidden APIs In this tutorial we'll be taking a look at scraping hidden APIs which are becoming more and more common in modern dynami... ](https://scrapfly.io/blog/posts/how-to-scrape-hidden-apis) [ python scrapeguide ### How to Scrape Realestate.com.au Property Listing Data We're taking yet another look at real estate websites. This time we're going down under! Realtestate.com.au is the bigge... ](https://scrapfly.io/blog/posts/how-to-scrape-realestate-com-au-property-listing-data) [ python scrapeguide ### How to Scrape Real Estate Property Data using Python Introduction to scraping real estate property data. What is it, why and how to scrape it? We'll also list dozens of popu... ](https://scrapfly.io/blog/posts/how-to-scrape-real-estate-property-data-using-python) Scale your web scraping effortlessly, **1,000 free credits** [Start Free](https://scrapfly.io/register)