     [Blog](https://scrapfly.io/blog)   /  [ai](https://scrapfly.io/blog/tag/ai)   /  [How to Scrape Google AI Mode in 2026](https://scrapfly.io/blog/posts/how-to-scrape-google-ai-mode)   

# How to Scrape Google AI Mode in 2026

 by [Hisham Medhat](https://scrapfly.io/blog/author/hisham) Nov 17, 2026 15 min read [\#ai](https://scrapfly.io/blog/tag/ai) [\#python](https://scrapfly.io/blog/tag/python) [\#scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [\#seo](https://scrapfly.io/blog/tag/seo) 

 [  ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode "Share on LinkedIn") [  ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode&text=How%20to%20Scrape%20Google%20AI%20Mode%20in%202026 "Share on X") [  ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode "Share on Facebook")    

 

 

Summarize this article with

 [  ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode) [  ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode) [  ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode) [  ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode) [  ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fhow-to-scrape-google-ai-mode) 



         

When it comes to scraping Google, the AI Mode page at `/search?q=...&udm=50` is a surface of its own. It's a conversational answer page, separate from the regular results page and the AI Overview box Google shows there. Unfortunately, the answer streams in only after the page loads, so the standard SERP recipe doesn't apply here.

We'll take a look at how we can scrape it in Python, starting from the official Scrapfly Google scraper. Every query we send will come back as one JSON object, so let's dive in!



[**Latest Google Scraper Code**github.com/scrapfly/scrapfly-scrapers/tree/main/google-scraper](https://github.com/scrapfly/scrapfly-scrapers/tree/main/google-scraper)

## Key Takeaways

- Google AI Mode has its own `/search?q=...&udm=50` page, apart from the AI Overview box on regular results.
- Scrapfly's `extraction_prompt` returns the answer and sources as JSON once the "Good response" button renders, with no class-name selectors to maintain.
- Sources arrive as `/goto` redirects, and one batch of unrendered requests turns them into real URLs.

**Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime.







## Before You Start: What This Google AI Mode Scraper Will Return

Here's a trimmed result from our live run on November 10, 2026, with the answer and the source cards Google returned for one query:

json

```json
{
  "query": "best free python courses",
  "answer": "Here are the top free Python courses available, broken down by your learning style and goals:\n1. Harvard CS50’s Introduction to Programming with Python (CS50P)\n    + Best for: A structured, rigorous computer science approach to Py...",
  "sources": [
    {
      "snippet": "About this course - Python Programming MOOC 2026\nby University of Hels...",
      "title": "Python Programming MOOC 2026",
      "url": "https://programming-26.mooc.fi/"
    },
    {
      "snippet": "Recommended free online Python courses : r/learnpython\nFebruary 2, 202...",
      "title": "Reddit · r/learnpython",
      "url": "https://www.reddit.com/r/learnpython/comments/10rlgnu/recommended_free_online_python_courses/"
    }
  ],
  "captured_at": "2026-11-10T10:59:10.175611+00:00"
}
```



Each source card in the trimmed list above carries a title, a URL and a snippet. Note that what sets AI Mode apart from AI Overview or the Gemini app is mostly the URL and when Google shows an answer at all:

| Surface | URL pattern | Trigger | Citations | When to scrape |
|---|---|---|---|---|
| AI Mode | `/search?q=...&udm=50` | Always for `udm=50` queries | Source panel cards + inline links | This guide |
| AI Overview | `/search?q=...` (no udm) | Conditional, query-dependent | Inline links + small card list | [Standard Google SERP guide](https://scrapfly.io/blog/posts/how-to-scrape-google) |
| Gemini app/API | `gemini.google.com` or REST API | Authenticated session / API key | Conversational, no SERP surface | Different product; out of scope |



## Step 1: Install and Initialize the Scrapfly Python SDK

Our scraper needs Python 3.10 or newer for the `str | None` type hints, and only two Python packages: [scrapfly-sdk](https://pypi.org/project/scrapfly-sdk/) and `parsel`, which the SDK uses for the `response.selector` we'll call in Step 3.

We can install both of them via `pip`:

bash

```bash
pip install scrapfly-sdk parsel
```



Next, we'll create our Scrapfly client and a `BASE_CONFIG` dictionary which every request in the following steps will reuse:

python

```python
import asyncio
import json
import os
from datetime import datetime, timezone
from typing import List, TypedDict
from urllib.parse import quote

from scrapfly import ScrapeConfig, ScrapflyClient

SCRAPFLY = ScrapflyClient(key=os.environ["SCRAPFLY_KEY"])

BASE_CONFIG = {
    "unblocker": True,
    "country": "US",
    "render_js": True,
}
```



Our client picks up the API key from the `SCRAPFLY_KEY` environment variable, and we can find that key on the [Scrapfly dashboard](https://scrapfly.io/dashboard). Note that `unblocker` is the current name of the [anti-scraping protection](https://scrapfly.io/docs/scrape-api/unblocker) option. Meaning older code that passes `asp=True` still works the same way!



## Step 2: Build the Google AI Mode URL

So, which piece turns a regular Google search into an AI Mode search? Just the `udm=50` parameter! The official `scrape_ai_mode()` puts it in the URL right after the `q=` value.

To pin the interface language and the country for every run, we can use the URL parameters `hl` and `gl` as well. Let's take a look at the URL our function builds for one query:

python

```python
def build_ai_mode_url(query: str, hl: str = "en", gl: str = "US") -> str:
    return f"https://www.google.com/search?q={quote(query)}&udm=50&hl={hl}&gl={gl}"


print(build_ai_mode_url("best free python courses"))
```



Which prints the full AI Mode URL for our test query:

text

```text
https://www.google.com/search?q=best%20free%20python%20courses&udm=50&hl=en&gl=US
```





## Step 3: Fetch the Rendered AI Mode Page

Let's take a look at the official scraper, which waits for the "Good response" feedback button Google places under a rendered answer:

python

```python
import re

WAIT_FOR_ANSWER = 'button[aria-label="Good response"][aria-pressed="false"]'


async def fetch_ai_mode(query: str):
    url = build_ai_mode_url(query)
    print(url)
    response = await SCRAPFLY.async_scrape(
        ScrapeConfig(
            url,
            **BASE_CONFIG,
            wait_for_selector=WAIT_FOR_ANSWER,
            rendering_wait=5000,
        )
    )
    html = response.scrape_result["content"]
    print(f"{len(html):,} characters of HTML")
    print("feedback buttons:", len(response.selector.css(WAIT_FOR_ANSWER)))
    print("source links:", len(set(re.findall(r"/goto\?url=([\w-]+)", html))))
    print("credits:", response.context["cost"]["total"])


asyncio.run(fetch_ai_mode("best free python courses"))
```



In the example above, we set [wait\_for\_selector](https://scrapfly.io/docs/scrape-api/javascript-rendering) to the button selector and give the page a 5-second `rendering_wait` as well. Here's what our run printed:

text

```text
https://www.google.com/search?q=best%20free%20python%20courses&udm=50&hl=en&gl=US
1,972,064 characters of HTML
feedback buttons: 1
source links: 4
credits: 6
```



That's 1,972,064 characters of HTML for only a few paragraphs of answer, with 4 unique `/goto` links. No heading or label on the page says which links sit in the source panel.



## Step 4: Extract the AI Mode Answer and Sources

The official scraper skips selectors altogether. Instead, the page goes to Scrapfly's [LLM extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) together with a plain English prompt, and JSON comes back in the response. Here's that prompt with the two result types we'll use:

python

```python
EXTRACTION_PROMPT = (
    "Extract a JSON object from this Google AI Mode search result page (udm=50). "
    "Ignore the top navigation, cookie banner, sign-in prompts, sidebar, "
    "related searches, and the follow-up input at the bottom.\n"
    "Fields:\n"
    "- query: the original search query shown above the AI answer\n"
    "- answer: the AI Mode answer text from the main answer column only. "
    "Return plain text with paragraph breaks. Do not include markdown links, "
    "bullet markdown syntax, tables, or inline citation URLs.\n"
    "- sources: array of source cards from the dedicated sources panel attached "
    "to the AI answer (small cards with a title, external link, and snippet). "
    "Include only those panel cards, not inline links mentioned inside the answer "
    "body and not sidebar or related-result links. Each item has:\n"
    "  - title: source page title\n"
    "  - url: full destination URL\n"
    "  - snippet: short excerpt on the card, or null if missing\n"
    "Return only the JSON object."
)


class AiModeSource(TypedDict):
    title: str
    url: str
    snippet: str | None


class AiModeResult(TypedDict):
    query: str
    answer: str
    sources: List[AiModeSource]
    captured_at: str
```



Here, our `EXTRACTION_PROMPT` limits `sources` to the cards in the panel, so inline links from the answer and sidebar results stay out of our list. The `AiModeSource` and `AiModeResult` classes describe the result we expect back.

The extraction runs inside the fetch request itself. So, our scrape function reuses the Step 3 config and only adds the `extraction_prompt` parameter:

python

```python
async def scrape_ai_mode(query: str, hl: str = "en", gl: str = "US") -> AiModeResult:
    url = build_ai_mode_url(query, hl, gl)
    response = await SCRAPFLY.async_scrape(
        ScrapeConfig(
            url,
            **BASE_CONFIG,
            extraction_prompt=EXTRACTION_PROMPT,
            wait_for_selector=WAIT_FOR_ANSWER,
            rendering_wait=5000,
        )
    )
    data = (response.scrape_result.get("extracted_data") or {}).get("data")
    if not isinstance(data, dict):
        raise ValueError(f"no AI Mode data extracted for {query!r}: {data!r}")
    data["captured_at"] = datetime.now(timezone.utc).isoformat()
    return data


result = asyncio.run(scrape_ai_mode("best free python courses"))
print(result["answer"][:300])
print(json.dumps(result["sources"][:2], indent=2, ensure_ascii=False))
print(len(result["sources"]), "sources")
```



Our function picks the JSON up from `extracted_data["data"]` and adds a `captured_at` timestamp before returning it. Here's the start of the answer and the first two sources it printed:

text

```text
Here are some of the best free Python courses available, broken down by your learning style and goals:
* 
* [Harvard's CS50's Introduction to Programming with Python](https://www.google.com/goto?url=CAESgQEBxCJF1xMgqNRbh28H7ciHKthkl_KT430aihcypjPaxPcI3y9OWMFkXG8BRHOr592xxoDsLuKHzo5AubkaN0Wf3omlTx1bv
[
  {
    "snippet": "CS50's Introduction to Programming with Python | Harvard University\n9 weeks, 3-9 hours/week, online, self-paced, free, introductory level, English language/transcript. Focuses on reading, writing, t...",
    "title": "Harvard University",
    "url": "https://www.google.com/goto?url=CAESgQEBxCJF1xMgqNRbh28H7ciHKthkl_KT430aihcypjPaxPcI3y9OWMFkXG8BRHOr592xxoDsLuKHzo5AubkaN0Wf3omlTx1bvurrmlb4Xi6uCK0QvQBMDEM9Msu8G19_jvBJ6jsMAEL47QS0Uz7AxbZR1I3hEkGzpj4vzujbQ98aWlk"
  },
  {
    "snippet": "Learn Python, Data Viz, Pandas & More | Tutorials - Kaggle\nLearn the most important language for data science. Solve short hands-on challenges to perfect your data manipulation skills.",
    "title": "Kaggle",
    "url": "https://www.google.com/goto?url=CAESWQHEIkXXauTdeSBcpJtSjnP3kvFTTCWknMBirV5ArkUZfRNWFxoCpsUxCmLEKIWATi7EkYndsHzcpHvsVlGxmzcjSWB_w8XD0ySMHVSr4O25RGVKERtvDfEJ"
  }
]
3 sources
```



Our run printed `3 sources`, and every `url` among them points at `google.com/goto?url=` with an opaque token.



## Step 5: Resolve the Google /goto Source URLs

This is where the official `scrape_ai_mode()` stops. The same repo does resolve `/goto` links already, though only for regular search results. Every `/goto` token redirects to the real page. So, a plain request without a browser is enough, and the URL it ends on is the source we're after:

python

```python
GOTO_PREFIX = "/goto?url="


async def resolve_goto_redirects(sources: List[AiModeSource]) -> List[AiModeSource]:
    resolved = {source["url"]: None for source in sources if GOTO_PREFIX in source["url"]}
    if not resolved:
        return sources

    config = {**BASE_CONFIG, "render_js": False}
    to_scrape = [
        ScrapeConfig(url, **config, raise_on_upstream_error=False, retry=False)
        for url in resolved
    ]
    async for response in SCRAPFLY.concurrent_scrape(to_scrape):
        if isinstance(response, Exception):
            error, response = response, getattr(response, "api_response", None)
            if response is None:
                print(f"failed to resolve a goto URL: {error}")
                continue
        requested = response.result["config"]["url"]
        destination = response.scrape_result["url"]
        if GOTO_PREFIX in destination:
            print(f"goto URL did not redirect: {requested}")
            continue
        resolved[requested] = destination

    for source in sources:
        source["url"] = resolved.get(source["url"]) or source["url"]
    return sources
```



Here, `resolve_goto_redirects()` sends all the `/goto` URLs as one [concurrent\_scrape](https://scrapfly.io/docs/sdk/python) batch with `render_js` turned off, and stores the URL each response landed on.

The catch is that a destination page can still fail to load, which happened to one source in an earlier run. That's alright though, as the redirect has already happened by then. So, the exception's `api_response` still holds the redirected URL, which our loop reads with `getattr()`.



Scrapfly

#### Scale your web scraping effortlessly

Scrapfly handles proxies, browsers, and anti-bot bypass — so you can focus on data.

[Try Free →](https://scrapfly.io/register)

## Step 6: Put the Google AI Mode Scraper Together

To put it all together, our `scrape_ai_mode()` function from Step 4 gets a `resolve_urls` flag, which runs the resolver right before we stamp the capture time:

python

```python
async def scrape_ai_mode(
    query: str, hl: str = "en", gl: str = "US", resolve_urls: bool = True
) -> AiModeResult:
    url = build_ai_mode_url(query, hl, gl)
    response = await SCRAPFLY.async_scrape(
        ScrapeConfig(
            url,
            **BASE_CONFIG,
            extraction_prompt=EXTRACTION_PROMPT,
            wait_for_selector=WAIT_FOR_ANSWER,
            rendering_wait=5000,
        )
    )
    data = (response.scrape_result.get("extracted_data") or {}).get("data")
    if not isinstance(data, dict):
        raise ValueError(f"no AI Mode data extracted for {query!r}: {data!r}")
    if resolve_urls:
        data["sources"] = await resolve_goto_redirects(data["sources"])
    # the extraction can return the same source card more than once
    data["sources"] = list({source["url"]: source for source in data["sources"]}.values())
    data["captured_at"] = datetime.now(timezone.utc).isoformat()
    return data


if __name__ == "__main__":
    result = asyncio.run(scrape_ai_mode("best free python courses"))
    print(json.dumps(result, indent=2, ensure_ascii=False))
```



Here's what our run printed, with the answer and snippets trimmed and the keys reordered for reading:

json

```json
{
  "query": "best free python courses",
  "answer": "Here are the top free Python courses available, broken down by your learning style and goals:\n1. Harvard CS50’s Introduction to Programming with Python (CS50P)\n    + Best for: A structured, rigorous computer science approach to Py...",
  "sources": [
    {"title": "Python Programming MOOC 2026", "url": "https://programming-26.mooc.fi/", "snippet": "About this course - Python Programming MOOC 2026\nby Universi..."},
    {"title": "Reddit · r/learnpython", "url": "https://www.reddit.com/r/learnpython/comments/10rlgnu/recommended_free_online_python_courses/", "snippet": "Recommended free online Python courses : r/learnpython\nFebru..."}
  ],
  "captured_at": "2026-11-10T10:59:10.175611+00:00"
}
```



This run came back with two source cards, a different pair from the Step 4 run since AI Mode answers change between runs, and both now carry their real URLs. The titles are still card labels like "Reddit · r/learnpython", though each URL leads to the exact page, down to the thread ID in the Reddit link!



## Step 7: Handle Blocking and Empty AI Mode Responses

Unfortunately, Google AI Mode doesn't always hand over a clean answer. When it doesn't, we'll usually see one of the errors our code already raises or prints, or Google's own block page:

| Symptom | Likely cause | What to do |
|---|---|---|
| CAPTCHA or `/sorry/` page | Google bot challenge | Keep `unblocker=True` and `country="US"`, lower the request rate, retry with backoff |
| Timeout on `wait_for_selector` | The answer never finished, or Google served a different layout | Check the page in a real browser; confirm the feedback button selector still matches |
| `ValueError: no AI Mode data extracted` | The extraction returned nothing for this page | Log the query, retry once, then inspect the HTML from the Step 3 fetch |
| `goto URL did not redirect` or `failed to resolve` | The redirect request was blocked or timed out | Keep the `/goto` link and retry the resolution later |
| Only 1 or 2 sources | Google showed fewer source cards for this answer | Treat it as a snapshot of that answer, not a parser bug |

Three different queries in our runs got 7, 1 and 2 source cards, and "best web scraping api" went from 7 sources in one run to the 6 in the tracking output below. So, a short source list reflects the answer Google picked more than our scraper.

For more on bypassing anti-bot protection in general, see our [anti-bot bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection-when-web-scraping) and the [Web Scraping API](https://scrapfly.io/products/web-scraping-api) product page.



## Optional: Track Google AI Mode Brand Citations Over Time

Now that one query returns clean JSON, tracking citations comes down to a loop over a list of prompts. Our tracker checks whether a domain appears in each answer's sources and appends every run to SQLite:

python

```python
import sqlite3
from urllib.parse import urlparse


async def track_brand_citations(prompts: list[str], brand_domain: str, db_path: str = "ai_mode_citations.db") -> float:
    matches = 0
    with sqlite3.connect(db_path) as conn:
        conn.execute(
            "CREATE TABLE IF NOT EXISTS runs (captured_at TEXT, prompt TEXT, sources TEXT, domain_matched INTEGER)"
        )
        for prompt in prompts:
            try:
                result = await scrape_ai_mode(prompt)
            except ValueError as error:
                print(f"{prompt[:40]:40} SKIP ({error})")
                continue
            domains = [urlparse(source["url"]).netloc for source in result["sources"]]
            matched = int(any(domain.endswith(brand_domain) for domain in domains))
            conn.execute(
                "INSERT INTO runs VALUES (?, ?, ?, ?)",
                (result["captured_at"], prompt, json.dumps(result["sources"]), matched),
            )
            matches += matched
            print(f"{prompt[:40]:40} sources={len(domains)} matched={matched}")
    rate = matches / len(prompts) if prompts else 0.0
    print(f"citation rate for {brand_domain}: {matches}/{len(prompts)} = {rate:.0%}")
    return rate


asyncio.run(
    track_brand_citations(
        prompts=["best python web scraping libraries", "how to scrape google search results", "best web scraping api"],
        brand_domain="youtube.com",
    )
)
```



Note that we match on the resolved hostname, as a `/goto` link would never match. Here's the output of our YouTube run:

text

```text
best python web scraping libraries       sources=7 matched=1
how to scrape google search results      sources=8 matched=1
best web scraping api                    sources=6 matched=1
citation rate for youtube.com: 3/3 = 100%
```



As we can see, YouTube made it into the sources of all 3 answers!



## Adaptation Note: Scraping Google AI Overview Instead of AI Mode

Unlike AI Mode, AI Overview is a conditional answer box which Google shows above some of the standard `/search?q=...` results. There's no `udm=50` in its URL and the page layout differs too. So, an AI Overview scraper fits better as an extension of the standard Google SERP recipe than as a copy of ours.



## FAQ

Can I scrape Google AI Mode without Playwright?

Yes. Our scraper doesn't launch a browser locally, as Scrapfly renders the `udm=50` page on its side. Our `wait_for_selector` setting also makes it wait for the "Good response" button before sending the page back.







What data can I extract from Google AI Mode?

As we've seen in the Step 6 output, it's the answer text along with source titles, URLs and snippets. Other data visible on the page can be added as a new field to the extraction prompt.







Do I need to log in to scrape Google AI Mode?

No. Our scraper never logs in, and the public AI Mode page loads without a Google account.







Why do I sometimes get "Can't generate an AI Overview right now" errors?

The message comes from Google, and we treat it as a transient issue there rather than a bug in our scraper.









## Summary

We've taken a look at how to scrape Google AI Mode with Scrapfly. Our scraper fetched the `udm=50` page with `unblocker` and `render_js` enabled, waited for the "Good response" button and let an extraction prompt return the answer with its source cards.

Finally, a second batch of unrendered requests turned the `/goto` tokens into real URLs, which is what makes citation tracking by domain possible!

Unfortunately, Google AI Mode answers keep changing, and even the source count shifted between two runs of one query. So, why not delegate the rendering, unblocking and extraction to Scrapfly?



### Power your scraping with Scrapfly

Forget about getting blocked. Scrapfly handles anti-bot bypasses, browser rendering, and proxy rotation so you can focus on the data.



[Try for FREE!](https://scrapfly.io/register)



Legal Disclaimer and Precautions

This tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect:

- Do not scrape at rates that could damage the website.
- Do not scrape data that's not available publicly.
- Do not store PII of EU citizens protected by GDPR.
- Do not repurpose *entire* public datasets which can be illegal in some countries.

Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer.

 

   [  Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents















 

  Table of Contents

- [Key Takeaways](#key-takeaways)
- [Before You Start: What This Google AI Mode Scraper Will Return](#before-you-start-what-this-google-ai-mode-scraper-will-return)
- [Step 1: Install and Initialize the Scrapfly Python SDK](#step-1-install-and-initialize-the-scrapfly-python-sdk)
- [Step 2: Build the Google AI Mode URL](#step-2-build-the-google-ai-mode-url)
- [Step 3: Fetch the Rendered AI Mode Page](#step-3-fetch-the-rendered-ai-mode-page)
- [Step 4: Extract the AI Mode Answer and Sources](#step-4-extract-the-ai-mode-answer-and-sources)
- [Step 5: Resolve the Google /goto Source URLs](#step-5-resolve-the-google-goto-source-urls)
- [Step 6: Put the Google AI Mode Scraper Together](#step-6-put-the-google-ai-mode-scraper-together)
- [Step 7: Handle Blocking and Empty AI Mode Responses](#step-7-handle-blocking-and-empty-ai-mode-responses)
- [Optional: Track Google AI Mode Brand Citations Over Time](#optional-track-google-ai-mode-brand-citations-over-time)
- [Adaptation Note: Scraping Google AI Overview Instead of AI Mode](#adaptation-note-scraping-google-ai-overview-instead-of-ai-mode)
- [FAQ](#faq)
- [Summary](#summary)
 
    Join the Newsletter  Get monthly web scraping insights 

 

  



Scale Your Web Scraping

Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits.

  No credit card required  1,000 free API credits  Anti-bot bypass included 

 [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) 

 Not ready? [Get our newsletter](#newsletter-modal) instead. 

 

 

## Related Articles

 [     

 python data-parsing 

### How to Parse Web Data with Python and Beautifulsoup

Beautifulsoup is one the most popular libraries in web scraping. In this tutorial, we'll take a hand-on overview of how ...

 

 ](https://scrapfly.io/blog/posts/web-scraping-with-python-beautifulsoup) [     

 python data-parsing 

### Quick Intro to Parsing JSON with JMESPath in Python

Introduction to JMESPath - JSON query language which is used in web scraping to parse JSON datasets for scrape data.

 

 ](https://scrapfly.io/blog/posts/parse-json-jmespath-python) [     

 python hidden-api 

### 3 Best Open-Source Job Scrapers for Multi-Board Data in 2026

Three open-source job scrapers ranked by board coverage, schema, and dated maintenance, plus a source-available Scrapfly...

 

 ](https://scrapfly.io/blog/posts/best-open-source-job-scrapers) 

  



   



 Scale your web scraping effortlessly, **1,000 free credits** [Start Free](https://scrapfly.io/register)