Cloud Browser File Downloads

Beta Feature: Cloud Browser is currently in beta and available to staff members only.

Cloud Browser supports automatic file download handling through a custom CDP extension. When the browser triggers a file download (PDFs, images, spreadsheets, etc.), you can retrieve the downloaded files using special CDP commands.

How It Works

When a navigation or user action triggers a file download in the browser:

The browser intercepts the download request
The file is saved to a temporary directory on the remote browser
Download events are emitted so you can track progress
You can retrieve the file content using the ScrapiumBrowser.getDownloads command

Key Points

Files are returned as base64-encoded strings
Maximum download size is 25 MB per file
Downloads are automatically cleaned up when the session ends
Multiple file downloads are fully supported in a single session
Browser download popups and confirmations are automatically bypassed for seamless automation

Automation-Friendly: Cloud Browser automatically disables browser download protection mechanisms (such as multiple download confirmation popups) that would normally interrupt automation scripts. Your downloads will proceed without any user interaction required.

CDP Commands

Cloud Browser extends the standard CDP protocol with custom ScrapiumBrowser commands for file download handling:

Command	Description
`ScrapiumBrowser.getDownloads`	Retrieve all downloaded files as base64-encoded content. Optionally delete files after retrieval.
`ScrapiumBrowser.getDownloadsMetadatas`	Get metadata (filename and size) for all downloaded files without retrieving content.

ScrapiumBrowser.getDownloads

Retrieves all files currently in the download directory, encoded as base64 strings. This command takes no parameters.

Response

{
  "files": {
    "document.pdf": "JVBERi0xLjQKJe...",
    "report.xlsx": "UEsDBBQAAAA..."
  }
}

ScrapiumBrowser.getDownloadsMetadatas

Retrieves metadata for all downloaded files without transferring the actual content. Useful for checking what files are available before downloading.

Response

{
  "metadata": {
    "document.pdf": 245760,
    "report.xlsx": 102400
  }
}

Code Examples

Download a file after clicking a download button:

const puppeteer = require('puppeteer-core');

const API_KEY = '';
const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter`;

async function downloadFile() {
    const browser = await puppeteer.connect({ browserWSEndpoint: BROWSER_WS });
    const page = await browser.newPage();

    // Navigate and trigger download
    await page.goto('https://web-scraping.dev/file-download');
    await page.click('#download-btn');
    await new Promise(resolve => setTimeout(resolve, 3000));

    // Get the CDP session and retrieve downloads
    const client = await page.createCDPSession();
    const result = await client.send('ScrapiumBrowser.getDownloads');

    for (const [filename, base64Content] of Object.entries(result.files)) {
        const buffer = Buffer.from(base64Content, 'base64');
        require('fs').writeFileSync(filename, buffer);
        console.log(`Saved: ${filename} (${buffer.length} bytes)`);
    }
    await browser.close();
}

downloadFile();

const { chromium } = require('playwright');

const API_KEY = '';
const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter`;

async function downloadFile() {
    const browser = await chromium.connectOverCDP(BROWSER_WS);
    const context = browser.contexts()[0];
    const page = context.pages()[0] || await context.newPage();
    const cdpSession = await page.context().newCDPSession(page);

    // Navigate and trigger download
    await page.goto('https://web-scraping.dev/file-download');
    await page.click('#download-btn');
    await page.waitForTimeout(3000);

    // Retrieve and save files
    const downloads = await cdpSession.send('ScrapiumBrowser.getDownloads');
    for (const [filename, base64Content] of Object.entries(downloads.files)) {
        const buffer = Buffer.from(base64Content, 'base64');
        require('fs').writeFileSync(filename, buffer);
        console.log(`Saved ${filename} (${buffer.length} bytes)`);
    }
    await browser.close();
}

downloadFile();

import base64
from playwright.sync_api import sync_playwright

API_KEY = ''
BROWSER_WS = f'wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter'

def download_file():
    with sync_playwright() as p:
        browser = p.chromium.connect_over_cdp(BROWSER_WS)
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else context.new_page()
        cdp_session = context.new_cdp_session(page)

        # Navigate and trigger download
        page.goto('https://web-scraping.dev/file-download')
        page.click('#download-btn')
        page.wait_for_timeout(3000)

        # Retrieve and save files
        downloads = cdp_session.send('ScrapiumBrowser.getDownloads')
        for filename, base64_content in downloads['files'].items():
            file_bytes = base64.b64decode(base64_content)
            with open(filename, 'wb') as f:
                f.write(file_bytes)
            print(f'Saved {filename} ({len(file_bytes)} bytes)')
        browser.close()

download_file()

import asyncio
import base64
from playwright.async_api import async_playwright

API_KEY = ''
BROWSER_WS = f'wss://browser.scrapfly.io?api_key={API_KEY}&proxy_pool=datacenter'

async def download_file():
    async with async_playwright() as p:
        browser = await p.chromium.connect_over_cdp(BROWSER_WS)
        context = browser.contexts[0]
        page = context.pages[0] if context.pages else await context.new_page()
        cdp_session = await context.new_cdp_session(page)

        # Navigate and trigger download
        await page.goto('https://web-scraping.dev/file-download')
        await page.click('#download-btn')
        await page.wait_for_timeout(3000)

        # Retrieve and save files
        downloads = await cdp_session.send('ScrapiumBrowser.getDownloads')
        for filename, base64_content in downloads['files'].items():
            file_bytes = base64.b64decode(base64_content)
            with open(filename, 'wb') as f:
                f.write(file_bytes)
            print(f'Saved {filename} ({len(file_bytes)} bytes)')
        await browser.close()

asyncio.run(download_file())

Monitoring Download Progress

Cloud Browser emits standard Chrome CDP events that you can listen to for tracking download progress:

Event	Description
`Browser.downloadWillBegin`	Fired when a download is about to start. Contains URL and suggested filename.
`Browser.downloadProgress`	Fired periodically during download. Contains progress state and bytes received.

Listening to Download Events

const puppeteer = require('puppeteer-core');

const API_KEY = '';
const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter`;

async function monitorDownloads() {
    const browser = await puppeteer.connect({
        browserWSEndpoint: BROWSER_WS,
    });

    const page = await browser.newPage();
    const client = await page.createCDPSession();

    // Listen for download start
    client.on('Browser.downloadWillBegin', (event) => {
        console.log('Download starting:', {
            url: event.url,
            filename: event.suggestedFilename,
            guid: event.guid
        });
    });

    // Listen for download progress
    client.on('Browser.downloadProgress', (event) => {
        console.log('Download progress:', {
            guid: event.guid,
            state: event.state,  // 'inProgress', 'completed', 'canceled'
            receivedBytes: event.receivedBytes,
            totalBytes: event.totalBytes
        });

        if (event.state === 'completed') {
            console.log(`Download completed: ${event.receivedBytes} bytes`);
        }
    });

    // Navigate and trigger download
    await page.goto('https://web-scraping.dev/file-download');
    await page.click('#download-btn');

    // Wait for download to complete
    await new Promise(resolve => setTimeout(resolve, 5000));

    // Retrieve the file
    const result = await client.send('ScrapiumBrowser.getDownloads');
    console.log('Downloaded files:', Object.keys(result.files));

    await browser.close();
}

monitorDownloads();

Multiple File Downloads

Cloud Browser fully supports downloading multiple files in a single session. Unlike regular browsers that may prompt for confirmation when triggering multiple downloads, Cloud Browser automatically accepts all downloads without interruption.

Example: Downloading Multiple Files

const puppeteer = require('puppeteer-core');

const API_KEY = '';
const BROWSER_WS = `wss://browser.scrapfly.io?api_key=${API_KEY}&proxy_pool=datacenter`;

async function downloadMultipleFiles() {
    const browser = await puppeteer.connect({
        browserWSEndpoint: BROWSER_WS,
    });

    const page = await browser.newPage();
    const client = await page.createCDPSession();

    await page.goto('https://web-scraping.dev/file-download');

    // Trigger multiple downloads - no confirmation popups will appear
    await page.click('#download-btn');
    await page.click('#download-pdf');
    await page.click('#download-csv');

    // Wait for all downloads to complete
    await new Promise(resolve => setTimeout(resolve, 5000));

    // Retrieve all downloaded files at once
    const result = await client.send('ScrapiumBrowser.getDownloads');

    console.log(`Downloaded ${Object.keys(result.files).length} files:`);
    for (const [filename, base64Content] of Object.entries(result.files)) {
        const buffer = Buffer.from(base64Content, 'base64');
        require('fs').writeFileSync(filename, buffer);
        console.log(`  - ${filename} (${buffer.length} bytes)`);
    }

    await browser.close();
}

downloadMultipleFiles();

Tip: When downloading multiple files, you can use getDownloadsMetadatas to check how many files are ready before retrieving them all with getDownloads.

Common Use Cases

PDF Downloads

Download PDFs generated by web applications, such as invoices, reports, or tickets that are created dynamically after form submission or authentication.

Export Files

Retrieve data exports (CSV, Excel, JSON) from dashboards and analytics platforms that require browser interaction to generate.

Generated Images

Download images that are generated on-demand, such as charts, QR codes, or dynamically created graphics.

Protected Documents

Access documents behind authentication or CAPTCHA protection that can only be downloaded through a real browser session.

Best Practices

Wait for Downloads to Complete

Always wait for the download to complete before calling getDownloads. You can either use a fixed timeout, listen for the Browser.downloadProgress event with state completed, or poll getDownloadsMetadatas until files appear.

Automatic Cleanup

Downloaded files are automatically cleaned up when the browser session ends (either when you close the CDP connection or when the session times out with auto_close=true). You don't need to manually delete files unless you want to clear downloads during a long-running session.

Check File Size First

For large files, call getDownloadsMetadatas first to check the file size. Remember that base64 encoding increases size by approximately 33%, so a 25 MB file will transfer as ~33 MB of base64 data.

Handle Download Failures

Downloads can fail or be canceled. Listen to the Browser.downloadProgress event and check for state: 'canceled' or state: 'failed' to handle errors gracefully.

Limitations

Maximum file size: 25 MB per file
Session-scoped: Downloads are only available during the session that triggered them
Auto-cleanup: All downloads are deleted when the browser session ends
Transfer overhead: Base64 encoding adds ~33% to transfer size

Cloud Browser Getting Started - Connection and basic usage
Puppeteer Integration - Full Puppeteer setup guide
Playwright Integration - Full Playwright setup guide
Cloud Browser Billing - Pricing and cost optimization
Error Reference - Troubleshooting guide

Cloud Browser File Downloads

How It Works

Key Points

CDP Commands

ScrapiumBrowser.getDownloads

Response

ScrapiumBrowser.getDownloadsMetadatas

Response

Code Examples

Monitoring Download Progress

Listening to Download Events

Multiple File Downloads

Example: Downloading Multiple Files

Common Use Cases

PDF Downloads

Export Files

Generated Images

Protected Documents

Best Practices

Wait for Downloads to Complete

Automatic Cleanup

Check File Size First

Handle Download Failures

Limitations

Related Documentation

Summary