How to open Python http responses in a web browser?

To view Python's HTTP responses in a web browser we can save the contents to a temporary file and open it in the default web browser using Python's webbrowser module:

import webbrowser
from tempfile import NamedTemporaryFile

# this can work with any response object of any http client like:
import requests
import httpx

def view_in_browser(response):
    """open httpx or requests Response object in default browser"""
    # first - save content to a temporary file:
    with NamedTemporaryFile("wb", delete=False, suffix=".html") as file:
        file.write(response.content)
    # open temporary file in a new browser tab as a web page
    webbrowser.open_new_tab(f"file://{file.name}")
    # - or new window
    # webbrowser.open_new(f"file://{file.name}")
    # - or current active tab
    # webbrowser.open(f"file://{file.name}")
    
# example use:
response = requests.get("http://scrapfly.io/")
response = httpx.get("http://scrapfly.io/")  
view_in_browser(response)

This is a great tool for developing web scrapers as it allows to easily visualize and debug the scraper process as well as to use the browser's developer tools.

Related Posts

How to Scrape RightMove Real Estate Property Data with Python

In this scrape guide we'll be taking a look at scraping RightMove.co.uk - one of the most popular real estate listing websites in the United Kingdom. We'll be scraping hidden web data and backend APIs directly using Python.

How to Scrape Google Search with Python

In this scrape guide we'll be taking a look at how to scrape Google Search - the biggest index of public web. We'll cover dynamic HTML parsing and SERP collection itself.

Quick Intro to Parsing JSON with JSONPath in Python

JSONPath is a path expression language for JSON. It is used to query data from JSON datasets and it is similar to XPath query language for XML documents. Parsing HTML