How to open Python http responses in a web browser?

To view Python's HTTP responses in a web browser we can save the contents to a temporary file and open it in the default web browser using Python's webbrowser module:

import webbrowser
from tempfile import NamedTemporaryFile

# this can work with any response object of any http client like:
import requests
import httpx

def view_in_browser(response):
    """open httpx or requests Response object in default browser"""
    # first - save content to a temporary file:
    with NamedTemporaryFile("wb", delete=False, suffix=".html") as file:
        file.write(response.content)
    # open temporary file in a new browser tab as a web page
    webbrowser.open_new_tab(f"file://{file.name}")
    # - or new window
    # webbrowser.open_new(f"file://{file.name}")
    # - or current active tab
    # webbrowser.open(f"file://{file.name}")
    
# example use:
response = requests.get("http://scrapfly.io/")
response = httpx.get("http://scrapfly.io/")  
view_in_browser(response)

This is a great tool for developing web scrapers as it allows to easily visualize and debug the scraper process as well as to use the browser's developer tools.

Question tagged: Python, requests, httpx

Related Posts

How to Scrape Google SEO Keyword Data and Rankings

In this article, we’ll take a look at SEO web scraping, what it is and how to use it for better SEO keyword optimization. We’ll also create an SEO keyword scraper that scrapes Google search rankings and suggested keywords.

How to Effectively Use User Agents for Web Scraping

In this article, we’ll take a look at the User-Agent header, what it is and how to use it in web scraping. We'll also generate and rotate user agents to avoid web scraping blocking.

How to Observe E-Commerce Trends using Web Scraping

In this example web scraping project we'll be taking a look at monitoring E-Commerce trends using Python, web scraping and data visualization tools.