To view Python's HTTP responses in a web browser we can save the contents to a temporary file and open it in the default web browser using Python's webbrowser module:
import webbrowser
from tempfile import NamedTemporaryFile
# this can work with any response object of any http client like:
import requests
import httpx
def view_in_browser(response):
"""open httpx or requests Response object in default browser"""
# first - save content to a temporary file:
with NamedTemporaryFile("wb", delete=False, suffix=".html") as file:
file.write(response.content)
# open temporary file in a new browser tab as a web page
webbrowser.open_new_tab(f"file://{file.name}")
# - or new window
# webbrowser.open_new(f"file://{file.name}")
# - or current active tab
# webbrowser.open(f"file://{file.name}")
# example use:
response = requests.get("http://scrapfly.io/")
response = httpx.get("http://scrapfly.io/")
view_in_browser(response)
This is a great tool for developing web scrapers as it allows to easily visualize and debug the scraper process as well as to use the browser's developer tools.
