Python Knowledgebase

HTTP2 is still relatively new protocol version that is not yet widely supported. Here are the options for HTTP2 client in Python.

When it comes to these 3 popular http client packages they have different strenghts. Here's how to choose the right fit.

To use proxies with Python's httpx library the proxies parameter can be used for http, https and socks5 proxies. Here's how.

To scrape all images from a given website python with beautifulsoup and httpx can be used. Here's an example.

To select dictionary keys recursively in Python the nested-lookup package can be used. Here's how.

There are several popular options when it comes to JSON dataset parsing in Python. The most popular packages are Jmespath and Jsonpath.

cURL through libcurl is a popular library used in HTTP connections and can be used with Python through wrapper libraries like pycurl.

To preview Python http responses we can use temporary files and the built-in webbrowser module. Here's how.

To scrape tables to Excel spreadsheet we can use bs4, requets and xlsxwriter packages for Python. Here's how.

To click on a pop up dialog or an alert in Playwright we can use dialog event capture using `page.on()` method. Here's how.

To check whether an HTML element is present on the page using Playwright the page.locator() method can be used. Here's how.

selenium error "chromedriver executable needs to be in PATH" means that chrome driver is not installed or reachable - here's how to fix it.

selenium error "geckodriver executable needs to be in PATH" means that gecko driver is not installed or reachable - here's how to fix it.

Python's ConnectTimeout exception is caused when connection can't be established fast enough. Here's how to fix it.

Python requests.ReadTimeout is caused when resources cannot be read fast enough. Here's how to fix it.

Python requests.MissingSchema exception is caused by missing URL detaisl. Here's how to fix it.

Python's requests.SSLError is caused when encryption certificates mismatch for HTTPS type of URLs. Here's how to fix it.

Python's requests.TooManyRedirects exception is raised when server continues to redirect >30 times. Here's how to fix it.

To save session between script runs we can save and load requests session cookies to disk. Here's how to do in Python requests.

To take page screenshots in playwright we can use page.screenshot() method. Here's how to select areas and how to screenshot them in playwright.

There are 2 ways to determine URL file type: guess by url extension using mimetypes module or do a HTTP HEAD request. Here's how.

To scroll to a specific HTML element in selenium scrollIntoView() javascript function can be used. Here's how to call it in Selenium.

To increase Selenium's performance we can block images. To do that with Chrome browser "prefs" launch option can be used. Here's how.