Python Knowledgebase

To scrape tables to Excel spreadsheet we can use bs4, requets and xlsxwriter packages for Python. Here's how.

To handle alert-type pop ups in Playwright the on "dialog" event can be captured and interacted with in both Python and NodeJS playwright clients

To check whether an HTML element is present on the page using Playwright the page.locator() method can be used. Here's how.

selenium error "chromedriver executable needs to be in PATH" means that chrome driver is not installed or reachable - here's how to fix it.

selenium error "geckodriver executable needs to be in PATH" means that gecko driver is not installed or reachable - here's how to fix it.

Python's ConnectTimeout exception is caused when connection can't be established fast enough. Here's how to fix it.

Python requests.ReadTimeout is caused when resources cannot be read fast enough. Here's how to fix it.

Python "requests.MissingSchema" exception is usually caused by a missing protocol part in the URL. Most commonly when relative URL is used.

Python's requests.SSLError is caused when encryption certificates mismatch for HTTPS type of URLs. Here's how to fix it.

Python's requests.TooManyRedirects exception is raised when server continues to redirect >30 times. Here's how to fix it.

To save session between script runs we can save and load requests session cookies to disk. Here's how to do in Python requests.

To take page screenshots in playwright we can use page.screenshot() method. Here's how to select areas and how to screenshot them in playwright.

There are 2 ways to determine URL file type: guess by url extension using mimetypes module or do a HTTP HEAD request. Here's how.

In Selenium, the scrollIntoView JavaScript function can be used to scroll to a specific HTML element. Here's how to use it in Selenium.

To increase Selenium's performance we can block images. To do that with Chrome browser "prefs" launch option can be used. Here's how.