🚀 We are hiring! See open positions

Selenium Knowledgebase

Selenium is one of the most popular tools for web scraping and web automation. It allows developers to control real web browsers like Chrome, Firefox, and Safari programmatically, making it a powerful choice for scraping dynamic websites that rely on JavaScript.

While Selenium is quite old it still holds up in web scraping and web automation tasks, particularly because of big community and ecosystem around it. It supports multiple programming languages, including Python, Java, C#, and JavaScript, making it accessible to a wide range of developers.

See below for more on Selenium in the context of web scraping and data programming 👇

How to click on cookie popups and modal alerts in Selenium?

To click on modal alerts like cookie popups in Selenium we can either find the button and click it or remove the modal elements. Here's how.

#selenium

How to handle popup dialogs in Selenium?

To click on a pop-up alert using Selenium the alert_is_present method can be used to wait for and interact with alerts. Here's how.

#selenium

How to capture background requests and responses in Selenium?

To capture background requests and response selenium needs to be extended with Selenium-wire. Here's how to do it.

#selenium

How to scroll to the bottom of the page with Selenium?

To scroll to the very bottom of the page the javascript evaluation feature can be used within a while loop. Here's how.

#selenium

How to block resources in Selenium and Python?

To block http resources in selenium we need an external proxy. Here's how to setup mitmproxy to block requests and responses in Selenium.

#selenium

Selenium: geckodriver executable needs to be in PATH?

selenium error "geckodriver executable needs to be in PATH" means that gecko driver is not installed or reachable - here's how to fix it.

#selenium
#python

Selenium: chromedriver executable needs to be in PATH?

selenium error "chromedriver executable needs to be in PATH" means that chrome driver is not installed or reachable - here's how to fix it.

#selenium
#python

How to block image loading in Selenium?

To increase Selenium's performance we can block images. To do that with Chrome browser "prefs" launch option can be used. Here's how.

#selenium
#python

Articles Related to Selenium