How to find elements by CSS selector in Selenium

CSS selectors are one of the most popular ways to parse HTML pages when web scraping. Using Selenium, to find elements by CSS selectors we can use driver.find_element() and driver.find_elements() methods:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://httpbin.dev/html")

element = driver.find_element(By.CSS_SELECTOR, 'p')
# then we can get the element text
print(element.text)
"Availing himself of the mild, summer-cool weather that now reigned in these latitudes..."
# we can also get tag name and attributes:
print(element.tag_name)
print(element.get_attribute("class"))

# for multiple elements we need to iterate
for element in driver.find_elements(By.CSS_SELECTOR, 'p'):
    print(element.text)

driver.close()

Also see: How to find elements by XPath in Selenium

Question tagged: Selenium, Css Selectors, Data Parsing

Related Posts

How to Scrape With Headless Firefox

Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.

Selenium Wire Tutorial: Intercept Background Requests

In this guide, we'll explore web scraping with Selenium Wire. We'll define what it is, how to install it, and how to use it to inspect and manipulate background requests.

Web Scraping Dynamic Web Pages With Scrapy Selenium

Learn how to scrape dynamic web pages with Scrapy Selenium. You will also learn how to use Scrapy Selenium for common scraping use cases, such as waiting for elements, clicking buttons and scrolling.