How to find elements by XPath in Selenium

XPath selectors are one of the most popular ways to parse HTML pages when web scraping. Using Selenium, to find elements by XPath we can use driver.find_element() and driver.find_elements() methods:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://httpbin.dev/html")

element = driver.find_element(By.XPATH, '//p')
# then we can get the element text
print(element.text)
"Availing himself of the mild, summer-cool weather that now reigned in these latitudes..."
# we can also get tag name and attributes:
print(element.tag_name)
print(element.get_attribute("class"))

# for multiple elements we need to iterate
for element in driver.find_elements(By.XPATH, '//p'):
    print(element.text)

driver.close()

Also see: How to find elements by CSS selector in Selenium

Question tagged: Selenium, Headless Browsers, Python

Related Posts

How to Use Chrome Extensions with Playwright, Puppeteer and Selenium

In this article, we'll explore different useful Chrome extensions for web scraping. We'll also explain how to install Chrome extensions with various headless browser libraries, such as Selenium, Playwright and Puppeteer.

Intro to Web Scraping using Selenium Grid

In this guide, you will learn about installing and configuring Selenium Grid with Docker and how to use it for web scraping at scale.

How to Scrape Google Maps

We'll take a look at to find businesses through Google Maps search system and how to scrape their details using either Selenium, Playwright or ScrapFly's javascript rendering feature - all of that in Python.