How to select element with one of many names in XPath?

The XPath syntax allows interaction with web elements' attributes, such as class, id, href, and others, through the @ XPath expression. This enables us to select any element in the web page DOM based on its attribute values with XPath selectors.
To select elements based on their attribute values, we can follow either of two XPath expressions:

  • Select the desired element by its exact attribute value, such as [@attribute='value']
  • Select by partial attribute value using XPath contains, such as [contains(@attribute, 'value')]

Let's go over practical examples of applying the above XPath queries to extract data on both HTML and XML documents.

Using XPath @ expression

The @ XPath expression selects a web element using the exact element's attribute:

<!-- select all links by selecting the @href attributes --> <html> <a href="/categories/1">category</a> <a href="/product/1">product 1</a> <a href="/product/2">product 2</a> <a href="/product/3">product 3</a> </html>

Here, we use the XPath query to select all elements containing the href tag.

Using XPath contains expression

The contains XPath function enables selecting a particular web element based on its partial text match:

<!-- select only product links by checking @href attribute --> <html> <a href="/categories/1">category</a> <a href="/product/1">product 1</a> <a href="/product/2">product 2</a> <a href="/product/3">product 3</a> </html>

Above, we select all href attributes by using a partial text search. For further details on XPath selectors, refer to our dedicated guide.

Parsing HTML with Xpath

Take a deep dive into XPath selectors through an expression cheatsheet and explore concepts using an interactive XPath tester. We'll also cover implementations, common idioms, and tips when it comes to XPath in web scraping.

xpath guide banner

XML

Note that the mentioned XPath functions can also be applied with the XML path language. Here's an example of parsing an XML document:

<product> <variant> <price>99$</price> <rate type="product A">4.5</rate> </variant> <variant> <price>120$</price> <rate type="product B">3.5</rate> </variant> </product>

We can also use the XPath contains expression to text by partial attribute text:

<product> <variant> <price>99$</price> <rate type="product A">4.5</rate> </variant> <variant> <price>120$</price> <rate type="product B">3.5</rate> </variant> </product>

For more on parsing XML with XPath and CSS selectors, refer to our dedicated guide.

How to Parse XML

Learn about XML parsing. We'll start by defining XML files, their format, and how to navigate them for data extraction. Finally, we'll explain how to parse XML in Python through a practical web scraping example.

xml parsing guide banner

Provided by Scrapfly

This knowledgebase is provided by Scrapfly data APIs, check us out! 👇