XPath vs CSS selectors: what's the difference?

XPath and CSS selectors are both used to parse HTML when web scraping. Both of these are small path languages that perform the same function. So, what's the difference?

Generally, CSS selectors are more brief and popular as it's the same language used to apply styles in web development. However, XPath selectors are more powerful though a bit more complex. In particular XPath offers these 3 powerful features that CSS selectors do not:

  • Traverse up the HTML tree, i.e. select element parent nodes.
  • Find elements by text.
  • Easily define custom functions and more built-in functions like regular expression matching.

When web scraping, it's best to mix both to take advantage of the strengths of these two different tools. For example, let's take a look at this example page and how CSS ad XPath selectors can be used to their strengths:

<div class="product">
  <div class="price">
    <div data-price="22.84">$22.84</div>
  </div>
  <div>
    <div>Company Name inc.</div>
    <div>
      <div>website: <a href="http://example.com">example.com</a></div>
    </div>
  </div>
</div>

To extract the price we can use a simple CSS selector:

.product>.price::attr(data-price)

However, CSS selectors cannot find elements by their text value or navigate up the HTML tree. So, to select Company Name inc. we'd have better luck with XPath:

//div[contains(text(),'website:')]/../../div[1]/text()

In the example above we're finding a div element that contains text website:, then select its grandparent and the first div child which is the name of the company!

To summarize XPath vs CSS selectors - both are great tools for parsing HTML though CSS selectors are briefer and easier to use while XPath is more powerful but more verbose and complex. Luckily, most programming languages support both and when web scraping we're best to mix these two technologies!

Related Posts

Web Scraping With Ruby

Introduction to web scraping with Ruby. How to handle http connections, parse html files for data, best practices, tips and an example project.

Parsing HTML with Xpath

Introduction to xpath in the context of web-scraping. How to extract data from HTML documents using xpath, best practices and available tools.

Web Scraping With PHP 101

Introduction to web scraping with PHP. How to handle http connections, parse html files for data, best practices, tips and an example project.