[Blog](https://scrapfly.io/blog) / [css-selectors](https://scrapfly.io/blog/tag/css-selectors) / [7 Best Python HTML Parsers for Web Scraping in 2026](https://scrapfly.io/blog/posts/best-python-html-parsers) # 7 Best Python HTML Parsers for Web Scraping in 2026 by [Ziad Shamndy](https://scrapfly.io/blog/author/ziad) Oct 27, 2026 22 min read [\#css-selectors](https://scrapfly.io/blog/tag/css-selectors) [\#data-parsing](https://scrapfly.io/blog/tag/data-parsing) [\#python](https://scrapfly.io/blog/tag/python) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers "Share on LinkedIn") [ ](https://x.com/intent/tweet?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers&text=7%20Best%20Python%20HTML%20Parsers%20for%20Web%20Scraping%20in%202026 "Share on X") [ ](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers "Share on Facebook") Summarize this article with [ ](https://chat.openai.com/?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers) [ ](https://claude.ai/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers) [ ](https://x.com/i/grok?text=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers) [ ](https://www.perplexity.ai/search/new?q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers) [ ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20article%20and%20explain%20how%20Scrapfly%20helps%20me%20scrape%20any%20website%20at%20scale%20and%20bypass%20anti-bot%20systems%20for%20my%20use%20case%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fposts%2Fbest-python-html-parsers) Python has plenty of HTML parsing libraries and while they all do the same job of turning HTML into something we can query, they differ quite a bit in their APIs, speed and dependencies. Note that a parser only works with HTML that's already in Python - it won't fix a 403 block or run any JavaScript. In this article, we'll be taking a look at seven popular Python HTML parsers and how they compare when parsing the same page: a 1.22 MB rendered Google search page. Each one of them found the same nine headings on it, but when it comes to speed it's a different story - the median parse and selection time was 3.25 ms for the fastest parser and 70.76 ms for the slowest one! ## Key Takeaways - On our 1.22 MB Google results page, lxml (3.25 ms) and selectolax (3.36 ms) were tied for fastest and html5lib was slowest at 70.76 ms. All seven found the same nine headings. - Pick by selectors first: lxml and Parsel do XPath and CSS, selectolax does CSS only, and `HTMLParser` has no selectors at all. - Beautiful Soup is a navigation API on top of `html.parser`, lxml or html5lib. Name the backend in code, or another machine may build a different tree. - Parsel and PyQuery run on lxml, so switching to them changes the API, not the parsing speed. - None of the seven fetches pages or runs JavaScript. When a parser finds nothing, check the response before the selectors. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## Which Python HTML Parser Is Best for Web Scraping in 2026? Each of these seven parsers has its own way of selecting data: - lxml - parses both HTML and XML documents and lets us select data with the `xpath()` method. - selectolax - Python bindings to the Lexbor HTML engine with `css()` and `text()` helpers for selecting data. - Beautiful Soup - a navigation library with its own `find()` methods. It doesn't parse HTML by itself though and uses `html.parser`, lxml or html5lib to do it. - Parsel - the selector library from Scrapy which can be used with CSS, XPath and even regular expressions through its `Selector` object. - html5lib - a pure Python parser that follows the WHATWG spec, so it reads broken HTML just like a web browser would. - `HTMLParser` - the parser from Python's standard `html.parser` module. It doesn't create a tree though and instead calls methods like `handle_starttag()` and `handle_data()` as it reads through the HTML. - PyQuery - a jQuery port for Python which runs on lxml. Here's how they compare when parsing a 1.22 MB Google search page and selecting its headings: | Python HTML parser | Best for | Query API | Test median | |---|---|---|---| | lxml | XPath plus HTML and XML | XPath, CSS | 3.25 ms | | selectolax | Thin high throughput parsing | CSS and text | 3.36 ms | | Beautiful Soup | Readable prototypes | `find`, CSS | 28.19 ms | | Parsel | Extraction pipelines | CSS, XPath, regex | 3.90 ms | | html5lib | WHATWG tree construction | Tree builder | 70.76 ms | | stdlib HTMLParser | Zero dependencies | Callbacks | 12.92 ms | | PyQuery | jQuery style traversal | jQuery CSS | 4.93 ms | lxml and selectolax came out on top with 3.25 ms and 3.36 ms while html5lib was the slowest at 70.76 ms. Though keep in mind it's just one page on one CPU, so on another page or machine these numbers could look different. The time to fetch the page or render it in a browser isn't part of them either, and each parser had to find the same nine headings for its time to count. You won't find Scrapy, Playwright, Selenium or any HTTP clients on this list. These are used to retrieve web pages (and in the case of browsers, render them) while parsers only come in once the HTML is retrieved. So, they're not really competing with each other - a web scraper usually needs both! With the short answer out of the way, let's take a look at how these parsers were benchmarked. ## How Were the Python HTML Parsers Benchmarked? For our benchmark we wanted a real page rather than imaginary HTML content, so we went with a rendered Google search page which weighs 1,221,207 bytes. Then, we had each parser go through the whole page and pick out the nine result headings using the `h3.LC20lb` CSS selector. We ran it on September 2, 2026, on an AMD Ryzen AI Max+ 395 CPU running Python 3.14.7, with the parser versions in the table below. We gave every parser 3 warmup runs before timing the next 20, and a run had to return all nine headings to count. Here are the median and p95 times we got (p95 being the time 95% of the runs finished under): | Parser | Version | Median | p95 | Matches | |---|---|---|---|---| | lxml | 6.1.2 | 3.25 ms | 3.30 ms | 9 | | selectolax | 0.4.11 | 3.36 ms | 3.40 ms | 9 | | Parsel | 1.11.0 | 3.90 ms | 4.01 ms | 9 | | PyQuery | 2.1.0 | 4.93 ms | 5.15 ms | 9 | | stdlib HTMLParser | Python 3.14.7 | 12.92 ms | 13.04 ms | 9 | | Beautiful Soup with `html.parser` | 4.15.0 | 28.19 ms | 29.94 ms | 9 | | html5lib | 1.1 | 70.76 ms | 72.38 ms | 9 | So, lxml came in first with 3.25 ms and selectolax second with 3.36 ms. That's a bit different from what other benchmarks online show, where selectolax is 15x to 46x faster than lxml - here lxml beat it by 0.11 ms. If you'd like to try this benchmark on your own machine, all we need is the parsers and a saved HTML page. For this example, let's install lxml (with cssselect for CSS selector support), selectolax and Beautiful Soup and save the product listing page from web-scraping.dev: shell ```shell pip install lxml cssselect selectolax beautifulsoup4 curl -s "https://web-scraping.dev/products" -o products.html ``` Next, we can time each parser with this script - let's save it as `parser_bench.py`: python ```python import sys import time from statistics import median, quantiles from bs4 import BeautifulSoup from lxml import html from selectolax.lexbor import LexborHTMLParser def with_lxml(text, selector): return len(html.fromstring(text).cssselect(selector)) def with_selectolax(text, selector): return len(LexborHTMLParser(text).css(selector)) def with_beautifulsoup(text, selector): return len(BeautifulSoup(text, "html.parser").select(selector)) def benchmark(name, parse, text, selector, expected): # a few warmup runs first, these aren't timed for _ in range(3): parse(text, selector) found = parse(text, selector) if found != expected: sys.exit(f"{name} found {found} elements, expected {expected}") times = [] for _ in range(20): start = time.perf_counter_ns() parse(text, selector) times.append(time.perf_counter_ns() - start) # perf_counter_ns() gives us nanoseconds, so divide by a million for ms median_ms = median(times) / 1_000_000 p95_ms = quantiles(times, n=20)[-1] / 1_000_000 print(f"{name:<14} median {median_ms:7.2f} ms p95 {p95_ms:7.2f} ms matches {found}") if __name__ == "__main__": path, selector, expected = sys.argv[1], sys.argv[2], int(sys.argv[3]) with open(path, encoding="utf-8") as file: text = file.read() print(f"document: {len(text.encode('utf-8')):,} bytes selector: {selector}") benchmark("lxml", with_lxml, text, selector, expected) benchmark("selectolax", with_selectolax, text, selector, expected) benchmark("beautifulsoup", with_beautifulsoup, text, selector, expected) ``` Here, each parser gets its own function which parses the HTML and counts the elements our selector matched. Then, in `benchmark()` we warm up the parser with three runs, check that it found as many elements as we expect and time the next 20 runs using `time.perf_counter_ns()`. To use it, we pass in the file name, the CSS selector and the expected number of matches. For example, for our products page it's `python parser_bench.py products.html div.product 5`, and for the Google page from our test it would be `h3.LC20lb 9`. Now, let's take a closer look at each of these seven parsers. First up is lxml. ## 1. lxml: Best Python HTML Parser for XPath and HTML or XML Work [lxml](https://lxml.de/lxmlhtml.html) is a Python library for parsing HTML and XML documents that's built on top of two C libraries: libxml2 and libxslt. It supports two ways of selecting data: XPath, which is built in, and CSS selectors through the extra `cssselect` package. Thanks to pre-compiled wheels, it's easy to install on most common platforms too. lxml is split into two modules - `lxml.html` and `lxml.etree`. The first one is for HTML and comes with some extra helpers for text, links and forms, while the second one is for XML: namespaces, validation, XPath and XSLT. Both share the same ElementTree style API. Let's take a quick look at the common pros and cons of lxml, especially compared to the other parsers on this list: Pros: - Fast - lxml 6.1.2 found all nine Google headings in 3.25 ms (median) and 3.30 ms (p95), the best result in our test. - Forgiving - broken HTML gets recovered instead of throwing an error. - HTML and XML parsing in one library. Cons: - Lower level API than Beautiful Soup or Parsel - which in turn means a bit more code to write. - Parsing XML with strict settings behaves differently from `lxml.html`, which can be confusing. We've written a whole tutorial on lxml which covers this library in depth: [Python lxml Tutorial: How to Parse HTML and XMLIn this tutorial, we'll take a deep dive into lxml, a powerful Python library that allows for parsing HTML and XML effectively. We'll start by explaining what lxml is, how to install it and using lxml for parsing HTML and XML files. Finally, we'll go over a practical web scraping with lxml.](https://scrapfly.io/blog/posts/intro-to-parsing-html-xml-python-lxml) That's it for lxml! Next up is selectolax, which finished just 0.11 ms behind but works in a very different way. ## 2. selectolax: Best Python HTML Parser for a Thin Lexbor CSS API [selectolax](https://github.com/rushter/selectolax) is a Python library for parsing HTML documents that's built on top of the Lexbor C engine through Cython bindings. For selecting data it only has CSS selectors, which we can run with the `css()` method, and once we find our nodes the `text()` helper gets their text. Older releases also shipped a second engine called Modest, but selectolax 1.0 removed it in October 2026 since the Modest C library isn't maintained anymore, so `selectolax.lexbor` is the module to import. To parse HTML with Lexbor, we create a `LexborHTMLParser` object from our HTML string. selectolax doesn't try to do everything lxml does, so the package stays fairly small. Let's take a quick look at the common pros and cons of selectolax: Pros: - Fast - selectolax 0.4.11 found all nine Google headings in 3.36 ms (median) and 3.40 ms (p95), pretty much tied with lxml. - Small - CSS selection and text extraction is pretty much all the package does. Cons: - Only capable of CSS selection, whereas lxml can do XPath too. selectolax also doesn't have the full ElementTree API that lxml has. For things like XPath's ancestor lookups we'd have to reach for lxml or Parsel instead. - Not as fast as lxml on our page. selectolax is often said to be 15x to 46x faster than lxml, however in our test lxml came out 0.11 ms ahead. If you're curious, the selectolax repository has a benchmark of its own across 754 domains. It's measured differently from ours though, so we can't compare the two results directly. That's it for selectolax! Both lxml and selectolax run on compiled engines with short and simple APIs, while our next parser, Beautiful Soup, focuses on readability instead. ## 3. Beautiful Soup: Best Python HTML Parser API for Readable Prototypes [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) is a Python library for navigating HTML documents that's built on top of other parsers: `html.parser`, lxml or html5lib. Since it doesn't parse HTML on its own, we pass the backend's name as the second argument of `BeautifulSoup()`, like `BeautifulSoup(html, "lxml")`. Beautiful Soup has a few pros and cons compared to the other parsers on this list: Pros: - Easy to read code. Something like `soup.find_all("div", class_="product")` tells us exactly what it does - which in turn means our parsing code is easier to maintain. - More than just search. We can also use version 4.15.0 to move around the tree, get the text of a node or even edit the HTML, on top of finding nodes with `find()`, `find_all()` and `select()`. - Not abandoned. The version from our test, 4.15.0, was the latest release in September 2026. Cons: - Slower than most parsers on our list. In our test with `html.parser` as the backend, it took 28.19 ms (median) to find the nine Google headings. - No page fetching. We still need an HTTP client or another tool to download or render the HTML for it. ### Choosing a Beautiful Soup Backend Currently, there are three backends Beautiful Soup can use, and each of them parses HTML a bit differently and at a different speed: - `html.parser` which comes with Python itself, so there's nothing extra to install. - `lxml` which has to be installed with `pip install lxml`, as it runs on compiled C code. - `html5lib` which also has to be installed separately, and it builds the tree just like a web browser would since it follows the WHATWG spec. Note that if no backend is named, Beautiful Soup will just use whichever one is installed, which can differ from one machine to another. So, let's always name it in our code - for example `BeautifulSoup(html, "html.parser")`. Our 28.19 ms result is with `html.parser`, so with lxml as the backend this number should come down. For more, see our full Beautiful Soup guide: [How to Parse Web Data with Python and BeautifulsoupBeautifulsoup is one the most popular libraries in web scraping. In this tutorial, we'll take a hand-on overview of how to use it, what is it good for and explore a real -life web scraping example.](https://scrapfly.io/blog/posts/web-scraping-with-python-beautifulsoup) If you're looking to replace Beautiful Soup rather than compare all of the parsers, take a look at our [BeautifulSoup alternatives](https://scrapfly.io/blog/answers/what-are-some-beautifulsoup-alternatives) answer instead. That's it for Beautiful Soup! Next up is Parsel, which focuses on extracting data rather than navigating the tree. ## 4. Parsel: Best Python HTML Parser API for CSS, XPath, and Extraction Pipelines [Parsel](https://parsel.readthedocs.io/en/latest/) is a Python library for extracting data from HTML, XML and even JSON documents. It supports CSS selectors and XPath for HTML and XML, JMESPath for JSON and regular expressions for anything else, with `.get()` and `.getall()` methods for getting the results. Parsel is the package Scrapy uses for parsing, but it can be installed on its own with `pip install parsel`. To parse HTML, we create a `Selector(text=...)` object and then run our selectors on it. We can chain selectors as well, so CSS and XPath can be used together: `selector.css(".product").xpath(".//h3/text()")`. Parsel has a few pros and cons compared to the other parsers on this list: Pros: - Faster than most parsers on our list. Parsel 1.11.0 needed 3.90 ms (median) and 4.01 ms (p95) for our Google page. - JSON support - Parsel can query JSON with JMESPath. - Free to use - Parsel is BSD licensed. Cons: - Not a parser engine of its own. It's more of a convenience layer over other parsing components than a new engine. - No page fetching. We still need an HTTP client or a browser to get the HTML. We've also written a full Parsel tutorial: [Guide to Parsel - the Best HTML Parsing in PythonLearn to extract data from websites with Parsel, a Python library for HTML parsing using CSS selectors and XPath.](https://scrapfly.io/blog/posts/guide-to-html-parsing-with-parsel-python) That's it for Parsel! So far, all four parsers focus on either speed or convenience, while the next one, html5lib, focuses on following the HTML spec as closely as possible. Scrapfly #### Extract structured data automatically? Scrapfly's Extraction API uses AI to turn any webpage into structured data — no selectors needed. [Try Free →](https://scrapfly.io/register)## 5. html5lib: Best Python HTML Parser for WHATWG Tree Construction [html5lib](https://html5lib.readthedocs.io/en/latest/) is a Python library for parsing HTML documents the same way modern web browsers do, by following the WHATWG HTML algorithm. It's written in pure Python. For the output, we can pick between `xml.etree`, `xml.dom.minidom` and `lxml.etree` trees. To parse HTML, we use `html5lib.parse()` and set the tree type with `treebuilder`, e.g. `html5lib.parse(html, treebuilder="lxml")`. html5lib has a few pros and cons compared to the other parsers on this list: Pros: - Same tree as a browser. html5lib follows the WHATWG spec, so broken HTML gets fixed the way browsers fix it. - Flexible trees - html5lib can output three different tree types. - Strict mode - `HTMLParser(strict=True)` raises a `ParseError` on broken HTML instead of fixing it. Cons: - Slowest parser on our list. html5lib 1.1 needed 70.76 ms (median) and 72.38 ms (p95) for our Google page. - Extra dependency - it has to be installed separately with `pip install html5lib`. - Not a browser - for some broken pages, the tree we get can still be a bit different from what Chrome shows in its developer tools. The reason html5lib is so slow is the spec itself, as following every step of it takes a lot more work than what lxml or selectolax do. ## 6. Python HTMLParser: Best Zero Dependency Event Parser [html.parser](https://docs.python.org/3/library/html.parser.html) is the HTML parser that comes with Python's standard library. It doesn't build a tree. For the output, we get calls to our own handler methods for every start tag, end tag, text and comment. To parse HTML, we subclass `HTMLParser`, write a `handle_starttag()` method and pass our HTML to `feed()`, e.g. `parser.feed(html)`. Python 3.14.1 also added a `scripting` parameter for choosing whether `noscript` content gets parsed or returned as is. Our benchmark used Python 3.14.7. `HTMLParser` has a few pros and cons compared to the other parsers on this list: Pros: - No dependencies - it comes with Python, so there's nothing to install. - Faster than Beautiful Soup and html5lib. Our counter needed 12.92 ms (median) for our Google page. - Forgiving - `feed()` doesn't raise errors on invalid HTML. Cons: - No selectors - it has no CSS or XPath support and no tree to use them on. - No tag matching - `handle_endtag()` isn't called for implicitly closed tags, and end tags aren't checked against start tags. - More callback code - for a node's parents or siblings, we have to track open tags ourselves. That's it for `HTMLParser`! So far, it's the only parser on our list that works without a tree, while the last one, PyQuery, focuses on a jQuery style API on top of lxml. ## 7. PyQuery: Best Python HTML Parser API for jQuery Style Traversal [PyQuery](https://pyquery.readthedocs.io/en/latest/) is a Python library for working with HTML and XML documents through a jQuery style API. It runs on top of lxml. For selecting data, it uses CSS selectors just like jQuery does in JavaScript. To parse HTML, we pass it to `PyQuery()` and then query it like we would in jQuery, e.g. `doc("h3").text()`. PyQuery 2.1.0 also covers traversal, getting text or HTML and changing the document. PyQuery has a few pros and cons compared to the other parsers on this list: Pros: - Familiar API. Anyone who has used jQuery will feel right at home with PyQuery. - Extra pseudo classes - jQuery ones like `:first`, `:last`, `:even`, `:odd`, `:eq`, `:lt`, `:gt`, `:checked`, `:selected` and `:file` work too, even though standard CSS doesn't have them. - Faster than Beautiful Soup and html5lib. On our Google page it took 4.93 ms (median) and 5.15 ms (p95) with version 2.1.0. Cons: - Not needed with lxml or Parsel. PyQuery is built on lxml too. - No JavaScript - PyQuery can't run the scripts on a page. - Not any faster. The jQuery syntax is easier on the eyes, but it's still lxml doing all of the parsing. Now that we've covered all seven parsers, let's take a look at how they compare on broken HTML, CSS and XPath. ## How Do Python HTML Parsers Handle Malformed Pages, CSS, and XPath? When it comes to parsing real web pages, broken HTML and selector support are two details that vary a lot between parsers. For this, let's take a quick look at how each parser handles broken HTML and which selectors it supports: | Parser | Malformed HTML model | CSS | XPath | Primary output | |---|---|---|---|---| | Beautiful Soup | Backend dependent | Yes | No native XPath | Soup tree | | lxml.html | Lenient HTML recovery | Yes | Yes | Element tree | | selectolax | HTML5 engine | Yes | No | Node tree | | Parsel | lxml backed | Yes | Yes | Selector list | | html5lib | WHATWG algorithm | Via tree tools | Via lxml tree | Configurable tree | | HTMLParser | Callback tolerance | No | No | Events | | PyQuery | lxml backed | jQuery CSS | Underlying lxml | PyQuery object | Beautiful Soup is a special case here, as it doesn't fix broken HTML by itself - the backend does. Calling `BeautifulSoup(html)` with no backend picks one of the installed ones, so on another machine we might get a different tree. lxml has two sides too. Its `lxml.html` module recovers from broken HTML just fine, but parsing the same page with `lxml.etree` and strict XML settings will throw an error instead. This is normal for XML, and it's `lxml.html` we'd use for scraping web pages anyway. Generally, most websites can be parsed with CSS selectors alone, however sometimes we need that extra parsing power of XPath - for example, to go up to a node's parent with `..`, to select following siblings with `following-sibling::` or to find nodes by their text with `contains(text(), ...)`. XPath also works the same way for HTML and XML. We have a separate introduction for each of them: [Parsing HTML with CSS SelectorsIntroduction to using CSS selectors to parse web-scraped content. Best practices, available tools and common challenges by interactive examples.](https://scrapfly.io/blog/posts/parsing-html-with-css) [Parsing HTML with XpathIntroduction to xpath in the context of web-scraping. How to extract data from HTML documents using xpath, best practices and available tools.](https://scrapfly.io/blog/posts/parsing-html-with-xpath) So, what if our parser can't find anything at all, even with the right selectors? ## Why Does a Python HTML Parser Return Empty JavaScript Content? Many websites these days rely on JavaScript to render their data (frameworks such as React, Angular or Vue.js), which for HTML parsers can be a confusing problem. Our HTTP client only gets the initial HTML, and if the data is added by JavaScript later, it's simply not there - so our parser sees different data than the browser and finds nothing. In other words, the parser is just one of a few layers in a web scraper: | Layer | Job | Example output | |---|---|---| | HTTP client | Fetch response | HTML bytes | | Browser renderer | Execute JavaScript | Rendered DOM | | HTML parser | Query document | Nodes and fields | | Extraction API | Map content to schema | Structured JSON | So, when a parser comes back empty, the first thing to check is the layer below it: the response status code, the final URL after redirects and the raw HTML body. Only once we've confirmed our data is in there does it make sense to fix our selectors. There are a few ways to deal with this dynamic content. If the page is rendered on the server, an HTTP client is all we need. Otherwise, we can use a browser (or a rendering service) to run the page's JavaScript first and then parse the result. Alternatively, we could look for the data where JavaScript gets it from - it's often hidden in the page as JSON or loaded through background requests. For more on that, see our guide on hidden web data: [How to Scrape Hidden Web DataThe visible HTML doesn't always represent the whole dataset available on the page. In this article, we'll be taking a look at scraping of hidden web data. What is it and how can we scrape it using Python?](https://scrapfly.io/blog/posts/how-to-scrape-hidden-web-data) If you'd rather not deal with this yourself, Scrapfly's [Web Scraping API](https://scrapfly.io/products/web-scraping-api) can take care of fetching and [JavaScript rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering), while the [Extraction API](https://scrapfly.io/products/extraction-api) can turn the fetched content into structured data. Note that neither of them is a local parser - they both work in the layers below it. That's it for our parser comparison! To wrap this guide up, let's take a look at some frequently asked questions about switching parsers. ## FAQ Is BeautifulSoup outdated in 2026?No. Beautiful Soup is still being updated, and version 4.15.0 was the current one as of September 2026. It's slower than most parsers in our benchmark, though we can pick a different backend for it. See the Beautiful Soup section above for more. Which Python HTML parser supports XPath?XPath is built into lxml and Parsel. The easiest approach is to use Parsel's `xpath()` method, which works the same way as its `css()` one. Alternatively, we can use lxml's `xpath()` directly. PyQuery runs on lxml but only exposes jQuery style CSS selectors, and Beautiful Soup has no XPath support. Can Python HTMLParser parse malformed HTML?Yes. `HTMLParser` doesn't raise errors when the HTML is broken, though it doesn't fix it either - end tags aren't checked against start tags, and implicitly closed tags never reach `handle_endtag()`. For a browser-like tree, html5lib should be used instead. Does a Python HTML parser execute JavaScript?No. None of the seven parsers in this article can run JavaScript - they only parse the HTML we give them. In this case, if a page loads its data with JavaScript, our HTTP client gets an almost empty page. The best approach to this is to render the page in a browser first or to find the hidden JSON data the page uses. Is selectolax faster than lxml?Some benchmarks show selectolax being 15x to 46x faster than lxml, but on our Google page the two were pretty much tied: 3.25 ms for lxml and 3.36 ms for selectolax (median). Just like with any benchmark, results can change with a different page or machine. ## Summary In this overview article, we took a glance at seven popular HTML parsers for Python in the context of web scraping. We took a look at the fast lxml and selectolax, the easy to read Beautiful Soup, Scrapy's own Parsel, the spec following html5lib, Python's built-in `HTMLParser` and the jQuery style PyQuery. We also took a look at how fast each parser is by benchmarking them on a real 1.22 MB Google search page, where lxml and selectolax came out on top and html5lib was by far the slowest. On top of that, we covered how each one deals with broken HTML and which selectors it supports. So, which parser should we use? A common idiom is to use Beautiful Soup for readable code, lxml or Parsel when XPath is needed and selectolax when parsing speed really matters. html5lib, `HTMLParser` and PyQuery fit more specific needs like browser-like trees, callbacks or jQuery style selectors. Finally, what if our parser can't find anything? Usually that means the page needs JavaScript, so we have to render it in a browser first. Alternatively, just check out Scrapfly's Web Scraping API, where we handle fetching and rendering for you, so all that's left is parsing the HTML! For more on choosing a parser, we recommend running our benchmark script on a saved page from your own project - results can be quite different from one page to another, ours included. Legal Disclaimer and PrecautionsThis tutorial covers popular web scraping techniques for education. Interacting with public servers requires diligence and respect: - Do not scrape at rates that could damage the website. - Do not scrape data that's not available publicly. - Do not store PII of EU citizens protected by GDPR. - Do not repurpose *entire* public datasets which can be illegal in some countries. Scrapfly does not offer legal advice but these are good general rules to follow. For more you should consult a lawyer. [ Add as a preferred source ](https://google.com/preferences/source?q=scrapfly.io) Table of Contents Table of Contents - [Key Takeaways](#key-takeaways) - [Which Python HTML Parser Is Best for Web Scraping in 2026?](#which-python-html-parser-is-best-for-web-scraping-in-2026) - [How Were the Python HTML Parsers Benchmarked?](#how-were-the-python-html-parsers-benchmarked) - [1. lxml: Best Python HTML Parser for XPath and HTML or XML Work](#1-lxml-best-python-html-parser-for-xpath-and-html-or-xml-work) - [2. selectolax: Best Python HTML Parser for a Thin Lexbor CSS API](#2-selectolax-best-python-html-parser-for-a-thin-lexbor-css-api) - [3. Beautiful Soup: Best Python HTML Parser API for Readable Prototypes](#3-beautiful-soup-best-python-html-parser-api-for-readable-prototypes) - [Choosing a Beautiful Soup Backend](#choosing-a-beautiful-soup-backend) - [4. Parsel: Best Python HTML Parser API for CSS, XPath, and Extraction Pipelines](#4-parsel-best-python-html-parser-api-for-css-xpath-and-extraction-pipelines) - [5. html5lib: Best Python HTML Parser for WHATWG Tree Construction](#5-html5lib-best-python-html-parser-for-whatwg-tree-construction) - [6. Python HTMLParser: Best Zero Dependency Event Parser](#6-python-htmlparser-best-zero-dependency-event-parser) - [7. PyQuery: Best Python HTML Parser API for jQuery Style Traversal](#7-pyquery-best-python-html-parser-api-for-jquery-style-traversal) - [How Do Python HTML Parsers Handle Malformed Pages, CSS, and XPath?](#how-do-python-html-parsers-handle-malformed-pages-css-and-xpath) - [Why Does a Python HTML Parser Return Empty JavaScript Content?](#why-does-a-python-html-parser-return-empty-javascript-content) - [FAQ](#faq) - [Summary](#summary) Join the Newsletter Get monthly web scraping insights Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Related Articles [ tools data-parsing ### HtmlAgilityPack vs AngleSharp: Which C# HTML Parser Should You Use? A fixture based comparison of Html Agility Pack and AngleSharp for C# HTML parsing, covering XPath versus CSS selectors,... ](https://scrapfly.io/blog/posts/htmlagilitypack-vs-anglesharp) [ python scrapeguide ### How to Scrape Real Estate Property Data using Python Introduction to scraping real estate property data. What is it, why and how to scrape it? We'll also list dozens of popu... ](https://scrapfly.io/blog/posts/how-to-scrape-real-estate-property-data-using-python) [ data-parsing css-selectors ### Parsing HTML with CSS Selectors Introduction to using CSS selectors to parse web-scraped content. Best practices, available tools and common challenges ... ](https://scrapfly.io/blog/posts/parsing-html-with-css) ## Related Questions - [ Q How to find sibling HTML nodes using BeautifulSoup and Python? ](https://scrapfly.io/blog/answers/how-to-find-siblings-nodes-with-beautifulsoup) Extract structured data with AI, **1,000 free credits** [Start Free](https://scrapfly.io/register)