In this article, we'll explore the FlareSolverr tool and how to use it to get around Cloudflare while scraping. We'll start by explaining what FlareSolverr is, how it works, how to install and use it. Let's get started!
HTTP headers are often presented in varying case though usually in Pacal-Case like
Content-Type. According to HTTP specification header names are case-insensitive, so
content-type is the same as
However, various browsers treat this issue differently. For example, for HTTP1.1 protocol, Chrome and Firefox will show the header name in the same case as it was sent by the server or Pascal-Case. This means when web scraping using HTTP1.1 it's important to replicate the exact case of expected headers to prevent the scraper from being blocked.
For HTTP2+ all headers are required to be lowercase so the scraper should always send headers in lowercase when scraping through HTTP2-capable clients.