What case should HTTP headers be in? Lowercase or Pascal-Case?

HTTP headers are often presented in varying case though usually in Pacal-Case like Content-Type. According to HTTP specification header names are case-insensitive, so content-type is the same as Content-Type.

However, various browsers treat this issue differently. For example, for HTTP1.1 protocol, Chrome and Firefox will show the header name in the same case as it was sent by the server or Pascal-Case. This means when web scraping using HTTP1.1 it's important to replicate the exact case of expected headers to prevent the scraper from being blocked.

For HTTP2+ all headers are required to be lowercase so the scraper should always send headers in lowercase when scraping through HTTP2-capable clients.

Question tagged: HTTP

Related Posts

How to Effectively Use User Agents for Web Scraping

In this article, we’ll take a look at the User-Agent header, what it is and how to use it in web scraping. We'll also generate and rotate user agents to avoid web scraping blocking.

How to Scrape in Another Language, Currency or Location

Localization allows for adapting websites content by changing language and currency. So, how do we scrape it? We'll take a look at the most common methods for changing language, currency and other locality details in web scraping.

How to Avoid Web Scraper IP Blocking?

How IP addresses are used in web scraping blocking. Understanding IP metadata and fingerprinting techniques to avoid web scraper blocks.