How to Set User Agent With cURL?

by mostafa Mar 13, 2024

The User-Agent header is one of the essential headers, which identifies the request sender's device with various details, such as the device type, operating system, browser name, and version. Missing this header or misconfiguring it can lead to request blocking.

To change the cURL User-Agent, we can use the -A cURL option:

curl -A "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0" https://httpbin.dev/headers

The response will include the modified cURL User-Agent:

{
  "headers": {
    "Accept": [
      "*/*"
    ],
    "Accept-Encoding": [
      "gzip"
    ],
    "Host": [
      "httpbin.dev"
    ],
    "User-Agent": [
      "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
    ]
  }
}

Another alternative to setting User-Agent with cURL is passing through as a header using the -H cURL option:

curl -H "User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0" https://httpbin.dev/headers

Finally, we can rotate the cURL User-Agent using bash:

user_agents=(
  "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
  "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/113.0"
  "Mozilla/5.0 (Windows NT 6.1; rv:109.0) Gecko/20100101 Firefox/113.0"
)

# get a random user agent
get_random_user_agent() {
  local random_index=$((RANDOM % ${#user_agents[@]}))
  echo "${user_agents[random_index]}"
}

user_agent=$(get_random_user_agent)

curl -A "$user_agent" https://httpbin.dev/headers

For more details on cURL, refer to our dedicated guide.

How to Use cURL For Web Scraping

In this article, we'll go over a step-by-step guide on sending and configuring HTTP requests with cURL. We'll also explore advanced usages of cURL for web scraping, such as scraping dynamic pages and avoiding getting blocked.

How to Use cURL For Web Scraping

Related Articles

Guide to using JSON with cURL

Learn how to send JSON with `cURL` using files, inline data, environment variables, and `jq`. Includes real-world examples for Slack & Google Translate.

CURL
Guide to using JSON with cURL

How to Ignore cURL SSL Errors

Learn to handle SSL errors in cURL, including using self-signed certificates. Explore common issues, safe practices.

CURL
SSL
How to Ignore cURL SSL Errors

How to Use cURL to Download Files

Master file downloads with curl and discover advanced use cases.

CURL
TOOLS
How to Use cURL to Download Files

cURL vs Wget: Key Differences Explained

curl and wget are both popular terminal tools but often used for different tasks - let's take a look at the differences.

CURL
HTTP
TOOLS
cURL vs Wget: Key Differences Explained

How to Use cURL GET Requests

Here's everything you need to know about cURL GET requests and some common pitfalls you should avoid.

CURL
How to Use cURL GET Requests

Sending HTTP Requests With Curlie: A better cURL

In this guide, we'll explore Curlie, a better cURL version. We'll start by defining what Curlie is and how it compares to cURL. We'll also go over a step-by-step guide on using and configuring Curlie to send HTTP requests.

CURL
HTTP
TOOLS
Sending HTTP Requests With Curlie: A better cURL