How To Send cURL POST Requests?

To send a cURL POST request, we can use the following syntax:

curl -X POST \
     -H "Headers" \
     -d "Body" \
     https://httpbin.dev/post

The above snippet represents three options required to send POST requests with cURL:
-X: Specifies the request HTTP method, POST in this case.
-H: For adding request headers, which are often required for POST requests.
-d: For adding the cURL request body.

Common cURL POST Request Examples

Now that we have explored the syntax required for sending cURL POST requests let's apply it to common use cases.

Sending JSON Body

To pass a JSON body with cURL POST requests, we can use the the -d option followed by the data object:

curl -X POST -H "Content-Type: application/json" -d '{"key1": "value1", "key2": "value2"}' https://httpbin.dev/post

Here, we modify the request Content-Type header and JSON body to the request object, which are returned with the response:

{
  "args": {},
  "headers": {
    ....
    "Content-Type": [
      "application/json"
    ],
  },
  "method": "POST",
  "data": "{\"key1\": \"value1\", \"key2\": \"value2\"}",
  "json": {
    "key1": "value1",
    "key2": "value2"
  }
}

Sending XML Body

Similar to the above snippet, we can send a cURL POST request with XML data through the -d option. We'll also change the Content-Type header to the equivalent of XML:

curl -X POST \
     -H "Content-Type: application/xml"
     -d "<?xml version="1.0" encoding="UTF-8"?><product><id>1</id><name type="title">Box of Chocolate Candy</name></product>"
     https://httpbin.dev/post

Sending Files

To make cURL POST files, we can use the -F option followed by the file path to upload it:

curl -X POST -F "file=@/file/path/file.img" https://httpbin.dev/post

Note that the -F argument only accepts one file path. To pass a list of files, each file has to be attached using a dedicated argument.

Sending Form Requests

Form POST requests are commonly required to retrieve tokens or data behind forms. Let's replicate this form example using cURL POST requests:

curl -X POST \
     -d 'custname=Scrapfly&custtel=&custemail=&size=small&topping=cheese&delivery=13%3A45&comments=' \
     https://httpbin.dev/post

Authenticate cURL POST Requests

Let's make cURL POST requests support basic HTTP authentication. For this, we can use the basic option:

curl -X POST --user user:passwd --basic https://httpbin.dev/basic-auth/user/passwd

The response of the above cURL request indicates that it's successfully authenticated:

{
  "authorized": true,
  "user": "user"
}

For further details on different mechanisms for handling authentication with cURL requests, refer to our dedicated guide.

Further cURL Options

Sending cURL POST requests often require additional configuration for the request. Here are additional cURL arguments and their usage:

Argument Full Option Description
-x --proxy Use proxy with cURL, either as HTTP or SOCKS
-Z --parallel Send multiple cURL requests in parallel
-L --location Make cURL follow request redirects
-O --output Save the response binary data to a local directory
-I --head Use the request HEAD HTTP method
-b --cookie Add request cookies
-v --verbose Sends the request in verbose mode, which include further debugging details
-k --insecure Skip SSL verification for HTTPs connection

For further details on sending requests with cURL, refer to our dedicated guide.

How to Use cURL For Web Scraping?

In this article, we'll go over a step-by-step guide on sending and configuring HTTP requests with cURL. We'll also explore advanced usages of cURL for web scraping, such as scraping dynamic pages and avoiding getting blocked.

curl web scraping article banner

Provided by Scrapfly

This knowledgebase is provided by Scrapfly data APIs, check us out! 👇