HTTP error codes can be confusing, especially when they disrupt your web scraping or automation tasks. One such error is the HTTP 405 error, which signals a problem with how a request is being made.
This article breaks down the meaning of the 405 error, its causes, and what it might mean if you're being blocked. We'll also explore ways to bypass this issue using Scrapfly.
What is HTTP Error 405?
HTTP error 405, "Method Not Allowed", occurs when a client attempts to interact with a server using an HTTP method that the server does not support for the requested resource. In simple terms, it’s like knocking on the wrong door; the server understands your request but can't process it because the request method you're using isn't allowed for that resource.
What are HTTP 405 Error Causes?
The main cause of the http status 405 is using an incorrect HTTP method. Each resource on a server is configured to handle certain types of requests (methods), such as:
GET: Retrieves data from the server.
POST: Sends data to the server for processing.
PUT: Updates a resource.
DELETE: Removes a resource.
HEAD: Retrieves only the headers of a resource.
For example, if you try to update data using a GET method or retrieve information using POST, you'll likely trigger a http code 405. This issue often occurs when the user is unaware of the available methods for a given resource.
To avoid this, it’s essential to understand what methods are allowed and use the correct one for your task, especially when working with APIs or interacting with websites programmatically.
Changing HTTP Request Method
Most tools and programming libraries used to send HTTP requests default to setting the HTTP method to GET if not specified. Most 405 errors occur due to the user being unfamiliar with how to change the default request method. Let's explore how to change the HTTP request method using different tools and libraries.
cURL
Python (requests)
Javascript (fetch)
PHP (Guzzle)
Ruby (Typhoeus)
Go
Rust (reqwest)
curl -X GET "https://httpbin.dev/get"
# post plain text
curl -X POST https://httpbin.dev/post -d "my data" -H "Content-Type: text/plain"
# post json
curl -X POST https://httpbin.dev/post -d '{"name": "my query"}' -H "Content-Type: application/json"
curl -X PUT "https://httpbin.dev/put"
curl -X DELETE "https://httpbin.dev/delete"
curl -X HEAD "https://httpbin.dev/head"
import requests
response = requests.get("https://httpbin.dev/get")
# post plain/text data
response = requests.post("https://httpbin.dev/post", data="my data")
# or application/json
response = requests.post("https://httpbin.dev/post", json={"name": "my query"})
response = requests.delete("https://httpbin.dev/delete")
response = requests.head("https://httpbin.dev/head")
print(response.text)
async function makeRequests() {
// GET request
let response = await fetch("https://httpbin.dev/get");
let data = await response.text();
console.log(data);
// POST request with plain text data
response = await fetch("https://httpbin.dev/post", {
method: "POST",
headers: {
"Content-Type": "text/plain"
},
body: "my data"
});
data = await response.text();
console.log(data);
// POST request with JSON data
response = await fetch("https://httpbin.dev/post", {
method: "POST",
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({ name: "my query" })
});
data = await response.text();
console.log(data);
// DELETE request
response = await fetch("https://httpbin.dev/delete", {
method: "DELETE"
});
data = await response.text();
console.log(data);
// HEAD request
response = await fetch("https://httpbin.dev/head", {
method: "HEAD"
});
console.log("HEAD request status:", response.status);
}
makeRequests();
<?php
require 'vendor/autoload.php'; // Make sure Guzzle is installed via Composer
use GuzzleHttp\Client;
$client = new Client();
try {
// GET request
$response = $client->request('GET', 'https://httpbin.dev/get');
echo $response->getBody();
// POST request with plain text data
$response = $client->request('POST', 'https://httpbin.dev/post', [
'body' => 'my data',
'headers' => ['Content-Type' => 'text/plain']
]);
echo $response->getBody();
// POST request with JSON data
$response = $client->request('POST', 'https://httpbin.dev/post', [
'json' => ['name' => 'my query']
]);
echo $response->getBody();
// DELETE request
$response = $client->request('DELETE', 'https://httpbin.dev/delete');
echo $response->getBody();
// HEAD request
$response = $client->request('HEAD', 'https://httpbin.dev/head');
echo "HEAD request status: " . $response->getStatusCode();
} catch (\Exception $e) {
echo "Error: " . $e->getMessage();
}
?>
require 'typhoeus'
require 'json'
# GET request
response = Typhoeus.get("https://httpbin.dev/get")
puts response.body
# POST request with plain text data
response = Typhoeus.post("https://httpbin.dev/post", body: "my data", headers: { "Content-Type" => "text/plain" })
puts response.body
# POST request with JSON data
response = Typhoeus.post("https://httpbin.dev/post", body: { name: "my query" }.to_json, headers: { "Content-Type" => "application/json" })
puts response.body
# DELETE request
response = Typhoeus.delete("https://httpbin.dev/delete")
puts response.body
# HEAD request
response = Typhoeus.head("https://httpbin.dev/head")
puts "HEAD request status: #{response.code}"
use reqwest::blocking::{Client};
use std::collections::HashMap;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let client = Client::new();
// GET request
let res = client.get("https://httpbin.dev/get").send()?;
println!("{}", res.text()?);
// POST request with plain text data
let res = client.post("https://httpbin.dev/post")
.body("my data")
.header("Content-Type", "text/plain")
.send()?;
println!("{}", res.text()?);
// POST request with JSON data
let mut json_data = HashMap::new();
json_data.insert("name", "my query");
let res = client.post("https://httpbin.dev/post")
.json(&json_data)
.send()?;
println!("{}", res.text()?);
// DELETE request
let res = client.delete("https://httpbin.dev/delete").send()?;
println!("{}", res.text()?);
// HEAD request
let res = client.head("https://httpbin.dev/head").send()?;
println!("HEAD request status: {}", res.status());
Ok(())
}
Practical Example
To understand 405 errors more, let's use the web-scraping.dev API to demonstrate how we can get a 405 http code deliberately.
We will be using cURL to send a POST request to the endpoint /api/review which is made to only accept GET requests.
The -v is the verbose output parameter which allows us to see the http status code returned.
In the curl output, we will see the http status code returned and the Allow header which lists the set of methods supported by a resource.
* Request completely sent off
< HTTP/2 405
< allow: GET
We can also see the http error message returned in the response body
{"detail":"Method Not Allowed"}
You can learn more about sending GET requests with cURL in our dedicated article:
405 Can Mean You're Blocked
While a 405 http error is technically about incorrect methods, it can also indicate that your connection is being blocked deliberately. Websites sometimes misconfigure their responses or misuse status codes to confuse bots and automated systems. In some cases, a 405 error could be masking an attempt to block your activity.
For instance, websites might throw random error codes, including 405, as part of anti-bot mechanisms. This tactic is commonly employed to hinder automated scraping tools, leading to a false sense of malfunction. If you're scraping data and encounter a 405 error, it might not just be about using the wrong method—it could signal that the website is intentionally blocking your connection.
Bypassing 405 blocking
To bypass 405 blocking you can start with scraping tools that fortify your requests against common detection techniques like:
curl-impersonate - can enhance cURL client fingerprint to mimic a real web browser.
It takes Scrapfly several full-time engineers to maintain this system, so you don't have to!
Scrapfly API also allows full customization of your HTTP requests with custom headers, methods, cookies and other HTTP parameters. This makes it easy to prevent 405 errors caused by wrong http methods or headers. You can learn more about request customization and much more in our Scrapfly API docs
Summary
In summary, HTTP 405 errors can arise from something as simple as using the wrong HTTP method, but they can also indicate more deliberate blocks put in place by websites. Understanding the root cause of a 405 error is key to resolving it. If you're dealing with intentional blocks, tools like Scrapfly can provide the advanced capabilities needed to bypass them, keeping your scraping efforts on track.