How to use proxies with NodeJS axios?

Javascript's axios is a popular HTTP client often used when web scraping with nodejs.
To use proxies with axios we can use the proxy parameter in get and postj methods:

const axios = require('axios');
axios.get('https://httpbin.dev/ip', {
  proxy: {
    host: "160.11.12.13",
    port: "8080",
    auth: {
      username: "username",
      password: "password",
    }
  }
})
.then((response) => {
  console.log(response.data);
})
.catch((error) => {
  console.error(error);
});

Note that axios does not support automatic proxy use through terminal environment variables HTTP_PROXY, HTTPS_PROXY or ALL_PROXY.

Axios also does not support SOCKS proxies directly but it can be enabled through socks-proxy-agent library:

const axios = require('axios');
const SocksProxyAgent = require('socks-proxy-agent');

const proxy = 'socks5://localhost:1080';

const httpAgent = new SocksProxyAgent(proxy);
const httpsAgent = new SocksProxyAgent(proxy);

axios.get('http://example.com', {
  httpAgent,
  httpsAgent,
})
.then((response) => {
  console.log(response.data);
})
.catch((error) => {
  console.error(error);
});

Finally, when web scraping, it's best to rotate proxies for each request. For that see our article: How to Rotate Proxies in Web Scraping

Question tagged: NodeJS, HTTP

Related Posts

How to Scrape With Headless Firefox

Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.

How to Use Chrome Extensions with Playwright, Puppeteer and Selenium

In this article, we'll explore different useful Chrome extensions for web scraping. We'll also explain how to install Chrome extensions with various headless browser libraries, such as Selenium, Playwright and Puppeteer.

How to Scrape Sitemaps to Discover Scraping Targets

Usually to find scrape targets we look at site search or category pages but there's a better way - sitemaps! In this tutorial, we'll be taking a look at how to find and scrape sitemaps for target locations.