[Answers](https://scrapfly.io/blog) / [http](https://scrapfly.io/blog/tag/http) / [What is Asynchronous Web Scraping?](https://scrapfly.io/blog/answers/what-is-asynchronous-web-scraping) # What is Asynchronous Web Scraping? by [Bernardas Alisauskas](https://scrapfly.io/blog/author/bernardas) Apr 18, 2026 2 min read [\#http](https://scrapfly.io/blog/tag/http) [ ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping "Share on LinkedIn") Asynchronous web scraping is a programming technique that allows for running multiple scrape tasks in effective parallel. Asynchronous programming is especially important in web scraping as web scraping programs have a lot of waiting time. In other words, every time a web scraper requests a web page, it has to wait for the response. This waiting time can be relatively long, especially when scraping large amounts of web pages. For example, let's take a look at this synchronous scraping example in Python: python```python import httpx from time import time _start = time() pages = [ "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", ] for page in pages: httpx.get(page) print(f"finished scraping {len(pages)} pages in {time() - _start:.2f} seconds") "finished scraping 5 pages in 15.46 seconds" ``` Here we have a list of 5 web pages that load in 2 seconds each. If we run this code, we'll see that it completes in **~15 seconds** every time. This is because our code waits for each page to fully complete scraping before moving on even if the program itself does nothing but wait for the server to respond. In contrast, asynchronous web scraping allows for running multiple scrape tasks in effective parallel: python```python import httpx import asyncio from time import time async def run(): _start = time() async with httpx.AsyncClient() as client: pages = [ "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", "https://httpbin.dev/delay/2", ] # run all requests concurrently using asyncio.gather await asyncio.gather(*[client.get(page) for page in pages]) print(f"finished scraping {len(pages)} pages in {time() - _start:.2f} seconds") asyncio.run(run()) "finished scraping 5 pages in 2.93 seconds" ``` This Python example uses `httpx.AsyncClient` and `asyncio` to eliminate the waiting time by running all requests in parallel. As a result, the code completes in **2-3 seconds** every time. --- Asynchronous programming is an ideal fit for web scraping and one of the easiest ways to speed up web scraping. For more see: [Web Scraping Speed: Processes, Threads and AsyncScaling web scrapers can be difficult - in this article we'll go over the core principles like subprocesses, threads and asyncio and how all of that can be used to speed up web scrapers dozens to hundreds of times.](https://scrapfly.io/blog/posts/web-scraping-speed) Scale Your Web Scraping Anti-bot bypass, browser rendering, and rotating proxies, all in one API. Start with 1,000 free credits. No credit card required 1,000 free API credits Anti-bot bypass included [Start Free](https://scrapfly.io/register) [View Docs](https://scrapfly.io/docs/onboarding) Not ready? Get our newsletter instead. ## Explore this Article with AI [ ChatGPT ](https://chat.openai.com/?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping) [ Gemini ](https://www.google.com/search?udm=50&aep=11&q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping) [ Grok ](https://x.com/i/grok?text=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping) [ Perplexity ](https://www.perplexity.ai/search/new?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping) [ Claude ](https://claude.ai/new?q=Summarize%20this%20page%3A%20https%3A%2F%2Fscrapfly.io%2Fblog%2Fanswers%2Fwhat-is-asynchronous-web-scraping) ## Related Articles [ http ### What is HTTP 407 Status Code and How to Fix it Imagine trying to access a website, only to be stopped in your tracks by a frustrating error code. If you've encountered... ](https://scrapfly.io/blog/posts/what-is-http-407-status-code-and-how-to-fix-it) [ blocking ### 5 Tools to Scrape Without Blocking and How it All Works Tutorial on how to avoid web scraper blocking. What is javascript and TLS (JA3) fingerprinting and what role request hea... ](https://scrapfly.io/blog/posts/how-to-scrape-without-getting-blocked-tutorial) [ ai tools ### What Is Browser as a Service? Cloud-hosted browser infrastructure accessible via API for scalable web automation, scraping, and AI agents with managed... ](https://scrapfly.io/blog/posts/what-is-a-browser-as-a-service) ## Related Questions - [ Q How to use CSS Selectors in Nim ? ](https://scrapfly.io/blog/answers/how-to-use-css-selectors-in-nim) - [ Q What is cURL and how is it used in web scraping? ](https://scrapfly.io/blog/answers/what-is-curl-and-how-is-it-used-in-web-scraping) - [ Q How to select elements by attribute using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-containing-value-css-selectors) - [ Q How to count selections in XPath and why? ](https://scrapfly.io/blog/answers/how-to-count-selectors-in-xpath-and-why) Scale your web scraping effortlessly, **1,000 free credits** [Start Free](https://scrapfly.io/register)