// Articles by Ziad Shamndy
How to Find All URLs on a Domain
Learn how to efficiently find all URLs on a domain using Python and web crawling. Guide on how to crawl entire domain to collect a...
What is Googlebot User Agent String?
Learn about Googlebot user agents, how to verify them, block unwanted crawlers, and optimize your site for better indexing and SEO...
Alternatives to Cloudscraper to Bypass Cloudflare
Learn why Cloudscraper is outdated and explore modern alternatives for bypassing Cloudflare protections effectively and ethically.
How to Capture and Convert a Screenshot to PDF
Quick guide on how to effectively capture web screenshots as PDF documents
Playwright Examples for Web Scraping and Automation
Learn Playwright with Python and JavaScript examples for automating browsers like Chromium, WebKit, and Firefox.
Web Scraping with Playwright and JavaScript
Learn about Playwright - a browser automation toolkit for server side Javascript like NodeJS, Deno or Bun.
How to use wget in Python
Learn how to use wget in Python through subprocess calls and what are other options.
Ultimate Guide to JSON Parsing in Python
Learn JSON parsing in Python with this ultimate guide. Explore basic and advanced techniques using json, and tools like ijson and ...
Guide to Axios Headers
Learn about Javascript's Axios headers. How to configure, update, inspect headers in request and responses, how to set defaults an...
Guide to Parsel - the Best HTML Parsing in Python
Learn to extract data from websites with Parsel, a Python library for HTML parsing using CSS selectors and XPath.
What is HTTP 401 Error and How to Fix it
Discover the HTTP 401 error meaning, its causes, and solutions in this comprehensive guide. Learn how 401 unauthorized errors occu...
JSONL vs JSON
Learn the differences between JSON and JSONLines, their use cases, and efficiency. Why JSONLines excels in web scraping and real-t...