🚀 We are hiring! See open positions

How to find HTML elements by text value with BeautifulSoup

by scrapecrow Oct 26, 2022

Using Python and Beautifulsoup we can find any HTML element by partial or exact text value using find / find_all method by passing regular expressions object to the text parameter:

import re
import bs4

soup = bs4.BeautifulSoup('<a>Twitter link</a>')

# case sensitive:
soup.find("a", text=re.compile("Twitter"))  # will find 1st ocurrance 
soup.find_all("a", text=re.compile("Twitter"))  # will find all ocurrances
# case insensitive:
soup.find("a", text=re.compile("twitter", re.I))
soup.find_all("a", text=re.compile("twitter", re.I))

Related Articles

Build a Documentation Chatbot That Works on ANY Website

Build an AI chatbot from any docs site using Scrapfly Crawler API, LangChain, and Streamlit. Works on Cloudflare-protected sites.

PYTHON
AI
RAG
CRAWLING
Build a Documentation Chatbot That Works on ANY Website

Scraper API vs Crawler API - When to Use Each for AI

Scraper API for pages, Crawler API for domains. Learn when to use each Scrapfly API for AI training, RAG applications, and web scraping at scale.

WEB-SCRAPING
TOOLS
API
AI
Scraper API vs Crawler API - When to Use Each for AI

LangChain Web Scraping: Build AI Agents & RAG Applications

Learn to integrate LangChain with Scrapfly for web scraping. Build AI agents and RAG applications that extract, process, and understand web data at scale.

AI
PYTHON
LANGCHAIN
SCRAPING
LangChain Web Scraping: Build AI Agents & RAG Applications

Best Web Scraping Tools in 2026

Comprehensive guide to choosing web scraping tools for production. Learn the scraping pipeline framework and how to combine tools like Scrapfly, BeautifulSoup, Playwright, and Scrapy for reliable data extraction at scale.

PYTHON
NODEJS
WEB-SCRAPING
Best Web Scraping Tools in 2026

How to Scrape Facebook: Marketplace and Events

Complete guide to scraping Facebook data including Marketplace listings and Events. Covers authentication, anti-bot bypass, and production-ready techniques.

PYTHON
PLAYWRIGHT
SCRAPEGUIDE
How to Scrape Facebook: Marketplace and Events

Crawl4AI Explained: The AI-Friendly Web Crawling Framework

Discover Crawl4AI, the AI-friendly web crawling framework. Learn features, installation, and intelligent web scraping for LLMs.

WEB-SCRAPING
AI
PYTHON
CRAWLING
Crawl4AI Explained: The AI-Friendly Web Crawling Framework