How to Scrape With Headless Firefox
Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.
To check whether an HTML element is present on the page in Playwright and Python we can use the page.locator()
or page.is_visible()
functions:
with sync_playwright() as pw:
browser = pw.chromium.launch(headless=False)
context = browser.new_context(viewport={"width": 1920, "height": 1080})
page = context.new_page()
# go to url
page.goto("https://scrapfly.io/")
# use .locator() with CSS or XPath selectors:
elements = page.locator("div.post-content")
if elements.count() > 0:
print(f"found {elements.count()} elements")
visible = sum([handle.is_visible() for handle in elements.element_handles()])
print(f"out of which {visible} are visible")
Note that this method will not wait for the element to appear on the page. For waiting see How to wait for page to load in Playwright?