How to click on cookie popups and modal alerts in Playwright?

by scrapecrow Jul 10, 2023

Modal pop-us are most commonly encountered as cookie consent popups or request to login popups. They are created using custom javascript that hide the content on page load and show some sort of message like this one:

cookie consent popup on web-scraping.dev/login page

There are multiple ways to handle modal pop-ups:

  1. We can click on one of the values like "OK" or "Yes"
  2. We can delete the modal element from the DOM

For example, let's take a look at web-scraping.dev/login page which on page load throws a cookie pop-up:

from playwright.sync_api import sync_playwright, TimeoutError

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    page = browser.new_page()
    page.goto("https://web-scraping.dev/login")

    # Option #1 - use page.click() to click on the button
    try:
        page.click("#cookie-ok", timeout=2_000)
    except TimeoutError:
        print("no cookie popup")
    
    # Option #2 - delete the popup HTML
    #   remove pop up
    cookie_modal = page.query_selector("#cookieModal")
    if cookie_modal:
        cookie_modal.evaluate("el => el.remove()")
    #   remove grey backgdrop which covers the screen
    modal_backdrop = page.query_selector(".modal-backdrop")
    if modal_backdrop:
        modal_backdrop.evaluate("el => el.remove()")

Above, we explore two ways to handle modal pop-ups: clicking a button that would dismiss it and hard removing them from the DOM. Generally, the first approach is more reliable as the real button click can have functionality attached to it like setting a cookie so the pop-up doesn't appear again. For cases when it's a login requirement or advertisement the second approach is more suited.

Related Articles

Bypass Proxy Detection with Browser Fingerprint Impersonation

Stop proxy blocks with browser fingerprint impersonation using this guide for Playwright, Selenium, curl-impersonate & Scrapfly

PROXIES
SELENIUM
PLAYWRIGHT
PUPPETEER
BLOCKING
Bypass Proxy Detection with Browser Fingerprint Impersonation

Playwright Examples for Web Scraping and Automation

Learn Playwright with Python and JavaScript examples for automating browsers like Chromium, WebKit, and Firefox.

PLAYWRIGHT
PYTHON
NODEJS
Playwright Examples for Web Scraping and Automation

Web Scraping with Playwright and JavaScript

Learn about Playwright - a browser automation toolkit for server side Javascript like NodeJS, Deno or Bun.

PLAYWRIGHT
HEADLESS-BROWSER
NODEJS
Web Scraping with Playwright and JavaScript

Playwright vs Selenium

Explore the key differences between Playwright vs Selenium in terms of performance, web scraping, and automation testing for modern web applications.

HEADLESS-BROWSER
PLAYWRIGHT
SELENIUM
Playwright vs Selenium

What is a Headless Browser? Top 5 Headless Browser Tools

Quick overview of new emerging tech of browser automation - what exactly are these tools and how are they used in web scraping?

HEADLESS-BROWSER
PLAYWRIGHT
SELENIUM
PUPPETEER
What is a Headless Browser? Top 5 Headless Browser Tools

How to Scrape With Headless Firefox

Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.

HEADLESS-BROWSER
PUPPETEER
SELENIUM
NODEJS
PLAYWRIGHT
PYTHON
How to Scrape With Headless Firefox