How to Scrape Google Maps
We'll take a look at to find businesses through Google Maps search system and how to scrape their details using either Selenium, Playwright or ScrapFly's javascript rendering feature - all of that in Python.
To handle browser dialog pop ups in Playwright like this one:
We need to attach a dialog
handler to the page
object which can be done using page.on("dialog", handler)
method:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(headless=False)
page = browser.new_page()
# create a dialog handler that will check message text and press yes/no
def handle_dialog(dialog):
if "clear your cart" in dialog.message:
print(f'clicking "Yes" to {dialog.message}')
dialog.accept() # press "Yes"
else:
dialog.dismiss() # press "No"
page.on("dialog", handle_dialog)
# to test this, let's try web-scraping.dev cart system:
# add something to cart
page.goto("https://web-scraping.dev/product/1")
page.click(".add-to-cart")
# try clearing cart which raises a dialog that says "are you sure you want to clear your cart?"
page.goto("https://web-scraping.dev/cart")
page.wait_for_selector(".cart-full .cart-item")
page.click(".cart-full .cart-clear")
print(f'items in cart: {page.query_selector(".cart-item .cart-title")}') # should be None
In the example above, attach a dialog handler to our page
object which checks whether the dialog message contains the text "clear your cart" and if so, it clicks "Yes" to clear the cart. Otherwise, it clicks "No" to cancel the dialog.