How to save and load cookies in Python requests?

When web scraping, we might want to pause our scraping session by saving cookies and resume scraping later. Using requests library, to save and load cookies we can use dict_from_cookiejar and cookierjar_from_dict utility functions:

from pathlib import Path
import json
import requests

# to save cookies:
session = requests.session()
session.get("")  # get some cookies
cookies = requests.utils.dict_from_cookiejar(session.cookies)  # turn cookiejar into dict
Path("cookies.json").write_text(json.dumps(cookies))  # save them to file as JSON

# to retrieve cookies:
session = requests.session()
cookies = json.loads(Path("cookies.json").read_text())  # save them to file as JSON
cookies = requests.utils.cookiejar_from_dict(cookies)  # turn dict to cookiejar
session.cookies.update(cookies)  # load cookiejar to current session
print(session.get("").text)  # test it
Question tagged: requests, HTTP, Python, Popular