Warc Store

View as markdown

Copy for LLM

Copy for LLM Open in ChatGPT Open in Claude Open in Perplexity

Coming Soon...

Warc Store will allow you to store scraped data in WARC format. Everything are managed for you, fully integrated with Scrapfly API. Each store are scoped per environment and project. Setup partitioning and ready to go !

WARC Format is commonly used in Big Data and implemented in multiple languages / projects, thanks to open source. You can load stores in Spark or any ETL to process / reprocess data.

Warc Store will be billed in "Pay as you go" based on GB. Warc Store can store up to many TB.