Projects and Environments

Scrapfly Projects feature is an easy way to organize Scrapfly resources. This feature is ideal for managing multiple web scraping projects with its own budgets, quotas and configurations.

Each project gives you two dedicated environments: LIVE and TEST. Each environment has a dedicated API key. The goal of each environment is to separate the production and sandbox. So you can confidently work/test/try under TEST without worrying about production.

By default, project "default" is created on your account which cannot be deleted. All of the existing projects and their details can be viewed and modified using the Projects Dashboard:

web interface for managing projects
The Number of Available Projects Depends on Your Plan
FREE DISCOVERY PRO STARTUP ENTERPRISE
1 projects 2 projects 5 projects 15 projects 50 projects

See in Your dashboard

Scopes

All resources listed below are scoped per project per environment

  • Monitoring
  • Webhook
  • Spider
  • Schedule
  • Throttling
  • Integration

Authentication & Usage

Web Platform

The project selector is available in most pages of the web interface. It allows you to select the project and environment you want to work on. The projects page allows to create, manage and delete projects.

Deleted projects are not recoverable - all of the data will be lost!

API

Each project and environment has a dedicated API key so to switch between projects and environments just change the used key.

Quota

Each project can have can be configured with a quota. The quota is a way to limit the usage of the project for:

  • API credit limit
  • Budget limit
  • Concurrency limit
  • Whether extra API credit use is allowed

Security

For security, each project can be configured to authorize specific IPs and CIDR. Meaning, only allowed IP/CIDR will be able to use the project.

Summary