Projects and Environments

Scrapfly Projects feature is an easy way to organize Scrapfly resources. This feature is ideal for managing multiple web scraping projects with its own budgets, quotas and configurations.

Each project gives you two dedicated environments: LIVE and TEST. Each environment has a dedicated API key. The goal of each environment is to separate the production and sandbox. So you can confidently work/test/try under TEST without worrying about production.

By default, project "default" is created on your account which cannot be deleted. All of the existing projects and their details can be viewed and modified using the Projects Dashboard:

web interface for managing projects
The Number of Available Projects Depends on Your Plan
FREE DISCOVERY PRO STARTUP ENTERPRISE
1 projects 2 projects 5 projects 15 projects 50 projects

See in Your dashboard

API Keys and Key Reset

Each project has two dedicated API keys for LIVE and TEST environments and can be always found on the top part of your scrapfly dashboard.

For invalidating API keys visit the Projects Dashboard and select Actions->Regenerate API Key option. Alternatively the keys can be reset in each individual project's control panel which you can access by clicking on the project name.

Scopes

All resources listed below are scoped per project per environment

  • Monitoring
  • Webhook
  • Spider
  • Schedule
  • Throttling
  • Integration

Authentication & Usage

Web Platform

The project selector is available in most pages of the web interface. It allows you to select the project and environment you want to work on. The projects page allows to create, manage and delete projects.

Deleted projects are not recoverable - all of the data will be lost!

API

Each project and environment has a dedicated API key so to switch between projects and environments just change the used key.

Quota

Each project can have can be configured with a quota. The quota is a way to limit the usage of the project for:

  • API credit limit
  • Budget limit
  • Concurrency limit
  • Whether extra API credit use is allowed

Security

For security, each project can be configured to authorize specific IPs and CIDR. Meaning, only allowed IP/CIDR will be able to use the project.

Summary