# Scrapfly Documentation ## Table of Contents ### Dashboard - [Intro](https://scrapfly.io/docs) - [Project](https://scrapfly.io/docs/project) - [Account](https://scrapfly.io/docs/account) - [Workspace & Team](https://scrapfly.io/docs/workspace-and-team) - [Billing](https://scrapfly.io/docs/billing) ### Products #### MCP Server - [Getting Started](https://scrapfly.io/docs/mcp/getting-started) - [Tools & API Spec](https://scrapfly.io/docs/mcp/tools) - [Authentication](https://scrapfly.io/docs/mcp/authentication) - [Examples & Use Cases](https://scrapfly.io/docs/mcp/examples) - [FAQ](https://scrapfly.io/docs/mcp/faq) ##### Integrations - [Overview](https://scrapfly.io/docs/mcp/integrations) - [Claude Desktop](https://scrapfly.io/docs/mcp/integrations/claude-desktop) - [Claude Code](https://scrapfly.io/docs/mcp/integrations/claude-code) - [ChatGPT](https://scrapfly.io/docs/mcp/integrations/chatgpt) - [Cursor](https://scrapfly.io/docs/mcp/integrations/cursor) - [Cline](https://scrapfly.io/docs/mcp/integrations/cline) - [Windsurf](https://scrapfly.io/docs/mcp/integrations/windsurf) - [Zed](https://scrapfly.io/docs/mcp/integrations/zed) - [Roo Code](https://scrapfly.io/docs/mcp/integrations/roo-code) - [VS Code](https://scrapfly.io/docs/mcp/integrations/vscode) - [LangChain](https://scrapfly.io/docs/mcp/integrations/langchain) - [LlamaIndex](https://scrapfly.io/docs/mcp/integrations/llamaindex) - [CrewAI](https://scrapfly.io/docs/mcp/integrations/crewai) - [OpenAI](https://scrapfly.io/docs/mcp/integrations/openai) - [n8n](https://scrapfly.io/docs/mcp/integrations/n8n) - [Make](https://scrapfly.io/docs/mcp/integrations/make) - [Zapier](https://scrapfly.io/docs/mcp/integrations/zapier) - [Vapi AI](https://scrapfly.io/docs/mcp/integrations/vapi) - [Agent Builder](https://scrapfly.io/docs/mcp/integrations/agent-builder) - [Custom Client](https://scrapfly.io/docs/mcp/integrations/custom-client) #### Web Scraping API - [Getting Started](https://scrapfly.io/docs/scrape-api/getting-started) - [API Specification]() - [Monitoring](https://scrapfly.io/docs/monitoring) - [Customize Request](https://scrapfly.io/docs/scrape-api/custom) - [Debug](https://scrapfly.io/docs/scrape-api/debug) - [Anti Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) - [Proxy](https://scrapfly.io/docs/scrape-api/proxy) - [Proxy Mode](https://scrapfly.io/docs/scrape-api/proxy-mode) - [Proxy Mode - Screaming Frog](https://scrapfly.io/docs/scrape-api/proxy-mode/screaming-frog) - [Proxy Mode - Apify](https://scrapfly.io/docs/scrape-api/proxy-mode/apify) - [(Auto) Data Extraction](https://scrapfly.io/docs/scrape-api/extraction) - [Javascript Rendering](https://scrapfly.io/docs/scrape-api/javascript-rendering) - [Javascript Scenario](https://scrapfly.io/docs/scrape-api/javascript-scenario) - [SSL](https://scrapfly.io/docs/scrape-api/ssl) - [DNS](https://scrapfly.io/docs/scrape-api/dns) - [Cache](https://scrapfly.io/docs/scrape-api/cache) - [Batch (Multi-URL Scraping)](https://scrapfly.io/docs/scrape-api/batch) - [Session](https://scrapfly.io/docs/scrape-api/session) - [Webhook](https://scrapfly.io/docs/scrape-api/webhook) - [Schedule](https://scrapfly.io/docs/scrape-api/schedule) - [Screenshot](https://scrapfly.io/docs/scrape-api/screenshot) - [Errors](https://scrapfly.io/docs/scrape-api/errors) - [Timeout](https://scrapfly.io/docs/scrape-api/understand-timeout) - [Throttling](https://scrapfly.io/docs/throttling) - [Troubleshoot](https://scrapfly.io/docs/scrape-api/troubleshoot) - [Billing](https://scrapfly.io/docs/scrape-api/billing) - [FAQ](https://scrapfly.io/docs/scrape-api/faq) #### Crawler API - [Getting Started](https://scrapfly.io/docs/crawler-api/getting-started) - [API Specification]() - [Retrieving Results](https://scrapfly.io/docs/crawler-api/results) - [WARC Format](https://scrapfly.io/docs/crawler-api/warc-format) - [Data Extraction](https://scrapfly.io/docs/crawler-api/extraction-rules) - [Webhook](https://scrapfly.io/docs/crawler-api/webhook) - [Schedule](https://scrapfly.io/docs/crawler-api/schedule) - [Billing](https://scrapfly.io/docs/crawler-api/billing) - [Errors](https://scrapfly.io/docs/crawler-api/errors) - [Troubleshoot](https://scrapfly.io/docs/crawler-api/troubleshoot) - [FAQ](https://scrapfly.io/docs/crawler-api/faq) #### Screenshot API - [Getting Started](https://scrapfly.io/docs/screenshot-api/getting-started) - [API Specification]() - [Accessibility Testing](https://scrapfly.io/docs/screenshot-api/accessibility) - [Webhook](https://scrapfly.io/docs/screenshot-api/webhook) - [Schedule](https://scrapfly.io/docs/screenshot-api/schedule) - [Billing](https://scrapfly.io/docs/screenshot-api/billing) - [Errors](https://scrapfly.io/docs/screenshot-api/errors) #### Extraction API - [Getting Started](https://scrapfly.io/docs/extraction-api/getting-started) - [API Specification]() - [Rules Template](https://scrapfly.io/docs/extraction-api/rules-and-template) - [LLM Extraction](https://scrapfly.io/docs/extraction-api/llm-prompt) - [AI Auto Extraction](https://scrapfly.io/docs/extraction-api/automatic-ai) - [Webhook](https://scrapfly.io/docs/extraction-api/webhook) - [Billing](https://scrapfly.io/docs/extraction-api/billing) - [Errors](https://scrapfly.io/docs/extraction-api/errors) - [FAQ](https://scrapfly.io/docs/extraction-api/faq) #### Data API #### Proxy Saver - [Getting Started](https://scrapfly.io/docs/proxy-saver/getting-started) - [Fingerprints](https://scrapfly.io/docs/proxy-saver/fingerprints) - [Optimizations](https://scrapfly.io/docs/proxy-saver/optimizations) - [SSL Certificates](https://scrapfly.io/docs/proxy-saver/certificates) - [Protocols](https://scrapfly.io/docs/proxy-saver/protocols) - [Pacfile](https://scrapfly.io/docs/proxy-saver/pacfile) - [Secure Credentials](https://scrapfly.io/docs/proxy-saver/security) - [Billing](https://scrapfly.io/docs/proxy-saver/billing) #### Cloud Browser API - [Getting Started](https://scrapfly.io/docs/cloud-browser-api/getting-started) - [Proxy & Geo-Targeting](https://scrapfly.io/docs/cloud-browser-api/proxy) - [Unblock API](https://scrapfly.io/docs/cloud-browser-api/unblock) - [Captcha Solver](https://scrapfly.io/docs/cloud-browser-api/captcha-solver) - [File Downloads](https://scrapfly.io/docs/cloud-browser-api/file-downloads) - [Session Resume](https://scrapfly.io/docs/cloud-browser-api/session-resume) - [Human-in-the-Loop](https://scrapfly.io/docs/cloud-browser-api/human-in-the-loop) - [Debug Mode](https://scrapfly.io/docs/cloud-browser-api/debug-mode) - [Browser Extensions](https://scrapfly.io/docs/cloud-browser-api/extensions) - [Native Browser MCP](https://scrapfly.io/docs/cloud-browser-api/mcp) - [DevTools Protocol](https://scrapfly.io/docs/cloud-browser-api/cdp-reference) ##### Integrations - [Puppeteer](https://scrapfly.io/docs/cloud-browser-api/puppeteer) - [Playwright](https://scrapfly.io/docs/cloud-browser-api/playwright) - [Selenium](https://scrapfly.io/docs/cloud-browser-api/selenium) - [Vercel Agent Browser](https://scrapfly.io/docs/cloud-browser-api/agent-browser) - [Browser Use](https://scrapfly.io/docs/cloud-browser-api/browser-use) - [Stagehand](https://scrapfly.io/docs/cloud-browser-api/stagehand) - [Billing](https://scrapfly.io/docs/cloud-browser-api/billing) - [Errors](https://scrapfly.io/docs/cloud-browser-api/errors) ### Tools - [Antibot Detector](https://scrapfly.io/docs/tools/antibot-detector) ### SDK - [Golang](https://scrapfly.io/docs/sdk/golang) - [Python](https://scrapfly.io/docs/sdk/python) - [Rust](https://scrapfly.io/docs/sdk/rust) - [TypeScript](https://scrapfly.io/docs/sdk/typescript) - [Scrapy](https://scrapfly.io/docs/sdk/scrapy) ### Integrations - [Getting Started](https://scrapfly.io/docs/integration/getting-started) - [LangChain](https://scrapfly.io/docs/integration/langchain) - [LlamaIndex](https://scrapfly.io/docs/integration/llamaindex) - [CrewAI](https://scrapfly.io/docs/integration/crewai) - [Zapier](https://scrapfly.io/docs/integration/zapier) - [Make](https://scrapfly.io/docs/integration/make) - [n8n](https://scrapfly.io/docs/integration/n8n) ### Academy - [Overview](https://scrapfly.io/academy) - [Web Scraping Overview](https://scrapfly.io/academy/scraping-overview) - [Tools](https://scrapfly.io/academy/tools-overview) - [Reverse Engineering](https://scrapfly.io/academy/reverse-engineering) - [Static Scraping](https://scrapfly.io/academy/static-scraping) - [HTML Parsing](https://scrapfly.io/academy/html-parsing) - [Dynamic Scraping](https://scrapfly.io/academy/dynamic-scraping) - [Hidden API Scraping](https://scrapfly.io/academy/hidden-api-scraping) - [Headless Browsers](https://scrapfly.io/academy/headless-browsers) - [Hidden Web Data](https://scrapfly.io/academy/hidden-web-data) - [JSON Parsing](https://scrapfly.io/academy/json-parsing) - [Data Processing](https://scrapfly.io/academy/data-processing) - [Scaling](https://scrapfly.io/academy/scaling) - [Walkthrough Summary](https://scrapfly.io/academy/walkthrough-summary) - [Scraper Blocking](https://scrapfly.io/academy/scraper-blocking) - [Proxies](https://scrapfly.io/academy/proxies) --- # Webhook Payload Reference When an alert transitions to TRIGGERED (or resolves), Scrapfly sends an HTTP POST to every webhook URL you have configured on that alert. This page documents the exact JSON payload, the headers sent with each delivery, and how to verify the HMAC signature. **Header alignment note:** The alert webhook HTTP headers are designed to match the `X-Scrapfly-Webhook-*` convention used by scrape webhooks. However, the HTTP-delivery layer for alert webhooks routes through a separate AMQP consumer (`alert_webhook` channel) rather than the scrape-engine webhook dispatcher. The header contract below is the intended design; if you are in the early-access period and observe different headers in practice, please report this to your account manager so the consumer can be aligned before GA. ## JSON payload Every delivery sends a JSON body with this shape: ``` { "event_id": "01J0AB12CDEFGH3JKLMNPQRS56", "alert_uuid": "01HXYZ123456789ABCDEF01234", "project_uuid": "01HPQR987654321ZYXWVU01234", "name": "Block rate > 20% on web-scraping.dev", "transition": "ok_to_triggered", "fired_at": "2026-05-17T12:34:56Z", "metric": { "id": "scrape.block_rate", "dimensions": { "env": "LIVE", "origin": "WEB_SCRAPING_API", "root_domain": "web-scraping.dev" }, "window_m": 5, "value": 34.5, "threshold": 20.0, "comparator": "gt" }, "links": { "dashboard": "https://scrapfly.io/dashboard/alerts/01HXYZ123456789ABCDEF01234", "graph": "https://scrapfly.io/dashboard/alerts/01HXYZ123456789ABCDEF01234/graph" } } ``` ### Field reference | Field | Type | Description | |---|---|---| | `event_id` | string (ULID) | Stable per-event identifier. Use this as your deduplication key: the same event\_id is sent on every retry, so storing it lets you reject duplicates. Also mirrored in the `X-Scrapfly-Webhook-Id` header. | | `alert_uuid` | string (ULID) | Unique identifier for the alert definition | | `project_uuid` | string (ULID) | Project the alert belongs to | | `name` | string | Human-readable alert name you set at creation time | | `transition` | string | State change that triggered this notification. Format is `_to_` (lowercase) across the alert state machine, plus two special values. Common examples: - `ok_to_triggered`: alert just fired - `triggered_to_recovering`: metric back in threshold, recovery window started - `recovering_to_ok`: alert resolved - `recovering_to_triggered`: breach returned before recovery window elapsed - `triggered_to_ok`: alert resolved (recovery\_minutes = 0) - `re_notify`: still triggered, re-notify interval elapsed - `test_fire`: synthetic notification from the test endpoint See [State machine](https://scrapfly.io/docs/alerting/state-machine) for the complete transition table. | | `fired_at` | string (RFC 3339) | Timestamp when this notification was generated | | `metric.id` | string | Metric ID from the [metric registry](https://scrapfly.io/docs/alerting/metric-reference) | | `metric.dimensions` | object | Dimension filter key/value pairs applied to this alert (may be empty) | | `metric.window_m` | integer | Evaluation window in minutes used for this observation | | `metric.value` | float | Raw metric value at the time the notification was triggered | | `metric.threshold` | float | Threshold you configured on the alert | | `metric.comparator` | string enum | `gt`, `lt`, `gte`, `lte`, `eq`, `neq` | | `links.dashboard` | string (URL) | Direct link to the alert detail page in the dashboard | | `links.graph` | string (URL) | Direct link to the metric graph for this alert | ## HTTP headers Every alert webhook delivery includes these headers: | Header | Example value | Description | |---|---|---| | `Content-Type` | `application/json` | Always JSON for alert webhooks | | `User-Agent` | `Scrapfly/Webhook` | Identify Scrapfly in your server logs | | `X-Scrapfly-Webhook-Timestamp` | `1747531296` | Unix-seconds timestamp at which Scrapfly signed the request. Included in the signed string (see verification below) so you can reject stale deliveries for replay protection. | | `X-Scrapfly-Webhook-Signature` | `3A9F2C...D8E1` | HMAC-SHA256 of `"."`, encoded as uppercase hex. Key is the UTF-8 encoding of your signing secret. See the verification section for the exact algorithm. | | `X-Scrapfly-Webhook-Signature-Lowercase` | `3a9f2c...d8e1` | Same digest, lowercase. Provided for runtimes and proxies that normalize header values to lowercase. | | `X-Scrapfly-Webhook-Resource-Type` | `alert` | Use this header to distinguish alert deliveries from scrape webhook deliveries if a single endpoint receives both. Alert deliveries always set this to `alert`. | | `X-Scrapfly-Webhook-Id` | `01J0AB...` | Mirrors the JSON body's `event_id`. The same event\_id is sent on every retry of the same event; use it as your deduplication key. NOT the alert\_uuid. | **Existing scrape webhook users:** If your endpoint already verifies the `X-Scrapfly-Webhook-Signature` header for scrape callbacks, the HMAC algorithm and key encoding are identical. The only difference is the header `X-Scrapfly-Webhook-Resource-Type` changes from `scrape` to `alert`, and the JSON body shape is different. ## Verifying the signature Scrapfly signs each delivery with HMAC-SHA256 over a string that combines the timestamp header and the raw request body. Including the timestamp in the signature lets you reject stale or replayed requests. The signing key is the UTF-8 encoding of your webhook signing secret (set in the alert create / edit form). The signed message is `"."` (literal dot between them). Do not parse and re-serialize the JSON body before signing; that changes the byte sequence. Algorithm summary: 1. Read the raw request body (before JSON parsing). 2. Read `X-Scrapfly-Webhook-Timestamp` (Unix seconds). 3. Optionally reject the delivery if the timestamp is more than a few minutes from your current clock (replay protection). 4. Compute `HMAC-SHA256(key=secret.encode('utf-8'), msg=(timestamp + "." + raw_body).encode())`. 5. Hex-encode the digest (uppercase). 6. Use a **constant-time comparison** to compare with the value in `X-Scrapfly-Webhook-Signature`. Python cURL (test delivery) Flask example. The same pattern works with FastAPI, Django, or any WSGI/ASGI framework that exposes raw request bytes. ``` import hmac import hashlib import time from flask import Flask, request, abort app = Flask(__name__) SIGNING_SECRET = "your-webhook-signing-secret" # from alert settings MAX_SKEW_SECONDS = 300 # reject deliveries older than 5 minutes @app.route("/scrapfly/alert", methods=["POST"]) def receive_alert(): # 1. Read raw bytes BEFORE any JSON parsing raw_body = request.get_data() # 2. Replay protection: reject stale deliveries ts = request.headers.get("X-Scrapfly-Webhook-Timestamp", "") if not ts.isdigit() or abs(int(time.time()) - int(ts)) > MAX_SKEW_SECONDS: abort(401, "Stale or missing timestamp") # 3. Compute HMAC-SHA256 over "." signed_message = ts.encode("utf-8") + b"." + raw_body computed = hmac.new( SIGNING_SECRET.encode("utf-8"), signed_message, hashlib.sha256, ).hexdigest().upper() # 4. Constant-time comparison (prevents timing attacks) received = request.headers.get("X-Scrapfly-Webhook-Signature", "") if not hmac.compare_digest(computed, received): abort(401, "Signature mismatch") # 5. Parse and handle the payload payload = request.get_json() resource_type = request.headers.get("X-Scrapfly-Webhook-Resource-Type", "") if resource_type == "alert": transition = payload["transition"] alert_name = payload["name"] metric_value = payload["metric"]["value"] print(f"Alert [{transition}]: {alert_name} = {metric_value}") return "", 200 ``` Use the test-fire endpoint to send a synthetic notification through all channels configured on an alert without affecting state: ``` # Replace ALERT_UUID with the UUID from the alert detail page # and with your scp-live-... project key. curl -X POST "https://api.scrapfly.io/alert/ALERT_UUID/test?key=" \ -H "Content-Type: application/json" ``` The test delivery sends the same headers and signing scheme as a real notification, with `transition` set to `test_fire`. This lets you verify HMAC signing end-to-end without waiting for a real alert to fire. ## Deduplication Webhook deliveries can be retried on timeout or server-side 5xx responses. Every retry of the same event ships the same `event_id`, so it is the only key you need: ``` dedup_key = payload["event_id"] # or equivalently: dedup_key = request.headers["X-Scrapfly-Webhook-Id"] ``` Do NOT dedup on `alert_uuid` alone: a single alert fires many distinct events over its lifetime (ok\_to\_triggered, re\_notify after each interval, triggered\_to\_ok) and they all share the same alert\_uuid but each has a unique event\_id. ## Routing multi-product webhook endpoints A single webhook URL can receive deliveries from multiple Scrapfly products (scrape callbacks, alert notifications, crawler callbacks, etc.). Use the `X-Scrapfly-Webhook-Resource-Type` header to route: ``` resource_type = request.headers.get("X-Scrapfly-Webhook-Resource-Type", "") if resource_type == "scrape": handle_scrape_callback(payload) elif resource_type == "alert": handle_alert_notification(payload) elif resource_type == "crawler": handle_crawler_callback(payload) else: # Unknown type: log and ignore for forward compatibility pass ``` [ Metric Reference ](https://scrapfly.io/docs/alerting/metric-reference) [ Anti-Spam Controls ](https://scrapfly.io/docs/alerting/anti-spam)