Proxy Saver Protocols
The Proxy Saver supports all standard and modern proxy protocols, ensuring compatibility with any browser or web scraping tool.
Below are the details and specificities of each protocol's usage.
While all the provided examples account for the possibility that the Proxy Saver certificates may not be properly installed, the recommended approach is to
configure them correctly
rather than bypassing SSL errors.
HTTP is the simplest proxy protocol and is universally compatible with all browsers and libraries.
Below are some examples:
curl \
--insecure \
--proxy http://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything
const {chromium} = require("playwright");
(async () => {
try {
const browser = await chromium.launch({
proxy: {
server: 'http://proxy-saver.scrapfly.io:3333',
username: 'proxyId-XXX',
password: 'scp-live-XXX'
}
});
const context = await browser.newContext({ignoreHTTPSErrors: true});
const page = await context.newPage();
await page.goto("https://httpbin.dev/anything");
const html = await page.content();
console.log(html);
await browser.close();
} catch (error) {
console.error("An error occurred:", error);
}
})();
from playwright.sync_api import sync_playwright
def main():
try:
with sync_playwright() as p:
browser = p.chromium.launch(
proxy={
"server": "http://proxy-saver.scrapfly.io:3333",
"username": "proxyId-XXX",
"password": "scp-live-XXX",
}
)
context = browser.new_context(ignore_https_errors=True)
page = context.new_page()
page.goto("https://httpbin.dev/anything")
html = page.content()
print(html)
except Exception as e:
print("An error occurred:", e)
finally:
browser.close()
if __name__ == "__main__":
main()
Using HTTP as a proxy protocol is straightforward, but opting for HTTPS ensures that your credentials are securely encrypted from the very start of the CONNECT
phase.
By default, curl will use HTTP/1.1 over TLS unless you have a recent version that supports HTTP/2 and explicitly specify the necessary flags.
Chrome, on the other hand, defaults to HTTP/2 but will gracefully downgrade to HTTP/1.1 over TLS if needed.
The Proxy Saver natively supports HTTP/2 while also maintaining compatibility with HTTP/1.1 over TLS to accommodate the broadest range of browser implementations.
curl \
--insecure --proxy-insecure \
--proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything
curl \
--insecure --proxy-insecure \
--proxy-http2 \
--proxy https://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything
const {chromium} = require("playwright");
(async () => {
try {
const browser = await chromium.launch({
proxy: {
server: 'https://proxy-saver.scrapfly.io:3333',
username: 'proxyId-XXX',
password: 'scp-live-XXX'
},
args: ['--ignore-certificate-errors'],
});
const context = await browser.newContext({ignoreHTTPSErrors: true});
const page = await context.newPage();
await page.goto("https://httpbin.dev/anything");
const html = await page.content();
console.log(html);
await browser.close();
} catch (error) {
console.error("An error occurred:", error);
}
})();
from playwright.sync_api import sync_playwright
def main():
try:
with sync_playwright() as p:
browser = p.chromium.launch(
proxy={
"server": "https://proxy-saver.scrapfly.io:3333",
"username": "proxyId-XXX",
"password": "scp-live-XXX",
},
args=["--ignore-certificate-errors"]
)
context = browser.new_context(ignore_https_errors=True)
page = context.new_page()
page.goto("https://httpbin.dev/anything")
html = page.content()
print(html)
except Exception as e:
print("An error occurred:", e)
finally:
browser.close()
if __name__ == "__main__":
main()
Socks5/h is another popular protocol used for HTTP proxying
Socks5 auth mechanism
For browsers and libraries that support native SOCKS5 authentication, the Proxy Saver does as well, ensuring seamless integration.
curl \
--insecure \
--proxy socks5h://proxyId-XXX:scp-live-XXX@proxy-saver.scrapfly.io:3333 \
https://httpbin.dev/anything
However, not all browsers support this mechanism.
While Chrome and Chromium do not support the SOCKS5 authentication mechanism (and have no plans to), Scrapfly offers a reliable workaround.
For browsers and libraries that do not support SOCKS5 authentication natively, Scrapfly offers a simple workaround using a custom header.
Simply include your authentication string in the X-SAVER-AUTH header.
This custom authentication mechanism enables the use of the SOCKS5 protocol with any library that supports it.
Below are some usage examples:
curl \
--insecure \
--proxy socks5h://proxy-saver.scrapfly.io:3333 \
--header "X-SAVER-AUTH: proxyId-XXX:scp-live-XXX" \
https://httpbin.dev/anything
const { chromium } = require("playwright");
const xSaverAuth = { 'X-SAVER-AUTH': 'proxyId-XXX:scp-live-XXX'};
(async () => {
try {
const browser = await chromium.launch({
proxy: {
server: 'socks5://proxy-saver.scrapfly.io:3333',
}
});
const context = await browser.newContext({ ignoreHTTPSErrors: true });
await context.setExtraHTTPHeaders(xSaverAuth);
const page = await context.newPage();
await page.goto("https://httpbin.dev/anything");
const html = await page.content();
console.log(html);
await browser.close();
} catch (error) {
console.error("An error occurred:", error);
}
})();
from playwright.sync_api import sync_playwright
def main():
try:
with sync_playwright() as p:
browser = p.chromium.launch(
proxy={
"server": "socks5://proxy-saver.scrapfly.io:3333",
}
)
context = browser.new_context(ignore_https_errors=True)
context.set_extra_http_headers({
"X-SAVER-AUTH": "proxyId-XXX:scp-live-XXX"
})
page = context.new_page()
page.goto("https://httpbin.dev/anything")
html = page.content()
print(html)
except Exception as e:
print("An error occurred:", e)
finally:
browser.close()
if __name__ == "__main__":
main()
