Proxies for SEO tools

A crawler walks a site in hundreds of threads, and sooner or later any server answers it with a 429. A MIX pool up to 25,000 IPs spreads those threads across different addresses, so Screaming Frog, Netpeak Spider, A-Parser and Key Collector run at full speed instead of one thread with pauses. A test hour costs $1.

  • Traffic is never metered
  • SOCKS5 and HTTP on one port
  • Refund on packages from 3 days
See prices
SEO audit tools converge on a single proxy node that links out to website pages

The price for your volume

Pick a package, then move the period and thread sliders. Threads here are the same threads you set in your crawler: 1,200 threads means 1,200 URLs the tool pulls at once. The slider sits on that value.

MIX-Business is built for an SEO department or an agency: several projects a week, Sitebulb and SEO PowerSuite side by side, A-Parser for suggestions and SERPs, ScrapeBox for bulk checks. 16,000 addresses and 1,200 threads by default: that is what goes into a crawler when the crawl has to finish in an evening.

1 month
1,200+20%

A thread is one simultaneous connection. More threads — more done in the same time.

$120
≈ $4 / dayfor 30 days

$100 for a month+ $20 for threads

save $420 vs. daily
YESrefund of the remainder
  • 1 hour — $1
  • 1 day — $15
  • 3 days — $35
  • 1 week — $50
  • 2 weeks — $75
  • 1 month — $100
  • 3 months — $255
  • 6 months — $475
Buy proxies — MIX-Business

How to connect the proxies to SEO tools

  1. 1
    Buy an hour for $1 and enter the IP of the machine your tools run on.

    Access is tied to the computer or server itself, so in the proxy settings of Screaming Frog and Netpeak the login and password fields stay empty. You can register up to ten such machines.

  2. 2
    Grab the proxy list URL from your dashboard.

    It returns a text file in address-and-port form, one proxy per line. Screaming Frog and Netpeak expect the same list but as a file: save the response to .txt and point the settings at it.

  3. 3
    Spread the pool across your tools.

    A-Parser and ScrapeBox re-read the URL themselves and take every address from it. Key Collector, Sitebulb and SEO PowerSuite read a list file, so refresh it with a script every 10 minutes, example below.

  4. 4
    Set tool threads no higher than the port count in your package.

    500 on Basic, 1,000 on Business, 1,500 on Elite without add-ons. Every port is a separate proxy and one crawler thread occupies one port: setting more threads in Screaming Frog than you have ports buys nothing.

  5. 5
    Choose the protocol per tool.

    SOCKS5 is what Screaming Frog, A-Parser, ZennoPoster and ScrapeBox take. Netpeak Spider, Key Collector and SEO PowerSuite go over HTTP(S). Both protocols sit on the same port, you only switch it in the tool.

This is how the pool plugs into an audit of your own. The script reads addresses from urls.txt, pulls each page through a random proxy and stores the status code, redirect, title, h1, description, canonical and robots in audit.csv — the same things a crawler shows, only on your rules. Replace PROXY_LIST_URL_FROM_DASHBOARD with the URL from your dashboard.

# pip install requests[socks]
import csv, random, re, time, requests
from concurrent.futures import ThreadPoolExecutor

PROXY_LIST_URL = "PROXY_LIST_URL_FROM_DASHBOARD"
REFRESH_SEC = 600
THREADS = 1200
HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; SeoAudit/1.0)", "Accept-Language": "en-US,en;q=0.9"}

_pool, _loaded = [], 0

def pool():
    global _pool, _loaded
    if time.time() - _loaded > REFRESH_SEC:
        text = requests.get(PROXY_LIST_URL, timeout=15).text
        _pool = [line.strip() for line in text.splitlines() if ":" in line]
        _loaded = time.time()
    return _pool

def first(pattern, html):
    match = re.search(pattern, html, re.S | re.I)
    return re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", match.group(1))).strip() if match else ""

def audit(url, tries=3):
    for _ in range(tries):
        proxy = random.choice(pool())
        try:
            r = requests.get(url, headers=HEADERS, timeout=20, allow_redirects=False,
                             proxies={"http": f"socks5h://{proxy}", "https": f"socks5h://{proxy}"})
        except requests.RequestException:
            continue
        html = r.text if "html" in r.headers.get("Content-Type", "") else ""
        return {
            "url": url,
            "status": r.status_code,
            "redirect": r.headers.get("Location", ""),
            "title": first(r"<title[^>]*>(.*?)</title>", html),
            "h1": first(r"<h1[^>]*>(.*?)</h1>", html),
            "description": first(r'<meta[^>]+name="description"[^>]+content="([^"]*)"', html),
            "canonical": first(r'<link[^>]+rel="canonical"[^>]+href="([^"]*)"', html),
            "robots": first(r'<meta[^>]+name="robots"[^>]+content="([^"]*)"', html),
        }
    return {"url": url, "status": 0, "redirect": "", "title": "", "h1": "",
            "description": "", "canonical": "", "robots": ""}

urls = [line.strip() for line in open("urls.txt") if line.strip()]
fields = ["url", "status", "redirect", "title", "h1", "description", "canonical", "robots"]
with open("audit.csv", "w", newline="", encoding="utf-8") as out:
    writer = csv.DictWriter(out, fieldnames=fields, delimiter=";")
    writer.writeheader()
    with ThreadPoolExecutor(THREADS) as executor:
        for row in executor.map(audit, urls):
            writer.writerow(row)
            print(row["status"], row["url"], row["title"][:60])

Which package to take for SEO tools

MIX-Basic

8,000 IPs · 500–900 threads

from $50 a month

One project: a crawl of up to a hundred thousand URLs, keyword collection, a weekly link check.

MIX-Business

16,000 IPs · 1,000–1,400 threads

from $100 a month

An agency or SEO department: several crawlers at once, suggestion harvesting and bulk checks on one pool.

MIX-Elite

25,000 IPs · 1,500–9,900 threads

from $150 a month

Your own service or a large in-house team: millions of URLs per crawl, threads up to 9,900, around the clock.

Five settings that protect the pool

  • Cap the crawler by URLs per second per domain, not only by threads. The server counts the rate, and 1,200 threads into one domain with no limit is a guaranteed 429.
  • Set a 20-second timeout and two retries through another proxy: on a large crawl, hung requests eat more time than skipped pages.
  • Do not fetch images, fonts and CSS when you are checking meta tags. In Screaming Frog these are checkboxes in Configuration, and the crawl speeds up several times over.
  • Read robots.txt once and cache it: a crawler that pulls it before every page wastes half the pool.
  • Before a big crawl, pick the fastest addresses from the CSV; the script below sorts the list by the speed column.

Questions about proxies for SEO tools

01Which tools does this work with?
Any tool that accepts a proxy list: Screaming Frog SEO Spider, Netpeak Spider and Checker, Sitebulb, SEO PowerSuite, A-Parser, Key Collector, KeyAssort, ScrapeBox, GSA Search Engine Ranker, ZennoPoster, Xenu's Link Sleuth. Services such as Ahrefs and Semrush run on their own infrastructure and need no pool.
02How many threads should I set in Screaming Frog?
No more than the port count in your package: 500 on Basic, 1,000 on Business, 1,500 on Elite. And keep a URLs-per-second limit per domain, or you will hit the defences of the site you are crawling, not the pool.
03How does the crawler authenticate to the proxies?
It does not: authorisation goes by the IP of the machine you entered when ordering. In the proxy settings of Screaming Frog or Netpeak the login and password fields stay empty, and the machine list is edited in the dashboard.
04Is the pool enough for a million-URL crawl?
Yes, traffic is never metered and the page count does not affect the price. The bottleneck is not the pool but the request rate to one domain: a million URLs of a single site are better crawled with a limit and in several passes.
05Can I collect SERPs and suggestions by region?
Yes, that is what the country on every address in the list is for. Take the list as CSV: columns ip, port, country, speed, uptime, real_ip, semicolon-separated. The script below sorts the pool by the speed column and keeps the fastest addresses, and a country filter is a one-line addition.
import csv, io, requests

CSV_URL = "PROXY_LIST_URL_FROM_DASHBOARD.csv"
TAKE = 1200
OUT_FILE = "proxies_fast.txt"

text = requests.get(CSV_URL, timeout=15).text
rows = list(csv.DictReader(io.StringIO(text), delimiter=";"))
rows.sort(key=lambda r: int(r["speed"]), reverse=True)
picked = [f"{r['ip']}:{r['port']}" for r in rows[:TAKE]]

with open(OUT_FILE, "w") as f:
    f.write("\n".join(picked) + "\n")
print(f"{len(picked)} proxies saved to {OUT_FILE}")
06What if the crawler does not run well on this pool?
Take an hour for $1 and start an ordinary crawl on it: an hour shows both the speed and the share of 429 answers. Packages from 3 days are cancelled in the dashboard, the time used is rebilled at the regular rate and the remainder returns to your balance.

The rest about the service is collected in the general FAQ, every package with prices is on the home page.

Start with a $1 hour

Run your crawler on the pool for an hour, look at the crawl speed, then pick a period. On packages from 3 days the remainder goes back to your balance.

Test proxies for $1
LOGIN/REGISTER