Odds Scraper: Collecting Bookmaker Odds Through a Private Proxy Pool

Odds scraper jobs that poll many bookmakers need many addresses, so run your web scraping software through a private IPv4 pool with HTTP and SOCKS5 on one access.

Odds scraper is a program that collects betting odds from bookmaker sites or odds comparison pages and saves them as a table: match, market, bookmaker, price and the time it was read. Odds move all day, so a scraper earns its keep only when it reads the same pages again and again, and that is where request limits per IP come in.

One server polling hundreds of match pages every minute sends thousands of requests an hour from a single address. Sites answer with slow pages, 429 codes or empty tables long before the job is done.

Odds scraper Python setup with OddsHarvester

OddsHarvester is an open source Python tool that reads OddsPortal with a Playwright browser. It covers 11 sports, from football and tennis to cricket, and saves results as JSON or CSV. Two commands do most of the work: upcoming for matches on a given date and historic for past seasons of a league.

Proxies go in with the --proxy-url flag, which accepts http:// and socks5:// addresses and can be repeated. With several proxies the tool rotates them round robin across matches and drops one after 3 failures in a row. With WinGate the line needs no login: access is bound to your server IP, so each flag holds only the scheme, the IP and the port.

import random, subprocess, requests

LIST_URL = "your list link from the dashboard"
pool = requests.get(LIST_URL, timeout=20).text.split()
picked = random.sample(pool, 50)

cmd = ["oddsharvester", "upcoming", "-s", "football", "-d", "20261101",
       "-m", "1x2", "--headless", "-c", "10", "--request-delay", "1.5",
       "-f", "csv", "-o", "odds_football"]
for line in picked:
    cmd += ["--proxy-url", "socks5://" + line]

subprocess.run(cmd, check=True)

The defaults are 3 parallel pages and a 1 second delay between pages. Raise --concurrency step by step and keep the thread count of your plan above it, since every browser page holds its own connection. Fifty proxies for ten parallel pages leave spare lines for the ones that fail.

If your own scraper uses requests or httpx without a browser, put http:// in front of each line of the same list. Each site then sees a pool address on every request, which covers web scraping anonymity for your server IP. The pool answers HTTP, HTTPS and SOCKS5 on the same addresses, so one list serves both kinds of code.

Odds scraper for OddsPortal: what the pages hold

OddsPortal lists matches by sport, country and league, and each match page shows prices from many bookmakers for every market, such as 1x2, over and under, or handicaps. The tables are built with JavaScript, which is why plain HTTP requests return empty markup and scrapers drive a real browser.

A browser page loads scripts, styles and data calls along with the HTML, so a scraper of this kind uses far more traffic per match than a single request. With unmetered traffic the price stays the same whether a match page weighs 50 kilobytes or 5 megabytes.

The historic command takes a league and a season, for example england-premier-league and 2024-2025, and walks the result pages of that season. A 20 team league plays 380 matches a season, so one historic run is a good first job to check the pool settings and the delay before you switch to live polling.

How to scrape betting odds without hitting rate limits

Three settings decide whether a long run stays healthy: the delay between pages, the number of parallel pages and the number of addresses behind them. Keep the delay at a second or more, start with a handful of parallel pages and give every parallel page several addresses to rotate through.

Watch the result codes. A growing share of 429 responses on one site means the request rate there is too high, so lower concurrency for that site and leave the others running. A timeout on a single address happens in any pool, and a retry on the next line handles it.

Football betting odds scraper on a schedule

Football has the most matches and markets, so it sets the size of the job. A weekend card across ten leagues can hold 100 matches. A scraper that reads every match page every 5 minutes during the last 6 hours before kickoff makes about 7,200 page loads a day. Through one address that is a heavy stream. Spread over 12,000 to 15,000 private IPv4 from 3,000 to 5,000 networks, each address sees a page only now and then.

Start the job from cron or Task Scheduler and give each run a fresh sample of the pool list. The list refreshes every 15 minutes, so reading the link at the start of every run keeps the proxies current without extra code.

Scrape betting odds into Excel

With -f csv the output opens in Excel as is. For a running sheet, add a column with the run time, append every run to one file and build a pivot table by match and bookmaker. The movement of a price over the day then shows up as a row of values side by side, and a sharp drop at one bookmaker stands out at a glance.

Arbitrage betting finder: why odds have to be fresh

An arbitrage betting finder compares prices for the same market across bookmakers and flags cases where the best price for each outcome adds up to less than 100 percent implied probability. Such gaps close fast, so a finder needs every price read within the last minute or two, and the request rate per address is what holds it back.

A private pool splits that rate across thousands of IPv4. The 1,000 thread plan costs $15 a day, $75 a week or $190 a month and covers a finder that polls several bookmakers at once. With 3,000 threads the prices are $30, $180 and $600, and traffic is not metered on either plan.

Try it on your own targets first: the free test runs up to 2 hours on the 1,000 thread plan. Bind your server IP, load 50 lines into the scraper, run one league for an hour and count the finished match pages against your current setup. Payment is in crypto, and the pool opens right after it.