The Wellfound API: what breaks a run, and what it costs

blackfalcondata

Most people building against a Wellfound API can parse the response. The harder part is staying in. Requests leave without trouble, then responses arrive empty, truncated, or replaced by a challenge page the parser cannot handle.

Here is what our own Wellfound Actor has to manage, how much a run costs, and where it stops.

Doing it yourself

Building it yourself makes sense for a single snapshot or simple data where a failed run costs nothing. The work involved is less obvious. These specifics come directly from what our own scraper has to handle.

The front door. Wellfound sits behind Cloudflare. A single plain request measured on 2026-08-22 came back 200, so light use works. The protection is still there, and it starts refusing requests when one becomes thousands.

Getting past the gate. A 403 from Wellfound does not always mean refusal. It often means the session cookie has gone stale. The fix is to obtain a new cookie and repeat the request, not back off.

Rate limits. Wellfound applies rate limits per-source rather than per-account. Exponential backoff keeps the run slow. Without it, the run fails.

Proxies. One IP requesting a thousand Wellfound pages looks like a pattern, not traffic. Rotation makes that volume unremarkable, and it dominates the bill.

Pagination. A Wellfound pagination loop needs to tell the difference between "no more results" and "we stopped being served results". Both produce an empty page.

Silent breakage. Wellfound redesigns rarely break a scraper loudly. A selector fails, a field comes back empty, and the run still reports success. The problem only becomes visible when a report looks thin.

Deduplication. Wellfound has no "changed since" parameter. Run the scraper twice and the second run returns the first run's data again. Content hashes on your side turn those repeated records into a diff.

The real cost is upkeep. The first version takes an afternoon to write. After that, keeping it alive takes an hour every few weeks, forever. Failures stay quiet: an empty report reveals the problem, not a scraper error.

One measured run, not an estimate:

120 records in 91.0 seconds
cost: $0.00876 — about $0.07 per 1,000 records

The shortcut

If you do not want to own that work, the same result takes one call:

```bash curl -X POST "https://api.apify.com/v2/acts/blackfalcondata~wellfound-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"query": "..."}' ```

Or use Python:

```python from apify_client import ApifyClient

client = ApifyClient("YOUR_TOKEN") run = client.actor("blackfalcondata~wellfound-scraper").call(run_input={"query": "..."})

for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) ```

What comes back

Every row contains 71 fields, divided into several groups:

company — companyId, companySlug, companyName, companyLogo, companyTagline, companySize, +10 more
salary — salaryMin, salaryMax, salaryEquityMin, salaryEquityMax, salaryCurrency, salaryPeriod
job — jobType, jobBenefits, jobAddressLocality, jobAddressRegion, jobAddressCountry

What people use it for

Benchmarking. Pricing and salary fields support comparisons across the entire set instead of forcing you to eyeball one page of results.
Lead lists. Company-level fields let you isolate the relevant segment before anyone opens a CRM.
Change detection. Timestamps make it possible to diff runs and act on new records instead of reading everything again.
Geographic analysis. Location fields allow market-by-market comparisons without another data source.

Limits

Nothing here bypasses a Wellfound login. Only public pages are in scope, regardless of who writes the scraper.
A scraper that works today may not work in six weeks. Failure usually appears as silence, not an exception.
Proxy costs grow with volume. They are also the line item most people leave out when estimating the DIY route.

Practical considerations

Scraping Wellfound and the law. Reading publicly visible pages is generally treated differently from accessing data behind a login, and courts in the US have repeatedly declined to treat public scraping as unauthorised access. That is not legal advice, and a site's terms of service are a separate question from the law. Anything behind a login is out of scope either way.

403s when the page still loads in a browser. The check is not about the URL. A plain request to Wellfound from our probe on 2026-08-22 was answered by cloudflare — 0 of 3 attempts were refused. A browser passes because it carries a TLS fingerprint, a cookie and a header set that a bare HTTP client does not.

Proxy requirements. Anything beyond a small sample needs proxies. Our own input exposes `proxyConfiguration` because a single origin IP is the easiest thing in the world to rate-limit.

Breakage frequency. Expect to make changes whenever Wellfound ships a redesign, and expect a delay before anyone notices. Extraction fails quietly, returning empty fields rather than errors. Budget an hour every few weeks, not a fix-and-forget afternoon.

DIY cost. Building it yourself is cheaper only if your time is free. The measured run below cost $0.00876 for 120 records. The equivalent DIY cost includes proxy spend and the maintenance hour, and that maintenance hour recurs.

Try it

If you would rather not own the upkeep, run the Wellfound API on Apify. It already handles everything above. New accounts get $5 of free platform credit each month, enough to cover a real run of this size several times over.

Disclosure: we build and maintain this Actor, and the link above is an affiliate link.

The Wellfound API: what breaks a run, and what it costs

The Wellfound API: what breaks a run, and what it costs

Doing it yourself

The shortcut

What comes back

What people use it for

Limits

Practical considerations

Try it

Further reading

Report Page