All open roles

Backend Infrastructure Engineer (Anti-Bot)

Engineering · Posted 2 months ago

You'll work on the infrastructure that makes Firecrawl fast and reliable against a web that constantly changes and fights back. This is adversarial web-extraction work: staying unblocked against websites that are actively fighting to stop automated access. Anti-bot evasion, proxy rotation and management, browser/TLS fi...

Way of working
Hybrid
Location
San Francisco, CA, Bay Area, CA
Pay range
$235,000 to $260,000
Level
Mid
Experience
3+ years
Type
Full time
Visa sponsorship
Not offered for this role
The company
Technology,Information and Internet, Information and Internet · 20 to 50 people

Skills that matter here

GoPythonTypeScriptNode.jsProxy SystemsHeadless BrowsersJS frameworksREST APIs / WebhooksDNSRedisPostgreSQLKafkaKubernetesDockerAWSPrometheusTerraformGrafanaCI/CDGitHub Actions

What you would be doing

  • Architect and manage proxy rotation, IP/session management, and egress-node isolation at scale
  • Beat anti-bot and anti-scraping defenses — keep success rates high against sites that actively fight back
  • Build and maintain browser/TLS fingerprint management and CAPTCHA/challenge-solving infrastructure
  • Own the health, cost efficiency, and economics of the proxy layer
  • Debug hard, intermittent adversarial failures across the stack and make them stay fixed

What they are looking for

  • You've personally built proxy infrastructure, anti-bot evasion systems, or web-extraction pipelines against adversarial defenses at production scale
  • You're relentless about reliability and comfortable in messy, adversarial problem spaces
  • You can own a system end to end and keep it healthy in production
  • You move fast and close the loop - you'd rather ship, measure, and iterate than perfect on paper

The full description

Backend Infrastructure Engineer Role

You'll work on the infrastructure that makes Firecrawl fast and reliable against a web that constantly changes and fights back. This is adversarial web-extraction work: staying unblocked against websites that are actively fighting to stop automated access. Anti-bot evasion, proxy rotation and management, browser/TLS fingerprinting, CAPTCHA and challenge handling, and keeping high-volume extraction working against sites that change defenses on you — day in, day out. When Firecrawl "just works" on a site that tries hard to stop it, that's this team. You'll own real infrastructure from day one, not tickets in a backlog.

About Firecrawl

Firecrawl is the easiest way to turn the web into data AI agents can use. One API call converts any URL into clean, LLM-ready markdown or structured data - the boring-hard problem everyone building with LLMs eventually hits, solved.

We hit 8 figures in ARR in year one and more than doubled it in year two. We have 150K+ GitHub stars, and developers, agents, and category-defining AI companies build on us every day. Growth like this is rare, and we're just getting started.

We're a small team punching far above our weight. Everyone here owns a real piece of the product and company, end to end, and runs it themselves - no hiding behind process or headcount.

This is a place for people who want to work at the frontier: an AI company building the infrastructure other AI companies run on, not one bolting AI onto an existing product. We move fast, go deep, and are building the tools superintelligence will rely on to gather data from the web.

What You'll Do

- Architect and manage proxy rotation, IP/session management, and egress-node isolation at scale

- Beat anti-bot and anti-scraping defenses — keep success rates high against sites that actively fight back

- Build and maintain browser/TLS fingerprint management and CAPTCHA/challenge-solving infrastructure

- Own the health, cost efficiency, and economics of the proxy layer

- Debug hard, intermittent adversarial failures across the stack and make them stay fixed

What We're Looking For

- You've personally built proxy infrastructure, anti-bot evasion systems, or web-extraction pipelines against adversarial defenses at production scale

- You're relentless about reliability and comfortable in messy, adversarial problem spaces

- You can own a system end to end and keep it healthy in production

- You move fast and close the loop - you'd rather ship, measure, and iterate than perfect on paper

What We're NOT Looking For

- General backend / SRE / platform / infrastructure engineering without adversarial web-extraction experience

- Device, IoT, or telemetry networking (flaky connectivity is not the same as an adversarial site)

- Service mesh, datacenter networking, or distributed-systems-at-scale for its own sake

- Defensive bot-blocking — building the wall; we need people who get past it

- Crawler/scraper building without an adversarial anti-bot component

A Note On Pace

We operate at an absurd level of urgency because the window for what we're building won't stay open forever. If that excites you, keep reading. If it doesn't, no hard feelings — but this role probably isn't for you.

How hiring runs

  1. 1Intro Chat
  2. 2Technical Chat
  3. 3Founders Chat
  4. 4Paid Work Trial

Interested in this one?

There is no apply button here on purpose. Tell us about yourself, we book a short call, and if this role fits we walk you through the company and ask before anything is sent. Always free for you.

Tell us about you
Tell us about you