.png)
The Client
Our client is a residential-proxy provider — a company whose core product is a large pool of residential and datacenter exit IPs. Their customers increasingly wanted not just raw proxies but the data those proxies unlock, delivered as clean, structured content without building and running scraping infrastructure themselves.
The Client Request
The client needed a production, general-purpose scraping API to sit on top of their proxy network: give it any URL and get usable content back — HTML or Markdown — regardless of the anti-bot protection guarding the page.
Crucially, success and speed on hard, protected sites had to improve without switching the underlying proxy vendor, and the same capability had to be exposed to both developers over REST and LLM applications over MCP.
It also had to scale: sync and async processing, multi-page crawl and URL discovery, batch and webhooks — with rate limiting, per-plan admission control and full observability around it.
Want to discuss the project?
Result
The result is a single, production-ready scraping API that turns the client's proxy network into a data product: any URL in, clean HTML or Markdown out, using the lightest handling that works and escalating only when a site fights back.
It detects and adapts to the major commercial anti-bot systems — Cloudflare, DataDome, Akamai, PerimeterX, Imperva and the common CAPTCHA vendors — and keeps getting better on hard targets, improving success and speed without ever switching the proxy provider.
Around the engine sits a full production surface — sync, async, crawl, map, batch and webhooks; token-based billing with plan-based admission control; per-user and per-domain rate limiting; and end-to-end tracing, metrics and logs — all running on the client's own bare-metal Kubernetes, on a fully open-source stack, with no public cloud.
We partnered with PinoByte to build a custom payment gateway, and their technical expertise exceeded expectations. They delivered a plugin-based platform that enabled rapid integration of new payment providers, improving our onboarding speed by 3×. They also built an intuitive admin panel and a powerful fraud detection system.
Throughout our collaboration, PinoByte proved to be a reliable and professional technology partner.
