If the dream of web data collection is to go from a prompt to a quality web data pipeline in one shot, then the new tools and features we’re sharing today can move you one step closer.
Zyte MCP, Agent Skills and Markdown Output
Headless Browsers via CDP
Spending Limits by API key
We’ve been using these tools internally for months now, decreasing the development time of 1000’s of spiders by approximately 52% (see Shane's talk below). Now we’re making them available to all users.
Spoiler alert: Today, even with these tools, domain expertise and a quality QA process still determines the result.
New: Zyte MCP
Zyte MCP connects AI assistants (like Claude) to the Zyte API, enabling agents to perform web data retrieval and extraction tasks that standard tools often fail to handle.
It is designed for developers who want to build autonomous agents that can reliably operate on the web without manual intervention. Beyond simple fetching, the Zyte MCP grants agents access to browser automation, Scrapy Cloud tools, and usage tracking.
When you want to create (or update) Scrapy spiders, you can now use our AI skills and workflows to generate production-ready Scrapy code from prompts.Try these skills and IDE plugins.
The skill explores your target site, discovers available fields, and gets your sign-off on a schema before writing any code. Then it scaffolds the full Scrapy project (page objects, fixtures, spider) and runs a smoke test to confirm extraction works before handing it back.
New: Zyte Headless Browser via CDP
Running browsers at scale and keeping them unblocked eats time and money.
Now you can connect your Puppeteer or Playwright scripts via CDP to Zyte’s remote browsers for execution, and we will take care of proxies, browser fingerprints, TLS handshakes, CAPTCHAs, and all the rest for high success rates.
In addition to the existing page content output, Zyte API can now convert web pages into clean markdown, stripping away the HTML and noise, leaving you with just the content you need, ready for any application.
Both are great options if you want low token, high signal, low token inputs for LLMs.