Open-source Python library for SEO and SEM workflows, including crawling with Scrapy, robots.txt and sitemap parsing, SERP APIs, log file parsing and text analysis.
advertools.readthedocs.io
CrawlerOpen SourceFree tierMCP server
Screaming Frog MCP
Open-source headless MCP server that wraps the Screaming Frog SEO Spider command line, letting AI agents start crawls and export or analyse crawl data for scheduled audits and CI use. Community project, not published by Screaming Frog.
What it is
Open-source headless MCP server that wraps the Screaming Frog SEO Spider command line, letting AI agents start crawls and export or analyse crawl data for scheduled audits and CI use. Community project, not published by Screaming Frog.
It sits in the crawler category, and also covers technical seo., and has been around since 2026. The vendor also ships an MCP server, so the data can be queried directly from an AI assistant.
The Atlas does not rank tools. No scores, no "top 10", no affiliate links. Categories are descriptive, and two tools in the same category may solve very different problems. Trial the shortlist against your own site before deciding.
Alternatives & neighbours
9 tools sharing at least one category with Screaming Frog MCP.
SEO platform built on its own web crawler and link index, offering Site Explorer, Keywords Explorer, Site Audit, Rank Tracker and Content Explorer. Brand Radar adds AI-answer visibility tracking.
AI visibilityahrefs.com
Chinese webmaster tool site providing composite SEO diagnostics, Baidu weight and keyword ranking analysis, backlink and reciprocal-link checks, keyword mining, dead-link detection and historical site data, plus a paid data API.
aizhan.com
High-performance crawler for large websites with a focus on link graph analysis, PageRank-style internal link metrics and reproducible technical audits.
audisto.com
French link intelligence platform with its own crawl index, providing semantic value and induced strength metrics for pages and domains.
babbar.tech
Microsoft's free site owner console for Bing, providing search performance data, index and crawl diagnostics, URL submission, IndexNow support and keyword research.
bing.com
Enterprise SEO platform combining crawling, log analysis, keyword data and an activation layer that renders and submits pages to search engines. Has extended reporting to AI search surfaces.
AI visibilitybotify.com
Technology lookup service and extension reporting the software, hosting, analytics and advertising stack used by a website, with lead list exports.
builtwith.com
Chinese webmaster tool suite from Chinaz covering SEO overview queries, Baidu weight estimation, app and site ranking lookups, WHOIS, ICP filing checks, reciprocal-link verification and a data API.
chinaz.com