Operator of ChatGPT and of four documented web crawlers: OAI-SearchBot for ChatGPT search results, GPTBot for model training, ChatGPT-User for user-initiated fetches and OAI-AdsBot for ad landing page checks. Its published robots.txt policy, which lets publishers allow search visibility while blocking training, is now a core technical decision for every site owner.
Publishes crawler documentation and IP ranges separating search indexing from training use.
OpenAI is a search engine based in United States.
6 other search engines, plus others based in United States.
Operator of Claude and of three documented crawlers: ClaudeBot for training data, Claude-User for user-initiated page fetches and Claude-SearchBot for search quality. It publishes robots.txt blocking instructions and crawler IP ranges, making it one of the AI institutions whose crawler policy SEO and content teams must configure for explicitly.
Baidu's webmaster platform (百度搜索资源平台), providing submission APIs, indexing diagnostics and Chinese-language crawling guidance. It is the required entry point for any site attempting visibility in mainland China's largest search engine, where Google is unavailable.
Search engine built on a fully independent index of billions of pages rather than syndicated results, operated by Brave Software. Its independence makes it one of the few alternative indexes SEOs can test against, and it also sells index access via an API used by AI products.
Vietnamese browser and search engine built specifically for Vietnamese users, with local-language handling and a search API for developers. It holds a meaningful share of Vietnamese search and browser usage, so sites targeting Vietnam optimise for it alongside Google.
Privacy-focused search engine that combines its own crawling with results licensed from Bing, and publishes its DuckDuckBot crawler documentation. Its refusal to build user profiles removes personalisation from its results, which makes it a useful neutral reference SERP for rank checking.
Not-for-profit search engine founded in Berlin in December 2009 by Christian Kroll, dedicating all profits to tree planting and operating as a steward-owned certified B Corporation. It represents a meaningful share of privacy- and values-motivated European search traffic and, with Qwant, co-founded a European index initiative.