Reference library

Everything worth reading

188 references — the official documentation that is authoritative by definition, the RFCs and specifications underneath it, the books that are still worth the shelf space, the courses that teach rather than sell, and the studies people keep citing. 38 are marked as the canon: the ones to read before anything else.

188
References
38
Canonical
168
Free
11
Formats

Start here

If you read nothing else, read these. Primary sources first — the engines' own documentation beats every summary of it.

2024 Zero-Click Search Study

Rand Fishkin

Study2024Free

Clickstream-based analysis finding that for every 1,000 Google searches only about 360 clicks reach the open web in the US and about 374 in the EU, with the remainder ending in no click or a click to a Google-owned property.

Canonsparktoro.com

AI Features and Your Website

Google Search Central

DocumentationFree

Google's documentation on how AI Overviews and AI Mode source content, the eligibility requirements, and the preview controls site owners can use to limit inclusion.

Canondevelopers.google.com

Authoritative Sources in a Hyperlinked Environment (HITS)

Jon M. Kleinberg

Landmark PostFree

Kleinberg's paper introducing the hubs-and-authorities model of link analysis, developed contemporaneously with PageRank and still the reference point for topical authority arguments.

Canoncs.cornell.edu

Bing Webmaster Guidelines

Microsoft Bing

GuidelineFree

Bing's published expectations for content, technical implementation and link acquisition, plus the practices Bing treats as abuse. The main non-Google engine policy document in English.

Canonbing.com

Common Crawl

Common Crawl Foundation

DatasetFree

Free, openly licensed petabyte-scale archive of web crawl data with monthly releases of WARC, WAT and WET files plus columnar indexes. Heavily used for link graph research and LLM pretraining.

Canoncommoncrawl.org

Creating Helpful, Reliable, People-First Content

Google Search Central

GuidelineFree

Google's self-assessment questions on content quality, expertise and originality. Includes Google's public explanation of E-E-A-T and of who, how and why in content creation.

Canondevelopers.google.com

FTC's Endorsement Guides: What People Are Asking

US Federal Trade Commission

GuidelineFree

The FTC's practical guidance on disclosing material connections in endorsements, affiliate content, sponsored posts and reviews. The governing US rulebook for affiliate and influencer content in search results.

Canonftc.gov

GEO: Generative Engine Optimization

Pranjal Aggarwal et al.

Landmark Post2023Free

The paper that named generative engine optimization, proposing a benchmark and testing which content modifications increase a source's visibility inside generative search responses.

Canonarxiv.org

Google Crawler (User Agent) Overview

Google

DocumentationFree

Authoritative list of Google's crawlers, fetchers and user-triggered agents, including Googlebot, Google-Extended (AI training and grounding opt-out), GoogleOther and their robots.txt tokens and IP ranges.

Canondevelopers.google.com

Google Search Central Blog

Google Search Central

DocumentationFree

Google's official channel for announcing algorithm updates, documentation changes, new structured data types and crawler behaviour changes. The primary-source record of Google Search announcements since 2005.

Canondevelopers.google.com

2024 Zero-Click Search Study

Rand Fishkin

Study2024Intermediate

Clickstream-based analysis finding that for every 1,000 Google searches only about 360 clicks reach the open web in the US and about 374 in the EU, with the remainder ending in no click or a click to a Google-owned property.

CanonFreesparktoro.com

96.55% of Content Gets No Traffic from Google

Ahrefs

Study2023Intermediate

Large-scale study of pages in the Ahrefs index showing the extreme skew of organic traffic distribution and the correlation between referring domains and any traffic at all.

Freeahrefs.com

About Applebot

Apple

DocumentationIntermediate

Apple's support article covering Applebot and Applebot-Extended, the user agents behind Siri, Spotlight Suggestions and Apple Intelligence training data, and the robots.txt controls for each.

Freesupport.apple.com

Advanced Web Ranking Google Organic CTR Study

Advanced Web Ranking

StudyIntermediate

Regularly refreshed CTR curves segmented by position, device, country, query length and SERP feature presence, drawn from a large Search Console sample.

Freeadvancedwebranking.com

Ahrefs Academy

Ahrefs

Course

Ahrefs' free training hub teaching how to apply its crawler, backlink index and keyword data to practical digital marketing workflows.

Freeahrefs.com

Ahrefs SEO Course for Beginners

Ahrefs

CourseBeginner

Free structured beginner video course covering how search engines work, keyword research, on-page SEO, link building and technical fundamentals.

Freeahrefs.com

Ahrefs SEO Statistics

Ahrefs

StudyBeginner

Continuously updated compilation of sourced SEO statistics drawn from Ahrefs' own studies and third-party research, useful as a citation index rather than as primary research.

Freeahrefs.com

Ahrefs: SEO — The Complete Guide for Beginners

Ahrefs

DocumentationBeginner

Ahrefs' free long-form SEO guide covering keyword research, on-page, link building, technical SEO and measurement, with worked examples from its own data.

Freeahrefs.com

AI Features and Your Website

Google Search Central

DocumentationIntermediate

Google's documentation on how AI Overviews and AI Mode source content, the eligibility requirements, and the preview controls site owners can use to limit inclusion.

CanonFreedevelopers.google.com

AI Overviews Reduce Clicks by 34.5%

Ahrefs

StudyIntermediate

Ahrefs study comparing average CTR for top-ranking pages on queries with and without an AI Overview, one of the most-cited quantifications of AI Overview click loss.

Freeahrefs.com

ai.robots.txt Community Blocklist

ai-robots-txt project

DatasetIntermediate

Community-maintained open-source list of AI-related crawlers with ready-made robots.txt, .htaccess, nginx, Caddy, HAProxy and JSON outputs for blocking or auditing them.

Freegithub.com

ai.txt by Spawning

Spawning AI

StandardIntermediate

Spawning's proposed ai.txt convention, a robots.txt-style file letting site owners state opt-in and opt-out preferences for AI training use of their media and text.

Freesite.spawning.ai

Another Step to Reward High-Quality Sites (Penguin)

Matt Cutts

Landmark Post2012Intermediate

Google's April 2012 announcement of the webspam algorithm update later named Penguin, which targeted link schemes and keyword stuffing and reshaped the link building industry.

Freedevelopers.google.com

ASA / CAP Advertising Codes

Advertising Standards Authority (UK)

GuidelineIntermediate

The UK non-broadcast advertising codes governing marketing communications, including online claims, affiliate disclosure and influencer marketing. Enforced by the ASA.

Freeasa.org.uk

Attention Is All You Need

Ashish Vaswani et al.

Landmark Post2017Advanced

Introduced the Transformer architecture, which underpins BERT, MUM, T5 and the large language models that now generate AI search answers.

Freearxiv.org

Authoritative Sources in a Hyperlinked Environment (HITS)

Jon M. Kleinberg

Landmark PostAdvanced

Kleinberg's paper introducing the hubs-and-authorities model of link analysis, developed contemporaneously with PageRank and still the reference point for topical authority arguments.

CanonFreecs.cornell.edu

Backlinko SEO Marketing Hub

Backlinko

DocumentationBeginner

Free reference hub of SEO guides, definitions and data-backed studies, known for large-sample analyses of search results and CTR.

Freebacklinko.com

Baidu Search Resource Platform Documentation

Baidu

DocumentationAdvanced

Baidu's webmaster platform documentation and announcements, covering submission APIs, crawling behaviour and the algorithm updates Baidu publishes for Chinese-language sites.

Freeziyuan.baidu.com

Bing Webmaster Guidelines

Microsoft Bing

Guideline

Bing's published expectations for content, technical implementation and link acquisition, plus the practices Bing treats as abuse. The main non-Google engine policy document in English.

CanonFreebing.com

Bing Webmaster Tools Help Center

Microsoft Bing

Documentation

Product documentation for Bing Webmaster Tools, including site verification, URL submission, crawl control, the Site Explorer and the IndexNow integration.

Freebing.com

Block Search Indexing with noindex

Google Search Central

DocumentationBeginner

How to use the noindex rule via meta tag or X-Robots-Tag header, and why disallowing a URL in robots.txt prevents the rule from being seen.

Freedevelopers.google.com

Brave Search Crawler Documentation

Brave

DocumentationIntermediate

Brave's help page on its independent search index crawler. Notes that the crawler does not advertise a differentiated user agent, respects Googlebot blocks, and that delisting requires a robots noindex directive rather than robots.txt.

Freesearch.brave.com

BrightEdge Research Reports

BrightEdge

StudyIntermediate

Enterprise-panel research on channel share, SERP feature prevalence and AI Overview behaviour, drawn from BrightEdge's Data Cube of customer sites.

Freebrightedge.com

BrightLocal Local Consumer Review Survey

BrightLocal

StudyBeginner

The longest-running annual consumer survey on how people use online reviews when choosing local businesses, covering review volume, recency, star-rating thresholds and response expectations.

Freebrightlocal.com

Building a StoryBrand

Donald Miller

Book2017Beginner

Messaging framework applying narrative structure to marketing copy, commonly used to shape landing page and homepage messaging alongside search-driven content.

Paidopenlibrary.org

CCBot Crawler Documentation

Common Crawl

DocumentationIntermediate

Common Crawl's page describing CCBot, the crawler that builds the open web corpus widely used to pretrain large language models, and how to allow or block it in robots.txt.

Freecommoncrawl.org

Chrome UX Report (CrUX) Documentation

Google Chrome team

DocumentationAdvanced

Documentation for the CrUX dataset of real-user Core Web Vitals from opted-in Chrome users, covering the API, BigQuery tables, the CrUX History API and dashboard tooling.

Freedeveloper.chrome.com

Chrome UX Report on BigQuery

Google Chrome team

DatasetAdvanced

The public BigQuery dataset containing monthly Core Web Vitals distributions for millions of origins, queryable for competitive performance benchmarking at scale.

console.cloud.google.com

Cloudflare Bot Solutions Documentation

Cloudflare

DocumentationAdvanced

Cloudflare's documentation on bot classification, verified bots and the controls network operators use to allow, challenge or block AI crawlers at the edge.

developers.cloudflare.com

ClueWeb22 Dataset

Lemur Project / Carnegie Mellon University

DatasetAdvanced

Large research web corpus derived from commercial search engine crawls, distributed under licence for academic information retrieval and web search research.

Freelemurproject.org

Common Crawl

Common Crawl Foundation

DatasetAdvanced

Free, openly licensed petabyte-scale archive of web crawl data with monthly releases of WARC, WAT and WET files plus columnar indexes. Heavily used for link graph research and LLM pretraining.

CanonFreecommoncrawl.org

Conductor Academy

Conductor

CourseIntermediate

Free enterprise-oriented SEO and answer engine optimisation learning library covering organic strategy, content operations and measurement.

Freeconductor.com

Content Marketing Institute Research

Content Marketing Institute

StudyBeginner

Annual B2B and B2C content marketing benchmarks, budgets and trends surveys, the long-standing reference for content team staffing and investment norms.

Freecontentmarketinginstitute.com

Crawl Budget Management for Large Sites

Google

DocumentationAdvanced

Google's explanation of crawl capacity limit and crawl demand, and the practical levers large sites have to influence how much Googlebot crawls.

Freedevelopers.google.com

Create a News Sitemap

Google Search Central

DocumentationIntermediate

Requirements for Google News sitemaps, including the news-specific XML namespace, the two-day article window and publication metadata.

Freedevelopers.google.com

Creating Helpful, Reliable, People-First Content

Google Search Central

Guideline

Google's self-assessment questions on content quality, expertise and originality. Includes Google's public explanation of E-E-A-T and of who, how and why in content creation.

CanonFreedevelopers.google.com

Cumulative Layout Shift (CLS)

Google Chrome team

DocumentationIntermediate

Definition of CLS, the visual stability Core Web Vital, covering session windows, expected versus unexpected shifts and the layout shift score calculation.

Freeweb.dev

CXL (Institute)

CXL

CertificationAdvanced

Paid practitioner-taught training in conversion optimisation, growth marketing, analytics and technical marketing, with minidegree programmes that include search and content tracks.

Paidcxl.com

Debug Google Search Traffic Drops

Google Search Central

DocumentationIntermediate

Google's structured diagnostic for Search traffic declines, distinguishing technical issues, manual actions, algorithmic changes, seasonality and reporting glitches.

Freedevelopers.google.com

Dense Passage Retrieval for Open-Domain Question Answering

Vladimir Karpukhin et al.

Landmark Post2020Advanced

Showed that learned dense embeddings can outperform BM25 for first-stage retrieval. The technical basis for vector search and for retrieval inside AI answer engines.

Freearxiv.org

Do People Click on Links in Google AI Summaries?

Pew Research Center

Study2025Intermediate

Independent behavioural study of a representative US panel finding users are less likely to click any result when an AI summary is present. Valuable as non-vendor evidence on AI Overview click behaviour.

Freepewresearch.org

Do You Need an SEO? Tips for Hiring an SEO

Google Search Central

GuidelineBeginner

Google's advice for site owners evaluating SEO providers, including due-diligence questions and warning signs of unsafe practices.

Freedevelopers.google.com

Enroll in Analytics Academy Courses on Skillshop

Google

CourseBeginner

Google's official help page mapping the former Analytics Academy curriculum onto the current Skillshop learning paths for beginner, intermediate and advanced GA4 users.

Freesupport.google.com

Evaluating Page Experience for a Better Web

Sowmya Subramanian

Landmark Post2020Intermediate

Google's announcement of the page experience signal and the introduction of Core Web Vitals as measurable components of it.

Freedevelopers.google.com

Expanding AI Overviews and Introducing AI Mode

Robby Stein

Landmark Post2025Beginner

Google's announcement of AI Mode, a conversational, query-fan-out search experience, alongside the wider rollout of AI Overviews.

Freeblog.google

FTC's Endorsement Guides: What People Are Asking

US Federal Trade Commission

GuidelineIntermediate

The FTC's practical guidance on disclosing material connections in endorsements, affiliate content, sponsored posts and reviews. The governing US rulebook for affiliate and influencer content in search results.

CanonFreeftc.gov

GEO: Generative Engine Optimization

Pranjal Aggarwal et al.

Landmark Post2023Advanced

The paper that named generative engine optimization, proposing a benchmark and testing which content modifications increase a source's visibility inside generative search responses.

CanonFreearxiv.org

Google Analytics 4 Courses and Certification (Skillshop)

Google

Certification

The Google Analytics learning catalogue on Skillshop, including the Google Analytics Certification exam and beginner through advanced GA4 courses. Free to take; the certification expires and must be renewed.

Freeskillshop.exceedlms.com

Google Crawler (User Agent) Overview

Google

DocumentationIntermediate

Authoritative list of Google's crawlers, fetchers and user-triggered agents, including Googlebot, Google-Extended (AI training and grounding opt-out), GoogleOther and their robots.txt tokens and IP ranges.

CanonFreedevelopers.google.com

Google Crawling and Indexing Documentation

Google Search Central

DocumentationIntermediate

Hub of Google's technical documentation on how Googlebot discovers, crawls and indexes pages, covering robots.txt, sitemaps, canonicalization, redirects, mobile sites and JavaScript.

Freedevelopers.google.com

Google Search Central Blog

Google Search Central

Documentation

Google's official channel for announcing algorithm updates, documentation changes, new structured data types and crawler behaviour changes. The primary-source record of Google Search announcements since 2005.

CanonFreedevelopers.google.com

Google Search Central YouTube Channel

Google Search Central

Course

Google's official video channel with SEO office hours, Lightning Talks, the Search Console training series and JavaScript SEO explainers from Google engineers and search advocates.

Freeyoutube.com

Google Search Console Help Center

Google

Documentation

Official product help for Search Console, covering property types, the Performance report data model, Index Coverage states, URL Inspection and manual actions.

CanonFreesupport.google.com

Google Search Essentials

Google Search Central

Guideline

The successor to the Google Webmaster Guidelines. Defines the technical requirements, spam policies and key best practices a site must meet to be eligible for Google Search. Continuously updated.

CanonFreedevelopers.google.com

Google Skillshop

Google

Certification

Google's official free training and certification platform covering Google Ads, Google Analytics, Google Marketing Platform and related products. The certifications are the only Google-issued marketing credentials.

Freeskillshop.exceedlms.com

Google's Common Crawlers

Google

DocumentationIntermediate

Detail page for Google's common crawlers, listing user-agent strings, robots.txt tokens and reverse-DNS verification for Googlebot variants.

Freedevelopers.google.com

Guidelines for Representing Your Business on Google

Google

Guideline

The rules governing Google Business Profile listings: eligibility, naming, categories, service areas and prohibited practices. The foundational policy document for local SEO.

Freesupport.google.com

How To Use Search Console

Google Search Central

DocumentationBeginner

Task-based guide to Search Console for common roles, linking to property verification, performance reporting, indexing reports and URL inspection.

Freedevelopers.google.com

How to Write Meta Descriptions and Control Snippets

Google Search Central

DocumentationBeginner

Google's guidance on snippet generation, meta description quality and the directives that limit snippet length or suppress snippets entirely.

Freedevelopers.google.com

HTML Standard — Document Metadata

WHATWG

StandardAdvanced

The living HTML specification section defining head, title, base, link, meta and the standard metadata names, which is the normative basis for title tags and meta descriptions.

Freehtml.spec.whatwg.org

HTML Standard — Links

WHATWG

StandardAdvanced

The living HTML specification section defining the a and link elements, the href attribute and every standard link relation including nofollow, sponsored, ugc, canonical and alternate.

Freehtml.spec.whatwg.org

HTTP Archive Reports

HTTP Archive

DatasetAdvanced

Longitudinal trend reports on how the web is built and performs, backed by a public BigQuery dataset of monthly crawls of millions of URLs.

Freehttparchive.org

HTTPS as a Ranking Signal

Zineb Ait Bahajji, Gary Illyes

Landmark Post2014Beginner

Google's confirmation that HTTPS is a lightweight ranking signal, which triggered industry-wide migration to TLS.

Freedevelopers.google.com

HubSpot Academy SEO Courses

HubSpot

CertificationBeginner

Free HubSpot Academy courses and certifications on SEO, topic clusters and content strategy, now extended with answer engine optimisation material.

Freeacademy.hubspot.com

IAB Tech Lab Standards

IAB Tech Lab

StandardAdvanced

The technical standards library for digital advertising, including ads.txt, sellers.json, OpenRTB and transparency frameworks relevant to publisher monetisation alongside organic search.

Freeiabtechlab.com

IETF AI Preferences (aipref) Working Group

IETF

StandardAdvanced

IETF working group chartered to standardise a vocabulary and attachment mechanism for expressing preferences about automated processing of web content by AI systems. The formal successor track to ad hoc conventions such as ai.txt.

Freedatatracker.ietf.org

IndexNow Protocol Documentation

IndexNow

StandardIntermediate

Open protocol, backed by Microsoft Bing and Yandex, that lets sites push URL change notifications to participating search engines instead of waiting to be recrawled.

Freeindexnow.org

Influencing Title Links in Google Search

Google Search Central

DocumentationBeginner

Explains how Google generates the title link shown in results, which sources it draws on and why the title element is sometimes rewritten.

Freedevelopers.google.com

Information Architecture for the World Wide Web

Louis Rosenfeld, Peter Morville, Jorge Arango

BookIntermediate

The standard text on organising, labelling, navigating and searching information environments. Directly relevant to site structure, taxonomy design and internal linking. Multiple editions since 1998.

CanonPaidopenlibrary.org

Information Retrieval: Implementing and Evaluating Search Engines

Stefan Büttcher, Charles L. A. Clarke, Gordon V. Cormack

Book2010Advanced

Engineering-focused IR textbook emphasising index construction, compression, query evaluation efficiency and rigorous evaluation methodology. ISBN 9780262026512 (hardcover), 9780262528870 (paperback).

Paidmitpress.mit.edu

Interaction to Next Paint (INP)

Google Chrome team

DocumentationIntermediate

Definition of INP, the responsiveness metric that replaced First Input Delay as a Core Web Vital, including input delay, processing time and presentation delay components.

Freeweb.dev

Intro to How Structured Data Markup Works

Google Search Central

DocumentationBeginner

Google's primer on structured data formats (JSON-LD, microdata, RDFa), placement, testing and the relationship between markup and rich result eligibility.

Freedevelopers.google.com

Intro to Product Structured Data on Google

Google Search Central

DocumentationIntermediate

Reference for Product, Offer, AggregateRating and merchant listing markup used for product rich results and Google Shopping surfaces.

Freedevelopers.google.com

Introduction to Information Retrieval

Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze

Book2008Advanced

The standard graduate IR textbook, covering inverted indexes, tolerant retrieval, vector space models, probabilistic ranking, link analysis and evaluation. The full text is available free online from Stanford NLP. ISBN 9780521865715.

CanonFreenlp.stanford.edu

JSON-LD 1.1 W3C Recommendation

W3C

Standard2020Advanced

The W3C Recommendation defining JSON-LD, the serialization format Google recommends for structured data. Specifies @context, @type, @id, framing and expansion.

Freew3.org

JSON-LD Project Site

JSON-LD Community

DocumentationIntermediate

Community home for JSON-LD with the playground, tooling list and specification links. Useful for debugging structured data before deploying it.

Freejson-ld.org

Largest Contentful Paint (LCP)

Google Chrome team

DocumentationIntermediate

Definition and measurement methodology for LCP, including which elements qualify, the good/needs-improvement/poor thresholds and the sub-part breakdown for optimisation.

Freeweb.dev

Learn Core Web Vitals

Google Chrome team

CourseIntermediate

Structured learning path on web.dev collecting the Core Web Vitals metric definitions, measurement tooling and optimisation techniques into a single course.

Freeweb.dev

Learn Performance

Google Chrome team

CourseIntermediate

Free course covering web performance fundamentals from HTML and CSS delivery through image, font and JavaScript optimisation, with measurement in lab and field.

Freeweb.dev

LearningSEO.io

Aleyda Solis

Course

Free curated SEO learning roadmap organising vetted third-party guides, videos, tools and courses by topic and experience level.

Freelearningseo.io

Lighthouse Documentation

Google Chrome team

Tool GuideIntermediate

Official documentation for Lighthouse, the open-source lab auditing tool that scores performance, accessibility, best practices and SEO, and explains how each audit is computed.

Freedeveloper.chrome.com

LinkedIn Learning SEO Courses

LinkedIn

Course

Subscription video library of SEO courses spanning foundations, keyword strategy, technical SEO, local SEO and ecommerce, with completion certificates.

Paidlinkedin.com

Localized Versions of Your Pages (hreflang)

Google Search Central

DocumentationAdvanced

Google's canonical reference for hreflang annotations across HTML, HTTP headers and sitemaps, including x-default and common implementation errors.

CanonFreedevelopers.google.com

Lumar Academy

Lumar

CourseIntermediate

Free technical SEO and website intelligence learning resources from Lumar (formerly Deepcrawl), covering crawling, site health, log analysis and enterprise SEO governance.

Freelumar.io

MDN: HTTP Response Status Codes

Mozilla

DocumentationIntermediate

Practical reference for every HTTP status code with browser and crawler implications. The most commonly consulted working reference when auditing redirect and error handling.

Freedeveloper.mozilla.org

MDN: The link Element

Mozilla

DocumentationBeginner

Reference for the HTML link element and its rel values, including canonical, alternate, preload and preconnect, with attribute-level browser support notes.

Freedeveloper.mozilla.org

Meta Tags and Attributes that Google Supports

Google Search Central

DocumentationIntermediate

Complete list of meta tags and HTML attributes Google Search recognises, including google-site-verification, notranslate, nositelinkssearchbox and data-nosnippet.

Freedevelopers.google.com

Meta Web Crawlers Documentation

Meta

DocumentationIntermediate

Meta's developer documentation for its crawlers, including facebookexternalhit for link previews and meta-externalagent, and the robots.txt directives that control them.

Freedevelopers.facebook.com

Moz Academy

Moz

Certification

Moz's on-demand training platform with courses and certificates in SEO essentials, keyword research, technical SEO, local SEO and link building. Mix of free and paid tracks.

academy.moz.com

Moz Search Ranking Factors

Moz

StudyIntermediate

Moz's long-running ranking factors resource, historically built from practitioner surveys and correlation analysis. The reference point for how the industry has framed ranking factors since 2005.

Freemoz.com

Moz: Local Search Ranking Factors

Moz

StudyIntermediate

Moz's explainer and survey-derived summary of the signals that influence local pack and localised organic rankings.

Freemoz.com

MS MARCO

Microsoft

DatasetAdvanced

Large-scale dataset of real anonymised Bing queries with human-generated answers and passage relevance labels. The standard benchmark for passage ranking and dense retrieval research.

Freemicrosoft.github.io

MUM: A New AI Milestone for Understanding Information

Pandu Nayak

Landmark Post2021Intermediate

Google's introduction of the Multitask Unified Model, built on the T5 text-to-text architecture, trained across 75 languages and able to understand and generate language across modalities.

Freeblog.google

Naver Search Advisor Webmaster Guide

Naver

GuidelineAdvanced

Naver's official webmaster guide for the dominant South Korean search engine, covering site registration, collection rules, markup and content policy.

Freesearchadvisor.naver.com

Overview of OpenAI Crawlers

OpenAI

DocumentationIntermediate

OpenAI's official documentation of GPTBot (model training), OAI-SearchBot (search index) and ChatGPT-User (user-initiated fetches), with robots.txt tokens and published IP ranges.

CanonFreedevelopers.openai.com

PageSpeed Insights

Google

Tool GuideBeginner

Google's public tool combining CrUX field data with a Lighthouse lab run for a single URL or origin. The standard reference point when discussing Core Web Vitals with stakeholders.

Freepagespeed.web.dev

Passage Re-ranking with BERT

Rodrigo Nogueira, Kyunghyun Cho

Landmark Post2019Advanced

Demonstrated large gains from applying BERT as a re-ranker over retrieved passages on MS MARCO. Foundational to passage ranking as deployed in modern search systems.

Freearxiv.org

Perplexity Crawlers Documentation

Perplexity AI

DocumentationIntermediate

Perplexity's documentation of PerplexityBot and Perplexity-User, explaining which agent indexes content for its answer engine and which fetches pages on behalf of a user request.

Freedocs.perplexity.ai

Product-Led SEO: The Why Behind Building Your Organic Growth Strategy

Eli Schwartz

BookIntermediate

Argues that durable organic growth comes from building products and pages that serve genuine user demand rather than from following formulaic SEO checklists. Written for in-house practitioners and executives setting SEO strategy.

Paidelischwartz.co

Redirects and Google Search

Google Search Central

DocumentationIntermediate

Covers permanent and temporary server-side redirects, meta refresh, JavaScript redirects and how each is interpreted for canonicalization and site moves.

Freedevelopers.google.com

Remove Your Site Info from Google

Google Search Central

DocumentationIntermediate

Decision guide for temporary URL removals, permanent removals, outdated content removal and refresh requests in Search Console.

Freedevelopers.google.com

RFC 3986: Uniform Resource Identifier (URI) Generic Syntax

T. Berners-Lee, R. Fielding, L. Masinter

Standard2005Advanced

The specification governing URL structure, percent-encoding, relative reference resolution and normalization. The basis for correct canonical URL and parameter handling.

Freerfc-editor.org

RFC 6265: HTTP State Management Mechanism

A. Barth

Standard2011Advanced

The cookie specification. Relevant to SEO through cookie-based personalization, consent gating and crawler behaviour, since Googlebot does not persist cookies between requests.

Freerfc-editor.org

RFC 8288: Web Linking

M. Nottingham

Standard2017Advanced

Defines the Link HTTP header and the registry of link relation types, including canonical, alternate, prev and next. Obsoletes RFC 5988.

Freerfc-editor.org

RFC 9110: HTTP Semantics

R. Fielding, M. Nottingham, J. Reschke

Standard2022Advanced

The current authoritative definition of HTTP methods, status codes, and header fields. The primary reference for correct use of 301, 302, 304, 404, 410 and 503 in SEO work.

Freerfc-editor.org

RFC 9111: HTTP Caching

R. Fielding, M. Nottingham, J. Reschke

Standard2022Advanced

Defines Cache-Control, ETag, Last-Modified and conditional requests. Underpins crawl efficiency, CDN behaviour and correct handling of If-Modified-Since by crawlers.

Freerfc-editor.org

RFC 9309: Robots Exclusion Protocol

M. Koster, G. Illyes, H. Zeller, L. Sassman

Standard2022Advanced

The IETF Proposed Standard that finally formalised robots.txt after 25 years of de facto use, defining syntax, matching rules, caching behaviour and error handling.

CanonFreerfc-editor.org

Rich Results Test

Google

Tool GuideBeginner

Google's official validator for structured data eligibility, showing the rendered HTML Googlebot sees and which rich result types a page qualifies for.

Freesearch.google.com

Robots Meta Tags Specifications

Google Search Central

DocumentationIntermediate

Specification of the robots meta tag and X-Robots-Tag HTTP header directives Google supports, including noindex, nofollow, nosnippet, max-snippet, max-image-preview and noai-adjacent controls.

Freedevelopers.google.com

Robots.txt Introduction and Guide

Google Search Central

DocumentationIntermediate

Google's reference for what robots.txt can and cannot do, syntax supported by Googlebot, and the interaction between disallow rules and indexing.

Freedevelopers.google.com

Rolling Out the Mobile-Friendly Update

Google Search Central

Landmark Post2015Beginner

The April 2015 announcement of mobile-friendliness as a ranking signal on mobile search, the update the trade press called Mobilegeddon.

Freedevelopers.google.com

Schema Markup Validator

Schema.org

Tool GuideBeginner

The schema.org-hosted validator for generic structured data correctness, independent of whether a type is eligible for a Google rich result.

Freevalidator.schema.org

Schema.org Getting Started Guide

Schema.org

DocumentationBeginner

Introductory walkthrough of schema.org markup covering item types, properties, nesting and the differences between microdata, RDFa and JSON-LD encodings.

Freeschema.org

Schema.org Vocabulary

Schema.org (Google, Microsoft, Yahoo, Yandex)

StandardIntermediate

The shared structured data vocabulary maintained by the major search engines and a W3C community group. The authoritative type and property reference behind almost all rich results.

CanonFreeschema.org

Search Engine Journal State of SEO Report

Search Engine Journal

StudyBeginner

Annual practitioner survey covering budgets, team structures, tooling, priorities and the perceived impact of algorithm and AI-search changes.

Freesearchenginejournal.com

Search Engine Land: What Is SEO?

Search Engine Land

DocumentationBeginner

Search Engine Land's evergreen explainer and guide hub defining SEO, its disciplines and how the trade press categorises ranking factors and search updates.

Freesearchengineland.com

Search Engines: Information Retrieval in Practice

W. Bruce Croft, Donald Metzler, Trevor Strohman

Book2009Advanced

University textbook on how search engines are actually built: crawling, indexing, query processing, ranking models and evaluation. Companion site hosts slides and the Galago toolkit. ISBN 9780136072249.

CanonPaidciir.cs.umass.edu

Search Quality Evaluator Guidelines

Google

GuidelineAdvanced

The PDF handbook Google gives to its external search quality raters. Defines Page Quality, Needs Met, E-E-A-T, Your Money or Your Life topics and lowest-quality signals. The closest public statement of what Google considers a good result.

CanonFreeguidelines.raterhub.com

Secrets from the Google Algorithm Leak

Mike King

Landmark Post2024Advanced

The most detailed technical analysis of the leaked Google Content Warehouse API documentation, covering NavBoost, click signals, site authority, twiddlers and document scoring modules.

CanonFreeipullrank.com

Semrush Academy

Semrush

Course

Free course library and exam-based certifications covering SEO, PPC, content marketing, competitive research and the Semrush toolset itself.

Freesemrush.com

Semrush SEO Crash Course with Brian Dean

Brian Dean

CourseBeginner

Free multi-lesson video course on keyword research, on-page optimisation, link building and content strategy, taught by the founder of Backlinko.

Freesemrush.com

SEO Best Practices for Ecommerce Sites

Google Search Central

DocumentationIntermediate

Google's specialty documentation for online stores, covering URL structure for facets, pagination, product data, out-of-stock handling and merchant listings.

Freedevelopers.google.com

SEO by the Sea

Bill Slawski

Landmark PostAdvanced

The archive of Bill Slawski's patent analyses, which for two decades was the primary public source translating Google and Bing patents into SEO implications. Slawski died in 2022; the archive remains a reference body of work.

Freeseobythesea.com

SEO Starter Guide: The Basics

Google Search Central

DocumentationBeginner

Google's own introductory guide to search engine optimization, covering how Google finds and understands pages, how to influence appearance in results and which practices are unnecessary.

CanonFreedevelopers.google.com

Site Names in Google Search

Google Search Central

DocumentationBeginner

Requirements for influencing the site name displayed above results, including WebSite structured data and other supported signals.

Freedevelopers.google.com

Sitemaps XML Protocol 0.9

sitemaps.org

StandardIntermediate

The jointly supported sitemap specification from Google, Yahoo and Microsoft. Defines the XML schema, the 50,000 URL and 50 MB limits, sitemap index files and robots.txt autodiscovery.

CanonFreesitemaps.org

Spam Policies for Google Web Search

Google Search Central

Guideline

Google's enumeration of behaviours that can trigger ranking demotion or removal from Search, including cloaking, doorways, scaled content abuse, site reputation abuse, expired domain abuse and link spam.

CanonFreedevelopers.google.com

Structured Data Markup that Google Search Supports

Google Search Central

DocumentationIntermediate

The search gallery listing every structured data type that can produce a rich result in Google Search, with required and recommended properties for each.

CanonFreedevelopers.google.com

The /llms.txt File

Jeremy Howard

Standard2024Intermediate

Proposal for a markdown file at /llms.txt that gives large language models a curated, inference-time map of a site's most useful content. A community convention rather than a ratified standard; adoption by major AI providers remains limited.

CanonFreellmstxt.org

The 2024 Web Almanac

HTTP Archive

Study2024Advanced

The full 2024 edition of HTTP Archive's annual state of the web report, spanning performance, markup, media, security, CMS, accessibility and SEO chapters.

Freealmanac.httparchive.org

The Anatomy of a Large-Scale Hypertextual Web Search Engine

Sergey Brin, Lawrence Page

Landmark Post1998Advanced

The paper that introduced Google, describing PageRank, anchor text as a ranking signal, the crawling and indexing architecture and the repository design. The single most important document in SEO's intellectual history.

CanonFreeinfolab.stanford.edu

The Art of SEO: Mastering Search Engine Optimization

Eric Enge, Stephan Spencer, Jessie Stricchiola, Rand Fishkin

Book

The most comprehensive general SEO textbook, first published by O'Reilly in 2009 (ISBN 9780596518868) and revised across multiple editions. Covers search engine mechanics, strategy, keyword research, content, links, measurement and organisational issues.

CanonPaidopenlibrary.org

The Beginner's Guide to SEO

Moz

DocumentationBeginner

One of the longest-running free introductory SEO guides on the web, covering crawling and indexing, keyword research, on-page and technical optimisation, link building and measurement.

CanonFreemoz.com

The Open Graph Protocol

ogp.me

StandardBeginner

Specification for the og: meta properties that control how URLs are rendered when shared on social platforms and in many chat and AI interfaces.

Freeogp.me

The Web Almanac — SEO Chapter

HTTP Archive

StudyAdvanced

Annual, peer-reviewed analysis of SEO implementation across millions of real websites, measuring adoption of canonical tags, structured data, hreflang, robots directives and rendering patterns.

Freealmanac.httparchive.org

They Ask, You Answer

Marcus Sheridan

Book2016Beginner

Content marketing framework built on answering customers' real buying questions, including pricing, comparisons, problems and reviews. Widely used as the editorial basis for bottom-of-funnel SEO content programmes.

Paidopenlibrary.org

Traffic Think Tank

Search Engine Land / Semrush

CourseAdvanced

Paid SEO community and training programme combining an on-demand course library with a private practitioner community. Now operated as a Search Engine Land community.

Paidtrafficthinktank.com

UK Competition and Markets Authority

GOV.UK

GuidelineIntermediate

The UK regulator whose digital markets work covers search engine competition, self-preferencing and online choice architecture, including designations under the digital markets competition regime.

Freegov.uk

Understand JavaScript SEO Basics

Google Search Central

DocumentationAdvanced

Describes Google's three-stage crawl, render and index pipeline for JavaScript applications and the practices that make client-rendered content indexable.

CanonFreedevelopers.google.com

Understanding Google Page Experience

Google Search Central

DocumentationIntermediate

Google's current position on page experience signals, including Core Web Vitals, HTTPS and intrusive interstitials, and how they relate to ranking.

Freedevelopers.google.com

Understanding Searches Better Than Ever Before

Pandu Nayak

Landmark Post2019Intermediate

Google's announcement of BERT in Search, described at the time as the biggest leap forward in five years, affecting roughly one in ten English queries.

Freeblog.google

US 7,346,839 — Information Retrieval Based on Historical Data

Google Inc.

Patent2008Advanced

The historical data patent, describing document inception dates, content update rates, link growth rates, anchor text change and domain registration signals as scoring inputs. Filed December 2003, published March 2008.

Freefreepatentsonline.com

US 8,682,892 — Ranking Search Results

Google Inc.

Patent2014Advanced

Frequently cited in analyses of site-level quality scoring and of query-and-click based ranking modifiers. Filed September 2012, issued March 2014.

Freefreepatentsonline.com

Using robots.txt (Yandex)

Yandex

DocumentationIntermediate

Yandex's robots.txt reference, including directives such as Clean-param and Host that are specific to the Yandex robot.

Freeyandex.com

We Analyzed 11.8 Million Google Search Results

Brian Dean

StudyIntermediate

Correlation study across millions of results examining backlinks, content length, page speed, HTTPS, schema and other factors against ranking position. Correlational, not causal.

Freebacklinko.com

We Studied 200,000 AI Overviews

Semrush

StudyIntermediate

Large-sample analysis of AI Overview appearance by query type, length, intent and citation source, one of the primary datasets on which AI Overview optimisation advice is based.

Freesemrush.com

Web Content Accessibility Guidelines (WCAG) 2.1

W3C

Standard2018Advanced

The WCAG version referenced by most current accessibility legislation, including the European Accessibility Act and many public-sector procurement rules.

Freew3.org

Web Content Accessibility Guidelines (WCAG) 2.2

W3C

Standard2023Advanced

W3C Recommendation defining testable accessibility success criteria at levels A, AA and AAA. Overlaps heavily with SEO through semantic markup, alt text, heading structure and link text.

CanonFreew3.org

Web Vitals

Google Chrome team

DocumentationIntermediate

Canonical definition of the Web Vitals initiative and the Core Web Vitals set, with the current thresholds and the field-versus-lab measurement model.

CanonFreeweb.dev

What Is a Sitemap (Google Search Central)

Google Search Central

DocumentationBeginner

Google's guidance on when XML sitemaps help, the supported formats, sitemap index files and submission methods.

Freedevelopers.google.com

Where Do DuckDuckGo Search Results Come From?

DuckDuckGo

DocumentationBeginner

DuckDuckGo's explanation of its result sources, including its own DuckDuckBot crawler, Bing, and other partners. Clarifies why there is no separate DuckDuckGo submission process.

Freeduckduckgo.com

Whiteboard Friday

Moz

Landmark Post

Moz's long-running weekly video series, begun by Rand Fishkin, explaining SEO concepts at a whiteboard. Collectively one of the most-cited free educational bodies of work in the industry.

Freemoz.com

Whitespark Local Search Ranking Factors Report

Whitespark

StudyIntermediate

Annual expert survey ranking the relative importance of local pack and local organic ranking factors, including Google Business Profile signals, reviews, citations and links.

Freewhitespark.ca

Wikimedia Downloads (Wikipedia Dumps)

Wikimedia Foundation

DatasetAdvanced

Complete periodic exports of Wikipedia and sister projects, including page content, pagelinks and pageview data. A standard corpus for entity, knowledge graph and query understanding work.

Freedumps.wikimedia.org

Yandex Webmaster Help for Site Owners

Yandex

DocumentationIntermediate

Yandex's official English-language help for webmasters, covering site indexing, Yandex Webmaster tools, quality requirements and the behaviour of Yandex's robot.

Freeyandex.com

Yoast SEO Academy

Yoast

CourseBeginner

Training platform focused on WordPress-centric SEO, with courses on SEO copywriting, technical SEO, structured data and site structure. Some courses are free, others bundled with Yoast SEO Premium.

yoast.com

Browse by format

Links go straight to the source. The Atlas does not host, mirror or summarise these works — every entry links to the publisher, the author or the platform. Where something is paid, it says so.