Crawling IndexingDuplicate consolidation

Canonicalization

The process by which a search engine groups duplicate or near-duplicate URLs into a cluster and picks one representative URL to index and rank.

In full

Search engines cluster URLs that serve substantially the same content, then select a canonical using signals including the rel=canonical annotation, redirects, internal and external links, sitemap inclusion, HTTPS preference and URL cleanliness. The rel=canonical tag is a strong hint, not a directive: Google frequently chooses a different canonical, which Search Console reports as 'Duplicate, Google chose different canonical than user'. Non-canonical URLs in a cluster generally do not receive their own ranking.

Example

`/shoes?color=red`, `/shoes?color=red&utm_source=news` and `/shoes/red/` all resolve to the same page; Google indexes `/shoes/red/` and folds the others into it.

Related terms

rel="canonical"

A link annotation (in the head or an HTTP header) that tells search engines which URL you consider the preferred version of duplicate…

Duplicate content

Substantively identical or near-identical content available at more than one URL, within a site or across sites.

URL parameters

Key-value pairs after a question mark in a URL; they frequently create duplicate content and crawl waste when they do not change page…

301 redirect

A server-side permanent redirect that tells crawlers a URL has moved for good and that ranking signals should consolidate on the…

URL Inspection tool

A Search Console feature showing Google's indexed state for a URL plus a live test with rendered HTML, screenshot, and page resource…

Where to read more