Incrementality testing
Measuring the additional outcomes caused by an activity, by comparing a treated group against an untreated control, rather than inferring…
AnalyticsInterrupted time seriesCausalImpact
Causal impact analysis
A statistical method that estimates the effect of an intervention by forecasting what would have happened without it, using control series as a counterfactual.
In full
Implemented in Google's open-source CausalImpact package using Bayesian structural time-series models, it takes a pre-period, a set of correlated but unaffected control series, and an intervention date, then reports the difference between observed and predicted values with credible intervals. It is the standard tool for evaluating migrations, template changes and algorithm updates when a true control group is impossible. Results are only as good as the control series; using a control that was also affected invalidates the estimate.
Example
After a migration, analysis using unaffected site sections as controls estimates a 9% loss with a credible interval excluding zero.
Related terms
SEO split testing
Testing SEO changes by applying them to a random subset of similar pages and comparing outcomes against a held-back control group.
Site migration
Any change to a site's domain, URL structure, platform, template or content organisation that materially affects how search engines see it.
SEO forecasting
Projecting future organic performance from current rankings, expected position changes, demand trends and seasonality, usually as a range…
Algorithm volatility
Day-to-day fluctuation in rankings caused by continuous algorithm changes, index updates and testing, distinct from confirmed named updates.