## Definition

A subdivision of statistics encompassing the simultaneous observation and analysis of more than one outcome variable.* – **Wikipedia*

## Relevance in CRO

Multivariate testing is an extension of A/B testing. Instead of comparing a single change, it’s comparing multiple different changes simultaneously, with each change having it’s own set of variations. Think of it like this:

- Change 1: Variation 1A & Variation 1B
- Change 2: Variation 2A & Variation 2B

results in four possible combinations:

- 1A + 2A
- 1A + 2B
- 1B + 2A
- 1B + 2B

The total potential number of combination is the multiplicative total of each change’s variation count; in this case 2 x 2 = 4. Three changes each with two variations would be 2 x 2 x 2 = 8. Test results are evaluated based on which combination of changes is the best performer.

The value of an MVT is expedience. You Instead of having to run each change as a separate A/B test in sequence, they all get bundled up into a single MVT and run concurrently. The catch is that you will need a (much) larger amount of traffic in order for MVT results to reach a meaningful sample size. Traffic gets distributed across each combination, so in the above example of 4 combinations, that means a 25/25/25/25 split.

## Additional Reading

- When to Do Multivariate tests Instead of A/B/n Tests –
*CXL* - Multivariate Testing: Promises and Pitfalls for High-Traffic Websites –
*Conversion Sciences* - A Practical Guide to Multivariate Testing –
*VWO*