« Back to Glossary Index

## Definition

A subdivision of statistics encompassing the simultaneous observation and analysis of more than one outcome variable. – Wikipedia

## Relevance in CRO

Multivariate testing is an extension of A/B testing. Instead of comparing a single change, it’s comparing multiple different changes simultaneously, with each change having it’s own set of variations. Think of it like this:

• Change 1: Variation 1A & Variation 1B
• Change 2: Variation 2A & Variation 2B

results in four possible combinations:

• 1A + 2A
• 1A + 2B
• 1B + 2A
• 1B + 2B

The total potential number of combination is the multiplicative total of each change’s variation count; in this case 2 x 2 = 4. Three changes each with two variations would be 2 x 2 x 2 = 8. Test results are evaluated based on which combination of changes is the best performer.

The value of an MVT is expedience. You Instead of having to run each change as a separate A/B test in sequence, they all get bundled up into a single MVT and run concurrently. The catch is that you will need a (much) larger amount of traffic in order for MVT results to reach a meaningful sample size. Traffic gets distributed across each combination, so in the above example of 4 combinations, that means a 25/25/25/25 split.

## Additional Reading

« Back to Glossary Index