Definition
A type of experiment used to minimize effort in evaluating if it will be worth investing resources into building a non-existent feature or product.
Relevance in CRO
The term originates from an architectural practice of painting doors and windows on an otherwise featureless area in order to provide an aesthetically pleasing space. A practical example is printed mesh overlay placed on scaffolding to hide construction. A not-so-practical example is Wile E. Coyote painting a picture of a tunnel on a wall to try and trick the Roadrunner into running into it but is then hoisted by his own petard.
Essentially, it’s setting up a test that adds a fake feature to a UI in order to gauge user interest. You’re not concerned with the down funnel effects on conversion, merely user engagement with the fake feature. Say an e-commerce store is thinking about starting a subscription service. There’s a lot of logistical requirements involved in setting that up:
- billing system needs to be updated to handle recurring payments
- product supply chain needs to ensure adequate supply is available for regular fulfillment
- website and app UIs need to be updated to enable subscription purchases
- order management needs to be updated to send recurring orders to the warehouse for fulfillment
That’s a lot of work for a new feature that you’re not even sure will be successful. What happens if the subscription service is a bust and now the company is out the hundreds of thousands of dollars and hours invested into building it? The painted door test is a method for mitigating that risk. Using that same example, you could instead set up a test to add a “Subscribe and Save” button on important pages in the website conversion funnel. Here’s the catch… clicking the button doesn’t actually sign the user up for a subscription; instead it triggers a notification that tells the user that the feature isn’t available yet.
What you’re concerned with is seeing what percentage of users click on the button. Those clicks are a proxy for determining user interest in the feature. Then, you can perform an informed ROI analysis based on that information. Would it be worth building out the subscription feature if 5% of users clicked that button? 10%? 50%? What’s the point at which it would be worthwhile?
Additional Reading
- A Product Manager’s guide to Painted-Door Tests – Curtis Stainer
- The Painted Door Test – Brian Hall
- Why and how to run a fake door test — a UX case study – UX Collective