CROWeekly: Case Studies and Roundtables
What's Happening at Corvus CRO
New Case Study Portal Launch
Case studies are a bit of a mixed bag in the CRO space. For every thorough, rigorous example there seem to be a dozen more touting a wholly unrealistic 300% uplift. Often, what you don’t see is that lift was based off of a tiny sample size, like jumping from 1 conversion to 4 off of 50 visitors. While technically that IS a 300% increase, a sample size of 50 is far too small to find a meaningfully measurable impact; it’s a bad result backed up by bad stats. That being said, it’s rather difficult to prove you know what you’re talking about without putting your money where your mouth is. With that in mind, we’re launching a case study portal and publishing our first case study today: “Dead Clicks: When Less Makes More”.
What makes our case studies different? Well, we’re pushing for full transparency by including as much information as possible. Each case study explains our testing process and the stats behind the results, including:
- research conducted to identify the area of opportunity
- hypothesis that forms the basis of the test
- mockups of the variations
- stats used for planning and execution of tests, such as:
- run duration
- sample size
- minimum conversions
- minimum detectable effect (MDE)
- confidence threshold
In addition to that level of detail, uncropped screenshots of relevant KPIs taken straight from the testing tool are on display for all to see. Those are the exact numbers and graphs we use for result analysis. See something odd? Call us out on it! After all, we’re not afraid to “Eat Crow”.
Back in March when Covid lockdowns really started to hit in earnest, Manuel da Costa at Effective Experiments began hosting a weekly virtual CRO Roundtable. It’s an open forum for optimizers, testers, marketers, developers, etc. to meet and have stimulating, organic conversations on experimentation. Anyone that works in the CRO space is welcome, beginners and experts alike. We’ve been going strong for 25 weeks now with a diverse crowd of consultants, agency workers, company owners, and platform founders. Here’s a quick rundown of what each roundtable is like:
- Friendly, open sessions with organic discussion
- No presentations or pre-planned agenda
- No recording to ensure that everyone can speak freely and confidentiality is maintained
- Turning your camera on is preferred but not required; seeing your face helps create better connections so we can all humanize and empathize
Sound like something you’d be interested in? Then come join us over at conversionworld.co. You’ll get get event announcements and be able to RSVP. The CRO Roundtable meets every Friday at the same time, same place via Zoom. Hope to see you there.