Get started

Blog

CROWeekly: Reducing Project Data Bloat

What Happened at Corvus CRO Last Week

screenshot of a Kanban board
A Kanban board display we use for managing split tests

How Corvus CRO Manages Split Tests Efficiently

We use a Kanban board view in Airtable to monitor and manage experiments. Each experiment is an Airtable record, which translates to a card on the board. There are 5 channels for progress stages of an experiment life cycle:

  • Idea – a new potential expriment submitted via a hypothesis builder
  • Plan – calculate the experiment’s sample size, run time, and priority in the testing plan
  • Build – set up the experiment in the split testing platform
  • QC – review the experiment and approve it to go live
  • Active – the experiment is live
  • Complete – the experiment has concluded

Moving a card from one channel to another triggers automation for repeatable tasks like sending status notifications, running a report, or updating project information. We use Basecamp for project management, but this process can be applied to almost any project management platform that has API, like Asana, Jira, or Trello. The automation is set up to use API hooks for performing simple, repeatable, standardized processes like creating projects/tasks, adding notes, and assigning tasks. This significantly cuts down on overhead and eliminates a lot of tedious manual configuration drudgery from the project management process.

screenshot of a Kanban board
Our split testing project list was becoming a bit bloated

Updating Our Process to Reduce Data Bloat

The goal of any good project management process is to minimize overhead. One important component of project management is dealing with data bloat. A lot of project data has a shelf life; it’s relevant and useful for only specific period of time. Data bloat accumulates when tasks aren’t acted upon and data goes stale. As data piles up, it becomes harder and harder for users of the system to find relevant information quickly and actually get work done. This wastes time and causes frustration. Worse, information bloat can lower morale if ignored for too long. To help combat data bloat, we’re making a change to our project creation automation.

Previously, we had the automation set to create a new project for an experiment when it was moved into the Planning stage. A lot of projects were getting created. Projects for lower priority experiment ideas were just sitting open and unused, making it harder to find the active projects. Sometimes an experiment idea would be scrapped or modified before it  was ready to build. If a project had already been opened for that experiment then it had to be manually updated, creating unnecessary revision work.

Now the automation creates projects when an experiment idea progresses to the Build stage. Building an experiment is where the bulk of actionable tasks begin, so this change should help increase project data freshness. Projects will be created more closely to their “ready for work” state instead of sitting open and unattended. The tasks that go on during the planning stage have been migrated to a separate, repeatable weekly task.

CROWeekly: Startup Week and Meetups Galore

What Happened at Corvus CRO This Week

Startup Cincy Week 2019

Participating in Startup Cincy Week 2019

Startup Cincy Week is an annual, week-long series of presentations, roundtable discussions, networking, and job fairs celebrating of the growing entrepreneurial community here in Cincinnati, Ohio. The event brings together entrepreneurs, students, mentors, investors, and teachers from local StartUps, BigCos, and universities. Topics range from starting a business, securing funding, scaling, creativity, technology, design, and more.

This is the second year that Corvus CRO has participated in Startup Cincy Week, and the first year that we’ve presented! Being event volunteers is becoming an annual tradition for us. Here’s a shot of Matt working the registration desk on Tuesday morning:

Matt waves hi from the Startup Cincy Week registration desk

New for us this year was organizing a panel. Since our main focus here at Corvus CRO is User Experience, we wanted to contribute some design based content. We brought together some of local UX designer friends and colleagues to participate in a roundtable discussion titled “UX for Start-ups: Leveraging experience design to grow your business”. All the presentations were livestreamed and recorded on Cintrifuse’s Facebook page, so here’s a link to our session. Please check it out! A great big thank you to Beau Heubach and Katie Pohlman from Gaslight and Paul Armstrong from Alchemy for joining us.

We’re already looking forward to Startup Cincy Week 2020!

October A/B Testing Happy Hour

A/B Testing Happy Hour October 2019 Meetup banner, inspired by The Legend of Zelda
October 2019’s banner was inspired by the title screen for The Legend of Zelda on NES

16-Bit Barcade was the place to be again in October. Thanks to all who attended last month’s A/B Testing Happy Hour, and a special thank you to Jessica Hall at Tech Elevator for helping promote the meetup. Several students from the current bootcamp session showed up to discuss front end development and share interesting personal anecdotes about what brought them into tech.

If you’re interested in attending future happy hours or chatting with other fine folks that work in the CRO space, then please check out the Cincinnati Split Testers meetup group. November’s A/B Testing Happy Hour will be here before you know it!

October CodePen Meetup

Building perfect responsive layouts with HTML and CSS

Bryan King, our Director of Design, is one of the co-organizers for CodePen Cincinnati, a local meetup group focused on discussing and teaching the latest in front-end development. For the October meetup, he gave a fantastic presentation on building perfect responsive layouts using HTML and CSS. He performed a love coding exercise demonstrating the power and versatility of flexbox and CSS grid. As always, thank you to Dot Dash for hosting.

CROWeekly: A/B Testing With Pixels and Beer

What Happened at Corvus CRO This Week

A/B Testing Happy Hour

September AB Testing Happy Hour Meetup banner, design inspired by Sonic the Hedgehog

September’s A/B Testing Happy Hour had the best event turnout to date! A great big thank you to everyone that attended; hope to see you in October, and many more in the future. There were some wonderfully interesting conversations around the role of data science in testing, integrating and scaling testing programs within a company, and some good ol’ nuts-and-bolts test build process. Moving locations to 16-Bit Barcade really helped with accessibility. Unfortunately there was an issue with the beer taps, so we all had to make do with cans, bottles, and poured drinks. The bourbon drinkers of the group were totally cool with that.

Hosting at 16-bit gave us a fun theming idea to design video game inspired custom banners for each monthly event. September’s banner was based on the start screen for the original Sonic the Hedgehog on Sega Genesis. Voting is open to choose next month’s theme: go vote on Twitter!

If you’re ever in the Cincinnati area on the 2nd Wednesday of the month, come join us; we’d love to meet you in person. Your fist beer is on us! The October meetup has been scheduled. You can always join the Cincinnati Split Testers meetup group to get updates and event notifications.

CROWeekly: Bookmarklets, Barbecue, and Beer

What Happened at Corvus CRO This Week

The Experiment Overlay can now trigger variation switches for experiments with delayed activation
The Experiment Overlay now triggers variation switches for experiments with delayed activation

Variation Switching for Experiments With Delayed Activation

The Corvus CRO Experiment Overlay will now properly switch variation bucketing on client-side Convert Experiences experiments that do not activate on initial page load. Typically, these are experiments set up with JavaScript based targeting conditions or manually activated via JavaScript API. Some common use cases for these types of activations are: experiments that should run only when specific element(s) are present on a page, or in response to a user action like clicking a button or watching a video.

Grilling With the Team

Once a month, we have an informal company dinner hosted by one of our staff. It’s the hosts responsibility to prepare the meal and entertain at their home. Having dinner at someone’s home invites open, non-work related conversation. Connecting over food and drink is a major way that we strengthen our team bonds.

This month it was my turn to host. Here’s the menu:

  • citrus marinated pork loin
  • grilled asparagus
  • sun-dried tomato orzo salad

Mistakes were made: I ended up burning the asparagus tips, and we didn’t take any pictures. That’s okay though, mistakes are learning opportunities. Next month will be better, for sure.

A/B Testing Happy Hour

A/B testing happy hour

Another one of our monthly endeavors is organizing a local A/B Testing Happy Hour every second Wednesday of the month here in our home base of Cincinnati, OH. Last Wednesday we enjoyed some fine conversion optimization discussion coupled with blood orange raddlers and raspberry-lemon hefeweizen on the newly opened rooftop bar at Braxton Brewing. If you’re ever in the area, then please join us!

CROWeekly: Data Transparency with Client Portals

What Happened at Corvus CRO This Week

UI tabs for displaying Airtable views on a client portal
UI tabs for displaying Airtable views on a client portal

Higher Visibility for Test Management Data

We use Airtable for collating experiment data from all of it’s various sources: internal, client, testing software, analytics software, project management software, etc. This experiment data forms the backbone of our testing process; it’s used for things like hypothesis construction, test prioritization, project management, triggering automation, and reporting.

Because “Transparency and Honesty” is one of our core values, we want to make sure that clients can view all of their experiment data on demand, at any time. However, creating additional accounts isn’t a viable solution for us. Because Airtable’s pricing is per user, costs would quickly balloon up as program maturity rises and more people get involved in the testing process. Also, a high level of data integrity is critical to the testing process. More users with editing privileges means more potential for loss of data fidelity. For these two reasons, we’ve opted to not set up user accounts for clients.

Airtable offers a couple different ways of sharing data to non-users. Private links can be enabled per view, allowing anyone with a link access to see that view. Maintaining a list of links would be cumbersome, so there’s also the ability to embed a view via an iframe, which is our current solution for data transparency. We have individual portals set up for each client. Airtable views for that client’s data are placed into tabs. Clients can easily view all of their test program data from their portal.

If data transparency is as important to you as it is to us, then we’d love to talk. Is finding the right information difficult? Struggling to make sense of all your testing data sources? Corvus CRO can help. Schedule a free 30 minute consultation with us today.

CROWeekly: Program Management With Basecamp and Airtable

What Happened at Corvus CRO This Week

Screenshot of the Basecamp 2 project list interface, showing projects that were automatically created via integration with Airtable
Split Testing projects automatically created in Basecamp 2 when a test is moved to the planning stage in the Airtable Kanban board

Automating Basecamp Project Management Tasks

Basecamp is a fantastic project mangement platform with an unbeatable price point at only $100 a month. We use it to collaborate with several of our clients. Both versions of Basecamp (2 and 3) are really good at handling team communication; it handles messages, notifications, scheduling, to-do lists, document sharing, and most of the other important features of a project management tool.

That being said, Basecamp has a little bit of trouble looking at the “big picture” if you have lots of active projects. Something that’s easily possible with an optimization program, especially if you set up a separate project for experiment like we do. Our solution for broader program visibility is a Kanban board:

Screenshot of the Basecamp 2 project list interface, showing projects that were automatically created via integration with Airtable
A Kanban board in Airtable, also we’re running a lot of Facebook inbound traffic personalization experiments

Experiment progress is tracked through these activity stages:

  • Idea – New experiments submitted through a hypothesis builder form
  • Plan – Experiments prioritized through a P.I.E.E. ranking
  • Build – Experiments being built in the testing platform
  • QC – Experiments ready for proofing and final approval
  • Active – Live experiments
  • Complete – Finished experiments

Every experiment gets it’s own card on the Kanban board. Progressing a card through the Kanban stacks forms the core of our automation triggers. For example, when an card is moved to Complete, a Google Analytics report automatically runs to grab user counts, transaction counts, and revenue for the matching time period. That GA data is used to perform comparative analysis and experiment lifetime value estimation.

The newest addition to the Corvus CRO automation arsenal is Basecamp project creation and management. Every time a card is moved to the Plan stack, a new Basecamp project is created from a template; the template sets up the necessary to-do lists and discussion channels for experiment collaboration. Progression into other stacks adds additional to-dos, checks off tasks, uploads files, and/or sends notifications as necessary. The automation handles building the project management framework, eliminating a significant chunk of configuration tedium, and freeing up people to work on more interesting things.

Organization is key to any successful conversion optimization program. Trapped in configuration hell? Project management software causing more headaches than progress? Can’t get a handle on your split testing process? Corvus CRO can help. Schedule a free 30 minute consultation with us today.

CROWeekly: Result Data Scraper Tool Updates

What Happened at Corvus CRO This Week

Feature Update and Bux Fix for The Result Scraper Tool
Feature Update and Bux Fix for The Result Scraper Tool
Feature Update and Bux Fix for The Result Scraper Tool
Feature Update and Bux Fix for The Result Scraper Tool

Automatic Detection of Winning and Losing Variations

If the Primary KPI is set for an experiment record in Airtable and a matching goal is found, the Result Data Scraper tool will now automatically detect the best and worst performing variations. Manual selection will be displayed as a backup if the Primary KPI is not set or a matching goal is not found. This makes the scraper a little bit easier to use, automating another bit of data entry if possible. Properly declaring a Primary KPI during the idea stage of an experiment results in less work in the completion stage. An ounce of prevention is worth a pound of cure.

Bug Fix for High Conversion Counts

Discovered and resolved a bug in the result data scraper tool if both the amount of conversions and visitors are both over 1,000. The regex used for stripping non-numeric characters from the conversion and visitor count strings did not have the global flag applied. For high counts on both conversion and visitors (for example: 3,521/119,347 conversions/visitors) only the first comma was being stripped. This was causing an error when attempting to push the visitor count into Airtable, since only numeric data would be accepted by that field. More robust string parsing and sanitization has been put in place to resolve this issue and help future-proof against other potential string parsing and conversion issues.

CROWeekly: Simplifying Split Test Asset Organization

What Happened at Corvus CRO This Week

Folders that were created automatically in Google Drive for each split test

Automating File Organization through Split Test Project Management Process

File organization is one of those tasks that always seems to fall through the cracks. Everyone has their own personal methods for setting up folders and naming files. Compound that with cloud storage, which is great for file sharing but can easily create more issues when multiple users can manipulate files and folders. A solution intended to foster collaboration ends up creating more friction. Asset management policies are something that should be established and adhered to at an organizational level.

One big problem with asset management policies is that it’s tedious for individual users to adhere to. While not particularly difficult, cutting corners is easier when you’re in the thick of production. File and folder names with “new”, “new new”, “final”, etc. start cropping up. When does a file stop being “new”? 1 week later? 1 month? 6 months?

Corvus CRO is taking one small step towards simplifying asset management by automatically creating a matching asset folder any time a split test moves into the “build” (production) stage. Folders are placed in an “Experiment Assets” parent folder, which is within the client’s root. Having a consistent location makes them easy to locate. The auto-generated folders have the same name as the split test, making it easy to cross-reference with the test itself.

While not comprehensive by any means, this is one small step towards a more robust asset management system. It sets a baseline, makes production tasks just a little bit easier, and opens up opportunity for automating away other tedium in the future. Not everything needs to be a giant leap forward. Lots of little steps can get you to the same place, but the fall isn’t as far if one doesn’t work out.

Organization is key to any successful conversion optimization program. Trapped in a web of asset files? Have a bunch of folder names that don’t make sense? Can’t get a handle on your split testing process? Corvus CRO can help. Schedule a free 30 minute consultation with us today.

CROWeekly: SessionCam Partnership

What Happened at Corvus CRO This Week

Corvus CRO is a SessionCam solutions partner
Corvus CRO is now a SessionCam solutions partner

Partnering with SessionCam to Easily Identify Friction and Opportunity

Corvus CRO is now a SessionCam solutions partner! SessionCam is a SaaS platform that provides session replay, heatmaps, and conversion funnels.

Behavioral analysis tools are necessary to the success of a conversion optimization program. They provide critically important data visualization around how users actually interact with a website. SessionCam was founded in 2009 and is one of the most mature solutions available on the market today.

The true advantage of SessionCam lies in their built-in process automation coupled with a machine learning engine. Their system identifies and quantifies valuable insight, exposing high value opportunities with minimal effort. Their system can automatically map customer journeys and detect website errors, providing significant time and effort savings; no more manual funnel construction. SessionCam’s opportunities engine automatically identifies anomalies in website traffic and user behavior that need attention, printing them by the associated lost revenue.

SessionCam and Corvus CRO can level up your conversion optimization efforts. Already using SessionCam but need help managing it? Want to get a split testing program started? Unsure which behavioral analysis tool best fits your needs? Corvus CRO can help. Schedule a free 30 minute consultation with us today.