Measure what Matters
The always-entertaining Jared Spool spoke recently in Atlanta to ask "Is Design Metrically Opposed?" I enjoyed his perspectives on designing digital experiences and how to measure the right things.
How to Drive Conversions Up
The Conversion Rate formula is pretty simple, with two factors; the number of people who convert (sign up, buy, etc) divided by the number of site (or page) visitors.
“Stop marketing. Because having too many visitors will drive your conversion rate down.” - Jared Spool
The inherent problem with ratios is that you can't uniformly control both of the factors being measured. One way, he said, to increase conversion rate (CR) is to stop driving people to your site. This sounds silly, of course, but manipulating the numbers affects the ratio which make it a flawed metric for success. Conversion rates overlook the customer behaviors that convert site traffic into paying customers. Which leads us to....
Be Careful with KPIs
More about UX KPI's on the User Interface Engineering site
In many cases, key performance indicators actually make the customer experience worse. This calls out the importance of mapping out the end-to-end customer journey to better understand how moving one lever impacts the experience in other areas. KPIs can be valuable -- but they must be tailored to the business and their customer's journey.
"Generic KPIs produce generic results. If we really want something that touches the core of what makes our business special, it should be a metric that only applies to what we're doing." - Jared Spool
Optimize Testing
Spool gave several examples of how industry leaders, Amazon and Netflix run their extensive testing with MVT. They systematically refine winning test variations to not only to learn what works but why? The key to a scalable, repeatable testing model is to understand both why the winning designs worked better and why the losing designed performed worse.
Satisfaction Surveys
The word satisfy is a neutral word. It's like a restaurant survey asking patrons "was your meal edible or unedible?" The questions must measure something that is meaningful to the audience. Asking variations of satisfaction like "completely satisfied" or "extremely satisfied" is as relevant as "was your meal Extremely Edible?"Rather than using satisfied/unsatisfied as a success metric, the better way to measure experiences is from extreme frustration to extreme delight. The good news is that designing excitement generators -- those things that delight people -- often doesn't take much investment to create.
Measuring Spool's Entertainment Value
Throughout the evening, Spool riffed on everything from growth hacking -- which he called "the latest, politically-correct term for sleezy, dark patterns on the Internet" -- to why measuring time on page doesn't work ("because peeing gets in the way") -- to his never-ending war with United Airlines.
Making design decisions based on user feedback metrics leads to success. By the end of the night, I came away with a better appreciation of the measurement problem in designing digital experiences. So I definitely recommend seeing him speak if you can. Check his upcoming schedule to see if he's coming your way or contact him to have him speak at an event.