Ten thousand ponies! or How to build a dashboard

So usually you identify One Key Metric to measure success. Or maybe more. And those metrics are probably the right ones for your product/service – pageviews, signups, app downloads, dollars, ponies.

The thing is, when everything’s right and you’re growing like gangbusters, nothing really matters. You just stand in front of that giant screen that measures the One Metric in real-time, and down shots with every milestone (ten thousand ponies!).

But then you’re not doing so well. And so then you switch to detective mode and dive into your logs to make sense of what’s happening – or what’s not.

It’s going to be hard to do that every time something goes wrong. And yes, things will go wrong more than once. Because once your baby has launched, you’re going to obsess over every little negative point and trend, however fleeting.

You don’t need an event log. You need a dashboard. Something in addition to your single-point metric. That’ll help you find why that One Metric isn’t doing so well [1].

Three general principles that will likely help:

1. Model and track your ideal customer flow. Create a state diagram of how you want your customers to navigate through your app or your website. Now track state transitions, not state populations.

For our paid subscription service with a free trial, our One Metric was payment transactions. When that slowed, we just didn’t know why. We just knew a lot of people were signing up and now weren’t paying enough. Why didn’t they like the service?

Then we created the following state diagram [2].

state-transitions

 

Which then told us that of the people who stated the free trial, too few people were paying us (x%) and too many were letting their trials expire (y%). It wasn’t that they disliked the service as they just didn’t *do* anything [3].

2. Track over time. Track both your One Metric and customer flow every day (or minute?). When you tinker with something, you’ll see how it affects your customer behaviour. Today, and a week from now. Note: you can get carried away with tinkering this way.

3. Export, don’t just display. Week-on-week on-screen charts are great, but have both the OneMetric and customer flow snapshots exportable as CSV. Being able to find correlations between data sets will help your cause/effect analyses no end. Chimps can do pattern recognition too. But they can’t rock a vlookup.

So then. To a million ponies.

 

[1] Look, chances are you’ll muck around with your log file or raw data exports anyway, but your dashboard’s going to tell you exactly where to look.

[2] Very briefly: the states ending in ‘f’ were free trial states, the others were paid states; a customer began in SSF (service started, free) and either paid (SSP or service started, paid), or actively stopped during the trial (USF or unsubscribed, free), or had their free trial expire (PPF or payment pending, free), and so on.

[3] We first thought they just didn’t care (terrible!). When we surveyed a sample, we found that communication was a problem – we were doing a terrible job at telling them just how they could pay.

Idea, Product, Business

Your division starts off on a new project with a bang. And months later ends with a whimper. You shut the thing down, your folks are terribly frustrated but you don’t really know what went wrong. From my experience, here’s probably what happened:

First, there’s the idea.

Then, there’s the productization of that idea.

Finally, there’s the business built around that product.

But you cannot make money off an idea. You can only make money from a business. And you cannot take an idea to market. You can only take a product to market.

Too often, Management doesn’t see the difference between the three and misses asking the right questions at the stage those questions need to be asked. The result is usually a shoddy product, reactive execution, management by crisis, a demoralized team and money down the drain.

Been there before?

If you rush to build a business around nothing more than an idea, you’ll probably find that

Your audience doesn’t understand your product because you don’t understand the audience. Or you spent so much time early on thinking about monetization you didn’t think enough about adoption.  Or by the time you launched, your product was a year behind its competition. Or you built your mobile app for a platform your early adopters don’t use. Or you launched without customer support or the ability to collect feedback.  Or you give up on the product too quickly because it didn’t really really excite you to begin with.

Seem familiar?

Here’s what I think might help avoid these holes:

When evaluating an idea you’d ask

  • Has this problem already been solved?
  • Does our organization – firm, company, startup – understand this space?
  • Is this way of doing business in our DNA?
  • Does it personally excite you, o ringmaster?
  • When the product will be ready, will it still be relevant?

And then, once the idea’s passed evaluation and you’ve begun to build,

  • Who are your early adopters?
  • How do we put together the talent needed to get this out the door?
  • What platform do we develop for? (mobile platform, web platform)
  • What features do we bake in/leave out?
  • What is our go-to-market plan?
  • What kind of customer support should we have (phone/email/in-app form)
  • How will we collect feedback?
  • What parameters will we monitor?

Once you’ve launched, gotten traction, feedback and momentum,

  • How do we make money?
  • What does our product roadmap look like? How often do we release?

You’re likely better off asking these questions when they really matter. Not too early, not too late. #Ilearntthehardway.

Among the most challenging advice I’ve received

… is this (at the very start of building a new product):

Assume your product’s already been built. Now think about how you’re going to get adoption and usage among your audience.

Too often you get so caught up in the excitement of building something cool, you don’t tackle the hard problem of seeding it and getting usage among early adopters until you’re very close to – or at – launch. Then you’ve got a product that’s ready and no one but yourselves using it. Your go-to-market guy’s under tremendous pressure and your developer’s twiddling his/her thumbs. Your team can run out of momentum and enthusiasm really quickly and it’s very hard to get your mojo back.

This has happened to us before. And I’m still working on that advice.

Serendipity

I like dense walking cities like New York because they allow for serendipity. In Los Angeles serendipity is hard; you’re essentially teleporting from one place to another (slowly) in your car. There’s no deviation from the path, no unplanned, found awesomeness.

Nathan Bowers, UX Hero