Wednesday, July 23

There Is No Such Thing As An Easy A/B Lunch

"It is perhaps an all-too-human frailty to suppose that a favorable wind will blow forever." — Richard Bode

In the context of his book, First You Have To Row A Little Boat, Bode was writing about how almost impossible it is to imagine what it might be like to be caught in a dead calm while there is a breeze blowing hard against your sail or in your face or on your back. It's almost impossible to imagine it because our brains are mostly predisposed to see the most fleeting moments as infinitely constant.

When things are good, we think the honeymoon will never end. When things are bad, we readily embrace the pain as permanent. Never mind that most of us have lived long enough to know that the evidence doesn't bear either infinity out. We're generally inclined to indulge ourselves in deception.

Social media is not a science. It only feels like one.

Sure, some applications of social media seem to fall under the banner of science. Marketers are indeed in the business of observation and experimentation. They do attempt to study the structure of online communities and the behavior of people on a one to one, one to some, and one to many scale.

Some applications even attempt to apply scientific method to the mix, with A/B testing among the most prominent manifestations. There is only one problem with it. While A/B testing sometimes leads to a product development or marketing breakthrough, the operative word is sometimes.

The wind doesn't always blow in a favorable direction and sometimes it doesn't blow at all. Never mind that more and more data scientists are attempting to decipher public manipulation, but they frequently fail to appreciate that data has the propensity to manipulate its handlers too.

The biggest problem today, it seems, is that many data scientists have studied statistics but relatively few are practiced at applying scientific method in the physical or natural world (or psychological and sociological worlds for that matter). If they were, they might better appreciate the incongruity of choice — six studies of which were recently shared in an Econsultancy article by Ben Davis.

While some studies are stronger than others, a fair encapsulation of the research concludes that the choices offered, number of choices offered, order of the choices offered, and order of emotional triggers all influence A/B testing. Or, in other words, if A/B both suck, you prove nothing at all.

If you ask people whether they like big keys or little keys on a cellular phone, no one innovates touch screen technology. If you ask people which cola they like better during an A/B experiment, someone will eventually rediscover the recipe for New Coke. If you always listen to prescreen tests, every movie will have a happy ending.

But those examples are only the most straightforward research failures. Some hiccups are caused by the most subtle changes. The order information is presented (shoes before or/after a new dress). The timing of an interruption (when most people are online or when they are more receptive to share). The influence of the last destination they visited (did they leave feeling elated or aggravated).

There is no such thing as an easy lunch in marketing.

There are plenty of people who will tell you otherwise, but it's simply not true. Marketing is not a science, even if marketers love to sell science. It can be an asset but only if you think and think deep.

A few years ago, I had the privilege of working on franchise collateral for Capriotti's Sandwich Shop. I can't really speak to what they are doing now in terms of marketing, but I still love their sandwiches.

The challenge they had and probably still do, had a lot to do with psychology. Specifically, one of the questions that needed to be asked was how could they become part of the lunchtime decision-making process? The answer isn't as easy as you think.

When most people make decisions about what to have for lunch at the office the first A/B choice they create is fast food or sit down. The primary influencer at this stage is time, but it quickly turns toward taste. If fast food wins the consensus, then most people will run down the big brand list (McDonald's, Burger King, Wendy's, etc.) and make a decision based on preferences, experiences, and proximity.

Interestingly enough, KFC only gets a shot if someone says they don't want a burger. And other alternatives, like Subway, are added to the mix if someone insists on no fast food (a position thanks mostly to their Eat Fresh campaign). So where does Capriotti's fit?

A/B testing convinced some people that it fits everywhere because they consistently win on taste, but it really wasn't true. Sure, it won with loyalists, catering, or as a wild card but not where it needed to. To capture the average lunchtime customer, it comes down to the first round choice. Fast food or sit down? This sandwich shop is neither.

My solution was a bit different from the marketing firm that had contracted me onto their team. While they wanted to push award-winning sandwiches, I wanted to reframe the front end choice that there is lunch or Capriotti's, thereby pre-empting the fast food or sit down decision-making process.

But we didn't then and no one has since. So despite being voted the greatest sandwich in America, it's still niche and not mainstream no matter how many A/B tests they run. Why? As I said. There is no such thing as an easy lunch. Just because the winds of research keep blowing your organization in different directions doesn't mean it will always be there or push you to the destination you want. Someone has to aim for it.
blog comments powered by Disqus
 

Blog Archive

Google+ Followers

by Rich Becker Copyright © 2010 Designed by Bie Blogger Template