Test data and & failure to launch

Blog post   •   Apr 04, 2013 15:56 BST

Author: Vanessa Howard

“You can release a product that sounds good, but today, with social media, with people that are able to talk to each other around the world so very quickly, you really have to have something that delivers, that people can use, because that word of mouth is going to get out, so very quickly.”

So said John Miewald of Pathway Innovations and Technologies recently and it is a statement that should resonate with anyone working with consumer-facing technology.

With the stakes so high, it is surprising to learn from a presentation given by Richard Bender of Bender RBT Inc. that in the US alone, the estimated cost of defects to developers stands at over $21bn and that the cost of fixing them for users is running at over $38bn (Bender, 2012). That’s a chunk of change and doesn’t factor in reputational damage, with social media soon circulating damning reviews from frustrated users.

Whilst ‘Apple Maps Fail’ provided plenty of laughs by flagging up a supermarket as a hospital in Florida, failing to recognise that Stratford-upon-Avon exists and placing Auckland station in the sea, serious questions have been asked about the company’s competency and strategic focus.

Back to the statistics again – it is estimated that only 37% of business-critical software projects succeed in meeting their objectives, both in terms of use to the organisation and the user (Standish Group, 2011 Chaos Manifesto). That’s a lot of fail. So what are companies doing to address it?

First, there is now far more being spent on consultancies which focus wholly on user experience, as it is clear that launching something that doesn’t meet what consumers want or need now goes beyond issues of a poor return on investment, it risks losing out to the competition and may even jeopardise the future of a company – a factor in the collapse of a lot of High street brands as they failed to move quickly enough and offer  consumers online and mobile sales portals. 

Second, it reiterates once again that testing is an essential strategic concern for any consumer-facing product or service. Research can tell you what the product or service should do but only testing and development teams that are equipped with the rich, sophisticated data needed to perform rigorous, efficient test cycles can deliver it.