An Overview: Testing methods for Online-Shops

As you might have picked up already, in July we address E-Commerce testing a lot on our Testbirds blog. In the last weeks we already tackled down some different testing methods: we emphasized the importance of cross-device testing due to the ongoing mobile-shift, showed how an Exploratory Bug-Test is executed and explained how much optimisation potential lies in testing in general (if you missed it: with crowdtesting we once helped a client increase his conversion rate by 30%!). In this week’s e-com series blog post, we want to present you with a rather simple thing: An Overview. What are the most common testing methods for web shops. Whether if it’s when creating an online-shop from conception phase to prototyping and to post-release stage, or also when the web shop is already running or a relaunch is planned.

Card Sorting during Conception phase

Yes, an online-shop needs to have an appealing design with an outstanding portfolio of products and services. A solid structure that is intuitive and simple is equally important though, because websites that make it easy for users to navigate, are a necessity for success. To put it into marketing-terms: Customer-centricity is key, especially during conception phase. With a Testing Method called Card Sorting, the customers’ needs are included into the development process from the very beginning. The target group helps define an intuitive navigation and menu structure.

With Card Sorting, a representative tester group of a client’s target customers sort and categorise virtual cards to define a clear and logical navigation structure. Usually there are two different options: the 50-100 testers either receive topics and must cluster them into new topics, e.g. they create their own menu structure; or they receive pre-defined topics and must group different items and words under those topics. With both options the outcome is a comprehensive feedback towards clarity and structure, which gets analysed by Testbirds project managers who also formulate concrete recommendations for improvement.

Card Sorting in short:

• Card Sorting helps with the challenge of having a simple navigation despite a large amount of content
• 50-100 testers representing the client’s target group define a logical navigation structure
• Feedback towards clarity and structure
• Fast Execution: testing phase 1-3 days, results and analysis 3-5 days

Prototype Evaluation via Remote Interviews

A next step towards a well-functioning online-shop is prototyping. Whether if it’s a click-dummy that contains the first versions of designs and navigation structures or even just first pictures and PDF files of the final website, a Prototype Evaluation Test is outstandingly important, because uncertainty is a huge matter to contend with. Will the product be accepted by the target group? Are different features necessary and understandable? Operational blindness makes it nearly impossible for an unbiased view on the software, risking missing important details and developing an unsuccessful product. An effective way to gain some insights into the customers minds is asking the target group for constructive feedback before the development phase takes place. One common testing method to do so is by executing Remote Interviews.

In Remote Interviews, the target group specific selection of testers share their screen and their thoughts in a video conference about a client’s digital product. Normally in 30-minute sessions with one tester at a time. The client is able to remotely watch, communicate and guide the tester during the whole testing process and is so able to control the procedure by direct interaction. The biggest advantages of this test method is the flexibility and direct feedback that make readjustments possible even during the testing phase. The interviews get recorded and can be re-evaluated at a later point.

Remote Interviews in short:

• Prototype Evaluation through Remote Interviews shows what the target group thinks about the idea before the development phase starts
• Normally 5-10 testers share their thoughts about the prototype in direct remote conversations with the client
• Very flexible testing manuscript and fast feedback
• Duration of the testing phase is dependent on the number of testers, results and analysis can be delivered in 5-7 days

QA and UX during Development and Post-release

With every online-shop it is most crucial to make sure that functionality and user-friendliness are flawless and on par. Whether it is during the development phase or after release, or even if an online-shop is already running for many years, the most common testing methods are functionality tests and usability tests. We have clients, who need our help right after a relaunch of their online-shop, because the release was too error-prone due to a lack of testing beforehand. Another client had his web shop for several years, which looked fine at first glance, but still the crowd found 249 bugs on the site.

Usability and UX Study

If a website does not meet the expectations of its users, or if users are unable to easily fulfill the intended actions, alternatives offered by competitors are often just a click away. Seemingly small features like headlines or an image next to the “buy” button can have serious consequences. Therefore, aspects like text to image ratio can be hugely influential. Although developers and designers are experts when it comes to their company’s software and digital products, they will never have an unbiased view due to organisational blindness. In a Usability & UX Study feedback from the crowd regarding usability and user experience of a webshop gets collected by conducting up to 10 use cases with up to 5 qualitative and 8 quantitative questions. The tester group can be chosen from over 65 demographic criteria and of course is adapted to exactly represent a client’s target customer.

Usability and UX Study in short:

• Usability and UX Studies help figuring out how the target group experiences a webshop
• The recommended number of testers is 15-20
• Benefits are that user expectations and needs are recognized and user satisfaction can be improved
• Exemplary duration: Testing phase on the platform: 2-7 days, Results and analysis: 3-7 days

Exploratory Bugtest

It is no news how fatal it can be if the basic functions of a webshop don’t work, especially in the ever-growing complexity of the device-landscape. The main purpose of the shop is to sell products and if this purpose cannot be fulfilled, well there is no reason for it to exist. Online-shops hence need to be extensively checked for functional errors and bugs on all devices. In a blog post 2 weeks ago, we already explained how Exploratory Bugtests are executed in more detailed, for this blog post, we hence stick to a short summary:

Exploratory Bugtest in short:

• Exploratory Bugtests are executed to ensure the functionality of the software on all the different devices in the market
• Normally 10-20 testers per device type are in one tester group
• The outcome is overall feedback on functionality and bug categorization & prioritization
• Exemplary duration: Testing phase on the platform: 1-3 days, Results and analysis: 1-2 days

Testbirds Top Tipp

If time is an issue, for example right before the release of a web shop, but both QA and UX still need to be tested, one of the most popular Testing Service at Testbirds combines the two methods QA and UX. The Bugability Test functions as a sort of all-in-one service to simultaneously test both the functionality and get usability feedback with the help of the crowd.

Bugability Test

In a Bugability Test, a group of testers, that matches the target customer of the client, conducts up to 10 use cases, where they are asked to complete up to 3 qualitative and 5 quantitative questions and give feedback regarding the usability of the web shop. The testers also report defects in comprehensive bug reports with details on the respective test device, title, severity, category, description, actual/ expected result and deliver screenshots and/or videos. This combination of functionality and usability testing gives a 360°-Feedback to eliminate critical malfunctions on the website right before launch. The test set-up is done by Testbirds project managers who consolidate the results in detailed test reports with recommended courses of action.

Bugability Test in short:

• Bugability Tests give feedback to the general question: Do I have a high-quality digital product?
• The recommended number of testers is 20-35 testers
• Biggest benefit is the combination of functionality and usability feedback
• Exemplary duration: Testing phase on the platform: 2-4 days, Results and analysis: 2-4 day

Online-Shops need to be tested continuously

Retail sites are highly dynamic with new offers, new products, sales, etc., which makes webshop testing a complex and ongoing necessity. It therefore is most efficient to include the testing methods stated above permanently into the many development sprints. With many clients, at Testbirds we therefore entered long-term partnership for continuous testing, an approach we’ve learned is extremely successful.