Evaluating the Combination of two User-oriented Usability Evaluation Methods: Thinking-aloud and Questionnaires

The Usability Evaluation (UE) of a product has been a popular research topic in recent years. Usability is assessed by different methods at every step of software development. Through redesign of the product and repeated assessments, the usability of a software is improved. UE also plays an important role in the development of websites. Nowadays, with web 2.0, web applications, and dynamic websites, it is even more important to test the usability of a website than it was in the past. Usability is assessed through its major aspects. Based on the International Organization for Standardization (ISO) 9241, these are effectiveness, efficiency and satisfaction. Every UE method evaluates different aspects of usability, but there is no method that evaluates all three at once. The aim of our work was the provision of a combination of two well-known UE methods that solves this problem. Furthermore, we evaluated our method through a case study with 28 participants. The combination of methods we proposed is a combination of two different user-oriented usability evaluation methods: Thinking-aloud and questionnaires. We have shown, that both methods give us different information about the usability of a website and therefore locate different usability problems. With our combination of methods, we are able to get data about the effectiveness, efficiency and satisfaction of the website as well as data about the context in which the website is used. These information give us the ability to determine the usability of a website and locate usability problems. We identified and prioritized 15 usability problems in our case study. Additionally, we were able to derive concrete redesign suggestions for the tested website based on the user statements and comments during and after the usability tests.