Website usability testing, application usability testing, laboratory usability testing – analyzing user behavior and reactions are now a standard.
The necessity, indispensability, and usefulness of researching representative groups of users are indisputable.
Research helps to improve websites and mobile applications and makes it possible to solve problems on websites that users encounter during their use.
They can improve functionality, streamline processes, and support achieving goals on a website and performing typical tasks.
They are also extremely helpful in improving the experiences, user experience, and needs of the target group.
Of course, their diagnostic, analytical, and cognitive value depends on many factors. One of them is the appropriate choice of research method.
User Experience research has developed and adapted from social science methodologies (such as psychology), several methods, and techniques that have proven helpful in discovering usability problems on a website.
One of the most fundamental divisions of these methods and techniques is the method of their execution, which can be In-Person (Face-To-Face) or Remote.
The first method is traditional and has quite a long history.
The second is becoming increasingly popular because of the pandemic and the dynamic development of tools (platforms that support conducting usability research).
Remote Usability Testing or In-Person Usability Testing? In any location or in a lab?
What are the advantages and disadvantages of In-Person and Remote testing? What to pay attention to when choosing a research method?
Here are our questions. Today we will take a closer look at the nuances of the UX researcher's work.
Do you want to perform UX Research?
What is Usability Testing, and what are UX Tests?
While answering this question, I should mention a fundamental observation. It's about the approach to research, its results, and the reasons for doing it.
It is very common to treat testing (e.g., website usability testing) as a form of confirming the flawlessness of an application, which makes such an approach look like a mere need for quality control.
Of course, research also has such a function, but reducing its significance, role, functions, goals, and results only to such narrowly understood needs (narrowly exploiting its potential) is always to the detriment of the digital product and the User Experience.
The purpose of testing is to diagnose not only errors (such as the incorrect operation of a function) but also problems that are not errors in the strict sense.
The purpose of usability testing is to improve a product, not just to ensure it's flawless – it's important to remember that.
Especially since usability testing should be implemented at the earliest possible stage of digital product development so that improvements can be made, resulting in a lower financial cost for the entire project.
All right, but what exactly is usability testing? What is it all about?
Let's start with the definition of usability, which, according to ISO 924, is understood as the effectiveness, efficiency, and satisfaction with which users achieve goals in specific environments (e.g., mobile applications, web services, web applications).
Usability testing is a process based on scientific, academic methodologies and techniques that studies the reactions of potential users.
Usually, they are asked to complete a list of tasks and interact with the application interface.
The time required to complete tasks and achieve set goals, the number of mistakes made, reactions (e.g., behavioral, emotional, cognitive), and many other indicators help determine how much a digital product is:
- useful
- intuitive
- easy to use
- trouble-free
- compatible with the perceptions
- expectations
- experiences
- habits
- conventions
- preferences.
Even more importantly, usability testing allows us to understand the product's use cases, understand its future users more profoundly and multi-dimensionally, and offer them a product in line with their multidimensional preferences.
Usability testing plays a pivotal role in the development of user-centered digital products.
A positive definition of usability testing should also be supplemented with a negative definition, that is, a clear indication of what usability testing is not.
Usability testing is not:
- aggregating opinions about a product – this is the domain of marketing research
- just an A/B comparative test because it doesn't answer the question of why variant A is better than variant B or vice versa
- a variation of surveys, as surveys do not allow observation of activities
- a variation of focus research, which is aimed at learning about opinions but does not allow us to learn about behaviors
- a variation of Heatmaps, which involve creating a visual representation of the most popular and least popular sections, areas, and elements on a website.
Let's summarize; above all, usability testing allows us to:
- learn about behaviors (answer the question: What do users do?)
- understand behaviors (answer the question: Why do users do it?)
- identify the problems (answer the question: What is causing trouble for users?)
- determine the level of satisfaction resulting from the use of the product (answer the question: What is the source of satisfaction/dissatisfaction of users?).
Generally speaking, usability testing can be divided into the following types of research:
- Moderated vs. Unmoderated
- Remote vs. In-Person
- Explorative vs. Comparative
Naturally, this basic division creates a matrix of different methods in which different types are combined.
Hence, for example, we can use moderated remote testing, unmoderated remote testing, unmoderated in-person testing, or moderated in-person testing.
In-Person Usability Testing
In-Person Usability Tests are conducted in a Face-To-Face format in a laboratory where the user is asked to perform various types of tasks. Alone or in the presence of a researcher.
Like any research method, In-Person Usability Testing has its limitations as well as undeniable advantages.
The key advantages of In-Person Usability Testing include:
- ability to collect data from more sources (e.g., body language, facial reactions)
- ability to simultaneously observe physical, emotional, and verbal reactions (holistic approach to behavior, user reactions)
- ability to interact and deepen the testing with probing questions
- greater ability to control the dynamics and rhythm of the test and adjust its course to the individual needs and reactions of each test participant
- relatively easier availability of participants
- ability to control the test environment with the help of facilitators (no dependence on the quality of the Internet connection)
- ability to conduct them at any stage of the design process – they do not require the creation of a clickable user interface
- in the moderated variant, the researcher can question the answers and check the reactions
- ability to gain more direct insight into users' behaviors, needs, desires, and concerns
- ability to mobilize researchers better, who must be prepared for very different circumstances
- ability to test a project on a variety of devices.
The most important disadvantages of In-Person Usability Testing include:
- higher costs of conducting the test (because of the need to hire a testing laboratory, organize transportation, etc.)
- greater logistical problems associated with recruitment, execution of the study
- longer duration of the entire research project
- moderation and interpersonal skills of a researcher
- influence of a researcher on a respondent and thus on their behavior (e.g., inexperienced researchers often unintentionally impose their beliefs and expectations on the user)
- in the case of a large number of respondents, budgetary, organizational, logistical, and recruitment problems tend to intensify.
Remote Usability Testing
Remote Usability Testing is gaining popularity and is slowly becoming one of the leading methods, mainly due to increasingly improved tools (dedicated remote testing platforms) and much better price and organizational attractiveness.
In the article "Remote Usability Tests: Moderated and Unmoderated," published on the NN Group blog, Remote Usability Testing is defined as tests that are similar to traditional tests but differ in that the participant and moderator are in two different physical locations.
Study participants in Remote Usability Testing interact with the project (usually, the study lasts from 15 to a maximum of 30 minutes) in their own homes, offices, or another location, and an expert observes their actions remotely.
Remote Usability Testing is performed using screen-sharing software or specialized testing platforms.
Key advantages of Remote Usability Testing include:
- realism, the authenticity of the test – the natural context and the environment of the respondent have less influence on the course of the test and the respondent than tests in laboratory conditions
- greater relevance of the information obtained (especially if it is Destination-Based Testing)
- more flexibility in the selection of respondents
- lower costs, shorter execution time, and greater ease of obtaining respondents (some platforms offer access to their own databases of study participants, who can be selected according to preferred criteria)
- ability to examine more people
- ability to quickly and accurately test selected elements of an application
- ability to learn about typical usability problems
- ability to quickly test a prototype (with the proverbial 1 click)
- shorter time between test creation and results
- smaller organizational and logistical requirements (only personal equipment and Internet connection are necessary for the test)
- ability to study users from different countries and cultures.
The most important disadvantages of Remote Testing include:
- it's one-way nature, the possibility of communication problems
- greater risk of test failure due to factors beyond the control of the researcher and the respondent (such as a broken Internet connection)
- less control over the test environment
- the need to conduct pilot studies (especially in the case of Unmoderated Remote Usability Testing)
- the need for the very meticulous and thoughtful planning of the study in every detail
- limited duration of the test, forcing the selection of more straightforward tasks with a narrower scope
- greater risk of testing financially motivated users, "professional testers" who perform tasks routinely
- thinking aloud can be a rather difficult skill for respondents to maintain
- the difficulty in finding the optimal way of communication, the balance between listening to the respondents and questioning them
- lack of real-time respondent support in unmoderated studies
- hard-to-predict technical and operational issues with the research platform.
Iterative Usability Testing
No matter which testing method is chosen, we must also keep in mind an important point made in the article "Usability Testing," published on the Interaction Design Foundation (IDF) blog.
Usability Testing produces the best results if it is carried out methodically and according to specific guidelines, which include:
- Study plan – a precisely defined scope of the study; the researcher should know what they want to test, why they want to test it, and with what methods they want to conduct the test.
- User tasks – should be prioritized, realistic (e.g., achievable in the available standard time of one or two-quarters of an hour), clear and understandable, and structured within a scenario of the most natural use of the digital product under study.
- Recruitment of study participants – should aim to precisely define the target group and select its representatives with the help of screening questionnaires.
- Test Facilitation – we should create a suitable and optimal environment that does not give rise to technical, functional, or communication issues.
It is also crucial to maintain fluidity in the conducted tests by following the guidelines concerning the following:
- User evaluations – quantitative (e.g., time taken to complete a task, success rates) and qualitative (e.g., facial reactions, changes in body language).
- Reporting – should result from a meticulous, critical, reliable, and exhaustive examination of all data obtained during the test.
Remote Usability Testing vs. In-Person Usability Testing. Summary
- Usability is defined as the effectiveness, efficiency, and satisfaction with which users achieve goals in specific environments.
- User behavior testing, UX testing with users, and remote usability testing are standard activities during the development of digital products, the usability of which is often compared to web analytics.
- Usability testing (of a website, for example) helps improve functionality, streamline processes, and support users in achieving goals and completing tasks.
- The diagnostic, analytical, and cognitive value of the research depends, among other things, on the correct choice of the appropriate research method.
- Usability testing (of a web application, for example) is often treated as a form, a process of confirming the flawlessness of an application or its quality.
- Reducing testing to such narrowly defined needs is always to the detriment of the digital product and user experience.
- Usability testing is a process based on scientific methodologies and techniques. In most cases, the software and its effectiveness after examination lead to constructive conclusions and enable the implementation of beneficial changes.
- Usability testing, UX research, and remote UX research with users help to understand product use cases, offer users a product in line with their multidimensional preferences and rationalize the decision-making process.
- Of particular note is task-oriented testing, that is, the ability to conduct remote testing, analysis, and observation in a natural environment, which can be used to improve user experience and make websites more competitive.
- Usability testing makes it possible to learn about behavior, understand behavior, identify problems, and determine the level of satisfaction that comes from using a product.
- In-Person Usability Tests are conducted in a Face-To-Face format in a laboratory where the user is asked to perform various types of tasks.
- In Remote Usability Testing, respondents interact with the project using dedicated research tools, and an expert observes their actions remotely.
- UX testing with users produces the best results when done methodically and according to guidelines.
