Homepage > Journal > UX research of mobile and desktop applications. Important differences
Journal

UX research of mobile and desktop applications. Important differences

How you like that:

Since the number of new mobile platforms and models is increasing every year, should we not adapt our research methods to these changes? Since usage patterns are also changing, shouldn't we consider them in the research process?

And can User Experience research of mobile and desktop applications (e.g., usability testing) be conducted using the same methods? Are there any differences between them?

So today, we will talk about UX Research. It's nothing surprising that this topic comes up in the context of User Experience. Without research, there is no good and positive User Experience.

Without research, there are no good mobile or web apps. However, for research to lead to good solutions, it must be relevant to the subject of the study. How to make them that way? Soon you will find out everything. I invite you to read the article.

We Audit. We Research. We Design.

What does User Experience research involve?

Let's start with a quick refresher. User Experience research (especially on usability) aims to match the mobile or web application to its users' expectations, requirements and needs in the best possible way. On the one hand, they help understand users. On the other hand, they make it possible to understand the impact of a project (e.g., a prototype) on a user.

User Experience research rationalizes the design process and provides data, conclusions, analysis, and insights.

It equally makes use of qualitative and quantitative methods. They provide a way to find answers to questions that are very important from a User Experience perspective:

  • What expectations, habits, and experiences do users have?
  • What needs do they want to fulfill with the app?
  • What problems do they face?
  • How do they use the application?
  • What elements hinder, discourage, or prevent conversion?
  • What fears, uncertainties, or doubts does the application raise (FUD – Fear, Uncertainty, Doubt)?
  • How is the interface perceived and rated?
  • What is the user's previous experience with similar applications?

Of course, these are not all the questions that research can answer, but only the most common ones.

UX research also makes it possible to identify, pinpoint, and reveal the following:

  • mistakes made by users
  • errors occurring in the application
  • measures of success (e.g., the correctness of tasks performed, finalization of tasks)
  • cognitive load (e.g., speed of tasks execution, ease of use of the interface)
  • cognitive friction.

The most common methods include:

  • observations
  • IDI (Individual In-Depth Interview)
  • card sorting
  • focus groups
  • expert analysis/heuristic analysis
  • surveys
  • A/B tests
  • eye-tracking
  • click tracking
  • cognitive walk-through.

Research – particularly of mobile applications – will become increasingly important. Primarily due to the rise of mobile devices. All signs indicate that mobile phones will soon be our primary devices, used not only for entertainment but also for work.

ux research of mobile applications
The Maze application allows us to analyze click maps as early as the prototype stage.

The current pandemic situation reinforces this trend.

Another argument favoring dedicated UX research is the increasing complexity of functions, contexts of use, and usage purposes. And this creates significant problems related to usability, experiences, and satisfaction.

We should remember the growing number of applications, too. Nowadays, competitiveness is more a matter of ensuring good UX than creating innovative solutions or new functionality.

Offering a better experience is a matter of research, optimization, and continuous development.

It won't be a big exaggeration to say that the development of mobile and web applications is "a work-in-progress," and UX research is an integral part of it.

The peculiarities of using mobile and desktop devices

From the user's perspective, mobile and desktop devices provide different experiences that we must consider in the design and research process. We use smartphones in different contexts, for different purposes, and in entirely different ways. So, our relationship with these devices differs significantly.

We use apps installed on our phones in a wide variety of contexts. Private, public, personal, and professional.

The changing nature of the contexts, and their diversity, is a significant challenge for designers and researchers (I will write more about the problem of simulating conditions in mobile application research).

When we use smartphones, we change our location. Therefore our capabilities change (e.g., related to Internet access), and the way we use the phone changes (e.g., with only the thumb of one hand or with both hands).

A large amount of stimuli makes our attention highly dispersed. An excess of information burdens the working memory.

Consequently, interactions with the application interface are shorter, simpler, more spontaneous, impulsive, more shallow in terms of engagement, and more stressful.

Mobile devices are also treated much more personally and store private, intimate information and resources. Applications raise concerns about data protection and access to data related to, for example, health, finances, or plans.

Our phones also differ significantly in terms of physical and technological parameters (e.g., diagonals, screen resolutions, operating systems). Diverse capabilities also produce complex, similar, but not identical experiences (e.g., related to tactility).

As noted by the authors of the article "Badanie użyteczności oraz dostępności interfejsu w aplikacjach mobilnych" (Usability and accessibility testing of mobile application interfaces), currently, communication between the mobile device and the user takes place using elements such as voice, bum-ping, vibration, jostling, gestures, including one-touch and multi-touch interactions.

Compared to the experience produced by desktop computers, the mobile experience means a smaller workspace, smaller interface elements, and operation usually with a thumb that is less precise than a cursor or a mouse.

It also involves experiencing interruptions, discontinuities, distractors, and chaotic processes.

That is why mobile UX Design and the research of mobile applications must be focused not only on functionality but also on the determinants of the interaction between a user and the application interface.

UX research of mobile and web applications

Exploratory and test research are the two most important types of UX research. The first one is mainly used to observe behavior and problems through conversation. The latter is primarily used to observe application usage patterns. The first enables the discovery of needs, while the latter is used to reveal problems.

Web and mobile application research is based on the same principles. The quantitative and qualitative methods used (e.g., usability testing) complement each other.

The significant difference stems from the vastly different contexts and uses of these devices. And this should be reflected in choosing adequate testing methods for individual applications. The most significant differences are:

  • context and external environment in which the device is used
  • diversity of mobile devices
  • variety of operating systems
  • wide selection of screen sizes and resolutions
  • diversity of sensors and peripherals in devices
  • ways in which a device interacts with a user.

The differences mentioned above concerning the operation, functionality, and, above all, the context of use cause many problems, especially when the research aims to improve the usability of mobile applications. Among the major issues are:

  • The dilemma of context – conducting field research under natural or laboratory conditions.
  • The problem of session recording – external (e.g., facial reactions) and internal – recording the actions performed on the phone.
  • The problem of the multiplicity of prototypes – resulting from the wide variety of devices and platforms and the requirements they place on applications.

The most important problem from the perspective of reliability and validity of research is the dilemma of context. The research situation and typical usage for desktop devices differ little from each other.

For mobile devices, this difference is fundamental. So, is it possible to conduct usability testing of mobile applications in a laboratory setting?

The problem of laboratory simulation of context in mobile application research

According to the authors of the article "Mobile Usability Research – The Important Differences from the Desktop," we can conduct mobile application research in a laboratory setting. And most importantly, there is no concern about the accuracy of the results obtained.

The comparative research mentioned in the article revealed little difference. For the time being, there is no convincing evidence that undermines the quality of the results obtained under controlled variation of the context of mobile app usage.

Observing users' behavior, reactions, and actions as they perform tasks in applications is, of course, extremely difficult under changing conditions (especially if the change is due to a change in location or movement). Primarily due to organizational, technical, and methodological factors.

UX research of applications
Figma allows users to clone the prototype for mobile devices, making it possible to test under near-real conditions.

The idea of reconstructing context in the laboratory is nothing new in the social and natural sciences. In her article "Mobile user research methods," Lorraine Patterson points out several ways to recreate natural conditions for user interaction with a mobile app interface. These include:

  • Audiovisual distractors – music or video played in the laboratory.
  • Social distractors – research assistants who distract by their behavior or questions, etc.
  • Motion and obstacle simulators – treadmills, walking routes with obstacles, simulating the variable speed of movement.
  • Stop/start techniques – enable to simulate the involuntary task interruptions.

Taking care of context variability is the main difference we should keep in mind. Research on desktop and mobile applications differs not so much in methods but in the conditions under which observations, measurements, and documentation are made.

Research of mobile applications conducted in a neutral context that doesn't provide significant changes in the situation, conditions, and strong stimuli will be incorrect.

There are several very important factors in favor of conducting tests under laboratory conditions:

  • reliability and validity of results
  • ease of organization and execution
  • more control over variables
  • lower costs
  • greater potential for the use of recording equipment.

The use of Nielsen heuristics in mobile app research

One of the more popular methods for testing mobile applications is heuristic analysis. This method is well-known and often used in web application research. The heuristic analysis makes it possible to diagnose usability problems more accurately especially concerning interfaces' usability.

It has become popular because of its speed of execution and low cost. The accuracy of the results is obtained with the involvement of a small amount of time and specialists.

For desktop devices, the most popular are, of course, Nielsen Heuristics. They help detect usability problems in 10 dimensions, including visibility, customization, control, consistency, and error prevention.

Nielsen's heuristics were initially developed with desktop devices in mind, so it's necessary to modify them a little for mobile application research.

The adaptation of popular heuristics was undertaken by two authors, Rebecca Baker and Xiaoning Sun, in the article "Empirical Development of Heuristics for Touch Interfaces," they presented some interesting modifications worth using in the heuristic analysis of mobile applications.

The authors propose to supplement the classic heuristics with three additional ones.

The first is Inconsistent Interaction Design, which occurs when users cannot use existing knowledge gained from using similar applications. Because of analogy and experience, they expect results, but these don't appear in the expected form. This situation causes confusion and frustration and is a source of task abandonment.

The second problem is Inaccurate Selection, figuratively referred to by the authors as the Fat Fingers problem. Although tactility is, in a manual sense, more straightforward than operating a mouse, it isn't as precise as the method of interaction known from desktop computers.

Therefore, it is crucial to study the size of the interface elements, their mutual distances, the errors that may occur due to their mismatch with the sensitivity of the device, and the size of a typical thumb.

The lack of help and navigational aids (e.g., search box, home page icon) is a major inconvenience in the use of mobile applications. Like the previous two, the third problem is also a source of confusion and frustration.

UX research of web and mobile applications. Summary

  1. UX research aims to tailor a mobile or web application as closely as possible to its users' expectations, requirements, and needs.
  2. It rationalizes the design process and provides data, conclusions, analysis, and insights.
  3. UX research makes equal use of qualitative and quantitative methods.
  4. The development of mobile and web applications is an example of a typical "work-in-progress" in which UX research is integral.
  5. Mobile and desktop devices provide different experiences that we must consider in the design and research process.
  6. The mobile experience means a smaller workspace, smaller interface elements, and operation with a thumb that is less precise than a mouse. It also involves experiencing interruptions, discontinuities, distractors, and chaotic processes.
  7. In summary, web and mobile application research is based on the same principles.
  8. The significant difference stems from the vastly different contexts and uses of these devices.
  9. The most important problem from the perspective of reliability and validity of research is the dilemma of context.
  10. There is no convincing evidence that undermines the quality of the results obtained under controlled variation of the context of mobile app usage.
  11. Research on desktop and mobile applications differs not so much in methods but in the conditions under which observations, measurements, and documentation are made.
  12. Mobile application research conducted in a neutral context that doesn't provide significant changes in the situation, conditions, and strong stimuli will be incorrect.
How you like that:
Journal / Redaktor
Author: Radek
UX Writer and researcher by education + experience. Collects The Story's knowledge and shares it on the Journal.
Reviewer: Dymitr Romanowski

Are you interested in working with us? Take a look at our Portfolio