Modern science relies on research problems, questions, hypotheses, experiments, replication of research, and meta-analysis.
Research methods, UX research, and usability testing (qualitative research, quantitative methods) are based on research questions.
To conduct UX research is, first and foremost, to ask good research questions.
Relevant problems and research questions are the starting point and form the horizon of our knowledge of the world and phenomena.
Problems and research questions organize knowledge and establish research trends for years to come (including within User Experience research), bringing to life entire schools of research or more detailed approaches.
They are simply necessary.
This is no different in the field of UX Research — the academic studies conducted within the broader field of Human-Computer Interaction (HCI) — and the more business-oriented, pragmatic ones.
In this article, we will look at the "scientific background," how knowledge is created, how research is born, and what role research questions play in investigation, definition, discovery, and understanding.
We will highlight best practices regarding the formulation of research questions. We will also show the differences between research problems and research questions and between research questions and research hypotheses.
At the end, we will present examples of research questions. In particular, we will focus on the research questions posed in the context of User Experience.
If you are interested in qualitative research, quantitative research, UX audits, usability testing, benchmarking, A/B testing, or other types of UX research (e.g., eye-tracking). You definitely also need to know about the issue of research questions.
In the following article, you can find everything you wanted to know about research questions but were afraid to ask.
We invite you to read on!
Do you want to perform UX Research?
What are research questions?
A research question is a question around which research is focused. It organizes it, structures it, and directs it. It is influenced not only by the research problem but also by choice of research method.
Research questions are the foundation, and as such, they are the starting point and the point of arrival because every study, on the one hand, uncovers something, reveals something, and allows for a better understanding.
On the other hand, it points out the "unknown" area and subsequent research problems and inspires new research questions and hypotheses.
The cyclical nature of this process is the essence of science. It is the driving force behind the development of the world of knowledge.
John W. Creswell, in his book "Research Design. Qualitative, Quantitative and Mixed Methods Approaches," starts by distinguishing between research questions in qualitative, quantitative, and mixed methods because of their differences.
In qualitative methods:
- Research questions are posed, but no assumptions are made (the study is conducted without expectations), or hypotheses (no predictions are made that take into account variables and statistical tests)
- Questions are posed in the most general way possible
- Research questions are asked in two forms — in the form of a main question, and additional questions
- Researchers seek to know as many factors as possible that can influence and condition a given phenomenon.
The purpose of posing research questions (including in the design process) in qualitative research is to "discover," "explain," or "explore" also during usability testing (e.g., of mobile apps for the target audience).
Catherine Marshall and Gretchen B. Rossman, in their book "Designing Qualitative Research," distinguished and divided research questions in qualitative research into:
- Contextual questions that seek to describe the nature of what already exists
- Descriptive questions which try to describe a phenomenon
- Emancipatory questions that aim to generate knowledge that will help people engage in social action, especially for those who are underprivileged
- Evaluative questions that assess the effectiveness of existing methods or paradigms
- Explanatory questions that seek to explain a phenomenon or explore the causes and relationships of what exists
- Exploratory questions, which deal with lesser-known areas of a given topic (e.g., the different viewpoints represented by different target groups)
- Generative questions that aim to provide new ideas to develop theories and activities
- Ideological questions that are used in research aimed at developing specific ideologies of a stance.
In quantitative research:
- Very detailed research questions are posed (regardless of who we want to study)
- Research questions are much more specific
- Hypotheses are made on the basis of a small number of variables
- Having expectations is common
- Quantitative hypotheses are used to capture the relationship between variables.
An important characteristic of questions in quantitative research is their precision.
Quantitative questions cannot be answered in the affirmative ("Yes") or negative ("No"); hence we cannot use words such as "is," "are," "do," or "does."
As Imed Bouchrika notes in his article "How to Write a Research Question: Types, Steps, and Examples," quantitative research questions can be divided into three types: descriptive, comparative, and relationship research questions.
Descriptive Research Questions usually start with the interrogative word "what" and are intended to provide answers to a single variable.
Comparative Research Questions aim to discover differences between groups where a given variable occurs.
Relationship Research Questions investigate and define trends and interactions between two or more variables.
In mixed methods:
- Specific research questions that are appropriate for mixed research methodology should be used
- Both quantitative and qualitative research questions are used.
We should also add Uwe Flick's remarks that he made in his book "Designing Qualitative Research." According to him, the success of research depends on the quality of the research question, which should be clear and emphatic.
We will write in a moment about the ways of formulating research questions.
The quality of the research question determines what data we will collect and what aspects and problems will be analyzed and interpreted.
According to Uwe Flick, a good research question defines the scope of necessary data that cannot be omitted from the research process.
How to formulate research questions?
Formulating research questions and asking research questions (e.g., during user interviews) are two of the most critical issues addressed when discussing the researcher's workshop.
A properly posed question makes it clear what we want to study — and, indirectly — why we want to study it.
Research questions, including UX research questions, should be:
- Clear, the more detailed and specific the question, the easier it will be to understand and identify what problem it indicates and what its scope is
- Concise, a research question should be appropriately balanced in terms of length. It should be long enough not to leave out any important issues and brief enough not to lose its point and clarity
- Complex, so that the simplification expressed in the answers "Yes/No" is not possible
- Argumentative so that they open a discussion rather than offer definitive conclusions
- Focused on a given problem and thus, they should be devoid of any digressions and unnecessary elements
- Realistic and rational so that it is possible to conduct research within the given time, context, budget, logistics, and organizational frame.
Most often, research questions are aimed at the following objectives:
- Description
- Exploration
- Explanation
- Testing, including usability testing
- Evaluation
- Action.
In the case of description and exploration, most often, a research question takes one of the following forms:
- What are the characteristics of X? (for example, a user interface)
- How does X change over time? (for example, users and their expectations)
In the case of explanation and testing, a research question, most often, takes one of the following forms:
- What is the relationship between X and Y? (for example, visible during eye-tracking studies with users)
- What role does X play in Y? (for example, a mobile application in the user's life)
In the case of evaluation and action, a research question, most often, takes one of the following forms:
- What are the advantages of X? (for example, graphical user interface)
- How effective is X? (for example, usability testing of digital products for a user)
Imed Bouchrika, quoted above, provides a very practical method for checking whether a research question is relevant.
A research question must meet five fundamental criteria.
It must be:
- Feasible — both in terms of realism, which we already mentioned, and limitations of the researcher themselves, whether they can face the challenge
- Interesting and/or useful — especially in the case of UX research questions regarding usability. The practical dimension of the knowledge obtained during the research is vital — although science should not have boundaries, not everything is worth knowing. Anti-awards are the best example of this (for instance, Darwin Awards)
- Novel — it should bring something new to the subject, to the knowledge about the problem in question
- Ethical — it cannot violate legal norms, informal social norms as well as rules, ethical norms that apply to researchers
- Relevant — novelty and freshness are important, but it is equally essential that the research provides value not only to the researcher but also to all or at least the majority of stakeholders.
Questions and research problems
A research problem is a much broader concept than a research question.
It is defined as a problem, issue, phenomenon, or mechanism that must be investigated, described, explained, understood, categorized, and embedded in the structure of existing knowledge.
As a part of research problems, research questions are formulated to concretize the research problems.
In the field of social sciences, six types of research problems are distinguished:
- Basic
- Theoretical
- Practical
- General
- Partial
- Specific
A research problem simply defines what we still do not know and what we should find out because of various objectives and benefits (teleological and practical).
A research problem focuses on pointing out what we already know and, even more importantly, what we still do not know.
Research problems have two sources:
- Heterogeneous — it is a reality itself in which something feels incomprehensible and needs explanation.
- Autogenic — it is a community of scientists who, based on the subject literature, see gaps in the knowledge that need to be filled.
Moreover, research problems based on the criterion of their usefulness and the purpose of the study can be divided into:
- Theoretical research problems — these are basic problems that are solved to develop research and provide tools for solving practical problems.
- Practical research problems — their purpose is primarily to realize important social goals; they are intended to improve the functioning of the social world (e.g., more useful user interfaces).
Research questions and hypotheses
While the research questions and hypotheses may look no different to a layperson at first glance, they are not the same.
Andrzej Jankowski, in his study "Jak stawiać hipotezy i pytania badawcze? Teoria, wyjaśnienia, przykłady" (How to write hypotheses and research questions? Theory, explanations, examples) defines a research hypothesis as a statement in which a supposition is expressed about (operation, structure, influence, dependence) some phenomenon, which should be confirmed or disproved by statistical analysis.
Hypotheses organize quantitative research, define its scope, and provide a link between theory and research methods and research as such.
A research problem, research hypotheses, and research questions are the triad (UX research is no exception) that forms the core of any study.
We can organize these three key elements of every research (including User Experience research) according to the criterion of their generality, and thus:
- A Research problem is the most general
- Research questions are more specific
- Hypotheses are the most concrete.
We should also add that research hypotheses are made for a particular purpose — namely so that they can be confirmed or refuted with the help of statistical tools and methods.
Hypothesis verification — if it is methodologically correct and no error has been made — means that we have (relative) certainty that a supposition is "true" or "false."
The relationship does or does not occur.
That said, the certainty level depends on the generality of the hypothesis itself.
As a rule, the more specific, concrete, and narrow the hypothesis, the more reliable the result of its verification (positive or negative).
Research questions, User Experience research — examples
We can find the best examples of research questions by reading research reports. It is also worth practicing posing research questions because it is very easy to make mistakes, especially if we are a novice researcher.
For example, a research question — formulated for quantitative research — that is too general, vague, or unspecific might read as follows:
How do people react to the interface?
The question is too general because it operates with a category of people that cannot be studied. It is physically, time-wise, organizationally, financially, logistically, etc., impossible to study several billion people.
The question is unclear, as it is unknown to whom and to what it refers.
It is unclear what type of reaction is of interest to the researcher. For example, whether they mean emotional, behavioral, conscious, involuntary, or automatic reactions.
The question is also unclear because it does not specify the concept of an interface. What kind of device is this interface on?
On what application? What type of interface? Reaction to which interface element is going to be measured?
Finally, the question does not imply the importance of the problem. It is impossible to deduce why such a problem is worth the effort.
The above research question, in order to sound much more concrete, understandable, clear, and realistic, should read like this:
In what time can American men between the ages of 25 and 30 find an ADDE chair in Ikea's online store using an internal search engine?
In qualitative research, research questions are often formulated to explore, evaluate, probe, discover, and describe users' expectations, impressions, and problems.
For example:
How would you rate your shopping experience in Ikea's online store?
Express your opinion using a scale of 1-10, where a rating of 1 means Very Bad Experience and 10 means Fantastic Experience.
How strongly do you agree with the following statement: "The ordering process at the Ikea store was easy."
Express your attitude with one of the statements from the following scale:
- I strongly disagree
- I disagree
- I neither agree nor disagree
- I agree
- I strongly agree
Of course, this is only a small sample of research questions. Depending on the research method, the purpose of the study, the research problem, the methodology, and the respondents, they will vary considerably.
UX Research Questions. Summary
- Modern science relies on research problems, questions, hypotheses, experiments, replication of research, and meta-analysis.
- Relevant problems and research questions are the starting point and form the horizon of our knowledge of the world and phenomena.
- How to formulate research questions — the purpose of posing research questions in qualitative research is to "discover," "explain," or "explore."
- Quantitative questions cannot be answered in the affirmative ("Yes") or negative ("No"); hence we cannot use words such as "is," "are," "do," or "does."
- Research questions can be divided into descriptive, comparative, and relationship questions.
- Research questions should be formulated in a clear, concise, complex, argumentative, realistic, and rational manner.
- A research problem is a much broader concept than a research question and is defined as a problem, an issue, a phenomenon, or a mechanism that needs to be investigated, described, explained, understood, categorized, and embedded in the structure of existing knowledge.
- A research problem focuses on pointing out what we already know and, even more importantly, what we still do not know.
- A research problem, hypotheses, and questions are the triad that forms the core of any study.
- The research process and the design process are intertwined, and their common link is precisely the research questions.
