Before embarking on any journey, having a good map means you’re most likely to reach your destination. It’s the same for a research project. By preparing a solid plan, you are more likely to achieve the desired outcomes and you can more accurately predict the time and cost to get there.
The following article provides some tips on choosing the right qualitative research approach, deciding how to segment and recruit participants, as well as mapping out the project timeline.
In this article, you should walk away with the following:
- A guide to answering these essential questions to frame your project:
- What type of research to focus on?
- What techniques and methodologies should be used?
- Who should conduct the research?
- Who will participate in the study?
- How long will testing take?
- What are the expected deliverables?
- What are the scheduling constraints?
- Examples of study characteristics, that range from in-depth to rapid research studies
- Example timelines for research study examples broken down by project phase
Setting the Stage
At the very onset of a research project, there are some important decisions to be made around the type of research you are doing and how you intend to do it.
Here are some of the questions you and your team must decide during the planning phase of the research.
1. Which is Right for Us, Generative or Evaluative?
Whether generative or evaluative, the type of research will depend on the purpose and goal of the study. For example, are you looking for new opportunities, seeking to validate design decisions, or make updates/changes to meet user needs?
If you are looking for new opportunities or insights into strategic business decisions, we recommend conducting generative studies. Generative Studies focus on what the user needs or hopes to accomplish, rather than how well a particular experience is currently performing. The gathered data tends to be more free-form, which allows for more open-ended brainstorming during analysis, where researchers can massage the data to reveal new ideas and insights.
On the other hand, Evaluative Studies are often recommended to measure and validate interfaces or features. Evaluative tends to give more targeted data that confirms (or doesn’t) existing goals and assumptions built into the interface. Rather than asking “What should the interface do?” It asks “How well does the interface do it?” Evaluative Studies are also well-suited for rapid testing cycles with a smaller set of participants.
2. What Techniques & Methodologies Should Be Used?
Once you know which type of research is right, you can think more about the techniques that could be applied.
Examples of techniques for qualitative research studies include:
- In-depth interviews
- Ethnographic studies
- Contextual inquiries
- Diary studies
- Concept testing
- Usability testing
Here are a few in-depth articles to give you a bit more background on popular UX research methods:
- A Comprehensive Guide to UX Research Methods, by Adobe
- When To Use Which User-Experience Research Method, by Nielsen Norman Group
Once you know which type and technique meet your requirements, you can begin to focus on some of the more tactical questions.
3. Who will Conduct the Research?
Deciding if your team is able to conduct the research internally or if it’s better suited to be wholly or partially outsourced to an outside research agency, will impact the proposed study’s timeline and depth.
Without a dedicated research team, it can be difficult for internal team members to break away from their day-to-day work to focus on a study. Leveraging an outsourced UX research firm can allow internal members to focus on the goals and implementation of findings while leaving the execution to an external team tasked with doing the grunt work of the actual research. If you are considering outsourcing any UX projects, check out this article on the 7 Things to Know When Outsourcing Your UX.
4. Who will Participate & How will You Recruit Them?
Finding the right subjects can make or break a project. It’s often one of the most overlooked aspects of a research project, which can easily overwhelm both your budget and your schedule.
When it comes to targeting and segmentation, you must have a clear understanding of who the different participants are and their current relationship to your product or the ecosystem in which your product exists. Some common segmentation groupings include:
- Customer vs. Prospect
- New Customer vs. Established Customer
- People new to your space vs. Experts in your space
- Active users vs. Inactive users
- Psychographic traits vs. Demographic traits
Different segments will bring different needs, knowledge, and assumptions to their sessions, so it’s essential to identify what you want to test and who are the best user types to target. Also, keep in mind, the more specific and layered your criteria is, the more challenging it will be to find people who meet those specific criteria.
When it comes to recruiting participants, there are several options available:
- Existing Customers – If you are looking for people who have an established relationship with your product, this is the first place to look. Customer rolls and other CRM repositories can provide a wealth of potential participants. However, be mindful of your existing communications policies before reaching out to them.
- Respondent Panels – Some agencies (gotomedia included) have their internal databases of potential participants. This type panel is a great way to jumpstart your recruitment
- Craigslist – Posting a study solicitation to public boards such as Craigslist can be a great way to access hard-to-reach or non-traditional users. Care should be taken, however, to craft your solicitation to avoid encouraging less than honest responses. Don’t be too obvious about what types of users you are hoping to recruit.
- Topical Forums – For niche audiences, posting a solicitation within topical forums can yield results. However, be acutely aware of any rules about posting—you don’t want to come off as spam.
- Social Media – If your company has a strong social media presence, you can leverage it to access participants. This network works great for reaching people who might be interested in your company / product but are not (yet) actual customers. Ideally, the viral effect will help spread the word.
- Website Intercepts – Companies such as Ethnio provide an easy-to-integrate popup on your site. This intercept can be an excellent tool for finding people who have a casual or developing interest in your product but are not yet customers.
- LinkedIn – This can be particularly useful for locating participants with specific job titles or professional backgrounds. The downside, however, is it tends to be limited to people within your network.
Most projects employ at least two of these methods. As a back of the napkin estimate, assume that it will take two hours to recruit each qualified participant. Sometimes it takes less, sometimes more, but on the whole, it is better to overestimate the required time to avoid challenging disruptions to your project schedule.
5. How Long will Testing Take?
As a rule, we like to assume three sessions per day—sometimes it’s faster, but given the vagaries of people’s schedules, three seem about right. Having this amount will allow you the time between sessions if someone is running late, a session needs to be extended, adjustments need to be made for technical or logistical mishaps, for setup/breakdown, bio breaks, lunch, and any post-session confabs.
6. What are the Expectations for Study Deliverables?
Deciding which deliverables are needed and how comprehensive they should be will help structure the interview/testing session and the depth and time required for the analysis, synthesis, and formatting. The simpler the deliverable, the less time necessary. The simplicity isn’t a matter of quality of results but is more about the required formats for the results to be effectively consumed by your company.
Here are some deliverables you might consider producing for your next study and a rough estimate of time frames it takes to deliver:
- Raw recordings = 1 day to consolidate
Recorded sessions you can share with team members to watch on their own time. These may be good for a confident enough team to do their analysis and are just looking for someone to conduct the sessions.
- Preliminary findings = 5 days
Workshops or quick meetings to discuss findings and next steps are great for faster rounds of research. These are ideal for agile teams ready to apply outcomes.
- Reports, slide decks = 10 days
Polished presentations of findings presented to a larger group that will be followed up by execution of prioritization of findings. These take preliminary findings and present them in a more visually compelling and formalized way and are better for teams that want to quickly understand the nature of the findings without getting into all the detail.
- Artifacts like journey maps, mental models, behavioral spectrums, etc. = 10 days
Visualizing findings for more straightforward clarification can add to the number of interviews needed, the time for analysis, and the artifacts’ creation. Artifacts like these are great to share with cross-functional teams, that require a firm grasp of the user and serve as reference material to validate against.
7. What Are the Scheduling Constraints?
Your answers to the previous questions may be impacted by the flexibility in your time frame. To gauge your available time frame, first consider the upcoming business deadlines or product releases that could be reliant on, or heavily influenced by, the research findings. How about any significant business growth or strategy decisions, like pivoting a product focus, that might impact how soon you need access to insights?
Whichever way your deadlines come about, we caution that it’s imperative to have at least a moderately aggressive timeline. This will help avoid the pitfalls that overly lengthy timelines bring about:
- Lack of urgency
- Loss of momentum and motivation
- Difficulties in scheduling participants
We learned that studies without firm deadlines tend to drag out unnecessarily. Work is wasted, and the opportunity for gaining new insights is lost.
Once you have the answers to these questions, you should have a rough estimate of the time and resources necessary to conduct the study. Deciding upon the pace will most likely be determined by the depth of testing as well as the team’s ability to efficiently apply learnings. The size of your team and business priorities will ultimately be a deciding factor in the number of back-to-back tests your internal teams can conduct.
To give you an idea of how the depth of studies can affect the pace we’ve outlined some of the top aspects of studies that apply more depth, as well as those with less and some sample timelines for each.
In-depth UX Research Studies
In-depth studies generally bring more formality and exploration to bear in the research, and may include one or more of the following characteristics:
- Generative research with more open dialogue that can surface opportunities and innovation, with possibly more ambiguous outcomes. In these cases, you might have more extended interview sessions and more participants.
- A more extended interaction between the user and the product or service. Consider how long your user’s typical interaction is with your product or service. This situation will have a bearing on how quickly you can get through your study. Also, consider how focused your user will need to be or the effort required of the users.
- Testing multiple groups of users or across various regions will impact the time required to recruit all the participants and the time necessary to analyze findings. With most studies, a minimum of 5-8 users is where you start to see emerging patterns, but you want a minimum number of users per segment when considering multiple segments.
Below we’ve outlined a couple of our typical timelines (phases and weeks) for more in-depth studies.
Healthcare – Evaluative Usability Study
This timeline is for an evaluative usability study where user interviews were approximately 90-minutes in length. During the session, users had to concentrate on what they were reading and entering into the application, which required a slower, more focused approach to analysis. This study had to work around external environmental deadlines. The recruitment consisted of new and current users.
Consumer App – Generative Product Opportunity Study
Here is an example of a generative study looking to explore new product opportunities. Participants consisted of current users that had to exert minimal effort to interact with the app. The bulk of the effort for this study was around the analysis and synthesis of findings and the presentation to stakeholders.
Consumer App – Evaluative Usability Study
This is an example of an evaluative usability study that relied upon various external recruitment and research facilities. Participants consisted of prospective users from multiple target segments across global regions. Testing consisted of 60-minute in-person interviews followed by a 5-7 day diary study.
With the Healthcare example above, you can see that the bulk of time was recruiting participants, conducting the interviews, and scrutinizing the data. This study turned out to be pretty straight-forward usability, which had longer session times, while the evaluative consumer app example, also a usability study, was more complicated. When looking at this example, you can see how the sessions were shorter. Still, we spent more time recruiting in the final two segments due to the language barrier than the first segment, adding to the effort and time spent in analyzing and synthesizing the data.
Rapid UX Research Studies
Below we’ve outlined a couple of our typical timelines (phases and weeks) for more rapid UX research studies. These projects tend to be more focused on answering specific, well-established questions but do so in an iterative fashion. Some characteristics of the rapid studies could include:
- Evaluative research that tackles quick iterative design decisions or systematic UX validation
- Generative research with a limited number of participants (minimum 5-8)
- A shorter interaction between the user and the product or service and less focus or effort expended by the user
- Limited user segments
- Limited interactions with external vendors
Consumer App – Evaluative Concept & Usability Testing
This is an example of an evaluative study that had a new focus each month to validate product designs and conceptual models. These studies focused on a small sample size of prospective and current users, with limited recruiting criteria. Sessions were 60-minutes in length.
Consumer & Business App – Validation Testing
Here we have an example of a usability test to validate the visual design. This test was limited in scope based on the visual designs that were available and lasted about 45-minutes. The recruitment consisted of about 15 participants in different segments but were all previous participants of a past study.
With these examples, you can see both have a very informal consolidation of findings. With the first example, more effort was spent on analyzing and synthesizing the findings to understand better how users interact with new product designs. In contrast, the second example is a very straight-forward test to validate a limited number of pages of a website design.
One critical challenge to developing your research methodology, is anticipating your needs before your project even begins. You may not know what is most interesting or fruitful in many cases until your research is underway.
At gotoresearch, we have found successful outcomes from various research styles, and one of the best ways to mitigate the risks of a wrong-sized research project is to follow an iterative approach. This approach means starting with shorter research cycles with less rigor to uncover key areas of interest and opportunity. Once these key areas begin to emerge, design longer and more rigorous research projects to dive deeper.
As your research efforts mature, you’ll likely find the best results by running both types of research simultaneously, on parallel tracks to create a pipeline of exploration that continually feeds your product design and development efforts.