AI-generated "synthetic users" are becoming a hot topic in UX research. These artificial profiles, created by large language models, promise to simulate real user behaviors and feedback. Companies like Synthetic Users say they can deliver faster, cheaper research by skipping the hassle of recruiting real participants. But can AI truly capture the nuanced, messy reality of human experience that makes UX research valuable?
What are synthetic users?
Synthetic users are AI-generated profiles designed to mimic specific user groups. Using LLMs trained on vast datasets, tools like Synthetic Users create personas with detailed backgrounds, goals, and behaviors. Researchers can define target demographics, set research goals, and generate simulated interviews or survey responses in minutes.
Picture this: a synthetic user representing a "25-year-old software engineer in Berlin" provides feedback on your prototype's usability. These tools offer serious scalability, producing responses from dozens or thousands of personas quickly, and they allow follow-up questions to dig deeper. The appeal is obvious.
The upside of synthetic users
Synthetic users offer compelling advantages, especially in early-stage research:
- Speed and scale: They eliminate the time-intensive process of recruiting and scheduling real participants, delivering insights in minutes rather than weeks. No more waiting around.
- Cost efficiency: Synthetic users reduce expenses tied to participant incentives and logistics, making them appealing when budgets are tight. Money talks.
- Exploratory research: They work well for generating hypotheses, testing initial concepts, or exploring user needs before investing in real-user studies. Think of them as a first draft.
- Data synthesis: AI can summarize vast datasets (like forum posts or academic literature) to create realistic profiles. Imagine a synthetic medical representative describing a typical workday based on thousands of real accounts. That's pretty impressive.
- Accessibility: They provide a starting point for teams with limited access to users, helping kickstart research where none might otherwise happen. Better than nothing.
For instance, synthetic users can simulate responses to survey questions about a new app's interface, letting designers iterate quickly before real-user testing.
The reality check
Despite their promise, synthetic users have significant drawbacks that limit their reliability:
- Lack of authentic emotion: AI lacks the emotional depth and contextual nuance of real humans. Synthetic responses often feel flat or overly optimistic, missing the unpredictable insights that emerge from real conversations. The magic is in the mess.
- Data dependency: Synthetic users rely entirely on the quality and scope of their training data. If that data is outdated or biased, the resulting profiles will misrepresent real user behavior. Garbage in, garbage out.
- Superficial insights: AI-generated feedback tends to lack the complexity of human experiences. Cultural influences, unexpected motivations, personal histories โ these are critical for meaningful UX design, and AI just can't capture them. The devil is in the details.
- They can't buy your product: As Pavel Samsonov notes, "an LLM can't buy your product." Synthetic users don't reflect real-world decision-making or economic behavior. Money where your mouth is.
- Risk of overreliance: Using synthetic users as a substitute for real research can lead to flawed assumptions. AI might "hallucinate" responses or reinforce existing biases rather than challenge them. Don't put all your eggs in one basket.
Here's the thing: a synthetic user might give you generic feedback on a healthcare app, but only real patients can reveal the emotional weight of navigating a chronic illness.
When synthetic users make sense
Synthetic users aren't a replacement for real-user research, but they can complement it in specific scenarios:
- Hypothesis generation: Use synthetic users to brainstorm ideas or identify potential pain points in early-stage design, then validate with real users. Start here, don't end here.
- Prototype testing: Test low-fidelity prototypes or concepts with synthetic users to refine designs before investing in usability studies. Polish before you present.
- Data-scarce environments: In domains like medical research, where recruiting diverse participants is challenging, synthetic users can fill gaps for initial exploration. When access is limited.
- Supplementary analysis: Use AI to summarize existing user data or generate interview guides, enhancing efficiency without bypassing human input. Work smarter, not harder.
A team designing a financial app might use synthetic users to explore initial feature preferences but rely on real users to understand trust-related concerns.
Best practices for responsible use
To use synthetic users responsibly, follow these guidelines:
- Always validate with real research: AI-generated insights need to be checked against real-user studies to ensure accuracy and depth. Trust but verify.
- Define clear inputs: Specify detailed user profiles and research goals to improve the relevance of synthetic responses. Vague inputs lead to generic outputs. Be specific.
- Use for early stages: Leverage synthetic users for ideation or low-stakes testing. Reserve real users for critical decision-making. Right tool for the right job.
- Check for bias: Critically evaluate AI outputs for signs of bias or oversimplification. Cross-reference with real-world data whenever possible. Question everything.
- Maintain empathy: Prioritize human-centered research to capture the emotional and contextual nuances that AI cannot replicate. Keep the human in human-centered design.
The path forward
Synthetic users are a double-edged sword. They offer efficiency and accessibility, particularly for teams with limited resources, but risk diluting the human-centered foundation of UX research. As one researcher put it, "Human-centered research comes from conducting research with โ you guessed it, humans." The clue is in the name.
The future lies in hybrid approaches, where AI handles tasks like data synthesis or hypothesis generation while real users remain the cornerstone of meaningful insights.
Rather than replacing human research, synthetic users should enhance efficiency in specific contexts. For example, Making Science's advanced synthetic user model integrates real user data from call centers to improve accuracy, but still emphasizes validation with real users. As AI evolves, its role in UX research will grow, but it must never overshadow the empathy, complexity, and authenticity that only real humans provide.
Key takeaways
- Synthetic users are AI-generated profiles that simulate user behaviors, offering speed and cost savings for UX research
- They excel in exploratory research, hypothesis generation, and data-scarce scenarios but lack the emotional depth and nuance of real users
- Limitations include superficial insights, data dependency, and the inability to reflect real-world purchasing behavior
- Use synthetic users as a complement to real-user research, not a substitute, and validate findings with human feedback
- Responsible use involves clear inputs, bias checks, and a commitment to maintaining empathy in the design process
Sources: Adapted from "Synthetic Users: Is there a place for AI-generated users in UX research?" by Anabella Ritchey, UX Collective, May 27, 2024. Additional insights from NN Group, Making Science, and IDEO.
