Every week I see another “proven framework” for prompting AI tools. Step one is always the same: define the role. “Act as a senior marketing strategist.” “You are an expert copywriter with 15 years of experience.” “Pretend you’re a seasoned developer.”
I stopped doing this months ago. I’ve shipped 6 iOS apps, won a solo hackathon against 27 teams, run client sessions, and built hundreds of prompts. Not a single role in sight.
My outputs didn’t suffer. If anything, they got sharper.
So let’s look at what the research actually says, because the “always define the role first” advice is being spread with zero evidence behind it.
What the Research Actually Found
In late 2024, researchers from Carnegie Mellon University, Stanford, and the University of Michigan published a peer-reviewed study with a title that says it all: “When ‘A Helpful Assistant’ Is Not Really Helpful.” (Full paper here, also on arXiv.)
They tested 162 roles across 4 popular families of LLMs and 2,410 factual questions. Their conclusion: adding personas to system prompts does not improve model performance on objective tasks compared to using no persona at all. In some cases it made things worse. ACL Anthology
The part that should make every “framework” post pause: while aggregating results from the best possible persona for each question did improve accuracy, automatically identifying which persona would help was near-impossible — results performed no better than random selection. ACL Anthology
Random. The role you pick might help. It might hurt. You have no reliable way to know which.
This finding was consistent across all four model families tested. It’s not a one-off result.
But What About the Studies That Say Roles Help?
Fair question. There are papers on the other side. PromptHub did a good breakdown of several of them.
One 2024 paper claimed a 10% performance improvement using something called “role immersion method” — but it used a multi-step framework with multiple LLM calls and an evaluator. It’s hard to know if the performance gain came from the role or from the extra processing steps. Prompt Hub
Another study, ExpertPrompting, showed gains — but only when the persona was highly specific, detailed, and auto-generated by the AI itself based on the task. Not “act as a marketing expert.” Something far more elaborate, built by the model, not typed by a human in three seconds.
The honest summary: persona prompting is generally not beneficial for accuracy-based tasks, especially with newer models. It works for style and tone. If you tell ChatGPT to talk like a cowboy, it will talk like a cowboy. But for quality of thinking and accurate output, the evidence isn’t there. Substack
The studies showing benefit for roles mostly use older models (GPT-3.5 era), elaborate multi-step frameworks, or highly detailed auto-generated personas. Not the one-liner “act as an expert” being taught in every prompting course.
Why Does This Advice Keep Spreading?
Because it feels logical. Humans respond differently when they’re in a role. A lawyer thinks differently than a teacher. So shouldn’t an AI?
Not really. Modern LLMs already have the knowledge. The role doesn’t unlock new information — it adds a costume over what’s already there, and according to the research, that costume is mostly decorative.
The advice also spread before the research existed. Early AI tutorials said “define the role” and it became gospel. It got repeated, turned into frameworks, sold in courses, posted on LinkedIn. Nobody went back to check if it worked.
What Actually Moves the Needle
The research consensus on what genuinely improves output quality comes down to three things.
Specificity over identity. Prompts should clearly articulate the desired outcome — ambiguity leads to unexpected or irrelevant outputs. Specifying the desired output format, whether a list, paragraph, or code snippet, improves response relevance. Prompting Guide
Compare these two prompts. “Act as an expert copywriter and write me an email.” vs. “Write a 150-word follow-up email to a CFO who didn’t respond to our proposal. Tone: direct, not pushy. Focus on one ROI stat. No jargon.” The second has no role. It has specificity, audience, constraints, format, and goal. That’s what does the work.
Context over costume. General instructions alone lead to overly broad responses. Prompts should provide sufficient detail to narrow the response space and align the output with the desired goal. PubMed Central Tell the AI what the situation is, who the audience is, what you’re trying to achieve. That’s context. A role is not context — it’s theater.
Format instructions. Be explicit about what you want the output to look like. Structure, length, tone, what to include, what to leave out. This is where your prompt tokens are better spent.
The One Exception Worth Knowing
Roles do work for style and tone tasks. If you want the AI to write in a specific voice, adopt a certain register, or match a character’s speech pattern, a role helps. “Write this like a cynical tech journalist” produces a different result than nothing.
But that’s style. Not accuracy. Not quality of thinking. Not usefulness of output. Know the difference before you spend the first line of every prompt on a job title.
What I Do Instead
I’ve been building with AI tools for months without a single role prompt. What I use instead: who the output is for, what the goal is (specific, not vague), what format I want, what constraints apply, and relevant context the AI can’t guess.
That’s it. No costume required.
The “define the role first” advice isn’t harmful — it’s just unnecessary, and it’s teaching people to optimize the wrong part of their prompt. You’re wasting your opening line on decoration instead of information.
Specificity is the skill. Not casting.
Want to go deeper? Start with the CMU/Stanford/Michigan paper and PromptHub’s breakdown of the competing studies.