When you build an application on an LLM, AI prompt engineering means writing and organising the instructions the model receives so that it consistently produces output that meets your requirements. The discipline is still relevant in 2026, but the work has moved. Several techniques that used to be standard advice now do little on current models. Specification, structure and measurement matter more than they did, because newer models follow instructions more literally and get swapped out more often.
What is prompt engineering in generative AI when you build an application?
OpenAI’s documentation describes prompt engineering as writing effective instructions so a model consistently generates content that meets your requirements, and notes that because output is non-deterministic the work is “a mix of art and science”. The Prompt Report survey (Schulhoff et al., 2024) catalogues 58 LLM prompting techniques and a 33-term vocabulary, and observes that the field still suffers from conflicting terminology. That fragmentation is one reason “prompt engineering” means very different things to different people.
The useful split is between prompting a chatbot and prompting a system. OpenAI suggests treating developer messages like a function definition that holds the system’s rules and business logic, and user messages like the arguments that function is applied to. In OpenAI’s model, developer messages also take priority over user messages. Once you see it that way, an application prompt is code. It needs a specification, versioning and tests, and you can’t judge it by reading one good answer. Typing better questions into ChatGPT is a separate skill, and our ChatGPT cheat sheet for engineering teams covers that side.
Is prompt engineering still relevant in 2026, or mostly folklore?
The tricks are overrated, but the discipline holds up. Wharton’s Generative AI Labs ran repeated trials of identical questions and found that popular prompt tweaks, such as saying “please” or giving orders, produced question-specific swings that mostly cancelled out across the dataset. Their conclusion was that aggregate model characteristics dominate over specific prompting strategies. The same report found that formatting instructions did affect performance consistently: removing them caused drops.
OpenAI’s accuracy guide calls prompt engineering “typically the best place to start”, and says it is often the only method needed for summarisation, translation and code generation. The same guide says many of OpenAI’s largest customer deployments used only prompt engineering and retrieval. Magic phrases are what’s fading. Clear specification is not.
Which prompting techniques actually improve LLM output quality?
The techniques that still pay off all make the requirement explicit and the prompt easy to parse. Each of the five below has current vendor guidance or measurement behind it.
- Explicit, specific instructions. A 2025 study of prompt underspecification found that accuracy on a requirement dropped by an average of 22.6% when the prompt left it unstated rather than spelling it out. Anthropic’s Claude documentation makes the same point: if you want more than the literal task, ask for it explicitly rather than hoping the model infers it.
- Curated few-shot examples. Anthropic still strongly advises few-shot prompting, but recommends a small set of diverse, canonical examples over a long list of edge cases. For reasoning models, OpenAI advises trying zero-shot first and adding examples only when needed, and warns that examples which disagree with the instructions can produce poor results.
- Delimiters and structure. OpenAI recommends Markdown and XML tags to mark where sections and supporting documents begin and end. OpenAI’s GPT-5 prompting guide reports that in Cursor’s testing, structured XML instruction specs improved instruction adherence.
- A deliberate section order. OpenAI’s suggested developer message runs identity, instructions, examples, then context, with request-specific context usually placed near the end.
- Schema-constrained output. OpenAI’s Structured Outputs feature ensures responses adhere to a supplied JSON Schema, so required keys are not omitted and enum values stay valid.
A role line in the system prompt sits in between. Anthropic says a role focuses Claude’s behaviour and tone, which is a reasonable use. Expecting a role to improve factual accuracy is a different matter, as the next section shows.
Which prompt engineering techniques still measurably improve output quality with current models, and which ones did newer models make unnecessary?
Three widely taught habits have weak or negative evidence on current models.
Expert personas. Google, Anthropic and OpenAI all recommend persona prompts, such as OpenAI’s “You are a world-class Python developer”. Wharton’s Generative AI Labs tested this on difficult factual benchmarks and found that expert personas did not reliably improve accuracy, even when the expertise matched the question’s domain. Low-knowledge personas often reduced accuracy. The authors suggest putting the effort into task-specific instructions, examples and evaluation workflows instead.
“Think step by step” on reasoning models. OpenAI’s reasoning guidance says chain-of-thought prompts are unnecessary for its o-series models, because they reason internally, and says such instructions can sometimes hinder performance. Meincke, Mollick et al. (2025) found that for models built to reason, chain-of-thought prompting gave marginal gains at best while significantly increasing time and tokens. For non-reasoning models it still helped a little on average, but it added variability and sometimes caused errors on questions the model would otherwise have answered correctly.
Anti-laziness nudges. Anthropic’s current docs tell developers to cut back on prompts that push for thoroughness or aggressive tool use, because newer Claude models are more proactive and may overtrigger. The same GPT-5 guide reports that Cursor found a “maximize_” context instruction written for older models made GPT-5 call search repeatedly on smaller tasks. Softening it improved the model’s choice between internal knowledge and tools.
| Technique | Status on current models | What the evidence says | Condition that changes it |
|---|---|---|---|
| Explicit requirements | Keep | Unstated requirements lose accuracy and regress more across updates | Very long requirement lists degrade too (see below) |
| Curated few-shot examples | Keep, curated | Strongly advised; diverse canonical examples beat edge-case lists | Reasoning models: try zero-shot first |
| XML or Markdown delimiters | Keep | Recommended by OpenAI; improved adherence in Cursor’s tests | Format effects vary by model |
| Structured Outputs | Keep for machine-read output | Guarantees schema adherence, not correctness | Strict formats can hurt reasoning |
| Expert persona for accuracy | Drop | No reliable gain on difficult factual benchmarks | A role can still set tone |
| “Think step by step” | Drop on reasoning models | Marginal gains, much higher token cost | Small gain on some non-reasoning models |
| Thoroughness nudges | Reduce | Cause overtriggering on newer models | Still needed at minimal reasoning effort |
One caveat cuts the other way. OpenAI’s GPT-5 guide says that at minimal reasoning effort, performance varies more with the prompt, and explicit prompted planning matters more because the model has fewer reasoning tokens for internal planning. Prompted planning still matters at minimal reasoning effort. Whether you need it now depends on the reasoning effort you run.
Designing system prompts and templates that stay consistent across inputs
Consistency comes from treating the template as an interface with a fixed shape, not from finding the perfect wording. Sclar et al. (2023) showed how much shape matters: meaning-preserving formatting changes alone produced differences of up to 76 accuracy points on LLaMA-2-13B in few-shot settings. That sensitivity persisted with larger models, more examples and instruction tuning. They also found that format performance correlates only weakly between models, which undermines comparing models on one fixed prompt format.
Our recommendation is a stable system section followed by a clearly delimited variable section. The ordering also pays off in cost and latency. OpenAI advises putting content you reuse at the start of the prompt so you get prompt-caching savings on repeated requests. When the inputs vary a lot in kind, Anthropic’s routing pattern classifies each input and sends it to a specialised prompt, because optimising one prompt for one kind of input can hurt performance on others.
Structured Outputs removes the need for strongly worded formatting instructions, but it has limits worth designing around. OpenAI notes that structured responses can still contain mistakes. It also notes that with unrelated user input, the model’s effort to fit the schema can produce hallucinations, so the prompt should say what to return when the input can’t produce a valid answer. A 2024 study found that format restrictions significantly degraded reasoning, and that stricter constraints degraded it more. Our judgement is to let the model reason in free text and constrain only the final fields the application parses. OpenAI illustrates schema-bound responses with an education example: a maths tutoring app whose output drives a UI.
Why do prompts that worked in a prototype break inside a real application?
Ad-hoc prompt development exposes only a narrow slice of possible behaviours, and the models themselves keep changing. OpenAI notes that even different snapshots within the same model family can produce different results.
The Tursio enterprise search case shows the pattern. The application’s prompts were stable on GPT-4-32k and protected by a large test suite, yet only 98% of tests passed on GPT-4.1 and 97.3% on GPT-4.5-preview, which broke the CI/CD pipeline. Each migration had a deadline: GPT-4-32k was deprecated with a one-year shutdown window, and GPT-4.5-preview with a window of three months. The authors traced many failures to GPT-4.1 following instructions more literally and no longer inferring implicit rules the older model had filled in.
Underspecification makes this worse. The 2025 underspecification study found that underspecified prompts were twice as likely to regress across model or prompt changes, sometimes with accuracy drops above 20%. The authors describe prompts and models evolving at the same time, sometimes without clear changelogs, so behaviour validated earlier stops holding. The obvious fix, writing everything down, has a ceiling. In the same study, gpt-4o averaged 85.0% accuracy when given 19 requirements together, against 98.7% on single requirements. About 11.4% of requirement pairs conflicted, meaning that specifying one cut the other’s accuracy by more than 5%. Contradictions are especially costly on GPT-5, which according to OpenAI spends reasoning tokens trying to reconcile them.
Our recommendation is to pin the application to a dated model snapshot, keep prompts under version control, and treat every model upgrade as a migration with its own regression run. The wider set of changes a prototype needs is covered in what it takes to move a generative AI prototype into production.
How do you evaluate whether a prompt is actually good?
A prompt is good if it passes a test set that represents your users’ tasks, across repeated runs and plausible format variations. One impressive answer proves nothing. Hamel Husain argues that unsuccessful AI products almost always fail to build robust evaluation systems, and that product evals measure something different from general model benchmarks.
A workable evaluation loop runs in this order:
- Write the requirements down. OpenAI notes that starting with prompt engineering forces you to define what accuracy means for your use case.
- Add cheap assertions. Husain’s Level 1 tests are fast assertions you run on every code change.
- Log traces and review them. Traces are the prerequisite for human and model-based evaluation at Level 2. Track how well model-based grading agrees with human judgement before you rely on it.
- Repeat runs and vary formats. Wharton’s repeated trials exposed inconsistencies that one-off testing hides. Sclar et al. recommend reporting a range of performance across plausible formats rather than a single number.
- Monitor every tracked requirement on each prompt or model change. The underspecification study recommends background validation so that drift is detected when you switch models.
- Keep A/B tests for significant product changes, since Husain places them at the expensive Level 3.
Manual spot checks don’t scale. The underspecification authors calculate that a conditional requirement relevant to 20% of examples and violated half the time needs more than 29 inspected examples to be caught with 95% probability. LLMs can help here by generating test cases, writing assertions and labelling data. The tooling side of this is covered in our guide to model development tools for generative AI.
How hard is prompt engineering in production, and where does the hard part move?
Writing a prompt that works on a demo is the easy part. Keeping a family of prompts correct, cheap and stable as the product grows is the hard part. Husain describes a product where prompt engineering gave rapid early progress and then plateaued. Fixing one failure mode surfaced another, nobody could see quality beyond vibe checks, and the prompts grew long and unwieldy trying to cover edge cases. He calls the evaluation system the place people almost always get stuck.
Cost changes the shape of the problem too. OpenAI notes that reasoning models are generally slower and more expensive than its GPT models, and the chain-of-thought study found that step-by-step answers need far more tokens than direct ones. So a technique that adds a point of quality can still fail on budget. That makes token count part of every evaluation, alongside accuracy. Optimisation can help on both fronts. In the underspecification study, a requirement-aware Bayesian prompt optimiser improved performance by 3.8% while cutting token usage by 41-45%. The Tursio authors name the operational side, safely migrating tuned prompts to new model versions without regressions, as prompt management, a skill distinct from tuning the prompt itself.
What is replacing prompt engineering in 2026: context engineering, retrieval or agents?
None of them replaces it. They widen the scope of what has to be engineered, and the prompt becomes one part of a larger input. Anthropic describes context engineering as “the natural progression of prompt engineering”. Prompt engineering covers writing and organising instructions, while context engineering covers curating the full set of tokens at inference, including everything that lands there outside the prompts. Anthropic lists that context as system instructions, tools, MCP, external data and message history.
The reason this matters is that long context degrades. Anthropic describes context rot: as tokens in the context window increase, recall from that context decreases, more gently in some models than others, but across all of them. Chroma’s 2025 study found performance degrading with input length in every experiment, and significantly higher performance on focused prompts than on full ones in all models tested. Its conclusion was that how information is presented matters more than whether it is present at all. For long documents, Anthropic’s prompting guide recommends placing the long-form data above the query and instructions. For agents that run beyond a single window, Anthropic describes compaction, structured note-taking and sub-agents, choosing between them by task shape.
Retrieval feeds the prompt, and agent tools need prompt engineering too. Retrieval is a way of putting proprietary or current text into the prompt, for example from a vector database. Anthropic recommends starting with simple prompts optimised against comprehensive evaluation, and adding multi-step agentic systems only when simpler solutions fall short. It also says tool definitions deserve as much prompt engineering attention as the main prompt. In its SWE-bench work it spent more time optimising tools than the overall prompt.
Is there still a future in prompt engineering as a skill or job role?
As a skill, yes. As a stand-alone title, we expect it to fold into ordinary software and product engineering, the way test writing did. The work now includes requirements analysis, evaluation design, version migration and context assembly. The Tursio authors frame prompt lifecycle management as the next dimension of LLM application development, with prompts continuously integrated against new model versions. People who can write a clear specification and build the evaluation that proves it holds will stay in demand under whatever title.
What happens to prompt engineering as models get better?
Better models remove some prompting work and make other work more exacting. Anthropic reports that smarter models need less prescriptive engineering, and expects treating context as a finite resource to remain central to reliable agents. Its Claude guidance already prefers general instructions such as “think thoroughly” over hand-written step-by-step plans. Newer models also follow instructions more literally, though. The Tursio authors found GPT-4.1 and GPT-4.5-preview needed more explicit, structured prompts than GPT-4-32k, and concluded that LLMs are getting “smarter in some ways, but dumber in others”. Models are increasingly helping with their own prompts too: OpenAI reports that early GPT-5 testers shipped prompt revisions generated by asking GPT-5 what to add or remove from a failing prompt. Forecasts beyond this are speculative, and how to evaluate predictions for AI in the future is a useful check before you plan around any of them.
When to change the prompt, add retrieval, or fine-tune
OpenAI separates two failure types. Context problems occur when the model lacks knowledge, has out-of-date knowledge or needs proprietary information. Behaviour problems show up as inconsistent formatting, wrong tone, or reasoning that isn’t followed consistently. The fix follows the failure type.
| What your evals show | First move | Next move if it plateaus | Evidence |
|---|---|---|---|
| Wrong or vague requirements, any output type | Rewrite and specify the prompt | Add curated examples | Start with prompting, then add instructions and examples |
| Missing, stale or proprietary facts | Add retrieval | Tune retrieval and prompt together | RAG beat unsupervised fine-tuning for knowledge injection (Ovadia et al., 2023) |
| Inconsistent format, style or rule-following | Tighten the prompt and schema | Fine-tune on representative examples | Fine-tuning suits syntax, style and rules |
| Both context and behaviour failures | Prompt first, then combine | RAG plus fine-tuning | The methods stack |
OpenAI’s worked example shows both directions. A gpt-3.5-turbo model fine-tuned on 1,000 examples scored 78 BLEU against 70 for GPT-4 with three few-shot examples, but adding retrieval to the fine-tuned GPT-4 model then cut accuracy by four points, because the task was a behaviour problem and the extra context added noise. If you do fine-tune a retrieval application, OpenAI advises including retrieved context in the training examples so they match production. Retrieval and fine-tuning each slow iteration and can introduce regressions. Our guide to fine-tuning generative AI models covers that path in depth.
So the rule is the one OpenAI states plainly: get as much accuracy as you can from basic methods before reaching for RAG or fine-tuning, and let the accuracy target drive the choice, not the perceived sophistication of the technique. That rule only works if an eval suite exists to tell you when prompting has actually run out.
FAQ
How to use AI for prompt engineering?
You can ask a strong model to critique a failing prompt and suggest what to add or remove. OpenAI reports GPT-5 users shipping revisions produced that way. LLMs can also generate test cases, write assertions and label data for the evals that tell you whether a revision helped.
Can you provide a guide for prompt engineering in OpenAI?
OpenAI’s own prompt engineering guide covers message roles, Markdown and XML structure, few-shot examples, context and the context window. It has a separate guide for reasoning models, which need different prompts.
