Working with Generative AI
Prompt Engineering: Getting More From the Model You Already Have
5 min read
Why prompts matter so much
Large language models are next-token predictors: given some text, they predict what's likely to come next. That sounds narrow, but it has a surprising consequence — the text you put in front of the model is the only lever you have over what comes out, short of retraining it.
Two people can use the exact same model and get wildly different results, purely because of how they ask. Prompt engineering is the discipline of designing that input deliberately rather than typing the first phrasing that comes to mind.
It's the cheapest, fastest, and lowest-risk of the three customization techniques covered in Three Ways to Customize AI — which is exactly why it should always be your starting point.
Anatomy of a well-engineered prompt
Strong prompts tend to layer several ingredients together. Not every prompt needs all of them, but recognizing the pieces helps you debug a prompt that isn't working.
- Role / persona — telling the model who it should act as ("You are a senior support agent...") shifts the tone, vocabulary, and assumptions it brings to the task.
- Context — the background facts the model needs but doesn't have. Without this, the model fills gaps with generic guesses.
- Examples (few-shot) — showing the model what a good answer looks like, rather than only describing it.
- Task / instruction — the actual thing you want done, stated plainly.
- Constraints / format — limits on length, tone, structure, or output format (e.g., "respond only in valid JSON").
Core techniques worth knowing
Zero-shot vs. few-shot prompting
A zero-shot prompt simply describes the task: "Classify this review as positive or negative." A few-shot prompt also shows examples of the task being done correctly:
"Review: 'Shipping was fast and the product works great.' → Positive Review: 'It broke after two days and support never replied.' → Negative Review: 'Decent for the price, but the manual is confusing.' → "
Few-shot prompting tends to produce more consistent, better-calibrated results because the model can pattern-match against your examples instead of guessing at your intent. The trade-off is a longer prompt — which costs more tokens and leaves less room for everything else.
Chain-of-thought prompting
Models are often better at multi-step reasoning when they're encouraged to "think out loud" before answering. Adding a phrase like "Let's work through this step by step" — or showing a worked example that reasons through to its answer — measurably improves performance on arithmetic, logic, and multi-hop questions. This technique, often called chain-of-thought (CoT) prompting, was popularized by a 2022 Google research paper that found it unlocked reasoning abilities in large models that simple instructions couldn't reach (Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," 2022).
Role and persona framing
Telling a model "You are an expert tax accountant" doesn't give it new knowledge — but it does shift which of its learned patterns it draws on, similar to how a person might speak differently in a courtroom than at a dinner party. This is one of the simplest, highest-leverage prompt techniques because it costs almost nothing to try.
System prompts vs. user prompts
Most modern chat-based APIs separate a system prompt (persistent instructions that frame the whole conversation) from user prompts (the individual messages a person sends). Putting durable instructions — tone, role, constraints — in the system prompt keeps them from competing with the user's actual question, and keeps them consistent across turns. The OpenAI prompt engineering guide and Anthropic's prompt engineering documentation both go deep on this distinction and are worth a skim if you're building anything production-facing.
Worked example: before and after
Before (vague, zero-shot):
"Write something about our new product update."
This could return almost anything — a tweet, a press release, a haiku. The model has no signal about audience, tone, length, or format.
After (layered, specific):
"You are a product marketing writer for a project-management SaaS company. Our new update lets users set recurring tasks. Write a two-sentence announcement for our changelog, in a friendly and confident tone, aimed at existing customers. End with a one-line call to action to try it."
The second version constrains the output enough that two different people running it would get results that are similar in shape, even if the wording differs — which is usually what you want in a real product.
When prompt engineering isn't enough
Prompt engineering can't give the model facts it was never trained on, and it can't make the model "remember" instructions across separate conversations — each new session starts from a blank slate (aside from whatever you put back into the prompt). When the limitation is missing knowledge, the next step up is retrieval-augmented generation. When the limitation is deep behavioral specialization that instructions alone can't reliably produce, the next step is fine-tuning.
Key takeaway
Prompt engineering is "programming in natural language" — and like any programming, being explicit, structured, and example-driven produces more reliable results than being vague. It costs nothing to experiment, so it should be the first tool you reach for, and the foundation the other two techniques build on top of.