Working with Generative AI
Three Ways to Customize AI: Prompt Engineering, RAG, and Fine-Tuning
5 min read
The adaptation problem
A foundation model like GPT-4 is trained on general-purpose internet text. It has broad capability but no knowledge of your specific domain, your company's tone, your proprietary data, or your internal processes.
To make it useful for a specific application, you need to adapt it. Three techniques offer different ways to do this — with very different trade-offs in cost, complexity, and control.
Each one sits at a different point on the easy-to-hard spectrum, and — critically — only one of them actually changes the model's weights. Keep that distinction in mind as you read: it's the thread that ties all three together.
Technique 1: Prompt engineering
Prompt engineering means changing how you communicate with the model — crafting the input text carefully to produce better outputs — without touching the model itself.
The model's weights stay fixed. The model's training data stays fixed. You're simply discovering how to talk to it more effectively.
What this looks like in practice
- Adding context: "You are a customer service agent for a software company. Answer in a friendly but concise tone."
- Providing examples: "Here are three examples of the response style I want: [examples]. Now respond to this query: [query]."
- Specifying constraints: "Respond in under 100 words. Do not use technical jargon."
When to use it
Prompt engineering is the right starting point for almost every AI application. It's free, instant, and iterative — you can test a new approach in seconds. For many use cases, a well-crafted prompt is sufficient.
Its limitation: the model has no memory of your instructions between conversations, and it can't access information that wasn't in its training data.
→ Go deeper on prompt engineering: techniques like few-shot prompting and chain-of-thought, with worked examples.
Technique 2: Retrieval-Augmented Generation (RAG)
RAG addresses the knowledge gap. Instead of retraining the model, you attach an external database that the model can query at response time.
When a user submits a question, the system:
- Searches the database for relevant documents
- Inserts those documents into the prompt as context
- Asks the model to answer based on that context
The model's weights don't change. But it now has access to information — your company's documentation, a legal database, a product catalog — that wasn't in its training data.
Visual suggestion: A flowchart: user query → database search → retrieved documents inserted into prompt → model response.
When to use it
RAG is ideal when the gap is about knowledge — the model needs access to specific, current, or proprietary information. It's far cheaper than fine-tuning and keeps the model's knowledge base updatable without retraining.
Its limitation: the model is working from retrieved excerpts, not from deep expertise in the domain. For complex reasoning tasks, shallow context retrieval may not be enough.
→ Go deeper on RAG: how retrieval pipelines actually work, vector embeddings, and where they break down.
Technique 3: Fine-tuning
Fine-tuning goes deeper. You take an existing model and continue training it on new, task-specific data — adjusting its internal weights to specialize its behavior.
This can produce substantial improvements in accuracy and speed for a specific task. A model fine-tuned on medical literature will answer clinical questions more reliably than a general-purpose model given the same information via RAG.
When to use it
Fine-tuning is appropriate when:
- A specific task requires consistently high accuracy
- A particular style or format must be deeply learned, not just instructed
- Latency matters and you want a smaller, faster specialized model
Its limitation: it's expensive — it requires curated training data, significant compute, and iterative evaluation. It's not iterative in the way prompt engineering is; each training run takes time and money.
→ Go deeper on fine-tuning: full fine-tuning vs. parameter-efficient methods like LoRA, and what a training run actually involves.
Comparison at a glance
| Aspect | Prompt Engineering | RAG | Fine-Tuning |
|---|---|---|---|
| Changes the model? | No | No | Yes |
| Cost | Low | Medium | High |
| Best for | Tone, format, behavior | Proprietary knowledge | Deep task specialization |
| Iteration speed | Immediate | Hours | Days to weeks |
| Starting point | Always | When knowledge gaps exist | After prompt engineering and RAG fall short |
Key takeaway
Prompt engineering adapts how you talk to the model — free, fast, and the right starting point. RAG extends the model's knowledge base with external data — ideal for proprietary or current information. Fine-tuning retrains the model's weights for deep specialization — powerful but expensive. Most real applications use some combination of all three.
What's next?
These three techniques assume you're working with an existing model. The next lesson looks at the class of models that makes this possible — foundation models, and why their emergence fundamentally changed how organizations build with AI.