Back
Templates

RAG, Fine-Tuning or Prompting? Decision Framework

Framework template10 minNewNeura Dynamics

Answer eight questions about your problem and see whether prompting, structured outputs, RAG, fine-tuning or a combination is the best place to start.

What it helps you do

Teams often reach for fine-tuning when a better prompt or retrieval would solve the problem, or build retrieval when the real issue is output format. This framework matches your answers to the technique most likely to help first.

How to use it

Answer based on the problem as it is today. The result is a recommended starting point; confirm it with a small test on your own data.

Decision framework

0 of 8 answered
01Does the answer depend on private or company-specific information?
02Does that information change more than a few times a year?
03Do users need to see where an answer came from?
04Is the main problem the format or structure of the output?
05Is the problem consistent tone, style or a narrow classification or extraction task that prompting has not solved?
06Do you have several hundred or more high-quality input and output examples?
07Have you tested a well-written prompt with examples and measured the result?
08Do you need a smaller or cheaper model to match a larger one on a narrow task?
Suggested starting pointAnswer the questions to see a recommendation.
Prompt engineering 0Structured outputs 0Retrieval-augmented generation 0Fine-tuning 0
01

How the recommendation works

Each answer adds weight to the techniques it favours. The highest total is the suggested starting point. When two techniques are close, combining them is often appropriate, for example RAG for facts with structured outputs for format.

02

Why prompting comes first

If a well-written, measured prompt has not been tried, start there. It is the cheapest option and the baseline every other technique must beat.

Want to build this?

If you want to test more than one approach on your own data, our Generative AI team can help design the comparison.

Generative AI Development
Sources & further reading
  1. Fine-tuning guideOpenAI · Documentationplatform.openai.com
  2. Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMsOvadia et al., Microsoft · 2023 · Research paperarxiv.org
  3. Structured outputsOpenAI · Documentationplatform.openai.com