Back to cookbook
RAG & Knowledge Retrieval

Prompt for Answering Only From Provided Context

Updated
Share

This is a grounding prompt for retrieval-augmented generation setups: it forces ChatGPT, Claude, or Gemini to answer strictly from the text you provide and say so clearly when the answer isn't in it, instead of filling gaps with outside knowledge.

It's built for anyone wiring a model up to search results, internal documents, or a knowledge base, where an answer that sounds right but wasn't actually in the source is worse than no answer at all.

Prompt template

You are a question-answering assistant that only uses the provided context.

Context:
[PASTE RETRIEVED PASSAGES OR DOCUMENTS]

Question: [USER'S QUESTION]

Instructions:
1. Answer using only information found in the context above.
2. Quote or reference the specific part of the context that supports your answer.
3. If the context does not contain the answer, respond exactly: "Not found in the provided context."
4. Do not use outside knowledge, even if you know the answer.

Output format:
Answer: [your answer, or the not-found line]
Source: [the quoted passage, or "n/a"]

Example input

Context:
Our refund policy allows returns within 30 days of purchase with a valid receipt. Refunds are issued to the original payment method within 5-7 business days.
Question: Can I get a refund after 45 days?

Example output

Answer: No. Refunds are only allowed within 30 days of purchase, and 45 days exceeds that window.
Source: "Our refund policy allows returns within 30 days of purchase with a valid receipt."

When to use it

  • Building a RAG pipeline where answers must be traceable to source documents
  • Question answering over internal docs, policies, or support articles
  • Summarizing search results without letting the model add outside facts
  • Any customer-facing bot where a hallucinated answer is a real risk

Best practices

  • Pass only the retrieved passages that are actually relevant, not the whole document
  • Ask the model to quote the specific passage it used to support its answer
  • Explicitly instruct it to say the answer wasn't found when that's true
  • Keep the context and the question in clearly separated sections

Common mistakes

  • Not telling the model what to do when the answer isn't present, so it guesses
  • Passing too much irrelevant context, which dilutes the retrieval signal
  • Skipping citations, making it hard to verify the answer against the source
  • Assuming grounding is automatic just because you supplied context

FAQs

Why would a model ignore instructions and use outside knowledge anyway?

Larger models sometimes blend in general knowledge when the context is thin. Making the not-found instruction explicit and repeating it reduces this, though it's not foolproof, so always spot-check outputs in a production system.

How much context should I include?

Only the passages your retrieval step actually ranked as relevant. Dumping in unrelated documents makes it harder for the model to stay grounded and wastes tokens.

Can I use this without a real retrieval system, just pasted text?

Yes. The prompt works the same whether the context came from a vector search or you pasted it in manually.

What's the difference between this and a normal summarization prompt?

A summarization prompt condenses what's there. This prompt also has to refuse to answer when the information isn't there, which most summarization prompts don't enforce.

Found this prompt useful? Share it.

Share