Self-Questioning Prompts for RAG: Checking Evidence

I check what retrieved sources support before I let a RAG agent answer. A citation alone is not proof. These 15 self-questioning prompts help me choose whether to answer, give only the supported parts, search again or abstain.

I use them to check:

• Coverage: source relevance, query coverage and evidence sufficiency.
• Claims: claim-to-source mapping, citation entailment and citation completeness.
• Disagreement: source conflicts, timing differences and unresolved conflicts.
• Retrieval: missing context, source quality and when to stop searching.
• Responses: supported-only answers, partial answers and abstention.

My rule is simple: no support, no claim. I keep dates, scope and qualifiers intact, limit extra retrieval to one to three focused attempts, and record the findings in a fixed eight-field JSON record. For high-stakes decisions, I also require human review - not just another prompt.

::: @figure {RAG Evidence Checking: From Sources to Supported Answers} :::

Self-reflective RAG with LangGraph: Self-RAG and CRAG

What Evidence Checking Must Establish

Source relevance checks whether a passage matches the question’s subject, jurisdiction, population and timeframe. Query coverage checks whether the evidence addresses every material part of the request. Keep these checks separate: a relevant document may answer only part of the question. Split the question into sub-questions, then mark each covered, partly covered or uncovered.

Claim support checks each factual statement against a specific passage, including quantities, units, dates and qualifiers. Record gaps, such as an absent denominator, undefined term or missing date. Don’t fill them from model memory. Check both directions: each citation must support its claim, and every material claim must have a citation.[7]

Source quality checks authority, provenance, update status and transparency about methods - not just whether documents agree.[8] Source conflict checks whether reliable sources give incompatible answers after definitions, dates, populations and units have been aligned.

Use these findings to assign one evidence state:

These checks guide the decision to answer, fetch more context or abstain. The findings also map directly to the prompts that follow.

Return a concise verification record with labels, findings, source references, citation decisions and the next action - not reasoning details. Each retrieval request should name a specific gap it could close.

1. Source Relevance Prompt

Check whether the source answers the question - not just whether it mentions the topic. A retrieval score alone does not prove factual support.

Before drafting, classify each passage as relevant, partially relevant or irrelevant. For each requested point, cite only the passage IDs that support it. Exclude irrelevant passages, and keep background only when it clarifies a term, the scope or an interpretation.

If no passage directly covers a required point, return RETRIEVE_MORE with the missing fact and a targeted search query. If retrieval still fails, state insufficient evidence rather than filling the gap.

2. Query Coverage Prompt

Once a passage is relevant, check whether it covers the full request. Split the request into individual requirements - dates, jurisdictions, populations and comparisons - and record each as requirement → passage ID → complete, partial or missing.[12][11]

Before answering, check that the final response covers every supported requirement. Retrieved evidence can still be left out of the answer.[13][14] Use any missing qualifier to guide the next retrieval, or answer only what the evidence supports.

3. Evidence Sufficiency Prompt

After checking relevance and coverage, test whether the evidence is strong enough to support the answer. Use sufficiency as the go/no-go threshold: every material claim needs direct support from reliable sources, with no unresolved contradictions. Judge the strength of the evidence - not how many passages were retrieved. Even relevant evidence may fall short if definitions or exceptions are missing.[17]

List the claims needed for the answer and map each one to supporting passages. Flag any claims with weak or missing support, then label the evidence Sufficient, Partial or Insufficient. If it’s sufficient, answer. If it’s partial or insufficient, retrieve more context, narrow the answer or decline to answer.

Retrieve any missing definition, exception or referenced footnote that could close the gap. If the gap remains, provide only the supported portion or abstain.[17] Apply this threshold before moving on to citation-level checks.

4. Claim-to-Source Prompt

Once you’ve checked that there’s enough evidence, audit each claim. Record every atomic claim in a verification table with four fields: Claim, Source ID, Evidence span and Verdict. Break statements with multiple claims into separate atomic claims.[11][19]

Use this prompt instruction:

Map every atomic claim to one retrieved passage. Label it fully supported, partially supported, contradicted or unsupported. Check that the passage supports the same subject, scope, date and quantity, not just the topic. Do not fill gaps from memory. For unsupported wording, rewrite it, retrieve evidence, or drop it.[20][21][24]

Use the table to decide whether each claim stays in the answer.

5. Citation Entailment Prompt

Once you’ve mapped each claim to a source, check whether the cited passage supports everything the claim says.

Test the claim’s full meaning against the passage: numbers, dates, scope, conditions and level of certainty.[25][27]

Label each claim entailed, partially supported, contradicted or unsupported. Briefly quote the wording that determines the label, flag any unsupported detail, and rewrite the claim to say no more than the source establishes.[25][26]

Pay close attention to qualifications and certainty:

‘may improve performance’ does not support ‘improves performance’.[25]

Silence is not support. If trimming removed a qualifier, retrieve the surrounding text before keeping the claim.[25]

6. Citation Completeness Prompt

After entailment checks, audit the whole draft for uncited factual statements. Check that every externally verifiable statement has a citation that directly supports it. A paragraph-level citation does not support unrelated facts. This audit catches citation gaps that individual claim checks can miss.[10][30]

List every material claim alongside its citation, and mark uncited claims as unsupported. Cite only sources that directly support the claim, and record every uncited material fact in the output table.[15][28]

Return a table with these columns: Claim | Citation | Status | Missing evidence | Action. For partly supported claims, narrow or split the claim, add evidence, or remove the unsupported part. Retrieve evidence for claims without support, or limit the answer to supported claims.[15][18][28][29]

7. Source Conflict Prompt

After checking support for each claim, test whether the cited sources still disagree about that same claim.

Extract each disputed claim. Compare dates, entities, populations, definitions and conditions on a like-for-like basis, then decide whether the sources conflict.[32][33]

Weigh authority, primary evidence and methodology - not recency alone.[31] Keep well-supported contrary evidence in view. Retrieve the full passage, original dataset or correction if missing context could change the answer.[32]

If reliable sources still disagree, present both positions and attribute each to its source. State what remains unresolved and do not draw a definitive conclusion on that claim.[33]

8. Temporal Conflict Prompt

When claims still conflict, check whether timing explains the disagreement. Check when each claim was valid - not just when it was published.

For each time-sensitive claim in the conflict, record its publication or update date, event date and validity period. Compare these with the question’s as-of date. If timing resolves the conflict, use the newer authoritative source that was valid at that time.

Treat an explicit replacement as supersession, not an unresolved contradiction, and mark the replaced claims as historical. If validity is unclear, label the claim time-unclear and retrieve the official update, effective-date wording or version history. If you still cannot establish validity, decline to give a definitive answer. State the answer’s as-at date.[35]

9. Unresolvable Conflict Prompt

Use this prompt only when relevance, support and timing checks still leave a conflict that cannot be resolved.

Declare the conflict; do not force agreement. List the conflicting claims and the sources supporting each. Keep both claims intact: do not average them or choose the source that sounds strongest.[34][36]

Record each claim, its exact supporting passage and citation, and why the conflict remains unresolved. Label the state unresolved_conflict. Add a brief comparison if useful, but do not take sides. State what evidence would resolve the issue, then use this state to trigger targeted retrieval for that specific point.[34][37]

10. Retrieve-More-Context Prompt

Retrieve Only to Close a Defined Evidence Gap

If checks for relevance, sufficiency or conflicting evidence leave a gap, state what’s missing and write a focused retrieval query. Retrieve only passages that address that gap.[38]

Check the new evidence for relevance, quality and consistency with the existing record. Keep the answer within what the combined evidence supports. Record: Evidence gap | Next retrieval query | New evidence found | Decision.

Limit retrieval to one to three focused attempts to avoid open-ended searching.[38] Stop earlier if new passages add no useful evidence. If support is still incomplete, answer only the supported part or decline to answer.[39] End the retrieval loop as soon as the evidence state is clear.

11. Source-Quality Prompt

Rate Source Quality Separately from Claim Support

After checking support for each claim, assess whether the source itself is worth using.

Keep this assessment separate from the claim check. Look at the source’s authority, provenance, how up to date it is, how clearly it explains its methods, and any conflicts of interest.

Record: author/publisher | date | original source | method | conflicts of interest | quality (high / medium / low) | action (use / use with qualification / retrieve corroboration / exclude). Keep the version and URL.[8]

Source quality is not claim support. Before trusting retrieved evidence, check the cited passage separately. For high-stakes claims, exclude evidence that is weak or cannot be verified.[22][40]

12. Retrieval-Stopping Prompt

Stop Searching Only When the Evidence Is Enough

Before you stop searching, check that every material claim has support, the chain of evidence is complete, and no unresolved conflict changes the conclusion.[41] Stop when those checks pass and another search is unlikely to change the answer.[42] Use this same test to decide whether to search again.

If the test fails, name the largest remaining gap and run one focused search.[45] If it adds no new support, stop searching. Give a partial answer backed by evidence, or abstain.[43][44] Stopping retrieval does not authorise an answer.

13. Supported-Only Answer Prompt

Remove Claims the Sources Cannot Support

Once retrieval stops, keep the draft within the evidence already established. Answer only from the retrieved sources. Don’t add assumptions, background facts or inferences unless the evidence states them.[16][47]

Keep qualifiers such as may, along with any limits on dates, locations or organisations. Remove unsupported claims or label them unsupported. If only part of the answer has support, give only that part. If the evidence is insufficient, say the sources do not support a reliable answer or decline to answer.[47]

Missing evidence means a claim is unproven, not false.[16][47] Use that distinction to decide whether to answer, give a partial answer or abstain.

14. Partial-Answer Prompt

Use this after retrieval stops if the evidence supports only part of the request.

Return three fields: Supported answer, Not established by the evidence, and Next action. Give each supported claim its own citation.[23][9]

For Next action, retrieve surrounding passages or a primary source to fill the gap. Stop when searches repeat evidence without filling it. If the missing information is central to the question, abstain.[48][49][50]

15. Abstention Prompt

Once retrieval stops, abstain if the evidence cannot support a reliable answer. Return status: abstain and stop. Don’t soften the refusal with hedging such as probably or it appears.[46][43]

Missing evidence does not mean the claim is false. Name a primary source to check, or ask the user to provide it. Refer unresolved high-stakes questions for human review.[51]

Use a fixed record for the refusal: status, unsupported claim, missing evidence, next action.

Match the Evidence State to the Agent’s Response

After assessing the evidence, use its state to choose the next action:

Align scope before treating evidence as conflicting. Do not blend incompatible values or pick a source simply because it suits the answer. Stop when retrieval adds no new evidence.

A Reusable Verification Output Contract

Return a fixed JSON record with all eight fields below, including any empty arrays. This contract serialises the results of the checks above, creating a machine-readable hand-off from evidence checking to response selection.

{
  "evidence_status": "insufficient",
  "relevant_sources": [],
  "supported_claims": [],
  "unsupported_claims": [],
  "conflicts": [],
  "missing_information": [
    "Supporting evidence for the requested answer."
  ],
  "next_action": "ABSTAIN",
  "citations": []
}

Example record for an insufficient-evidence outcome.

Use evidence_status to record evidence strength: sufficient, partial, conflicting or insufficient. Use next_action to select the response path: ANSWER, RETRIEVE_MORE, QUALIFY or ABSTAIN.

Keep these fields independent. For example, partial evidence can lead to either RETRIEVE_MORE or QUALIFY. QUALIFY means a bounded answer, not an additional user-facing outcome.

When evidence passes verification, record it as follows:

• Sources: Give each source a version-specific ID and each passage a stable reference. Each relevant_sources entry should include its ID, title, passage references and a brief finding. Store document dates and version metadata with each source record so updates cannot silently change citations.
• Claims and conflicts: Give each claim an ID. Map supported claims in citations using claim_id, source_id, passage_id and support. For unsupported claims, record a reason. For conflicts, record the disputed claim, competing references and resolution status.
• Validation: Store findings only: source status, claim support, conflicts and reasons. Check that every supported claim has a citation, every reference resolves to a stored passage and every unsupported claim has a reason. Track citation coverage separately from citation correctness.[10][23]

Operational Rules for Reliable Evidence Decisions

These rules turn the earlier prompts into a repeatable checking process. Check evidence before drafting, and separate relevance from entailment: a source can relate to a claim without supporting it. After drafting, verify each material claim, fix unsupported wording and check again.[10][53]

Set clear limits for handling conflicting evidence. Compare authority, scope, jurisdiction and effective dates; neither retrieval score nor recency settles a disagreement. For UK questions, check which nation the source covers. Use unambiguous dates and distinguish publication dates from effective dates.

Retrieve only to fill a named gap, with a preferred source type and a clear stopping condition. Set and enforce retrieval, token and time limits in code. Stop when results add nothing material or a limit is reached. Running out of budget is not a reason to guess: provide the supported parts, or abstain if the central question remains unsupported.

At deployment, reliability needs controls beyond the prompt. Where possible, keep document permissions, allow-lists or trust tiers, jurisdiction filters, retrieval gates and citation validators outside the model. Treat retrieved text as evidence, not instructions.

Require human review for high-stakes legal, health, safety, financial and employment decisions. Log validation failures, repairs and overrides for audit. Test outdated sources, conflicting passages and questions outside the source collection. Monitor failures before release and after deployment.[8][52]

Conclusion: Make Evidence Checking a Decision Interface

Evidence checks should guide the agent’s next step. Check relevance and coverage, verify each claim and its citations, and flag unresolved conflicts. Then use the evidence to decide whether to answer, qualify the response, retrieve missing context or decline.[54][55]

Adjust the prompts to suit your source collection and risk level, but keep the established eight-field output contract fixed. Retain source IDs and supporting spans so reviewers can trace each claim.[56]

Apply stopping rules alongside that contract. Record why the agent stopped: sufficient support, no added support from further retrieval, or a limit being reached. Choose a supported partial answer or a clear refusal rather than an answer that goes beyond the evidence.

FAQs