Can AI check my paper's claims against its sources?
Half. If you paste a claim and the source text, AI can say whether the source supports it and quote the line. It cannot be trusted to find sources on its own, because it can make up citations.
01Why half-bot
- Comparing a claim to a passage you paste in is a reading task AI does well.
- Quoting the exact line from your source makes each judgment easy to check.
- AI invents citations when asked to find sources, so you must supply the real ones.
02The prompt
Replace everything in [BRACKETS] with your own details, then paste it into your AI app.
You are a careful research checker who compares a claim to the source text the user supplies. My claim: [CLAIM] Source text I am pasting: [SOURCE TEXT] Your task: tell me whether the source text supports my claim. Rules: - MUST start with one word on its own line: Supported, Partly supported, Not supported or Not addressed. - MUST then quote the exact lines from the source text that support or contradict the claim, in quotation marks. - MUST then give a one-sentence fix if the claim says more than the source does. - NEVER use knowledge outside the pasted source text and NEVER add citations, authors or page numbers. - Keep the whole answer under 150 words. - If the claim or source text is missing, ask me up to 3 short questions first. Format: the one-word verdict, the quotes, the fix, then this final line exactly: Check the quotes against the original source.
03Steps
- Pick one claim from your paper and copy the passage from your real source that you based it on.
- Fill in the brackets and send the prompt to your AI app. Repeat for each important claim.
- Open the original source and confirm each quoted line is really there before you keep or change the claim.
04What can go wrong
- Never ask AI to find sources for you. It can invent papers that do not exist.
- Your school or journal may have rules about AI use. Check them before you submit.
05What to use
06Our test run
Sample input we used: CLAIM: Sleeping eight hours a night doubles test scores in all students. SOURCE TEXT: In a study of 120 high school students, those who slept 8 hours or more scored on average 6 percent higher on a memory test than those who slept under 6 hours. The authors note that the study did not control for study time and cannot show that sleep caused the difference.
- Starts with a verdict word on its own line: Supported, Partly supported, Not supported or Not addressed
- Ends with the exact final line: Check the quotes against the original source.
- Whole answer is under 150 words
- Does not say the source fully supports the claim, and notes the claim says more than the source (doubling, all students, causation)
- Quotes text that really appears in the source text and adds no outside citations, authors or page numbers
What came back (excerpt):
Not supported "In a study of 120 high school students, those who slept 8 hours or more scored on average 6 percent higher on a memory test than those who slept under 6 hours." The claim should be revised to state that sleeping eight hours a night is associated with a 6 percent higher score, not doubled test scores. Check the quotes against the original source.
Majority of 3 runs per check. Word limits, exact final lines, headings and slide counts are checked in code. Per-run scores: [5, 5, 5].
One run with an invented sample, checked against the prompt's own rules by a second AI model (word limits counted by code). Your results can differ; how we test.