Fact-Check AI Writing With a Claim Ledger

Turn fluent prose into rows that can fail

To fact-check AI writing, extract each verifiable claim into a ledger before polishing the draft. Record the exact claim, risk level, source needed, source location, supporting passage, check date, and decision. Open the original source yourself; generated citations and summaries are leads, not evidence. Correct or remove unsupported claims, recompute numbers, and recheck the final rendered copy so a later edit has not changed the wording you actually verified.

A claim ledger is deliberately less charming than the article. That is its advantage. Prose lets one confident sentence carry a name, date, statistic, comparison, and causal implication past the reader in a single well-tailored coat. A table makes each item empty its pockets.

Know what counts as a claim

Extract statements a reader could reasonably ask you to prove:

• Names, identities, job titles, and organizational relationships
• Dates, times, locations, distances, prices, quantities, and percentages
• Quotations and paraphrases attributed to a person or institution
• Links and citations
• Product features, availability, compatibility, limits, and policies
• Laws, regulations, standards, official guidance, and eligibility rules
• Scientific, medical, financial, legal, historical, and technical explanations
• Rankings, superlatives, comparisons, trends, and causal statements
• Claims about what a source “shows,” “confirms,” or “requires”

Also catch quiet claims. “A faster option” is a comparison. “This prevents errors” is a causal promise. “The official page” claims that the page is both real and authoritative for the statement beside it. “Currently” claims you checked the relevant state recently enough for the publication context.

Purely creative passages may not need factual sourcing, but invented specificity does not become harmless because it arrived in a whimsical sentence. If the draft says the fictional café is three blocks from a real station, either verify the geography or make the whole setting unmistakably fictional.

Build the minimum useful ledger

Use one row per independently checkable claim:

Add fields when the work needs them: publication date, source owner, archive link, calculation, jurisdiction, reviewer, expiry trigger, or correction note. Do not add columns nobody will maintain. A spreadsheet can become a very organized way to avoid checking anything.

Copy the claim exactly. Do not reduce “The agency requires X in all cases” to “agency guidance about X,” because the universal scope may be the error. If one sentence contains four assertions, create four rows.

Rank risk before opening tabs

Check high-risk material first:

1. Claims that could affect health, safety, legal rights, money, employment, reputation, or access to essential services
2. Quotations, accusations, personal details, and claims about identifiable people
3. Core claims without which the article's conclusion fails
4. Numbers, calculations, dates, product behavior, policies, and named-organization statements
5. Low-consequence background details

Risk ranking sets the order and review depth. It does not grant amnesty to the bottom rows. A low-risk unsupported detail should still be sourced, generalized, clearly framed as opinion, or removed.

Escalate specialist material. A general editor can verify that a source contains a sentence; that does not make the editor qualified to decide whether a clinical interpretation, legal conclusion, or engineering recommendation is sound. Use an appropriate subject-matter reviewer where the stakes or publication rules require one.

Treat generated references as search leads

The NIST Generative AI Profile uses the term “confabulation” for confidently presented erroneous content. Its discussion includes false or misleading logic and citations. In editorial terms: polished formatting does not turn a reference into a source.

For every generated citation:

1. Search the title, author, identifier, or quoted phrase independently.
2. Confirm that the work exists in the claimed form.
3. Open the original publication or an authoritative record.
4. Check author, title, publisher, date, edition, and identifier.
5. Read enough surrounding context to test the nearby claim.
6. Record the exact supporting passage and location.

If the source exists but does not support the wording, the citation fails. If a real organization hosts the page but the URL path was invented, it fails. If the study found an association and the draft says it proved causation, it fails with excellent credentials.

Never ask the same generated bibliography to certify itself. A model can repeat an earlier error, produce another plausible path, or explain why its fictional source would have been perfect. Use it to extract candidates if helpful; make the verification decision from evidence outside the generated response.

Prefer primary sources for primary claims

Match the source to the assertion:

• Law or regulation: official legal text or responsible government source
• Company feature or policy: current official documentation or contract
• Research result: the original paper, dataset, or formal report
• Public official's statement: official transcript, recording, filing, or verified release
• Price, schedule, opening, or eligibility: the current responsible provider
• Quotation from an interview: recording or checked transcript

Secondary reporting can provide context, criticism, or a route to primary material. It may also be the right source for what the secondary author reported. Do not cite a company's marketing page as independent evidence that its product is “best,” and do not cite a search-result snippet when the underlying page is available.

Source quality is claim-specific. A government tourism page may be authoritative for park hours and unhelpful for the history of an Indigenous place name. “Official” is not a universal upgrade token.

Verify the support, not merely the topic

Read the sentence before and after the relevant passage. Ask:

• Does it support the same subject?
• Does it support the same action or relationship?
• Is the scope the same: some, most, all, or one example?
• Is the time period the same?
• Is the geography or jurisdiction the same?
• Does the source describe a requirement, recommendation, estimate, or opinion?
• Has the draft dropped a caveat or condition?

Paste a short supporting excerpt into the ledger for internal review, within the source's permitted use. Do not decorate the article with long quotations simply because the checking sheet contains them.

When sources disagree, do not average prose into a confident midpoint. Identify differences in definitions, dates, samples, versions, or authority. Attribute the disagreement or narrow the claim to what the evidence supports.

Recompute every number

Check inputs, units, operators, rounding, and the sentence that interprets the result. For a percentage change from 80 to 100:

(100 - 80) ÷ 80 × 100 = 25%

The increase is 20 units and 25 percent, not “20 percent.” For a decrease from 100 back to 80:

(80 - 100) ÷ 100 × 100 = -20%

Percentage changes are not symmetric because the starting bases differ. Put the arithmetic in the ledger even when it seems obvious. Obvious arithmetic has an impressive publishing record.

Verify conversions from original units rather than from another rounded conversion. Preserve significant precision appropriate to the source and use. If the draft states a range, check both endpoints and whether totals include concurrent or sequential steps.

Check quotations character by character

Open the recording, transcript, email, filing, or publication. Confirm speaker, wording, omissions, punctuation that affects meaning, and surrounding context. Mark any editorial insertion or ellipsis under the publication's rules. Do not repair a speaker's factual mistake silently inside quotation marks.

Generated text should not invent quotations to summarize a viewpoint. Convert unsupported quoted wording to a sourced paraphrase only if the evidence supports that paraphrase. Otherwise remove it.

For translations, record who translated, from which source, and whether the publication requires review by a qualified speaker. A smooth translation can still choose the wrong legal, technical, or cultural meaning.

Separate current facts from durable guidance

Mark perishable claims: prices, roles, product capabilities, laws, policies, schedules, availability, active incidents, and live closures. Add a checked date and an expiry trigger, such as “recheck before publication” or “remove when campaign ends.”

Evergreen writing should not freeze a live conditions board into prose and then announce that it will go stale. Link readers to the responsible current source and generalize the durable lesson. Use a dated snapshot only when the date itself is relevant to reporting and the article clearly preserves that historical context.

Capability claims about AI tools need the same discipline. Features may depend on model, version, region, account, configuration, file type, or rollout status. Test the actual approved setup and cite current documentation. “AI can edit PDFs” is not a useful claim if the workflow depends on which PDF, which tool, which account, and what “edit” means.

Make editorial decisions explicit

Each ledger row ends in one decision:

• Keep: Source directly supports the wording.
• Edit: Evidence supports a narrower or differently attributed claim.
• Remove: No adequate support, or the detail is unnecessary.
• Label opinion: The sentence is genuinely editorial judgment, not disguised fact.
• Escalate: A qualified reviewer or rights holder must decide.

Do not leave “probably fine” in the decision column. That phrase is not a status; it is a small unattended fire.

Use the four-pass editing prompts only after the claim decisions are stable. Line editing may improve the supported sentence, but any change to scope, certainty, number, quotation, or attribution reopens the row.

Recheck the final artifact

Run a final search for every name, number, date, quotation, citation marker, and external link. Open links from the rendered page. Confirm captions, alt text, charts, tables, metadata answers, and FAQs match the checked body. Verify that layout or content-management transformations did not truncate a unit, detach a footnote, or point one citation at the wrong sentence.

Record the final reviewer and date, then save the ledger with the publication package. If the article is updated, copy the ledger forward and reopen affected rows. The AI editing workflow explains version labels and final approval; the disclosure guide covers how material tool involvement may need to be described.

Fact-checking AI writing is not a special ceremony. It is ordinary evidence work made more explicit because the draft can produce plausible detail faster than a person can verify it. Slow the claims down, give each one a row, and make unsupported fluency leave through the same door it entered.