Skill Hub — AI-Assisted Analysis

AI Output Review

Review any AI-generated output for hallucinations, inconsistency, and hidden assumptions before using it for work decisions.

What this skill is for

AI can generate code, analysis, recommendations, and summaries in seconds, but it invents facts, contradicts itself, and hides assumptions behind confident language. This skill provides a structured review workflow that catches these failures before the output is used in a document, a system, or a client conversation. The output is an AI Output Review Log that records what was checked, what was wrong, and what was corrected.

When to use this skill

After receiving any AI-generated output that you plan to forward, publish, or act on.
When AI produces code, configuration, or scripts that will run in a production or pre-production system.
When AI generates business analysis, requirements, or recommendations that will be presented to stakeholders.
When AI summarizes data, incidents, or project status and you need to confirm the facts before using the summary.
When AI provides an explanation of a system, process, or error and you are not certain the explanation is accurate.
When multiple AI outputs are combined and you need to check for contradictions between them.

Real work situations

AI generates a SAP table analysis with wrong field names

An AI produces a list of fields to check for a customer master data issue and includes ZZ_CUSTOMER_GROUP and Z_REGION_CODE as standard fields. Neither field exists in the target system. If the analyst copies this list into an investigation guide, the team will waste hours searching for fields that are not there, and the real root cause will remain unaddressed.

AI writes a shell script with deprecated commands

An AI generates a bash script to automate SAP transport imports and uses an obsolete command that was removed in the current kernel version. The script looks correct and runs in a test environment that still has the old command, but fails in production. The failure blocks a critical transport and requires emergency manual intervention.

AI summarizes a project status with fabricated milestones

An AI synthesizes a project status from chat logs and email threads and reports that the integration testing phase was completed last week. In reality, the phase was delayed and is still open. The false summary is sent to the steering committee, creating a credibility gap when the delay is discovered during the next review.

Inputs required

AI-generated output — the raw text, code, or analysis produced by the AI tool.
Source context — the original documents, data, tickets, or transcripts that the AI was supposed to analyze.
Domain reference — system documentation, configuration guides, or known-behavior references for the subject matter.
Previous review logs — patterns of past AI errors on similar tasks (optional).
Intended use — a clear statement of what the output will be used for, so the review can focus on the highest-risk claims.

Questions to ask

Which specific claims does the AI make, and what evidence supports each claim?
Which technical objects, field names, transaction codes, or APIs does the AI reference, and do they exist in this system?
What assumptions does the AI make about the environment, data, process, or timing?
Does the AI contradict itself, or does it contradict the source material?
What is missing from the AI output that a human expert would normally include?
What would happen if the most important claim in the AI output were wrong?
Does the AI use generic language that sounds correct but is not actionable or verifiable?

Working method

Capture the raw output. Save the AI output exactly as received, with metadata about the tool, model, date, and prompt used.
Define the review scope. Based on the intended use, decide which parts of the output are high-risk and which are low-risk. Focus review time on high-risk sections.
Flag factual claims. Highlight every specific claim about a system, a number, a date, a person, a process step, or a technical object.
Cross-reference with source material. Compare each factual claim against the source documents, system data, or known configuration. Mark each claim as verified, unverified, or false.
Check for internal consistency. Read the output as a whole. Look for contradictions between sections, conflicting numbers, or recommendations that conflict with stated constraints.
Surface hidden assumptions. Identify what the AI assumes to be true but never states explicitly. List each assumption and its risk if it is wrong.
Assess completeness. Compare the output against a checklist of what a competent human would include for this type of artifact. Flag missing elements.
Produce the review log. Document every check, every finding, and every correction. Include the original claim, the verification result, and the corrected version.
Decide on usage. Based on the review log, decide whether the output is usable as-is, usable after correction, or must be discarded and rewritten.

Decision rules

If a factual claim can be verified against source material, accept it. If it cannot be verified, flag it as unverified.
If a factual claim is proven false, remove it and log it as a hallucination. Do not try to edit it into a correct claim.
If the AI contradicts itself, trust the version that aligns with source material; if neither aligns, flag both as uncertain.
If an assumption is unstated but critical, elevate it to a risk and assign an owner to verify it.
If the output is missing a required element, add it manually or send the output back for regeneration with a clearer prompt.
If the AI uses generic language that cannot be verified, demand specific terms, numbers, or examples before accepting.
If the output is intended for a client, apply a stricter review standard than if it is for internal drafting.

Deliverables

AI Output Review Log — a record of every claim checked, its verification result, and the correction applied.
Confidence Map — a section-by-section label showing which parts of the output are trusted, corrected, or discarded.
Correction List — a concise list of changes made to the AI output before it is used.
Assumption and Risk Register — unstated assumptions that must be verified before the output is relied on.

Templates

AI Output Review Log

---
title: AI Output Review Log
ai_tool: [tool name / model]
date: [YYYY-MM-DD]
reviewer: [name]
intended_use: [document / code / presentation / decision]
---

## Source
- **Original prompt**: [summary or attached]
- **AI output length**: [pages / lines / tokens]
- **Source material**: [documents / tickets / data used]

## Claim Verification
| # | Section | Claim | Source Check | Result | Correction |
|---|---------|-------|--------------|--------|------------|
| 1 | [heading] | [exact claim] | [doc / system / known] | [Verified / False / Unverified] | [none / corrected / removed] |

## Consistency Check
- [ ] No internal contradictions
- [ ] No contradictions with source material
- [ ] No contradictions with known system behavior

## Assumption Register
| # | Assumption | Risk if Wrong | Owner to Verify |
|---|------------|-------------|-----------------|
| 1 | [assumption] | [consequence] | [name] |

## Confidence Map
| Section | Confidence | Notes |
|---------|------------|-------|
| [heading] | [High / Medium / Low / Discarded] | [reason] |

## Usage Decision
- [ ] Usable as-is
- [ ] Usable after correction
- [ ] Must be discarded and rewritten

Quality checklist

[ ] Every factual claim in the output is either verified or explicitly flagged as unverified.
[ ] Every hallucination or false claim is logged, not silently corrected.
[ ] The confidence map covers every section of the AI output.
[ ] The assumption register contains at least one entry, or a note that no assumptions were found.
[ ] The review log can be read by someone who did not see the AI output and still understand the findings.
[ ] The correction list is specific enough that another reviewer can apply the same changes.
[ ] A usage decision is recorded and signed off by the reviewer.

Common mistakes

Mistake: Copying AI output into a client document without any review. Consequence: A hallucinated claim reaches the client, damages credibility, and may trigger incorrect work.
Mistake: Reviewing only for grammar and tone while ignoring factual accuracy. Consequence: A technically polished but factually wrong output is approved and used.
Mistake: Trusting the AI because it sounds confident. Consequence: Invented field names, deprecated commands, or fabricated dates are treated as facts.
Mistake: Failing to log corrections so the same AI error pattern repeats on the next task. Consequence: The team never learns which prompts or topics produce unreliable output.

Weak output vs Strong output

Weak output — bad AI usage

A consultant receives a five-page AI-generated analysis of a SAP integration error, skims it for obvious nonsense, and forwards it to the client with a note that it was "AI-assisted." The analysis incorrectly states that the error is caused by a missing filter in the IDoc distribution model, when the actual cause is a misconfigured partner profile. The client acts on the wrong diagnosis and spends two days rebuilding the distribution model before discovering the real issue. The consultant has no record of what was checked and cannot explain why the wrong cause was sent.

Strong output — good AI usage

## AI Output Review Log — Integration Error Analysis

### Claim Verification
| # | Claim | Source Check | Result |
|---|-------|--------------|--------|
| 1 | Error caused by missing IDoc filter | Checked WE20 partner profile | FALSE — partner profile uses wrong message type |
| 2 | Message type is ORDERS | Checked EDP13 and partner profile | VERIFIED |
| 3 | Distribution model needs rebuild | Checked BD64 and source docs | UNVERIFIED — no evidence in logs |

### Correction Applied
- Removed the claim about the IDoc filter being the root cause.
- Added the correct root cause: partner profile message type mismatch in WE20.
- Flagged claim 3 as unverified and escalated to the integration team.

### Usage Decision
- Corrected version approved for client communication.
- Unverified claim held back pending integration team confirmation.

Agent instructions

AI Prompt Pattern

Role: You are an AI output review assistant.
Context: I have received AI-generated output for [type of work artifact]. I need you to review it systematically before I use it.
Tasks:
1. Flag every factual claim the AI makes, especially technical objects, numbers, dates, and named people.
2. Cross-reference each claim against the source material I provide.
3. Identify any claim that contradicts the source material or itself.
4. Surface hidden assumptions that the AI did not state explicitly.
5. Check for completeness: what would a human expert normally include that the AI omitted?
6. Produce an AI Output Review Log in the template format provided.
Constraints: Do not trust the AI's tone of confidence as evidence. Do not skip a claim because it "sounds right." If you cannot verify a claim, label it as unverified. Do not invent source material.

Agent dos

Ask for the source material and the intended use before starting the review.
Produce the review log as a separate artifact so the review itself is traceable.
Log every hallucination or false claim so the team can improve prompts or avoid the pattern.
Link findings to specific sections or line numbers in the AI output for easy correction.

Agent don'ts

Do not copy AI output into a deliverable without producing a review log.
Do not treat grammar and polish as evidence of factual accuracy.
Do not skip cross-referencing because the AI output is long or the deadline is tight.
Do not present your own assumptions as verified facts during the review.

Related skills

Validate AI-Generated Requirements — focused validation for requirements-specific AI output.
AI Accountability — defining ownership and review gates for AI-generated output.
Convert Notes to Requirements — turning raw input into structured drafts before AI review.
AI-Assisted Documentation Review — using AI to pre-review large documents with human sign-off.

Related Atlas pages

AI-Ready Process Documentation — how process documentation is structured for AI-assisted review.
AI Agent for SAP Support — operational context for AI-assisted analysis in SAP support.
Operational Memory for SAP AMS — building reviewable knowledge records from AI output.

Verification status and limitations

This skill is a public working interpretation of AI output review practices. It is not official vendor guidance, SAP documentation, or an AI governance framework. The review steps assume access to source material or system documentation; if neither is available, the verification results will be less reliable. AI tools and models change frequently; the error patterns described here are representative, not exhaustive. Use this skill as a structured starting point, not as a guarantee that all AI errors will be caught.