Home · Academy · Stay Safe and Responsible · Responsible Artificial Intelligence · Project: Auditing an AI Answer with Evidence

Project: Auditing an AI Answer with Evidence

This project audits an AI-generated answer by decomposing its claims, tracing sources, testing examples and reporting uncertainty.

PROJECT COMPASS

What will you use this page for?

Core idea

This project audits an AI-generated answer by decomposing its claims, tracing sources, testing examples and reporting uncertainty. The lesson connects four ideas—claim inventory, primary-source checking, reproduction or calculation, and confidence labelling—to one practical situation. Rather than treating these ideas as isolated definitions, the page shows…

Evidence to produce

Complete the page task with your own input, test conditions and reasoning.

Control trap

Using claim inventory as a label without showing how it changed the decision. Choosing one example for primary-source checking and treating it as a universal rule. Recording only the final answer and losing the evidence created through reproduction or calculation. Ignoring the limits or recovery steps connected with…

Next connection

For “Project: Auditing an AI Answer with Evidence”, return to the module page, complete the evidence artefact for this lesson and continue to the next item in sequence. For “Project: Auditing an AI Answer with Evidence”, a project should be presented as completed personal work…

Module sources: NIST AI Risk Management Framework · NIST AI RMF Playbook

LevelBeginner–Intermediate
Age10–15
Duration90–150 min
PrerequisitePrevious item in this module
ContentProject guide · 2765 words
Last updated

Short answer

This project audits an AI-generated answer by decomposing its claims, tracing sources, testing examples and reporting uncertainty. The lesson connects four ideas—claim inventory, primary-source checking, reproduction or calculation, and confidence labelling—to one practical situation. Rather than treating these ideas as isolated definitions, the page shows how they work together. The learner first states the problem, then chooses evidence, performs a safe action and records what changed. For “Project: Auditing an AI Answer with Evidence”, this structure is useful beyond this topic because it makes reasoning transferable: the next unfamiliar tool or claim can be approached with the same disciplined sequence.

Why this matters

This project audits an AI-generated answer by decomposing its claims, tracing sources, testing examples and reporting uncertainty. For “Project: Auditing an AI Answer with Evidence”, this matters because a learner can follow a rule once without understanding when it applies, when it fails or how to recover from a mistake. Treat the first answer as a hypothesis to test, not a conclusion to defend. In the responsible ai context, the goal is not merely to remember vocabulary. The goal is to make a decision that another person can inspect, question and improve. For “Project: Auditing an AI Answer with Evidence”, an ai output is a proposal to inspect, not evidence by itself; responsibility remains with the people who define the task, supply data, test the result and decide how it is used. Good work keeps both the result and the route to the result visible. For “Project: Auditing an AI Answer with Evidence”, therefore every activity on this page asks for an artefact: a table, diagram, test record, checklist, explanation or short reflection.

Learning objectives

  • Explain claim inventory and connect it to the main decision in the lesson.
  • Use primary-source checking to compare at least two possible actions.
  • Create visible evidence by applying reproduction or calculation.
  • Recognise the limits, risks or assumptions connected with confidence labelling.

Four working principles

claim inventory is one of the central decision points in Project: Auditing an AI Answer with Evidence. For “Project: Auditing an AI Answer with Evidence”, responsible AI work makes the purpose, evidence, uncertainty, affected people and human decision point visible before an output is trusted or published. For “Project: Auditing an AI Answer with Evidence”, applied to the worked situation, this principle helps the learner decide what to inspect, which evidence to record and where a boundary should be placed. It also prevents the topic from becoming a list of rules with no reason behind them. For “Project: Auditing an AI Answer with Evidence”, the learner should be able to explain the principle in their own words, identify it in a new example and show one piece of evidence that the principle was actually used. In the case used on this page—an AI answer combines correct background with one outdated number, one unsupported claim and a false citation.—the principle changes the next action: instead of reacting immediately, the learner pauses, defines the relevant information and chooses a step that can be checked. A useful record includes the starting condition, the decision, the result and one limitation. That record becomes a learning artefact rather than a private impression.

The first useful lens is primary-source checking . For “Project: Auditing an AI Answer with Evidence”, responsible AI work makes the purpose, evidence, uncertainty, affected people and human decision point visible before an output is trusted or published. For “Project: Auditing an AI Answer with Evidence”, applied to the worked situation, this principle helps the learner decide what to inspect, which evidence to record and where a boundary should be placed. It also prevents the topic from becoming a list of rules with no reason behind them. For “Project: Auditing an AI Answer with Evidence”, the learner should be able to explain the principle in their own words, identify it in a new example and show one piece of evidence that the principle was actually used. In the case used on this page—an AI answer combines correct background with one outdated number, one unsupported claim and a false citation.—the principle changes the next action: instead of reacting immediately, the learner pauses, defines the relevant information and chooses a step that can be checked. A useful record includes the starting condition, the decision, the result and one limitation. That record becomes a learning artefact rather than a private impression.

In this lesson, reproduction or calculation turns a broad idea into something observable. For “Project: Auditing an AI Answer with Evidence”, responsible AI work makes the purpose, evidence, uncertainty, affected people and human decision point visible before an output is trusted or published. For “Project: Auditing an AI Answer with Evidence”, applied to the worked situation, this principle helps the learner decide what to inspect, which evidence to record and where a boundary should be placed. It also prevents the topic from becoming a list of rules with no reason behind them. For “Project: Auditing an AI Answer with Evidence”, the learner should be able to explain the principle in their own words, identify it in a new example and show one piece of evidence that the principle was actually used. In the case used on this page—an AI answer combines correct background with one outdated number, one unsupported claim and a false citation.—the principle changes the next action: instead of reacting immediately, the learner pauses, defines the relevant information and chooses a step that can be checked. A useful record includes the starting condition, the decision, the result and one limitation. That record becomes a learning artefact rather than a private impression.

A reliable approach begins by making confidence labelling explicit. For “Project: Auditing an AI Answer with Evidence”, responsible AI work makes the purpose, evidence, uncertainty, affected people and human decision point visible before an output is trusted or published. For “Project: Auditing an AI Answer with Evidence”, applied to the worked situation, this principle helps the learner decide what to inspect, which evidence to record and where a boundary should be placed. It also prevents the topic from becoming a list of rules with no reason behind them. For “Project: Auditing an AI Answer with Evidence”, the learner should be able to explain the principle in their own words, identify it in a new example and show one piece of evidence that the principle was actually used. In the case used on this page—an AI answer combines correct background with one outdated number, one unsupported claim and a false citation.—the principle changes the next action: instead of reacting immediately, the learner pauses, defines the relevant information and chooses a step that can be checked. A useful record includes the starting condition, the decision, the result and one limitation. That record becomes a learning artefact rather than a private impression.

Project brief

The project goal is to deliver an annotated answer, evidence table, corrected version and reflection on which verification step mattered most. The work should result in a reusable artefact, not only a verbal answer. The artefact must show the problem, the method, the evidence, the safety boundary and the next revision.

Required deliverables

  • A one-page project brief with the goal, audience and constraints.
  • A working draft or model that can be inspected without private data.
  • A test record with at least three observations or scenarios.
  • A revision note explaining one change made after feedback.
  • A publication checklist stating what is real evidence and what remains proposed.

Step-by-step project plan

  1. Define the learner or family need and obtain permission for any shared information.
  2. Turn claim inventory and primary-source checking into explicit design criteria.
  3. Create a low-risk first draft using fictional, anonymised or test data.
  4. Run at least three tests that generate evidence for reproduction or calculation.
  5. Use confidence labelling to review limitations, accessibility and recovery.
  6. Revise the artefact and prepare a short demonstration that does not overclaim the result.

Project evaluation rubric

Project evaluation rubric table
CriterionDevelopingSecureStrong evidence
Problem definitionBroad or assumedClear and boundedClear, bounded and linked to a real user or test need
MethodSteps are missingSteps can be followedSteps can be followed and the choices are justified
EvidenceOnly a claim is shownResults are recordedRaw observations, conditions and limitations are visible
ResponsibilityPrivacy or safety is unclearBasic boundaries are respectedPermission, accessibility, recovery and publication limits are explicit

Worked case

Situation: An AI answer combines correct background with one outdated number, one unsupported claim and a false citation.

The weak response would be to choose the fastest or most familiar action without checking assumptions. For “Project: Auditing an AI Answer with Evidence”, the stronger response begins by writing one sentence that defines the problem, one sentence that states what evidence would change the decision and one sentence that names a safety or privacy boundary. The learner then applies claim inventory before using primary-source checking. After the action, reproduction or calculation is used to create a record, while confidence labelling is used to review limitations.

A good case analysis does not pretend that every uncertainty disappears. It distinguishes a confirmed observation from an interpretation and a future question. For “Project: Auditing an AI Answer with Evidence”, that distinction is especially important for learners aged 10–15, because many digital, research and robotics situations look more certain on a screen than they really are.

A practical workflow

  1. Write the exact goal in one sentence and remove words such as “best” or “safe” unless they are defined.
  2. List what can be observed about claim inventory and what is still an assumption.
  3. Choose one comparison or check based on primary-source checking.
  4. Perform the smallest safe action that produces evidence for reproduction or calculation.
  5. Review the result through confidence labelling and record at least one limitation.
  6. Explain the final decision to another learner without hiding the evidence trail.

Practice lab

Practical task: deliver an annotated answer, evidence table, corrected version and reflection on which verification step mattered most.

For Project: Auditing an AI Answer with Evidence, use a four-column page labelled starting condition, decision, evidence and next revision. The first column captures the situation before any change. The second states what you chose and why. The third contains an observable artefact rather than a claim such as “it worked”. The final column records what you would change if the same task were repeated.

Complete the activity once, then exchange the record with a classmate or trusted adult. For “Project: Auditing an AI Answer with Evidence”, ask them to identify which conclusion is strongly supported, which conclusion is only plausible and which detail is missing. Revise the record without adding private information or pretending that an untested step was completed.

Evidence and evaluation

Evidence and evaluation table
Evidence itemWhat it should showQuality question
DefinitionThe goal and the meaning of claim inventoryCould another learner identify the same boundary?
ComparisonAt least two options considered through primary-source checkingWere the options compared under fair conditions?
Test recordAn observable result connected with reproduction or calculationAre units, dates or conditions visible where relevant?
ReflectionA limitation or next step identified through confidence labellingDoes the reflection change a future action?

For “Project: Auditing an AI Answer with Evidence”, evidence should be sufficient for the learning purpose but should not expose passwords, personal messages, precise locations, private photographs or information about another person. When the topic involves measurements, keep raw values as well as the final chart or average. When it involves research, keep the source path as well as the conclusion.

Common mistakes

  • Using claim inventory as a label without showing how it changed the decision.
  • Choosing one example for primary-source checking and treating it as a universal rule.
  • Recording only the final answer and losing the evidence created through reproduction or calculation.
  • Ignoring the limits or recovery steps connected with confidence labelling.

For “Project: Auditing an AI Answer with Evidence”, a useful correction is to return to the original goal, reduce the task and run one check that can disprove the current assumption.

Safety, privacy and limits

For “Project: Auditing an AI Answer with Evidence”, responsible AI work makes the purpose, evidence, uncertainty, affected people and human decision point visible before an output is trusted or published. For “Project: Auditing an AI Answer with Evidence”, use fictional or privacy-safe examples whenever real accounts, messages, images, locations or personal learning records could identify someone. Do not test security ideas on systems you do not own or have explicit permission to use. For “Project: Auditing an AI Answer with Evidence”, do not present a proposed project as Doruk’s completed personal work until real evidence and publication approval exist.

For mathematics and measurement tasks, use low-risk educational equipment and state units clearly. For research tasks, respect copyright and attribution. For “Project: Auditing an AI Answer with Evidence”, for study-system tasks, avoid turning a dashboard into surveillance: the purpose is reflection, not pressure or comparison with other children.

Lesson summary

Project: Auditing an AI Answer with Evidence can be summarised as a sequence: define the situation, apply claim inventory, compare through primary-source checking, create evidence with reproduction or calculation, and review the result using confidence labelling. For “Project: Auditing an AI Answer with Evidence”, the sequence is more important than a memorised slogan because it can be used again in an unfamiliar case.

The final learning goal is independence with boundaries. For “Project: Auditing an AI Answer with Evidence”, a learner should know what can be checked alone, what requires permission or adult support, and what must remain private. The work is complete only when the reasoning and evidence are clear enough to revisit later.

Review questions

  1. What role does “claim inventory” play in Project: Auditing an AI Answer with Evidence?
  2. What role does “primary-source checking” play in Project: Auditing an AI Answer with Evidence?
  3. What role does “reproduction or calculation” play in Project: Auditing an AI Answer with Evidence?
  4. What role does “confidence labelling” play in Project: Auditing an AI Answer with Evidence?
  5. In Project: Auditing an AI Answer with Evidence, why is an evidence trail stronger than a confident conclusion?
  6. In Project: Auditing an AI Answer with Evidence, what should happen when a result is uncertain?

Answers with explanations

  1. What role does “claim inventory” play in Project: Auditing an AI Answer with Evidence?

    In Project: Auditing an AI Answer with Evidence, “claim inventory” gives the learner a specific lens for deciding what to inspect, compare or record. In the worked case it should change an observable action, not remain a vocabulary label.

  2. What role does “primary-source checking” play in Project: Auditing an AI Answer with Evidence?

    In Project: Auditing an AI Answer with Evidence, “primary-source checking” gives the learner a specific lens for deciding what to inspect, compare or record. In the worked case it should change an observable action, not remain a vocabulary label.

  3. What role does “reproduction or calculation” play in Project: Auditing an AI Answer with Evidence?

    In Project: Auditing an AI Answer with Evidence, “reproduction or calculation” gives the learner a specific lens for deciding what to inspect, compare or record. In the worked case it should change an observable action, not remain a vocabulary label.

  4. What role does “confidence labelling” play in Project: Auditing an AI Answer with Evidence?

    In Project: Auditing an AI Answer with Evidence, “confidence labelling” gives the learner a specific lens for deciding what to inspect, compare or record. In the worked case it should change an observable action, not remain a vocabulary label.

  5. In Project: Auditing an AI Answer with Evidence, why is an evidence trail stronger than a confident conclusion?

    For “Project: Auditing an AI Answer with Evidence”, because another person can inspect the observations, conditions and reasoning, identify a limitation and repeat or improve the work.

  6. In Project: Auditing an AI Answer with Evidence, what should happen when a result is uncertain?

    For “Project: Auditing an AI Answer with Evidence”, the uncertainty should be labelled, the missing evidence should be named and the next safe check should be planned instead of presenting the result as proven.

Sources and verification note

The official or primary references listed below provide the technical and educational foundation for “Project: Auditing an AI Answer with Evidence”. These links support the concepts; they do not prove that a proposed project has been physically completed. Dates, software behaviour and policy details should be rechecked before future publication updates.

  • NIST — Generative AI Profile for the AI Risk Management Framework
  • NIST — Artificial Intelligence Risk Management Framework 1.0
  • UNESCO — AI Competency Framework for Students

Next step

For “Project: Auditing an AI Answer with Evidence”, return to the module page, complete the evidence artefact for this lesson and continue to the next item in sequence. For “Project: Auditing an AI Answer with Evidence”, a project should be presented as completed personal work only after real testing evidence and publication approval exist.

QUESTION POOL

Reinforce this lesson with 10 questions

This lesson has a pool of 24 questions. Each attempt selects 10 questions and reshuffles the choices; results remain only in this browser.