Student Tech Learning Guide

Assessment Design in the Age of AI: Match Assistance to the Target

An AI Assistance Ladder for aligning unaided, bounded and open-tool assessment conditions with the exact capability a teacher intends to measure.

Teacher observing students complete different assessment stages with clearly bounded tools

The right amount of AI in an assessment depends on what the score is supposed to mean. If the target is fluent recall, AI invalidates the evidence. If the target is evaluating machine output, prohibiting AI removes the task. Many assessments contain both kinds of learning but state only one blanket rule.

Design by stage. Name the target, choose the assistance condition that preserves it, then collect enough process and transfer evidence to support the intended conclusion.

Write the Intended Inference First

Complete: “A successful performance will show that the student can ___, with access to ___, under ___ constraints.” This distinguishes knowledge from resource use and individual capability from team production.

Assessment validity is not a property of an attractive task. It concerns whether the evidence supports the interpretation and use of the result. When AI changes the work students perform, teachers must revisit that connection.

Use the Five-Level AI Assistance Ladder

1

Unaided baseline

No AI; measure recall, fluency, first-pass reasoning or an independently owned performance.

2

Bounded support

AI may clarify language, read text or organize access without solving the target.

3

Feedback only

Student attempts first; AI comments within a defined role and the student decides revisions.

4

Open production

AI may contribute ideas or artifacts; assessment emphasizes direction, verification and integration.

5

AI as object of critique

Students test outputs, compare failures and reason about impacts, data and limitations.

Assign a Level to Each Stage

Plan

Will the student define the problem unaided, compare AI suggestions or accept a generated plan?

Produce

Which representations, calculations, sentences or code must remain the student's?

Revise

May AI diagnose issues, and what judgments must the student explain?

Defend and transfer

What can the learner still explain or apply after tools are removed or conditions change?

A single project can legitimately move through several levels when each transition is visible.

Preserve an Unaided Sampling Window

Open-tool performance cannot by itself reveal what a student can do when the tool is unavailable, wrong or inappropriate. Preserve small unaided samples for prerequisite knowledge and ownership: a retrieval prompt, live worked step, brief explanation or changed example.

This need not recreate a high-stakes closed exam. A representative sample can complement a richer open-tool project and reveal what instruction is needed next.

Assess Decisions AI Does Not Own

When AI is allowed, shift criteria toward problem framing, constraint selection, evidence quality, verification, treatment of uncertainty, consequences and revision. Do not give most points to surface fluency that the tool can supply.

Require students to identify one AI suggestion they rejected and why. Rejection is not automatically good; the explanation reveals whether the student is exercising informed control.

Create Equitable Conditions

Open AI use can introduce differences in accounts, devices, language support, paid features and prior coaching. Provide an approved tool or equivalent non-AI route, common practice time and clear limits. Do not assess premium access.

Bounded supports and accommodations require separate planning. The question is whether the support removes an access barrier or performs the target skill; our AI accommodations guide develops this distinction.

Keep Humans Accountable for Consequential Judgments

The Department of Education's AI teaching and learning report recommends humans in the loop and warns against overtrust. AI can generate examples or feedback, but educators remain responsible for defining targets, interpreting evidence and providing recourse.

NIST's AI Risk Management Framework highlights validity, reliability, transparency, privacy and harmful-bias management. These concerns grow as the stakes of an assessment increase.

Document the Assessment Contract

Tell students

  • the capability being assessed;
  • the AI assistance level at each stage;
  • which work must be completed without AI;
  • what use must be disclosed and in what format;
  • how claims and sources must be verified;
  • what process evidence will be collected;
  • how they can ask about access, privacy or ambiguous cases.

A short example of permitted and nonpermitted use is more actionable than “use AI responsibly.”

Audit Whether the Score Still Means What You Claim

After the assessment, inspect a sample of performances. Did high scores reflect the target or better tool operation? Did the unaided sample agree with the claimed individual capability? Were some students disadvantaged by access or unclear rules? Did the process evidence change a judgment?

Revise the task, rubric or allowed assistance based on those findings. In the age of AI, assessment design is not a one-time policy decision; it is an evidence cycle.

Frequently Asked Questions