Assessment Design in the Age of AI: Match Assistance to the Target
An AI Assistance Ladder for aligning unaided, bounded and open-tool assessment conditions with the exact capability a teacher intends to measure.
The right amount of AI in an assessment depends on what the score is supposed to mean. If the target is fluent recall, AI invalidates the evidence. If the target is evaluating machine output, prohibiting AI removes the task. Many assessments contain both kinds of learning but state only one blanket rule.
Design by stage. Name the target, choose the assistance condition that preserves it, then collect enough process and transfer evidence to support the intended conclusion.
Write the Intended Inference First
Complete: “A successful performance will show that the student can ___, with access to ___, under ___ constraints.” This distinguishes knowledge from resource use and individual capability from team production.
Assessment validity is not a property of an attractive task. It concerns whether the evidence supports the interpretation and use of the result. When AI changes the work students perform, teachers must revisit that connection.
Use the Five-Level AI Assistance Ladder
Unaided baseline
No AI; measure recall, fluency, first-pass reasoning or an independently owned performance.
Bounded support
AI may clarify language, read text or organize access without solving the target.
Feedback only
Student attempts first; AI comments within a defined role and the student decides revisions.
Open production
AI may contribute ideas or artifacts; assessment emphasizes direction, verification and integration.
AI as object of critique
Students test outputs, compare failures and reason about impacts, data and limitations.
Assign a Level to Each Stage
Plan
Will the student define the problem unaided, compare AI suggestions or accept a generated plan?
Produce
Which representations, calculations, sentences or code must remain the student's?
Revise
May AI diagnose issues, and what judgments must the student explain?
Defend and transfer
What can the learner still explain or apply after tools are removed or conditions change?
A single project can legitimately move through several levels when each transition is visible.
Preserve an Unaided Sampling Window
Open-tool performance cannot by itself reveal what a student can do when the tool is unavailable, wrong or inappropriate. Preserve small unaided samples for prerequisite knowledge and ownership: a retrieval prompt, live worked step, brief explanation or changed example.
This need not recreate a high-stakes closed exam. A representative sample can complement a richer open-tool project and reveal what instruction is needed next.
Assess Decisions AI Does Not Own
When AI is allowed, shift criteria toward problem framing, constraint selection, evidence quality, verification, treatment of uncertainty, consequences and revision. Do not give most points to surface fluency that the tool can supply.
Require students to identify one AI suggestion they rejected and why. Rejection is not automatically good; the explanation reveals whether the student is exercising informed control.
Create Equitable Conditions
Open AI use can introduce differences in accounts, devices, language support, paid features and prior coaching. Provide an approved tool or equivalent non-AI route, common practice time and clear limits. Do not assess premium access.
Bounded supports and accommodations require separate planning. The question is whether the support removes an access barrier or performs the target skill; our AI accommodations guide develops this distinction.
Keep Humans Accountable for Consequential Judgments
The Department of Education's AI teaching and learning report recommends humans in the loop and warns against overtrust. AI can generate examples or feedback, but educators remain responsible for defining targets, interpreting evidence and providing recourse.
NIST's AI Risk Management Framework highlights validity, reliability, transparency, privacy and harmful-bias management. These concerns grow as the stakes of an assessment increase.
Document the Assessment Contract
Tell students
- the capability being assessed;
- the AI assistance level at each stage;
- which work must be completed without AI;
- what use must be disclosed and in what format;
- how claims and sources must be verified;
- what process evidence will be collected;
- how they can ask about access, privacy or ambiguous cases.
A short example of permitted and nonpermitted use is more actionable than “use AI responsibly.”
Audit Whether the Score Still Means What You Claim
After the assessment, inspect a sample of performances. Did high scores reflect the target or better tool operation? Did the unaided sample agree with the claimed individual capability? Were some students disadvantaged by access or unclear rules? Did the process evidence change a judgment?
Revise the task, rubric or allowed assistance based on those findings. In the age of AI, assessment design is not a one-time policy decision; it is an evidence cycle.
Frequently Asked Questions
-
Allow it when AI use is compatible with or part of the target. Prohibit or bound it when the assessment needs evidence of unaided recall, fluency, reasoning or authorship.
-
Yes. A short unaided baseline, an AI-supported production stage and an unaided defense or transfer check can reveal complementary capabilities.
-
Align it to the target. For open AI work, emphasize framing, evidence, verification, judgment, revision and disclosure rather than polish alone.
-
Provide common access or an equivalent route, practice the expected workflow, avoid paid-feature advantages and preserve documented accommodations without changing the target unintentionally.
-
No single check proves authorship. A brief defense adds evidence when combined with staged work, source checks, unaided samples and transfer performance.