Process Evidence: A Better Way to Assess Programming in the Age of AI

Why a final artifact may not answer every assessment question

Final code, tests and documentation are essential evidence about the submitted artifact. They may not show how a student framed an ambiguous requirement, responded to a failed approach, or can transfer a concept to a changed problem. AI tools can contribute substantially to an artifact, so the artifact alone may be insufficient when the learning outcome concerns those reasoning or adaptation abilities.

This is an assessment-design question, not a claim that AI makes final work worthless. The ACM Task Force report on generative AI and programming assessment surveys educator practice and describes assessment adaptations; it does not validate the TraceYield interpretation below. Npuls’ vision on assessment, examination and AI is broad sector guidance that emphasizes fair, valid and learning-focused assessment, not a ready-made programming rubric.

A TraceYield interpretation: trajectory evidence is not learning evidence by itself

TraceYield’s proposed interpretation is to keep four things separate: development trajectory → selected trajectory evidence → evidence relevant to a defined learning outcome → lecturer assessment/judgment. The trajectory provides context. A selected record may support a bounded claim about what happened. The lecturer decides whether that evidence is relevant and sufficient for the learning outcome, alongside the artifact and other assessment evidence.

This is a proposed way to organize evidence, not a validated educational methodology. TraceYield is pre-pilot; its public product descriptions present intended evidence views and connections. This article does not claim classroom validation or that the product determines learning.

Choose evidence from the learning outcome backwards

Start with one outcome and ask what a student must demonstrate. Select the smallest evidence set that can answer that question, and state what each item can and cannot support. If the outcome is translating requirements into a service design, a short framing note and a rationale for one design choice may be relevant. If it is debugging concurrency, a failure, the changed hypothesis, and a targeted test may matter. If it is testing, the test itself, its scope and the student’s explanation of what it does not cover may be central.

The final artifact remains part of the assessment. A process record should add evidence about a learning outcome that the artifact cannot answer by itself; it should not be collected merely because a tool can record it. The Npuls vision and the ACM report provide external context for AI-aware, learning-focused assessment. The mapping method and planner on this page are TraceYield’s proposed interpretation, not a finding established by those sources.

Illustrative example: preventing double bookings in an HBO-ICT application

This is a constructed teaching example, not real student data, a TraceYield record or a validated case study. The assignment is to change an existing equipment-booking service so two users cannot reserve the same device for overlapping times. The course outcome is: “Design and implement a data-integrity rule for a concurrent web application, and justify how the rule is verified.” AI use is permitted under the assignment brief.

Task and context. The student receives the existing service, its database schema, the booking requirements and a test environment. The repository currently checks availability in application code before inserting a booking.

AI proposal. The student asks an agent how to prevent overlapping reservations. The agent proposes checking for an existing booking, then inserting if none is found.

Student action and verification failure. The student adapts the proposal and adds a test that launches two overlapping booking requests close together. Both requests succeed in the test environment, demonstrating that the check-then-insert path can race.

Changed context and hypothesis. The student supplies the test result and schema to the agent, then records a revised hypothesis: the invariant must be enforced at the database boundary, not only by a prior application-level lookup.

Correction, implementation and verification. After considering alternatives, the student adds a database constraint appropriate to the selected storage model, handles the resulting conflict in the service, and updates the integration test. The student runs the targeted concurrency test and the existing booking tests, then explains what those checks cover and what they do not.

What the example’s evidence can and cannot support

A. What trajectory evidence directly supports: if the records exist and are authentic, the initial request was made; the agent proposed a check-then-insert approach; the student added a concurrency test; that test produced a failure; the student supplied the result as new context; the implementation changed; and specified tests returned specified results. Each statement should link to its source record or artifact.

B. What may be relevant to the learning outcome: the test and resulting change may be relevant to whether the student can identify a race condition and apply a data-integrity constraint. The database change, targeted test and the student’s explanation or demonstration provide different evidence. The lecturer decides whether their combination is relevant and sufficient for this course outcome.

C. What cannot be concluded from the trajectory alone: that the student understood concurrency, authored the implementation, worked independently, possesses general competence, acted honestly or should receive a particular mark. Those conclusions require other assessment evidence and lecturer judgment. The example illustrates a possible evidence design; it establishes no research result.

Use a small, purposeful evidence set—not a full conversation archive

For many assignments, a lecturer can review a short framing note, one selected decision or correction, the relevant artifact/test and a focused explanation. A sample of submissions may need a follow-up demonstration or transfer question, depending on the learning outcome and local assessment design. The lecturer does not need to read every prompt to ask whether a student can explain a selected design choice.

A selected checkpoint can also support formative feedback while the student can still change direction. For example, a lecturer can respond to a debugging note with a question about the new evidence or ask the student to test an alternative hypothesis. Such a checkpoint need not become a separately graded record.

Usually unnecessary or disproportionate: complete prompt transcripts, every keystroke, permanent monitoring, unrelated private conversations, raw token totals as a learning measure, or repeated documentation of routine actions that do not illuminate an outcome. If a record includes sensitive information, use a safer summary or omit it when the assessment question can be answered without it. Tell students in advance what is collected, why, who can see it and how it is retained.

A simple design test is to ask: Which outcome does this evidence illuminate? Can the student produce it proportionately? Can the assessor interpret it consistently? What privacy or access risk does it create? Remove an item when its purpose is unclear. This adapts existing advice in the article’s sources into a TraceYield planning aid; it has not been empirically validated as a scale or rubric.

Inference boundaries: state the narrowest defensible conclusion

A recorded correction can establish that a correction occurred if the record reliably supports that statement. It does not establish why the student made it or whether the student has general competence. A test result supports a claim about that test, data and environment; it does not prove the entire implementation correct. A final artifact shows what was submitted; it does not, by itself, show what the student understands.

Missing trajectory records are missing evidence. They are not evidence of misconduct. A record of AI use is not proof that the work was not learned; a record of no AI interaction is not proof of unaided authorship. If evidence is ambiguous, use a fair, outcome-aligned follow-up method rather than converting uncertainty into an accusation or score.

The lecturer’s judgment remains separate

TraceYield’s described role is to make selected development evidence easier to connect and review. The product is not an AI detector, automatic grader or authorship system. It does not decide whether evidence meets a learning outcome. That judgment remains with the lecturer under the course rubric, institutional procedure and relevant context.

Use the printable Process Evidence Planner to map one learning outcome to a task, final artifact, selected process evidence, explanation or demonstration, verification and the lecturer’s later judgment. For the broader assessment design, see the HBO-ICT assessment guide; for the specific question of understanding and responsibility, see the student ownership guide.