Why AI Detection Is Not Enough for Programming Assessment

Identify what kind of signal you have

An AI-code classifier compares code with patterns learned or defined from human- and machine-generated examples. An AI-text classifier analyzes prose; results for essays or reflections cannot be assumed to transfer to source code. Code-similarity tools compare a submission with other code and do not establish that either source was AI-generated. Provenance or AI-use telemetry can record an artifact’s known origin or interactions with a tool where those records are captured, but does not automatically show who made the relevant decisions or what they understood. Selected process evidence records parts of development work; it may inform a process question, not decide misconduct.

Read the output according to the tool’s definition

A product may return a label, score, threshold flag, confidence category, highlighted segment or similarity result. The meaning depends on that product and version. “80% AI” has no universal interpretation: unless the specific documentation defines and validates it as a probability of authorship, do not describe it as an 80% chance that AI wrote the submission. Turnitin describes its AI Writing Report percentage as qualifying prose in a long-form document and says its model does not reliably detect non-prose such as code; this is documentation about that text product, not evidence for or against a source-code detector. Turnitin’s report guide explains its scope.

Also check which class a study or tool calls positive before interpreting false-positive and false-negative rates. In this article, a false positive means human-produced code receives an AI-like flag; a false negative means AI-generated or AI-assisted code receives no such flag. Papers may define their positive class in the opposite direction, so their metric labels must be read in context.

Code-detection research is conditional evidence

Code-specific research does not support either a universal “detectors work” or “detectors never work” claim. Pan et al.’s ICSE-SEET 2024 study applied five mostly text-oriented AI-generated-content detectors to human Python solutions and ChatGPT-generated variants, including altered prompts. It tested those tools on code; it did not study mixed classroom submissions or validate all code classifiers. Xu and Sheng’s AAAI 2024 paper reports a specialized method evaluated on assignment-style code generated by text-davinci-003. Those results show what the tested methods could distinguish under those data and model conditions, not how a current detector will classify a particular course submission. Pan et al. and Xu and Sheng describe their respective experimental settings.

Later benchmark work broadened the conditions. Droid includes multiple languages, coding domains, generators and human-AI refined or adversarial examples; the authors report that performance varies across settings, with machine-refined and humanized examples remaining difficult for many tested systems. AICD Bench adds distribution shifts and human, machine, hybrid and adversarial classification tasks. Such work helps characterize research methods and their limits; it does not validate any commercial tool for a lecturer’s course or institutional decision. See Droid and AICD Bench for the benchmark conditions and authors’ findings.

A benchmark is not the classroom

A benchmark can label known human and generated samples, specify models and programming languages, and control prompts or transformations. A real submission may combine starter code, libraries, conventional patterns, partial assistance, human edits, team work, refactoring, generated tests, older code and several tools. The evaluated detector, threshold, task, language, model version, editing and artifact mix all affect how results should be interpreted. Validation is conditional on the setting tested; that does not make benchmarks useless, but benchmark performance does not automatically establish classroom predictive validity.

False positives and false negatives both matter. Even a benchmark with strong sensitivity and specificity cannot by itself tell a lecturer how trustworthy a flag is in a course: the share of flagged work that represents the condition of interest also depends on how common that condition is in the assessed population. No prevalence or error rate should be assumed from a different course, language, model or dataset.

Different signals support different claims

The table separates what a record can directly contribute from conclusions it does not establish on its own. These sources can be combined when relevant, but none is a universal authorship or learning test.

Mixed work does not fit a simple human-or-AI label

A programming artifact may contain human-written code, a suggestion accepted and edited, generated code later refactored, a human-designed approach implemented with AI, generated tests, or revisions by several contributors. A binary classifier may not match this history or the educational question. Do not derive a human share, AI share or student contribution percentage from its score.

A detector describes properties of the artifact under its classification method. It does not show who prompted a tool, selected an approach, changed a suggestion, reviewed or verified the result, or understood it. Use “flagged by the detector” unless stronger, separate evidence supports a more specific statement.

Constructed example: a flagged validation implementation

Constructed example — not student data, classroom research or TraceYield evidence. A student submits a conventional backend validation implementation. The assignment permits bounded AI use when students disclose material use. A configured AI-code detector flags the artifact above its review threshold.

What the result establishes: this detector classified this artifact above that threshold under its method. What remains unknown: whether AI was used; what, if anything, it contributed; whether the use met the assignment conditions; whether required disclosure was complete; who made the technical decisions; whether the student understands the implementation; and whether the learning outcome was met.

The lecturer can return to the actual question. If disclosure is required, review the student’s disclosure against the assignment conditions. If the outcome concerns validation, inspect a relevant test or development record and ask the student to explain a boundary case or make a small related adaptation. Each source answers a different part of the question; none turns the detector result into proof of authorship.

If the student can explain which boundary case the validator rejects and a relevant test exercises it, the lecturer may conclude that the explanation and test provide evidence about that validation outcome under the assessed conditions. The detector flag still does not establish who produced the code. If evidence remains insufficient for a learning judgment, state that limitation. Do not resolve uncertainty as either cheating or a false accusation without evidence.

When AI use is permitted, assess the stated conditions

If AI use is allowed, “Was AI involved?” may not be the main assessment question. The relevant questions may be whether the student followed the assignment conditions, made any required disclosure, can explain the decisions relevant to the outcome, and verified the submitted behavior. See what students should disclose about AI coding tools for the disclosure question and Student Ownership for bounded judgments about responsibility and understanding.

An explanation or small adaptation can provide evidence about a specific outcome under those conditions. It does not prove who authored the original artifact. For selecting development records relevant to an assessment question, see Process Evidence; a focused oral assessment is treated separately in Oral Defense.

Use a flag as a prompt for proportionate review

A detector may be useful for a bounded purpose, such as low-stakes triage or research under conditions that have been evaluated. Before relying on it, a course team would need to know the intended population, tested languages and tasks, threshold, error profile, tool version, and what action follows a flag. A benchmark or vendor claim alone does not answer whether the tool is suitable for a particular course. For a consequential decision, the detector result should not carry the judgment by itself.

Npuls’ guidance for AI-aware assessment says detection should be a starting point for conversation, not definitive evidence; it also describes giving students a chance to explain how work was produced. This is Dutch-sector assessment guidance, not a benchmark of source-code detectors. The guidance should not be read as a universal detector error rate or a code-specific validation study.

If uncertainty remains, return to the assessment question: inspect one relevant development record, ask about a specific design choice, ask the student to interpret a test, request a small related modification, or check required disclosure against the assignment conditions. Explain the concern and allow a response under the institution’s process. A follow-up may provide evidence about a defined learning outcome; it does not prove authorship.

Keep assessment, permitted use and misconduct distinct

A detector signal may lead to assessment uncertainty. Separate evidence may then bear on whether AI was used. That evidence must be compared with the applicable assignment rules before deciding whether a rule was violated; any misconduct judgment belongs to the institution’s procedure. A detector flag alone does not establish cheating. Likewise, no detector signal does not establish independent work or learning.

A similarity match is a separate issue: it may indicate resemblance that merits review, but it does not establish AI generation. An AI detector does not establish plagiarism, copying, collusion or contract cheating. Keep those questions distinct and use the evidence relevant to each one.

What development records can and cannot add

Where captured, TraceYield may surface selected task context, AI interactions, steering, decisions, corrections and verification events. Those records can provide context a classifier does not measure, but capture may be incomplete and a recorded interaction does not prove authorship, independent work, learning, policy compliance or misconduct. TraceYield does not detect cheating or replace lecturer and institutional judgment.

Use evidence for the stated assessment purpose and avoid collecting a complete interaction history merely because a tool can expose one. For the wider question of selecting records relevant to a learning outcome, see Process Evidence.