Is AI assessment accurate?
A similar total does not tell you whether every grading decision was right. The work, the rubric, and the feedback need to agree.
What we have measured.
In an early blind comparison of 43 submissions on one Stanford engineering assignment, AI and TA totals differed by an average of 11.4 points out of 300 in absolute terms.
This measures agreement on assignment totals. It does not establish that every rubric decision was correct, that either grader was always right, or that the result transfers to another course.
Why we focus on conceptual feedback →Inspect the decision.
Treemarks connects the submitted work, the selected rubric items, and the proposed score. Your teaching team can examine the evidence, correct a decision, and check matching work under the same ruling.
The feedback needs its own check: does it identify what is missing and give the student an appropriate next step? An accurate total can still come with unhelpful feedback. Course staff review the results and approve grade release.
What remains to test.
We need stronger comparisons with independent human judgments, evidence about errors that remain after review, and tests on new questions and courses.
Whether the feedback improves students’ understanding on later, unassisted work is a separate research question. The grading comparison does not establish learning gains.