When I explain the concept of Docent, a common first question I hear is if AI grades assignments, can't students just use AI to do their homework? They imagine a scenario where professors create assignments with AI, students complete them with AI, and graders assess them with AI as well.
But, let me clear that up—Docent doesn't work like that.
While we were crafting Docent, we figured out we really needed two things to make AI grading effective:
- A "gold answer," which is basically the perfect solution to the assignment.
- A "grading rubric," which is a guide on how to deduct points for mistakes.
The "gold answer" is our way of ensuring that even if an AI is doing the grading, students can't just whip up another AI to spit out the right answers. (For the future, we're considering adding a feature where Docent can tell if an assignment is easily solvable by an LLM, without having access to the gold answer.)
Now, developing a comprehensive grading rubric is a bit trickier. It's hard to guess all the ways students might slip up. In an "AI-less setting", we usually end up tweaking the rubric a few times over time, based on what we see after the assignment has been run a couple of times.
How can an LLM make our life easier? Docent is great when it comes to building these rubrics. Since it can handle grading hundreds of assignments at once, we quickly spot the common mistakes by simply asking Docent to grade the assignments and find the mistakes. We look at the identified mistakes, and we add them in the rubric. After adjusting the rubric, we ask Docent for a re-grade, and voila! After a few rounds, we end up with a solid rubric that catches most errors.
One additional cool thing about this whole process? Docent can summarize feedback from all the submissions and we can create a report on the most common slip-ups. We take this back to the classroom to chat about the tricky parts of the assignment and help everyone learn better.
It's like having a super-hard-working assistant who may not know how to grade at the beginning but is always willing and eager to help. They never complain if you ask them to regrade assignments, summarize findings, or provide feedback.
Use Docent, be lazy, and teach smarter, not harder!
