An October 2023 study revealed a discrepancy between the grades of students with last names closer to the end of the alphabet and those with last names closer to the start. By analyzing over 30 million Canvas grading records from the University of Michigan, researchers found students with lower alphabetically-ranked surnames — that is, names that begin with W, X, Y or Z — receive noticeably lower grades on their assignments and exams, more negative feedback from instructors and are more likely to submit post-grade complaints.
Rackham student Jiaxin Pei and School of Information alum Helen Wang co-authored the study with Jun Li, Ross School of Business associate professor of technology and operations. To address the differences in grade markings between students, the researchers proposed instructors use a newly-added feature in Canvas which randomizes the order of students’ assignments.
In an interview with The Michigan Daily, Pei and Wang elaborated on how their shared teaching experience at the University of Michigan inspired them to investigate the relationship between students’ surnames and reported grades.
“Grading is something both of us have a lot of experience with,” Pei said. “We have been (teaching assistants) and graded a lot of students’ assignments and exams. Both of us can relate to the fact that it’s actually a really time-consuming and difficult job to do.”
Wang explained that graders’ high workload causes them to develop fatigue by the time they grade assignments of students with lower-ranked last names, potentially accounting for the lower marked grades.
“In our paper, we argued that the potential bias we observed is caused by a mechanism called fatigue,” Wang said. “When graders have too much of a workload and too many students, they develop this fatigue, they get distracted and they will perform worse and worse with more and more. Given this mechanism, it would be great if we can reasonably control the workload of each grader and give them behavioral nudges to take rests, maybe every five minutes when they work more than 40 to 45 minutes or so.”
Wang said the study’s findings are broadly applicable across institutions, and that the creators of Canvas, a learning management tool used across the University, recently added a feature that randomly sorts students’ submissions in SpeedGrader, a Canvas tool used to quickly grade assignments which previously sorted students in alphabetical order.
“Sequential design, like sorting students from the initials A to Z, is not unique to the University of Michigan,” Wang said. “This design applies to all Canvas-like applications, no matter what school or university. The Canvas (staff), when they saw our study, actually made an improvement in May. They released a new version of Canvas that specifically allows for this random sorting of students in the system. It reflects the conscious idea between researchers and industry workers that the sorting of students is important.”
Wang also shared the reactions she saw from various University instructors and researchers when presenting the study at different research conferences.
“Most of (our audience) are researchers and university professors,” Wang said. “They will say, ‘Oh, I have never heard about that’, and then say, ‘That actually makes a lot of sense.’ People start to reflect, thinking ‘How am I grading my students? Am I doing something wrong? Am I actually treating all the students equally?’. I’m very happy that our research can actually motivate people to think about fair grading.”
In an interview with The Daily, Peter Bahr, associate professor of education, proposed an alternative to the study’s findings, a scenario where some instructors give students higher grades as they grade more assignments, not lower grades.
“What’s being described (in the study) is, on average, a modest net decline in grading,” Bahr said. “It could actually be that there’s a share of graders that tend to grade higher when they’re experiencing fatigue, in addition to graders who grade lower, and that it just appears like we have more lower graders. In either case, the problem is that the quality of the grading is declining, whether it’s going higher or lower. We don’t want either one of those. We want continuity in the quality of grading.”
One concern Bahr expressed about the study’s findings is that large instructor workloads may lead them to choose more structured assignments, which could suppress the amount of creative work students produce.
“Faced with a large amount of grading, there can be a tendency for instructors to choose more structured tasks in lieu of more creative or critical tasks like writing papers,” Bahr said. “I don’t think that’s a good outcome. Frankly, I think the process of writing papers is an important one. It’s not just the writing; it’s the way our minds develop and the way we think about a topic. I’d want to encourage others to continue using writing assignments as opposed to choosing multiple choice exams or something with less decision-making involved.”
Bahr said randomization alone may not address the systemic problem of maintaining consistent grades across students.
“What randomization would be doing is saying ‘let’s randomly distribute lower quality feedback,’” Bahr said. “What we actually want to do is improve the feedback overall, and improve the accuracy and consistency of the grading. That requires us to have more graders, more training and rubrics when we approach our grading.”
In an interview with The Daily, Batoul Abdallah, lecturer in teacher education, recommended that instructors concerned about grading quality should review their work after all assignments are graded.
“When you’re done with grading a set of papers, as much as you want to close the virtual book on it and be done, it might help to just look at the first and last few you graded and make sure that you know the expectations are the same across the board, to maintain consistency,” Abdallah said.
In an interview with The Daily, Mark Hoover, associate research scientist and lecturer in educational studies, said teaching smaller classes and assigning group projects have helped reduce over-exhausting graders.
“I can have a class of 24 students, with 4 people in a group, and now I’ve got six projects to read through,” Hoover said. “I tend to keep the projects a little shorter. They’re often 1000-1500 words, something like that. If I have four to six of those, then they’re all different enough. They’re very engaging. I don’t feel fatigued at all.”
Hoover emphasized the importance of the study in highlighting aspects of general grading systems not consciously paid attention to.
“I think it’s nice to see this kind of attention and digging into the practice,” Hoover said. “These things are often not very visible and not consciously thought about. There’s a lot that happens in teaching that just gets done, and people are socialized to it without much explicit instruction, or conscious, deliberate attention and thinking. Drawing things like this forward so that they can be thought about more explicitly is important.”
Daily Staff Reporter Thomas Gala-Garza can be reached at tmgala@umich.edu.
The post U-M researchers suggest end-of-alphabet last names receive lower grades appeared first on The Michigan Daily.
Leave a Reply