Quibblers and cynics might call AI a solution in search of a problem. We couldn’t possibly comment on such a stance—but we will note that the list of problems that stand waiting for AI-powered salvation does seem to be diminishing by the day. Today’s banished potential use case: remote exam supervision.
The exam in question was the annual entrance exam for the National Autonomous University of Mexico (UNAM), Mexico’s largest and most prestigious university. As El País explains, while UNAM will admit just over 50,000 students this year, more than half of those places go to students whose places are assured by virtue of attending one of 14 UNAM prep high schools. The remaining places—just under 22,000 this year—go to the best performers in the university’s annual entrance exam. As one might expect, then, the exam is highly competitive—nearly 160,000 applicants sat for it this year—and with stakes so high, it’s easy to see why any opportunity to cheat might prove irresistible.
With all that in mind, one might well ask why on earth UNAM decided that this, of all exams, was the one it would use to decide on whether an AI-supervised remote exam is a great idea. The answer to that question remains unclear, but what is clear is that things didn’t go especially well.
To protect against the basest urges of its prospective freshmen, UNAM turned to two key pieces of software. The first was LockDown, a browser published by a company called Respondus, which presumably chose the name before 2020. The browser is designed to basically prevent a user’s computer from doing anything other than the exam while it is in progress. The second program was a piece of AI-powered spyware proctoring software from a company called Territorium that promised to monitor applicants via webcam to ensure they weren’t cheating.
It appears that neither the browser nor the proctoring software was, um, 100% effective. Multiple outlets have reported on the ensuing debacle, but none captures it quite as starkly as NPR, whose report opens with protesters at UNAM’s Mexico City campus chanting, “We demand honesty, not artificial intelligence.” They may have a point: Across the previous five years of exams, just 0.9% of participants scored 110 or higher. In the locked-down, AI-monitored remote exam, that percentage shot up to 5.5%. Understandably, pretty much everyone involved immediately cried foul; students were furious, one expert estimated that at least half the participants cheated, and UNAM—with an entire poultry farm’s worth of egg on its face—appointed an expert committee to look into what had happened.
That committee handed down its findings this week, and they weren’t pretty for either Respondus or Territorium. (Why these companies all have faux-Roman names remains a mystery for the ages.) The committee recommended a “control exam” to verify the scores of students who won admission via the remote test—and also for anyone whose scores would have won them admission in any year between 2021 and 2025. This means that something like 58,000 students, or a third of those who sat the original exam, will be required to sit the additional test—which, unsurprisingly, will be conducted in person.
Read the full article here
