AI is breaking the academic sorting machine

I see the mathematicians panicking at what we can do with agentic AI.

Let us be clear on what we are talking about. In a few weeks, someone who is not a high-level mathematician can, with AI, produce the equivalent of a good PhD thesis in math. I could see a top 1% high school student, given enough of an AI token budget, just write the equivalent of a PhD thesis as a hobby.

If Joe Smith completed a PhD thesis in math back in 2022, it is now possible for a really smart high schooler to generate the same output while playing video games. Joe must not be too happy about it, especially if Joe is still looking for a prestigious faculty position.

Notice how we are not so excited. I mean, why aren’t we celebrating this incredible breakthrough? Finally, all the math problems we have can be solved faster! It is worth reflecting on why we don’t care.

Part of the issue is that most of academic research lost its customers years ago.

In part thanks to the arrival of “peer review” in the 1970s, we have long ago closed the research world into siloed communities. The purpose of the academic output is to sort people out for jobs. Write papers that are impressive and you may get a good professorship. If you don’t, then you will be flipping burgers. There is an incredible glut of people with academic credentials, but only so many jobs at the top. The glut has been continuously, and somewhat deliberately, increasing.

If you hold a PhD today, your chance of having a tenure-track or tenured position is about 10% and falling. Let me be clear. Any academic who shows worry for what the PhDs might do now should look in a mirror because you have been training too many for decades. And, also, if fewer people decide to go for the PhD that might be more than fine.

AI disrupts the sorting mechanism in mathematics… But do you think for a minute that it does not apply to mechanical engineering, sociology, etc.? Is that bad news? No. I think that it is excellent news and I have been saying so for years.

Here is what I wrote in 2024…

« AI’s ability to generate vast amounts of text raises concerns about a potential flood of irrelevant theoretical papers, further straining the evaluation system. Stonebraker’s (2018) call for rewarding problem-solving over publication needs revisiting. Perhaps the emphasis should be on the impact and significance of research, not just its passage through peer review—a skill replicable by AI. AI can pave the way for a “golden age” of scientific progress if we can develop new evaluation methods focused on problem-solving and real-world impact. The scientific community must adapt to the evolving landscape. By recognizing the limitations of peer review and prioritizing the pursuit of meaningful solutions, we can ensure that AI becomes a catalyst for scientific advancement, not a detriment. »

I predict that, over time, the focus will move away from “papers as the final output.”

Nobody wants a paper about cancer, we want to eradicate cancer. We should reward people who get results, no matter which tools they use. And solving a problem because it is impressive to do so is not enough.

« But Daniel, Mathematicians can’t cure cancer or give us antigravity. »

Maybe it is time they try. Frankly, the AI disruption might be precisely what we needed.

« But Daniel, won’t AI just replace all of us? »

I wish. But thus far, it is not happening. I have more AI accounts than most people have pencils and I am working 50 hours a week. Intelligence is not a scalar quantity. Once an AI can do something, I somehow always find more work to do.

Further reading. Daniel Lemire, Will AI Flood Us with Irrelevant Papers? Communications of the ACM, Vol. 67, No. 9 (September 2024).

Generated by RSStT. The copyright belongs to the original author.

Source