cheating

The Empire Strikes Back against AI Cheating

Joy Buchanan2 Comments

People are considering whether university evaluations can survive in the AI age. Hollis Robbins wrote on Substack: “How to limit unauthorized AI use in the classroom“

Robbins emphasizes class size and teaching load against the time of an instructor.  An instructor teaching 4 sections with 100 students each is very limited in their ability to monitor and prosecute AI teaching. It’s worse if this instructor is on a temporary contract.

Limited eyes and hands and human attention really are a constraint here, at least for now. Some people see AI tools in the hands of students as the end of education itself.

I have been tweeting my replies to this:

I don’t do remote exams, but I hear about improvements to remote proctoring technology. The arms race is not over.

Technology goes both ways. The phone students were using to cheat are now being marshalled as a “second camera” for remote test proctoring. Instructors are going to largely win this year if they take current technology seriously, for multiple choice and short answer evaluations.

The commercial Respondus program has just added Word extensions. This technology already exists and can run on the students’ laptops.

Right now, a clever student might still be able to shift their carbon-based eyes to a direction where the answer is displayed illicitly. And the instructor’s eyes can only monitor so many eyes. This is all so 2024. This conversation may be over soon. Human students can be placed under the supervision of machine eyes. Right now, we are still dealing with issues of false positives when machines flag students for cheating, but the machines are improving.

I believe that the roads will eventually be dominated by machine drivers and their unblinking eyes. Humans might drive cars for fun in the hinterlands, but it will no longer be considered a serious thing humans to do for work. Monitoring student cheating will become like truck driving. Human eyes are on the way out. We are going to become more cheat-proof than college has ever been before.

As a college professor, that will have implications for my job, although I can imagine a not-completely-negative future. Maybe I could do more fun work with students because the work of proctoring will be handled automatically. I have spent many many hours constructing tests that would be hard to cheat on and watching students take them. I take cheating seriously, and all the faculty at my business school work hard to protect the value of our degree. I predict that this will become a trivial part of teaching within 10 years.

Will students respond with various forms of hacking and deep fakes against such a system? Maybe. So far, in any arms race, Uncle Sam has been winning in the end for a century now.

If there is a will to do so, we could even bring back the research paper by having students work on a monitored computer that does not let them use AI to write. (We could almost do that already, but perhaps the true limiting factor is that, as I like to say, readers are that which is scarce.)

[Credit to my colleagues Art Carden and Anna Leigh Stone who have talked with me about test proctoring this semester.]

AI Can’t Cure a Flaccid Mind

Zachary Bartsch5 Comments

Many of my classes consist of a large writing component. I’ve designed the courses so that most students write the best paper that they’ll ever write in their life. Recently, I had reason to believe that a student was using AI or a paid service to write their paper. I couldn’t find conclusive evidence that they didn’t write it, but it ended up not mattering much in the end.

Continue reading →

Message To My Students: Don’t Use AI to Cheat (at least not yet)

Jeremy Horpedahl4 Comments

If you have spent any time on social media in the past week, you’ve probably noticed a lot of people using the new AI program called ChatGPT. Joy blogged about it recently too. It’s a fun thing to play with and often gives you very good (or at least interesting) responses to questions you ask. And it’s blown up on social media, probably because it’s free, responds instantly, and is easy to screenshot.

But as with all things AI, there are numerous concerns that come up, both theoretical and immediately real. One immediately real concern among academics is the possibility of cheating by students on homework, short writing assignments, or take-home exams. I don’t want to diminish these concerns, but I think for now they are overblown. Let me demonstrate by example.

This semester I am teaching an undergraduate course in Economic History. Two of the big topics we cover are the Industrial Revolution and the Great Depression. Specifically, we spend a lot of time discussing the various theories of the causes of these two events. On the exams, students are asked to, more or less, summarize these potential causes and discuss them.

How does ChatGPT do?

On the Industrial Revolution:

And on the Great Depression:

Now, it’s not that these answers are flat out wrong. The answers certainly list theories that have been discussed by at various times, including in the academic literature. But these answers just wouldn’t be very good for my class, primarily because they miss almost all of the theories that we have discussed in class as being likely causes. Moreover, the answers also list theories that we have discussed in class as probably not being correct.

These kinds of errors are especially true of the answer about the Great Depression, which reads like it was taken straight from a high school history textbook, ignoring almost everything economists have said about the topic. The answer for the Industrial Revolution doesn’t make this mistake as much as it misses most of the theories discussed by Koyama and Rubin, which was the main book we used to work through the literature. If a student gave an answer like the AI, it suggests to me that they didn’t even look at the chapter titles in K&R, which provide a roadmap of the main theories.

So, my message to students: don’t try to use this to answer questions in class, at least not right now. The program will certainly improve in the future, and perhaps it will eventually get very good at answering these kinds of academic questions.

But I also have a message to fellow academics: make sure that you are writing questions that aren’t easily answered by an AI. This can be hard to do, especially if you haven’t thought about it deeply, but ultimately thinking in this way should help you to write better exam and homework questions. This approach seems far superior to the one that the AI suggests.