Articles by Tim Hamlett

Technology

Education and ChatGPT: Time to return to bad old days?

Happily, I retired from the university teaching scene before ChatGPT and its mates came along, because it has apparently made life rather difficult. Generally, the most subtle assessment method, at least in the humanities, is the essay. You set the topic, send the student away, and assess the resulting masterpiece a week later. People in other necks of the woods have other choices, of course. Mathematicians can be asked to tackle a problem on the spot. Subjects which consist primarily of absorbing a lot of memorable material can be tested with multiple-choice questions, which have the additional advantage that they can be graded by a computer. The most picturesque assessment method was applied to trainee aircraft engineers. The class assembled at 9am, and each member was presented with a small engine. At 1pm the examiner returned, by which time the engine had to be dismantled entirely into its component parts. The class then had lunch, and then reassembled to reassemble their engines. At 5pm, the examiner returned and tried to start each engine. If it started first time you had passed, and if not… But I digress. The problem with setting essays now is that you do not know if the result was partly, or indeed wholly, written by artificial intelligence (AI). This has, in turn, produced a lot of interest in detection, which is sometimes possible. The person who told me about this problem recalled a case in which the essay included the word “albeit.” This is not the sort of word that comes up often in sociable chat, so the writer was asked what it meant… and could not answer. Clearly, though, there is going to be an ongoing arms race here between AI users and programmers, trying to produce ever more convincing forgeries, and teachers and other detectives looking for ways to spot traces of computer composition. The answer, I fear, is going to be a return to the primitive practices of the past. When I took my first degree, half a century ago, there was no continuous assessment, multiple choice was regarded as simplistic, and computers were monstrous machines which had to be fed punched cards. We wrote essays at least once a week, and these had to be submitted and discussed, but they were not part of the assessment. There was also a small examination each term intended to reassure your personal academic adviser that you were still alive and working, but this was not part of the assessment either. After three years of this regime, you encountered an ordeal more like the old Chinese civil service exam – in which candidates were locked up for a week and, according to legend, invited to write down everything they knew – than modern gentle testing methods. Starting on Wednesday, we had three-hour exams each morning and afternoon until Sunday, when you had a day off, a relic of the days when universities were mainly intended to train clergymen. The ordeal resumed on Monday and ended at lunchtime on Tuesday, at which point there was an understandable tendency for the survivors to get resoundingly drunk. The actual examination was quite Spartan. You were allowed (times had changed) to use a Biro. The paper was white and completely blank. People were agitating for lined paper to help authors whose writing tended to diverge gradually from the horizontal as they got down the page. This came eventually, but too late to help me. There was a list of about a dozen questions, of which in your three hours you were expected to attempt four. As the marks available were divided equally between your four answers, it was an elementary point of exam technique to get to four somehow. There were no “open books”, bringing in notes of any kind was cheating, and the examiners were not – with rare exceptions – the people who had taught you. We were not asked our opinions of the teachers, but because the examiners were applying consistent standards, the outcomes of individuals and groups could be compared. This was a stressful system, and I would not recommend copying it in all its details. It favoured glib bullshitters and people who could write quickly. It was a great preparation for journalism, which is perhaps not a recommendation. But as AI tramples its way across the academic landscape, it may be time to rediscover the merits of putting the students in a room with paper, pen, questions and a time limit. At least you know who has written what.