AI in Academia — An experiment with an annual report
A confession first: I have not used AI* in any serious capacity so far. There have been many an instance where there was an opportunity to play around and I have done so. Mostly chatGPT or similar or demonstrations of how well Gemini could summarize documents. I have not really used it for coding. I like my code simple since otherwise I cannot follow myself.
*really large language models, something separate from machine learning, which I have used professionally and teach about.
But the noise around AI has grown to a crescendo. To the point that our University has given us access to a Google account with Gemini specifically for AI usage. So I decided to a more controlled experiment. Not something that is open-ended and just generates the frizz of possibilities but something that would a) save me time, and b) save a headache that I have to go through annually and c)there is a decent experiment for me to set up.
The other trend in Academia, at least in the USA, is to ask for more and more status reporting and accountability. Professors are evaluated on an annual and often now on a longer period timescale as well. You get to justify your existence and employment over and over. This requires documentation and summary documents of that documentation.
Perfect! This is a task that I would like to avoid and involves meticulous summary of many many documents that I would love to pawn off to a machine. I read somewhere that AI generated text is for documents you don’t expect to be read really by people. Or by real people. I forget.
So I took my 2024 documentation pile and fed it into Gemini. And this is an experiment so I can check how well it did compared to the actual summary document I wrote with much sweat & swearing in 2025.
I fed it a CV, the various documents attesting I had reviewed a paper or similar, student evaluations, a prize certificate for advising, and the emails I had chucked into the “merit review 2024” as perhaps somewhat pertaining to what I was up to.
There are three categories in a typical evaluation: teaching, research (papers & grants), and service.
The Gemini generated document — after some fiddling with the prompt — generated a very convincing looking summary document for the first two. There was enough in the whole ensemble to make something that looked like what I had written up. Just with fewer spelling mistakes. The style was…very linked-in-y if you catch my drift. Call it Calvinism, call it Northern European upbringing (oh wait those are the same), call it my personality but it contained more explicit rah-rah than I would have done myself.
However, the mentoring of students and much of the service I had done was missing. So if you are using any “AI” to generate documents like that, please understand that this is where it will be underselling you to the administration. It looks like you could take on some more service work!
The reason is simple, it was not in the data. Student evaluations are easy to digest but supervising a student and their good experience with you is much much harder to capture in the documents like the ones I fed it. And putting thattogether is typically one of the more laborious parts of the annual reporting document. So unless you want to add a heap of personal data (here is access to my email accounts and photo library?), it will not show you the kind of work that makes the big impact on students, the reason they come to you for projects or take your classes. It doesn’t show your work on a public event or a committee.
Would it save me time? Maybe some. It would certainly get me started on this task-I-prefer-to-avoid. But there is a real risk that it may undersell what you’re trying to show. And if a potential cost-of-living-expenses “merit” raise is on the line, it might not be worth it. An actual human may read this.
