‘I can see the value of AI as a useful tool for things like writing abstracts for scientific papers and the like.’ That was me on the topic of AI generated writing before I experimented with it myself.
As a total novice, I had no idea what I could ask of an AI bot or how I should ask it. I signed up for a free mini course offered by creativity guru Dave Birss on LinkedIn on how to get started with ChatGPT to ‘upskill as a researcher and a writer.’ Perfect.
I quickly learned that instructions had to be detailed and lengthy. It wasn’t just ‘write me an essay’ or ‘write me an article’ on a topic. Creating a prompt for the chatbot involved writing a meticulous paragraph about the style and scope of the output writing, along with the intended audience and purpose of the writing – for example, was it to sell a product, to persuade readers to act or to simply entertain.
This long-winded instruction had to cover some points that should not be included in the output. This sounded odd to me. Even though essays in literature tend to not have bullet points or numbering, if I wanted a literary essay, I had to specify no bullet points or numbering. In the prompt examples that Birss gave, he always used a phrase about the output piece being ‘jargon free.’ For that I’m assuming if you don’t point it out, the bot gives you jargon.
During the course, for practice, I asked ChatGPT to write an informative 500-word summary of an article that I referred to in last week’s blog. When I put in some of the things I did not want, I included that the summary should be ‘jargon free.’ Within seconds, I received a summary that adequately picked up the main points of the article. But it was flat. It lacked a sense of the critical tone, and there was mention of the provocative examples in the original text. The only comment it made on style noted that the article was ‘jargon free.’ I wonder where it picked up that idea. All these shortcomings made it hard to imagine where such a summary might be used.
To be fair, Birss does point out that AI cannot create ready-to-use copy. Some human editing would be involved.
My second outing with AI involved asking it to help me author an essay that I’m working on for The Journal of Open Learning, where I’m the Book Reviews Editor. We have a special issue coming up on capacity building in online and distance learning. I thought it would be useful to readers to have a short editorial piece on books in this specialised area of education. As I hadn’t had much joy finding specific books on this topic in the usual scholarly search engines, I thought I’d hand it over to ChatGPT. I wrote a long prompt, explaining to the bot that this essay was for an academic audience. I even specifically named the journal. I added that I wanted to advise academics and scholars about recent books in capacity building for online and distance learning. I forgot to mention no bullet points or numbering.
The bot spewed out 1000 words for a generalist audience – perhaps first year undergraduates – that was full of definitions of online and distance learning and of capacity building. It also contained a paragraph on the value of reading books (that part was for high school students). No actual books were mentioned. The style and formatting, complete with a numbered list of diverse ways to find valuable resources, was more suitable to a business report.
I responded to this by asking the bot for some examples of recent books to accompany the article and the books had to have capacity building and online learning in their chapter headings or indexes. There, bot, chew on that one. Within 30 seconds, I was given a list of six book titles with the names of their authors – no dates or publishers were mentioned. I checked each book to get a full reference and to make sure they addressed the topic. One of the books had different authors for the title than what I had been given, and it was generally about online learning – no capacity building. Another book was by a well-known author, and the title was similar to, but not the exact same as, a book written 25 years ago. Four of the six books were completely made up. They simply did not exist.
My second adventure with AI was like the first, missing the mark and not giving me anything I could work with. I take back what I said earlier this year about AI writing scientific abstracts. AI as a writer’s tool is in its infancy. In my experiments, it has misunderstood the nuances of writing and the rigour expected of academic research and prose. For the time being, AI appears to have a place in writing for generalist audiences and might produce working drafts for business reports, marketing content, articles for the popular press, blogs – hey, wait a minute…
Paola Trimarco 