The Limits of AI for Writing

This time around, I thought I’d do some trial and error with ChatGPT to write my next album review. Obviously, a degree of editing was going to be required. But working with AI to create worthwhile original content is a slog, and I decided to abort the attempt quickly. Let’s look at these 3 samples and then explain my frustration.

In bold, I have 3 different prompts that I used for my next prog album review for Third by Soft Machine. The italicized text is what ChatGPT gave me; in all three cases, I stopped generating text once I saw what was being spit out. I’ll explain why after presenting the results:

Write a sophisticated review of the Soft Machine album "Third"

ChatGPT:

The Soft Machine's third studio album, aptly titled "Third," stands as a towering achievement in the realm of progressive rock. Released in 1970, the album marked a significant departure from the band's earlier, more whimsical sound and established them as one of the most innovative and experimental groups of the era.

The album's opening track, "Facelift," immediately sets the tone for the rest of the album with its disorienting time signature and intricate instrumentation. The song's

Write an impressionistic review of the Soft Machine album "Third"

ChatGPT:

The Soft Machine's "Third" is an otherworldly journey through the avant-garde landscape of progressive rock. Released in 1970, this album defies easy categorization with its complex arrangements, unconventional time signatures, and experimental soundscapes.

From the opening notes of "Facelift," the listener is plunged into a kaleidoscope of sounds that twist and turn with the band's improvisational prowess. The jazzy saxophone solos, frenetic drumming, and ethereal keyboards create an atmosphere that is both chaotic and beautiful.

"Slightly All The Time" is a standout track that showcases the band's ability to seamlessly blend different genres. The funky bassline, hypnotic drums, and soaring saxophone solos are punct

Write a review of the Soft Machine album "Third" in the style of kaiserediting.com

ChatGPT:

Soft Machine's "Third" album is a transcendental masterpiece that defies convention and pushes the boundaries of progressive rock. With its intricate arrangements, complex rhythms, and experimental soundscapes, this album is a true testament to the band's creative vision.

From the opening track "Facelift," the listener is immediately swept away by the band's virtuosic instrumental skills. The interplay between the saxophone, keyboards, and drums is nothing short of magical, creating a soundscape that is both disorienting and mesmerizing.

The album's standout track, "Slightly All The Time

First, let’s bullet point the standout problems and then talk about them:

Limited ability to follow my prompts as intended
Formulaic structures
Commonplace descriptions and ideas

Limited ability to follow my prompts as intended

What DID I intend?

The first was just testing the waters: what would ChatGPT produce if I asked for a review? I got boring basic details that anyone could write without listening to the album—year of release, genre, and the band’s reputation. Too generic

Prompt 2 was trying to get the AI to write in a style closer to how someone might respond to the album in the moment. Merriam Webster’s definition of “impressionistic” is “based on or involving impression as distinct from expertise or fact.” In other words, I was looking for a review that speaks to sensation and emotion, perhaps relating the music to non-musical experiences that the writer might have had.

“Impressionism is a style of painting developed in France between 1870 and 1900 which concentrated on showing the effects of light on things rather than on clear and exact detail.” - *Collins Dictionary*

The AI failed miserably to give me something “based on impression.” Instead, it described how a “listener” would be impacted by the music, but this is flimsy writing since no one can say how other people would feel; you can only describe your own reactions with authority.

Moreover, the AI failed to live up to other definitions that link impressionism to non-systematic and non-rational modes. All ChatGPT did was give me a similar review to the first one, just with some more fanciful, figurative wording. But it was still exactly as systematic and structured as the first, with the same general sense of meaning.

The third prompt is interesting because you’ll notice that it is an absolute failure to follow my instructions. The text that ChatGPT generated is nothing at all like my other reviews—there’s no commentary on a personal history with the music or how it connects to my other experiences or my idiosyncratic thoughts about music more generally. There’s no attempt to connect the music to anything beyond itself.

It didn’t even include a single dash!

This is all the evidence I need that asking an AI to write the review I would write myself is ludicrous. I won’t be doing it again.

Formulaic structures

The most offensive thing about my third result is that the AI gave me the exact same structure and style as the first 2 prompts.

This style in itself isn’t flawed. It’s fine as a textbook example of how to structure a document: begin by describing the subject with a general overview of what it’s like and its impact. Then, get into details by describing key elements with an eye to reinforcing the overall assessment of the first paragraph. Allow the commentary to build so as to develop and deepen the argument.

In this case, the AI named the album and release year, then described it in relation to the band’s earlier output while emphasizing the experimental and unique sound of the album. A second paragraph describes the first song in terms that are certainly accurate. I quit generating when I saw this happen three times in a row because I knew that I wasn’t getting anything interesting, not even worthwhile with heavy editing. I’d need to start from scratch to get at anything like the style and content of my other reviews.

Formulas provide solid structure, but without worthwhile content, all you have is empty phrasing. I describe 5-paragraph essays as a circular structure where the conclusion of the paper is nothing more than some commonplace ideas stated at the beginning. The essay goes full circle without deepening or developing the ideas, merely repeating some basic info.

But if ChatGPT is so consistent at nailing down what is arguably a right way to structure an essay, what’s the problem?

Well, as I tell my students, there is no right structure for an essay. The 5-paragraph style so many of us learned in high school is notorious for many reasons, but the biggest problem is that people often forget it is only a teaching tool. It’s a guide for beginners, so using it is a sign that the writer need not be taken too seriously as a writer—they’ve never learned to harness the tools of writing to truly communicate.

So long as ChatGPT is locked into the most basic formulas intended for novices, it’s not going to be interesting as a writer. Essays and articles that get published are not 5-paragraph essays. Why would they be if that style is so easily produced by anyone, and now AI? What matters is content, what you actually have to say. Complex and thoughtful thinking requires something other than a basic formula. There needs to be malleability, adaptability, non-conventional ways of thinking and doing. All the sorts of things AI is not programmed for and likely can’t be programmed for: there are too many human variables connected to our irrationality, preference for sensation over reason, our ability to generate and accept technical error in service of meaning and feeling.

Commonplace descriptions and ideas

Which brings us to the next issue: impeccable style and accuracy.

The AI provided what seem like perfectly grammatical and nicely-written sentences that describe the album and individual tracks. It’s all things that listeners of the album might agree on and there’s nothing objectionable or controversial in these assessments.

And that’s exactly the problem.

There’s no character to the writing except that it imitates the contemporary style of internet writing. ChatGPT can do no more and no less than obey the rules of style and grammar, present descriptions and information that is already out there online, and spit it out in a bland, if eloquent, way.

But let’s get back to the fact that I did this three times. Look at all the repeated phrases and the overall sameness of the content in all three cases. If I asked for three reviews by three different authors, the results would be wildly different. Each would focus on something different because they would all have different impressions and different histories with music and writing in general. One or more might even say something negative about the album. Just by way of example, here is the only sentence of the first draft of MY review for Soft Machine’s Third: “I had to take my earbuds out when ‘Facelift’ started.” I wrote that and then got the idea to use ChatGPT.

Nothing so personal or idiosyncratic appears in these reviews. Sure, there’s an attempt at sounding like a feeling person through the use of emotive terms like “whimsical,” “beautiful,” “disorienting,” and “magical.” But these are all so easily applied to music in general as to come off as completely generic, verging on meaningless. This is actually where the AI came closest to writing like me because I do rely on similar kinds of phrasing to describe music, though I make deliberate attempts to add novelty by playing with the language and rewriting the more cliched descriptions. But yea, I have described drumming as “frenetic” in a review or two.

I asked AI to give me three different results and got slight variations of the same things 3 times. There’s a danger here that hasn’t been articulated in anything I’ve read about AI. It’s easy to assume that AI has objectivity and presented a completely new script every time, especially because of all we hear about how the memory of previous results is always erased. But that doesn’t mean we really are getting 3 different writers if we present 3 different prompts.

No, what we get is just one writer: the AI with its programming and bias toward bland, formulaic writing that relies on the average view of things as articulated online. When I gave it the chance, the AI twice called “Slightly All the Time” the standout track of the album. Is that its opinion or just general consensus? If the former, can we trust the judgment given AI’s inability to experience music like a human? If the latter, all it’s doing is agreeing with popular opinion, so we’re not getting an original thought.

If we ask this same AI, or several of them, all programmed with more-or-less the same functionality, then we are doomed to reading only one author’s view of things, and that author is inherently uninteresting because it has no experiences, no identity, no feelings, no personality. It doesn’t even have an agenda that opens its writing up to worthwhile analysis. That is, no agenda other than that of the people programming it and feeding the prompts. Ultimately, this encompasses all of us through the averages of what is written online and public discourse more generally. If AI, ideally, has access to the sum total of human knowledge and experience, then we can’t possibly grow that knowledge or experience by relying on AI to create for us.

What does it all mean?

All these problems speak to the immense gulf between AI writing and the stuff put out by actual people. Many of us are focused on the I part of AI—intelligence. But it’s the artificial we need to understand better. AI can mimic human output by looking at finished products and generating things that on the surface are similar. But the process to get there is inorganic. There is no thought process at all, much less any process of personal reflection.

I generate a music review by repeatedly listening to the music, thinking about how it makes me feel, how it sounds or doesn’t sound like other music I know, how it lives up to my expectations or not, what the lyrics might mean, how my actions while listening impact my feelings, how the music relates to other experiences I’ve had, and countless other things I might not even think of consciously. AI can’t do any of these things because it has no experiences. Moreover, it can’t even listen to music in the way people do. Even if you believe that AI is or could become conscious, that consciousness would not be linked to the same kind of bodies people have, so its experiences would be alien to us at least in part. Arguably we’re all alien to ourselves and each other, but it’s a matter of kind, not just degree, when it comes to non-human intelligence, maybe.

When AI describes the listener and the way the instruments play against each other, it’s not describing a subjective experience of the music. At best it describes what is literally true of the song (its arrangements and style) and the way other people have described their experience of it.

One reason that AI writing is becoming so prominent and pushed by companies is that the overwhelming majority of what is written for the internet is intended to be bland, characterless, formulaic, anodyne, and impersonal, despite relying on style that is informal—a lot of “you focus,” conversational tone, and other elements that create an artificial sense of intimacy. In other words, AI writing is great for the marketing mindset of what a lot of the internet has become and what the forces pushing AI envision it to become even more so.

If you come to a music review looking for a merely functional description of what the album is, the names of the songs, their instruments and genre, maybe a sense of song structure and mood, then these AI reviews get the job done in the most efficient and functional way. If you’re interested in music, chances are that’s not the kind of review you’re looking for. It certainly doesn’t describe how music feels to you.

We should beware the endless reproduction of recycled text. Who demands it and who benefits from it?

If you want to understand how someone else has felt and thought about the music and through the music, then an AI review is worthless.

With this in mind, we can perhaps divide the value of AI in two depending on the goal of our writing and reading.

If it’s a matter of providing information—transmitting data to an audience—then AI will certainly be a valuable tool. There is no reason to prize the human collection and presentation of data over the AI’s efforts since the facts remain the same. AI might even present them more clearly and efficiently. Of course, a certain amount of human editing and fact checking is warranted for a variety of reasons.
If the writing and the reading is about experiencing, thinking, and feeling, then bringing in AI defeats the purpose of writing at all since it does not experience, think, or feel; it only imitates these, and pretty unconvincingly at that.

We can anticipate an immediate future where so much of what we read will be AI-generated. The economic implications and human costs of that need to be addressed immediately. Incidentally, UBI is the answer here, as it is for so many of our problems.

Part of this reckoning requires us to think about why human writing needs to be encouraged more than ever. AI can’t speak to the subjective day-to-day experiences of any human being, especially when it comes to people and groups whose experiences are not well-recorded online.

From a purely aesthetic view, the imagination and even the flaws in grammar and style that only people can accidentally produce will provide far more valuable guidance and companionship than the antiseptic perfection of AI writing.

In my reviews, I’ve written things that I know many music fans would find surprising or objectionable. AI avoids such conflict and unpopular opinions because it’s built on averages. As such, it never challenges, only reinforces commonplace thinking. Good writing is not flawless and does not reproduce popular thinking. I invite discussion and challenge by saying what I really think and feel about the music and by being up front about my biases and how and why my opinions change. This, AI is not programmed to do, cannot do, because it is not built to see the world in the way people do. We can’t expect it to speak the same language as us, even if it can use the same words and grammar.

If music is the language of the heart or the soul, whichever quotation you subscribe to, then it was pretty unfair of me to ask an AI that lacks either to write about music. But we ought to always ask ourselves whether a piece of writing requires a human touch or is just functional text. Keeping the difference in mind is going to be very important as we navigate the implications of AI for writing.