"Why is this outline so predictable?"

And other silly questions I should stop asking LLMs

I’ve been experimenting lately with LLMs as writing tools, and I keep being surprised how useful they are. I also keep being surprised how not useful they are.

From a prose standpoint, I think your average LLM is pretty useless at present. I’ve talked about this before, but the flow of your average LLM-generated scene is weird and off-putting. Most human editors would advise against interrupting every dialogue line with description, which makes it hard to sink into the scene, but that’s the LLM default. Like:

This is Claude 3 Opus, if you are the sort of person who cares about such things. Not sure how it settled on the names Lisa and Sarah

It reads as unsophisticated. And though you can iterate on this and provide more suggestions, saying things like “make it more literary” doesn’t really work, perhaps in part because “literary” is an amorphous and vibes-based term; you know it when you see it. The “improvement”:

I find the LLM much more useful as my ideas man—not to be trusted to generate something specific but to generate many possible options from which I can choose. Like so:

I continue to hope I will be able to use an LLM to help me write outlines, but I’m not yet convinced. Basically: Outlining is tedious. It takes an hour (minimum) to write a brief summary of a book, and then, only once you’ve finished, can you look at it and say, “Yeah, actually, this would not be a good story.”

It’s much more efficient to describe your premise to an LLM and tell it to write you a bullet point outline. I have by now done this dozens of times, and the result is usually the same: The beginning is fine, the climax is predictable, and the ending is saccharine. Exhibit A:

This seems fine as a beginning, though that prologue seems pretty vague

There is a 50% chance that any LLM plot climax will include viral fame. Do with that what you will

In fairness, it seems rude to be angry at a predictive algorithm for being too predictable. Also, endings are hard. Humans write bad, predictable, and saccharine endings too. So really, what the LLM is helping me do is come up with a dozen ways not to write my book. Which is useful! It’s nice to get the obvious ideas out of the way. It’s also a lot easier to be the one saying no than the one coming up with material. Plus, the LLM will not get annoyed when you say, “Frankly, this is a no good very bad idea,” which is something one should not say to their friends and colleagues.

So where do you get the good idea? If you’re aiming for something sophisticated and original, you can’t just tell your LLM to invent something sophisticated and original. The problem I’ve been having is that it is incredibly easy to trick myself into believing a brilliant idea is right around the corner if only I construct the correct series of prompts. This is a very effective way to waste a lot of time. I’m sure someone could (can/will/should) construct a model with tools specifically made for fiction writers, but at this juncture, you should probably expect to come up with all your best ideas (mostly) on your own.

I’ve found this experimentation both disappointing and heartening. It’s disappointing because it’d be neat if someone could spoon-feed me a recipe for an emotive and compelling bestselling novel. But it’s heartening because it’s nice to remember that feelings are difficult to quantify—that the question of how we appeal to human emotion is one best understood through the creative process itself.

Currently reading: The Spirit Catches You and You Fall Down by Anne Fadiman. Just started but this is fascinating so far!

Non-urgent thought of the week: Proper millennials use emojis with abandon; zillennial cuspers use emojis sparingly, as if they can recognize this is embarrassing but can’t help it; zoomers use emojis only ironically, or when on an app that automatically changes emoticons to emojis. This is the full taxonomy; fight me <3

If you liked this, consider sharing it with your favorite person. If you hated it, consider sharing it with your least favorite person.

Reply

or to participate.