Prompt Poetry
I subscribe to Andrew Ng’s weekly newsletter called The Batch. Andrew is one of the world’s leading authorities on AI, and he’s a wonderful educator too. His AI courses on Coursera are legendary.
In last week’s newsletter, Andrew talked about Sora, a new AI video generation tool from OpenAI. Some of the examples are simply amazing. And though the generated videos are short (a minute or less), they “raise the bar for detail and realism” as Andrew puts it. Rather than generate a single image, Sora produces high quality videos from text prompts. We have now entered the world of text-to-video models.
While the videos are indeed exceptional, the prompts are equally fascinating. In fact, they highlight an interesting phenomenon. Consider, for example, the prompt that generated the video of a woman walking in Tokyo:
A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
While you read this, did you notice how concise and evocative the writing was? The description of the woman’s dress and accessories is vivid, her manner of movement sharply articulated. And though it lacks a rhyming scheme, the prompt has the feel, dare I say, of a sonnet. And how about those compact sentences? Hemingway would be proud.
All this leads to a rather provocative thought. Does the future belong to the poets and short-story writers amongst us? Clearly, the ability to say a lot with just a few well-chosen words is going to be a valuable skill in this new world. In fact, it appears that those who possess vivid imaginations, love words, and find great joy crafting tight sentences will thrive and prosper on generative AI.
Poets have been doing this kind of work for millennia. Poetry’s delight lies in its ability to evoke mental images (beautiful and otherwise) with just a few words. The words are what fuel the mind’s imaginative faculties. Only now – with the help of generative AI – we can visualize evocative text almost instantly.
How might poets and writers take advantage of this new capability? Well, AI adds a new step to the write - rewrite cycle. Now it can be: write - visualize - rewrite. Stated simply, creatives can now quickly see how their choice of words alter the visual world they seek to create in the minds of their readers. Obviously, generative AI can never perfectly replicate the imaginative world an individual reader creates while reading a poem or story. Each human being is unique, made up of a richly nuanced and layered set of embodied experiences. And while reading, the brain uses that knowledge to visualize or make sense of a particular text.
Generative AI models, on the other hand, transform prompts into imaginative images or videos in a way that is far different from how our brains simulate a world, a place, or a scene. Even so, they offer us a baseline of shared experience derived from billions of written documents and images. In other words, their visual outputs are a fair approximation of the average person’s mental constructs.
In summary, concise, sharp, clear writing is more important than ever. Who could have ever imagined an outcome of this sort? Only a poet!