Michael C Hogan

Agile Product Development & Innovation Strategy


How does ChatGPT write?

Current LLMs don’t “understand”. They guess based on probabilities. The guesses sound statistically human. Sometimes the guesses are useful; sometimes they’re wrong (“hallucinated”). It often seems like more effort to verify LLM output than to just do the work from scratch. Output seems challenging to reproduce.

Here’s an overview by Steven Wolfram via Dave Winer’s RSS feed (Dave created RSS)…

(Source: stephenwolfram.com)

And the remarkable thing is that when ChatGPT does something like write an essay what it’s essentially doing is just asking over and over again “given the text so far, what should the next word be?”—and each time adding a word. (More precisely, as I’ll explain, it’s adding a “token”, which could be just a part of a word, which is why it can sometimes “make up new words”.)


Read the rest at https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/.