First, an LLM can easily write a program to calculate the number of rs. If you ask an LLM to do this, you will get the code back.
But the website ChatGPT.com has no way of executing this code, even if it was generated.
The second explanation is how LLMs work. They work on the word (technically token, but think word) level. They don’t see letters. The AI behind it literally can only see words. The way it generates output is it starts typing words, and then guesses what word is most likely to come next. So it literally does not know how many rs are in strawberry. The impressive part is how good this “guessing what word comes next” is at answering more complex questions.
ChatGPT used to actually do this. But they removed that feature for whatever reason. Now the server that the LLM runs on doesn’t isn’t provide the LLM a Python terminal, so the LLM can’t query it
The LLM isn’t aware of its own limitations in this regard. The specific problem of getting an LLM to know what characters a token comprises has not been the focus of training. It’s a totally different kind of error than other hallucinations, it’s almost entirely orthogonal, but other hallucinations are much more important to solve, whereas being able to count the number of letters in a word or add numbers together is not very important, since as you point out, there are already programs that can do that.
At the moment, you can compare this perhaps to the Paris in the the Spring illusion. Why don’t people know to double-check the number of 'the’s in a sentence? They could just use their fingers to block out adjacent words and read each word in isolation. They must be idiots and we shouldn’t trust humans in any domain.
The most convincing arguments that llms are like humans aren’t that llm’s are good, but that humans are just unrefrigerated meat and personhood is a delusion.
It’s a statistical model. It cannot synthesize information or problem solve, only show you a rough average of it’s library of inputs graphed by proximity to your input.
Congrats, you’ve discovered reductionism. The human brain also doesn’t know things, as it’s composed of electrical synapses made of molecules that obey the laws of physics and direct one’s mouth to make words in response to signals that come from the ears.
Not saying LLMs don’t know things, but your argument as to why they don’t know things has no merit.
Why doesn’t the LLM know to write (and run) a program to calculate the number of characters?
I feel like I’m missing something fundamental.
You didn’t get good answers so I’ll explain.
First, an LLM can easily write a program to calculate the number of
rs. If you ask an LLM to do this, you will get the code back.But the website ChatGPT.com has no way of executing this code, even if it was generated.
The second explanation is how LLMs work. They work on the word (technically token, but think word) level. They don’t see letters. The AI behind it literally can only see words. The way it generates output is it starts typing words, and then guesses what word is most likely to come next. So it literally does not know how many
rs are in strawberry. The impressive part is how good this “guessing what word comes next” is at answering more complex questions.But why can’t “query the python terminal” be trained into the LLM. It just needs some UI training.
ChatGPT used to actually do this. But they removed that feature for whatever reason. Now the server that the LLM runs on doesn’t isn’t provide the LLM a Python terminal, so the LLM can’t query it
The LLM isn’t aware of its own limitations in this regard. The specific problem of getting an LLM to know what characters a token comprises has not been the focus of training. It’s a totally different kind of error than other hallucinations, it’s almost entirely orthogonal, but other hallucinations are much more important to solve, whereas being able to count the number of letters in a word or add numbers together is not very important, since as you point out, there are already programs that can do that.
At the moment, you can compare this perhaps to the Paris in the the Spring illusion. Why don’t people know to double-check the number of 'the’s in a sentence? They could just use their fingers to block out adjacent words and read each word in isolation. They must be idiots and we shouldn’t trust humans in any domain.
The most convincing arguments that llms are like humans aren’t that llm’s are good, but that humans are just unrefrigerated meat and personhood is a delusion.
This might well be true yeah. But that’s still good news for AI companies who want to replace humans – bar’s lower than they thought.
And why we should fight them tooth and nail, yes.
They’re not just replacing us, they’re making us suck more so it’s an easy sell.
Well yeah. You’re preaching to the choir lol.
It doesn’t know things.
It’s a statistical model. It cannot synthesize information or problem solve, only show you a rough average of it’s library of inputs graphed by proximity to your input.
Congrats, you’ve discovered reductionism. The human brain also doesn’t know things, as it’s composed of electrical synapses made of molecules that obey the laws of physics and direct one’s mouth to make words in response to signals that come from the ears.
Not saying LLMs don’t know things, but your argument as to why they don’t know things has no merit.
Oh, that’s why everything else you said seemed a bit off.
sorry, I only have a regular brain, haven’t updated to the metaphysical edition :/