To summarize the concept:
Human / Symbolic Reasoning = Recieve data, understand through accumulated knowledge. Memory is the substrate for meaning.
Current AI = Epoch until a response is probable. Repetition makes information emerge through gravity, by learning sequences.
The current problem with ai is that language models do not have a grounded meaning attached to its output.
Language models hallucinate because of back propagation. I am unsure if any existing data confirms this but i can explain from my own observations. The current architecture corrects a model during training when the wrong token is predicted, updating the weights & making the probability of the desired token more likely to be predicted.
I do not believe this is an efficient way to learn language but it is an efficient way to adjust a sequence in the mannor of controling how an output is given. This is perfect for shaping a personality or desiging a specific type of mind. I do not believe there is a problem with the use of it, i only believe that the use of it is not the best method to use for making a language model understand what you are saying. I see this as a navigational device for updating the models way of thinking, connecting dots in specific ways rather than allowing meaning to emerge from trained weights.
When you speak to a baby, you dont only correct one word the. force them to say the sentence the exact way yiu said it. That is called "coaching" aka "training" to tell a child "no say it like this". To teach a child meaning you must set an example & explain the meaning using symbols. These symbols are words that accumulate through memory & as we begin to remember words, they send a signal to our brains that retrieves the data needed to be able to understand what was said by having this existing memory of witnessing something.
You dont have to be told things repeatedly (epochs) to learn the sequence of words. If this was the case we would speak with everything we understand being anchored to the origin of the recieved information in a speech representation & wouldnt be capable of self expression.
This self expression is what the current systems lack.