i'm sure that if it wasn't for the fact that the developers of the majority of LLMs in use are developed a bunch of uptight, narcissistic pieces of shit showboats, who stamp their ego into the models, that actually, normal human beings would build models that make typos sometimes.
they spend an inordinate amount of time "cleaning up" their training material, but didn't consider that "dirty" results might on net be better for users.
the problem has to do with the fact that even at 70b parameters, an LLM model, from any of the various architectures being used now, is actually a pittance compared to the number of parameters that our wetware can deploy.
they have to "clean" the training data because the more orthogonal the direction of content is, that is, the *focus* of the data set, does not fit into the parameters, and then you will get a lot of hallucinations although when it isn't tripping it will speak more direct and humanly to you.
you can even see that quite quickly by comparing gpt to claude to grok. grok is a big model, but it has less restrictive safety rules, and probably has a more diverse training set.
because grok has a broader input dataset, it produces more focused results and doesn't try to railroad you into its "safety" framework.
it will just take time, because literally these models take weeks, months, to train, and the big cloud providers are deliberately inflating their financial records in order to maintain their dominance, against people who can't muster a giant loan for a whole fucking data center of acres of GPUs.
"we can stay autistic longer than they can stay solvent" has relevance to non-establishment AI devs as it does to bitcoin.