It doesn't matter how many variables you have, you can't improve the modelling of a quadratic function with more than 3. You also don't improve your model with more data points past some amount.
This analogy describes some of my skepticism of the claim that we will create super intelligence by merely adding more data, more parameters, and more compute to LLMs.
Yes we will continue to see improvements in the ability to model human knowledge and reasoning, and yes it will be amplified by machine speed, but I don't think we somehow unlock a higher level of intelligence. We will plateau at some point and then the focus will shift more to applications of this still impressive but sub-human intelligence.
Login to reply