Just a non-technical aspect worth mentioning: there could be a remarkable *cultural* or even *artistic* component to this idea. Consider this: randomness and creativity are connected; if you use no randomness and follow a "paint by numbers" algorithm approach to creating something, it will usually be perceived as flat, stale. At the same time, it's deterministic, reliable. The way this can manifest in LLMs is that you can choose to make them completely deterministic, but their output will be boring and less useful, in many scenarios, because of that [1]. So the existence of "subliminal/covert channels" in the data streams that we will consume in future may be intimately connected with the creativity that is allowed. It's a very profound demonstration of the link between freedom and creativity. I may be overreaching but this connection could be profoundly important politically in the future. [1] as noted, there *are* benefits as well as disadvantages to what's called "greedy decoding" i.e. always pick the most likely token; for some tasks this works OK. But fascinatingly there are pathologies: a never-varying choice algorithm can lead to *infinite loops*. For anyone who studied nonlinear optimization techniques in the context of engineering, this might ring a bell: local minima can function as traps for search algorithms. Basically, a chatbot that has no randomness is too simple to navigate the noisiness of reality, unless the reality it's navigating is very clean and simple. View quoted note →