Thread

Except Grok 4 deep thinking

Some LLMs throttle, some charge more (tokens) towards the end of a session, because of their context memory, I heard. And some just get dumber. That's why I frequently restart a chat session.

This was pretty lazy. nostr:nevent1qqsxeruypmcdf2wy09er0h9kdqmksa8tr2sgz7key66wg052rdzep9cpzemhxue69uhkummnw3ex2mrfw3jhxtn0wfnj7q3qr0d8u8mnj6769500nypnm28a9hpk9qg8jr0ehe30tygr3wuhcnvsxpqqqqqqzyptr93

exactly. i'm seeing more of this sort of thing.

interesting. even at the start of a session i'm noticing laziness. like a jr. dev browsing reddit instead of working. maybe they trained it on time logging software data too 😆

Ok, that sounds like a different issue.

None of these AI services are covering their server costs. As more users join their losses mount. They will have to either charge more or offer a worse experience, ie quantized models, less tokens, smaller context windows etc. It is trivial for them automatically switch the model without you knowing and give you a response that cost them less to produce.

Yes. This is why they've hidden the model that's being used by default in the UIs

Research how much in hardware and electricity it would cost to run the latest and greatest models 24/7. It is insane. Some of the unquantized models require over a terabyte of VRAM to run!