I run a few models on 4090 locally. They are good enough for many tasks but there hallucination patterns are more obvious than for the larger paid models when it goes south. I mean to use langchain and vector databases on my extensive PDF library to see how much more I can get out of it.