You're forced to use way smaller models because you don't have the necessary RAM/VRAM (basically very fast memory) to store the largest model after it is read from your (much slower) hard drive. So the model has less parameters (basically a lesser amount of knowledge) and won't perform as well, especially for more complex tasks. It's also likely to be slower. You can download updated versions of the model with ollama once they're released.
Login to reply
Replies (1)
Oh and I recommend a web interface like openweb-ui that is easy to use. There are also many Desktop clients. Tell me your OS and I'll show you some options.