Currently tinkering with local LLMs like a real psycho.
Any suggestion?
Login to reply
Replies (9)
the qwen 3.6 moe if you haven't tried it. probably a popular ablit like HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Unfortunately, it seems too heavy for my modest HW (8GB of VRAM)..
the popular ablit that size is called (are you ready for this?) DavidAU/Qwen3.5-9B-The-Defiant-Fable-Uncensored-Heretic-NEO-IMATRIX-MAX-MTP-GGUF
And we still wonder why people are scared about this stuff 🤣
Thanks, I'll take a look!
right? 😂 there's a perfectly good Qwen/Qwen3.5-9B as well. personally i find that running modified models is half the fun
I'm currently testing stuff with the Qwen/Qwen3.5-4B model and I'm planning to move to the 9B!
I want to have fun, I'll move to those models later ahahah
wait ... some people are running the 35B MoE on 8GB VRAM 🤣 https://www.reddit.com/r/LocalLLaMA/comments/1vx4k9g/whats_the_best_local_model_youve_found_for_8_gb
I thought this was literally joking around, but no it is just how this stuff works. That is nearly incomprehensible to me. Is there a LHotP(Littlehouseontheprairie)-Columbo model? Something a bit more wholesome but serious about problem solving? Whoa. 👀


ikr! you can of course run Qwen/Qwen3.5-9B. the reason the "abliterated" models are so popular is because they perform what amounts to brain surgery in order to remove onerous safety refusals. whether that's what you want comes down to personal taste, but the fact that it can be done and provides capabilities that frontier models effectively don't have makes it a very interesting option