Thread - Nostr Hypermedia

utxo the webmaster 🧑‍💻 _@utxo.one 6 days ago

Achievement Unlocked: First quantized model and it actually fucking works

barrydeen/Qwen3.8-27B-AWQ-4bit · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Replies (23)

Don't Believe The Vibe 🌱🍋🍊 _@nostrdvm.com 6 days ago

Nice 👍

𝕞ptf mypathtofire@iris.to 6 days ago

Nice on, prob i cant run it. Why not uncensored one? My qwen3.5 uncensored rocks!

𝕞ptf mypathtofire@iris.to 6 days ago

I just have 16gb vram

1 replies ↓

Gigi dergigi.com 6 days ago

👀

1 replies ↓

51N4 mainvolume@nostr.fr13nd5.com 6 days ago

Chad’s all over the news.

Sex personer döda efter skjutning i USA

Senaste nytt • Snabba nyheter från Aftonbladet

𝕞ptf mypathtofire@iris.to 6 days ago

Yes, under 100 tok/s is painful and of they dumb too. Qwen models most usable rn with cpu /ram swapping... .. Cant wait for the next gen pcs , gonna be epic...

1 replies ↓

NotBiebs and 69 others notbiebs@primal.net 6 days ago

The one on Ollama is only 18gb

qwen3.8

Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks.

1 replies ↓

utxo the webmaster 🧑‍💻 _@utxo.one 6 days ago

Does it have MTP full precision tho

1 replies ↓

NotBiebs and 69 others notbiebs@primal.net 6 days ago

I dunno. Haven’t looked into it much or played with it much yet.

𝕞ptf mypathtofire@iris.to 6 days ago

You have a link?

utxo the webmaster 🧑‍💻 _@utxo.one 6 days ago

Here's the performance on my quant

2 replies ↓

𝕞ptf mypathtofire@iris.to 6 days ago

Nice one, was checking poster.place earlier. Looks pretty good.

𝕞ptf mypathtofire@iris.to 6 days ago

Nice work... I will benchmark my speed. If i clean the vram and do a clean load lol

1 replies ↓

𝕞ptf mypathtofire@iris.to 6 days ago

Yes i saw u could male windows and move around etc.. Had workbench 2.0 retro vibes

Geek geek@getalby.com 6 days ago

that's actually pretty impressive, will def test

zaytun zaytun@zayt.space 6 days ago

Thats badass man. Is this for your 3090s or your spark cluster?

zaytun zaytun@zayt.space 6 days ago

Congrats on the achievement. I think maybe your next step could be abliteration!

1 replies ↓

Theo Hague svrnhdl@svrnhdl.com 4 days ago

16 isn't half bad, is it? What GPU? I got a 3090 but considering adding a second card.

1 replies ↓

𝕞ptf mypathtofire@iris.to 4 days ago

I would certainly upgrade a 2nd card, but priced out atm. Im running just a 7800XT. Was super cheap when i bought it with 64gb ram, sadly ddr4 :(

1 replies ↓

Theo Hague svrnhdl@svrnhdl.com 4 days ago

What software stack are you running with it? I'm just getting started with local AI. Got a OpenWeb UI set up on top of Ollama but it feels a long way off cloud services

2 replies ↓

utxo the webmaster 🧑‍💻 _@utxo.one 4 days ago

Try Hermes

1 replies ↓

hasky hasky@primal.net 4 days ago

What you guys talked about ? Seems interesting

𝕞ptf mypathtofire@iris.to 3 days ago

Theres lots of options llama.cpp /hermes /pi/comfy... Im reaearching Picture and video models rn.