Have you tried Ollama and Open-WebUI on NixOS yet? I just added it to my config a few days ago and some models work fine using CPU with no GPU.