Cool, I like the design.
Why GPT-2 though? Did you see my example repo?
View quoted note →
I have 3 or 4 models people can pick from, ranging from a super-tiny CPU model that produces near-garbage (even worse than your GPT-2 I'd say) up to something needing a moderate chunk of RAM (12GB) that's still fast *enough*; but you probably wouldn't want to load it in a webpage. Or .. well, I guess there's some technical question about how you could have it pre-loaded and work for any client making a request. Huh, not sure.
Login to reply
Replies (2)
Hmm on reflection though, your "setup" (design?) is interesting: it's obviously not theoretically pure, but it's much closer to practical today: if the web server could serve as I was waffling about above, somehow, a very high quality model, it could let an anon client use it to generate a message, pass a link to the recipient ("pass a link" means also knowing the secret key, that's already in the design so no loss), who could then anonymously go and decrypt the message from the LLM output they see... but no, this kind of setup is veering towards the "why are you doing all this instead of just passing the secret message to your recipient at the start". Meteor's main value is if you have a long series of messages you want to pass, over time, to someone you don't want to risk talking to directly more than the initial step. That's where the "receiver needs to keep their copy of the LLM" comes in.
We both decided to implement! I had the llm do a python script first, and then wanted something to run on a html page that was easy for people to see as a demo. I just picked a model to run the test. Didn't put too much thought into it.