what the hopping fucking sideways magical tomfoolery genius is this?!?
arkinox's avatar arkinox
I want to show you why nostr relays and clients should support partial event ID queries with a cool new thing. send or come yes our like Paste these 6 words into https://approx.vercel.app and you will retrieve this note: View quoted note → I've mapped the 256 shortest and most common words in the English language to a unique byte. You can use a short string of innocuous looking words to link to any nostr event, ie, any data you want. Partial event queries have classically been part of the nostr protocol, but relay implementations have been enforcing limits on how short an event id query may be, with most relays requiring the full 64 hex characters. I am hoping that relay devs (and client devs) will reconsider this artificial limit when considering how useful partial queries can be. Uses: - Representing nostr events with ~6 words can enable you to link to a nostr event in adversarial environments where known nostr URLs are shadowbanned and long hex strings would raise suspicion. (I got banned from Github for putting my pubkey in my bio once, for example) - sharing ~6 words is shorter than sharing a full id/nevent/note1 - You can remember ~6 words to memorize a nostr event id - As most events will never collide after 6 or 7 bytes, partial queries are a powerful shorthand for data portability. Imagine being able to represent everything on the internet with only 7 bytes. You could literally memorize how to retrieve any data on the internet! I think this is too much of an opportunity to waste. Nostr has this unique advantage of making a universe of data accessible with minimal effort. Let's not suffocate it with arbitrary limits! Relay implementors, please let me know your feedback on this if you care to comment. Caveats: - Will 6 words ever represent more than 1 nostr event? Yes, it can happen. So then you use 7 or 8 words to be more specific. Git and Github use 7 bytes to refer to commits because it's so exceedingly rare that they ever repeat. In the case that there is a collision, picking between the 2 events retrieved is likely not a problem. - It's not as convenient to refer to an event with words as it is to link to an event, but links can be grepped and banned on centralized platforms. - This word list probably sucks, I made it in less than an hour, but there aren't any lists of 256 words that I could find that were specifically engineered to appear reasonably innocuous and likely evade algorithmic detection. BIP-39 uses 2048 words for 11 bits each which is too uneven for conveying an arbitrary number of bytes. Also, the BIP-39 words are not very subtle. Anyways, I kindly ask relay implementors to consider supporting partial event id queries so that this sort of powerful feature can remain a core part of the nostr protocol. Brought to you by @npub16lvp...7j3k
View quoted note →

Replies (54)

As said, above my head... But from my grug-brain understanding, it should only take 7-8 characters of a Nostr event ID before the chances that there is another event with the same leading characters is slim to none. That being the case, a Nostr client doesn't really need the whole thing in order to pull up the event, so long as it is searching the right relays for it. If the full event ID is `nevent1qvzqqqqqqypzq6z80pwapdkhn6st5d5uxfm5atcl82jzyhdwmurs7q656felnj6xqy2hwumn8ghj7un9d3shjtnyv9kh2uewd9hj7qgswaehxw309ahx7um5wghx6mmd9uqzqc8ztkrnk6rnwl6dgznhtvxf6fuhpkja3y2tkuqpvl6uaxekycz5xdhvag,` then clients should only need `1qvzqqqq` to find that specific event, and those characters could be converted to mnemonic words that can be memorized or otherwise hidden in a message. Nostr clients could even support looking up the event using those mnemonic words.
LOL! Note IDs, nevents, naddrs, and hex IDs... So many to try and keep straight! I think I've got it now, though. So you are proposing making a mnemonic phrase out of the first 6-8 characters of the HEX ID of, well, anything you want to, right? Could be the hex version of a user's npub, the hex version of a bookmark list's ID, or the hex version of a book ID published to Alexandria, and as long as the Nostr client knows where to look for it, since relay hints aren't encoded into the hex, only the nevent or naddr, the client should be able to find it with just that mnemonic phrase. And hex IDs are completely different from nevents and such. You can't just take the "nevent" prefix off of the front and what you are left with is the hex ID. It's a completely different set of characters. Jumble is kind enough to allow users to see the raw event information if they want to, and I believe the hex ID of your reply is: `a50bc29d3377e1b0f50a0f471ca7431d27489f013df620a90e313c5f10b1c2a9` While the nevent is: `nevent1qvzqqqqqqypzp68dx7vvdlltl7sg2qdv8838ze3tl5tq76y0jnz966fdsana6dz6qyt8wumn8ghj7mn0wd68yetvd96x2uewdaexwtcpr9mhxue69uhhwmm59e38y6t8dp6xymmvwshxuet59uqzpfgtc2wnxalpkr6s5r68rjn5x8f8fz0sz00kyz5suvfutugtrs4ffsug0m` But we could have clients display an option to copy an "event seed" or some similar title for it, which is a mnemonic phrase of human readable words that represent the first 6-8 characters of the hex ID. So, the question becomes, how will clients know where to find the referenced event without relay hints? Do we have "indexer" relays that just record hex IDs and what relays they were seen on, so clients can reach out to the indexer to tell them where to look for the event? Or do we use a DHT for that? Or do we make the phrase 12 words, with the first 6 being a mnemonic of the author's pubkey and the last 6 the mnemonic of the event, and then the client can just check the author's kind 10002 relay list to see where to look for the event?
Good questions! I like the "event seed" option you mention. Without relay hints, it's just a game of brute force searching relays. But perhaps we could encode the relay somehow. TBH that's a bit trickier. The primary ID of a relay is its URL. Encoding all of it would take too many words. But expressing the relay URL in plaintext would betray the purpose of the preceding 6 words, which would reduce this scheme's effectiveness against shadowbanning. Perhaps it's up to the user. Include the relay if you aren't in an adversarial environment, and exclude it if you are; without a relay hint you'll simply have to brute force search relays for it until you find it, which can be somewhat automated. If you are sharing an event in such a furtive way, you might want to broadcast your event to as many relays as you can in order to ease the pain of finding it.
That makes sense. Tools like NADAR exist already for searching all known relays for a particular event, and yes, an event you want to share in this way could be intentionally blasted far and wide to assist in finding it with just the first several characters of the hex. Runs against the philosophy of outbox, though. I think maybe sharing a 6 or 7 word string of the author's hex pubkey along with 6 or 7 words of the event's hex ID would probably do the trick, and be more in line with outbox, though. A user could pretty quickly commit to memory the phrase referring to their public key, so they would just need to add the phrase representing the particular event they want to share. Clients could then find the author's kind 10002, which should already be blasted far and wide in the relays, and look for the event itself on the author's listed outbox relays. I feel like this could help diminish the chances of collision as well. How likely is it there will be two events with the same author and the same leading 6 characters for their hex ID?
Ah, dev-speak. I am not there yet... Maybe one day, one step at a time, I will understand all that. I impress @ hodlbod often with the seeming contradiction of my depth of understanding of the protocol, somehow paired with my complete ignorance of how to write a single line of useful code.
We're all somewhere on the journey. I'm happy to answer questions any time if I happen to be further along a particular branch of the path. In case it's interesting to you, what I meant in non-dev speak is: Say I have some new Event Kind I want to introduce. I have in mind a handful of ways the kind might be used in an **eventual** app. I want to check that the design of the kind I have in mind will support those features. nak is basically a bare-bones nostr client - simple requests and simple responses all in text only - so if I have a relay running that understands my new Kind, I can use nak to make sure that the kind of requests I would send to the relay do what I hope. Since it's a CLI tool, once I have a little collection of interactions tested out in nak, I can basically write those commands into a script (with a bit of extra diagnostic printout, etc.) and now when I run that script it either goes well: "okay great, the Kind makes sense, the relay handles it properly, and the user interactions work like I thought" or things fail and I can look into it further. That "happy path" (the minimal actions a user might take in order to basically use the new features. not yet taking into account errors or weird edge cases, etc.) then acts as a kind of proof and assurance that "upgrading" those features into a client app UI is worthwhile and will work well. In a nutshell: that "prototype with nak first" approach is like 1,000x faster to whip up than going straight from Idea -> UI. And sometimes you falsify some hypothesis you had really early and don't waste your time determining which color your buttons should be :)
That could work, if the place he shared it doesn't censor NIP-05s. They are domain based and easy to figure out what they are for, other than the fact that they look like an email.
Alright, say I want to make myself useful in a Nostr building context. Given my understanding of the protocol, where should I start in learning how to build on it?
I'm game. Just need to get my account sorted with GitHub. It was flagged, likely due to signing up with an email alias instead of my actual email... Then figure out how to add a PR. 😂
the only thing I can think of is that there is an ambiguity, where for example 66675 could be the start of a pubkey, the end of a pubkey or the middle. to me the idea of partial searches is a good one regardless. implementation wise, it does add some complication depending on the database engine the relay you're writing uses, but not that much.
I think it would be reasonable to limit these queries to prefix only. No searching the middle of a string for a partial hex value.
...I guess you could post a note with one of these "seed phrases" to another note in plain text and then reference *this* note by phrase elsewhere - you could nest them arbitrarily. pointers to pointers. which seems pointless (mind the pun) at first, but could be an interesting primitive for like AR games and other stuff.
It's just unnecessary complexity. REQ filters are already too powerful, they should be simpler. The use cases are marginal and never important, often just "cool". They can all be implemented using custom relays with crazy logic, which is what we should be doing anyway for many other things.
Technically can't find an event even if you have the full ID anyway, because you don't know the relays the event is in. That's why we have nevent1 codes and relay and author hints in tags. Choosing the relays for each client action is the most important part of Nostr but people have assumed that part away way too many times by hardcoding popular relays and expecting all events to be there, which works as long as we're ridiculously small, but kills the value proposition of Nostr entirely. It's like assuming Bitcoin will always be zero: it's only true if the entire project is a failure, and in that case nothing matters anyway.
That was a major part of the reason I made the suggestion regarding partial pubkey queries. If you can query for a user's 10002 with just the first 6 characters of their pubkey, you can find their outbox relays where the event is likely stored, since 10002 events are intended to be shared far and wide.
That said, this could indeed be done with a custom relay. One that pulls in 10002 events from all known relays and allows for a query of just the first several characters of a pubkey. There is no need expect all relays to have this behavior. From there the client will have the relays it can expect to find the event, and relays DO currently support partial event ID queries, right @arkinox?
I didn't mean custom implementation for this purpose, I think you can probably do better with a custom relay implementation. I mean, if it's already custom you can assign a serial number to each event and map those numbers to real events in a custom database. Then convert those numbers into single or double-word mnemonics for users. Or is that boring, doesn't contain cryptographic tricks, centralized? Maybe it is, but I think boring is good and centralized is fine as long as it plugs into the rest of the Nostr ecosystem.