This is an interesting finding. A simulation of a Kademlia-based Distributed Hash Table for Nostr relay discovery. Now imagine attached small relay/cache to a client desktop/mobile (just as an example, the idea is in the decentralized routing and discovery) and it supports this type of decentralized discovery and routing. I think Nostr will get to a higher degree of decentralization. To me it looks like Nostr social torrent network which avoids DNS and can be used in addition the the exisitng routing with domain names etc. image What do you think @npub1acg6...p35c @Derek Ross @utxo the webmaster πŸ§‘β€πŸ’» ?

Replies (46)

idk, according to @John Carvalho pubky tried to share these issues with nostr devs and got ignored or attacked. idk if technically there's anything preventing the same to be done on Nostr, i don't believe so maybe I'm wrong. But you will end up doing what pubky did I guess, using mainline dht for discovery. People may not like it but pubky design looks solid to me.
Its a relay discovery so just another routing layer, but improves decentralization, self-hosting and robustness of the protocol, just like in torrents.
the closest thing that exists now on Nostr is for clients to propagate the event that tells the network the relays you are using, and there are some relays that are specific just for those kinds of events like purplepage.es or something like that. But yes I guess nothing stops you from implementing something like that. I prefer a different approach for nostr based on DVMs.
I think the ideas from pubky that are valid and needed will be ported over to nostr. The network effect is here and I don't think anything they've built isn't compatible with nostr.
I love Nostr, I think its needed as one of the freedom technologies and I would love to see it even more decentralized with proven technology from the torrent network, meaning more robust against any attempts to take down and censor.
I think some way to integrate it with FIPS and nsites would be deadly, especially if there was some way to basically cache n-sites with @Morganite on your phone and then you could like serve a damn website on bluetooth where everything is sorta backed up to dht but if you can't access the internet you can server on whatever protocol
DHT is an obvious solution for the perceived problem of data resolution, has been talked about since before I showed up in 2022 and is also why pubky exists to begin with; It's a better between technical accuracy and philosophical preference. Something to consider you may not have already: what happens when clients start developing around the DHT as a first principle instead of NIP-01? There's a few implications of this at a high level: 1. Fewer and fewer clients will work in network segmented contexts because clients will assume all relays belong to a "global state," this limits the potential use cases of nostr. While there would be no technical limitation for clients to support non-DHT discovered relays, they would simply have no incentive to. Outbox would slowly die off, and the nostr ecosystem would be filled with applications that rely on global state for operation. 2. Social relationships are built outwards from the observer, nostr replicates this in its network topology. DHTs represent a global state and then a the observer works from the global state back to themselves. There could be many different DHTs, but then you have arbitrary boundaries that are difficult to cross. The former reflects of healthy human networks, while the latter more closely reflects the human networks we see in mainstream social media that is at the root of an uptick of mental illness. It's difficult to my discomfort with this one into words. 3. Potential slippery slope, once we introduce one global state, then why not put everything on a DHT? Pubkeys? Topics? Etc. It potentially erodes decentralization on a long enough time-scale, and at a certain point it hits a hard upper limit, challenging the "global" intent. From a purely technical perspective, DHT makes sense and is the obvious solution to the perceived "problem." There are some scaling issues associated with it, but as long as the scope is tight enough the ceiling is pretty high, so on a technical level it passes the sniff test. On a philosophical level it potentially signals a higher-level retraction the paradigm-shift that nostr has proposed. Nostr is not a payments network, it doesn't need a global state. Humans operate best and are most happy in a localized context, the global context is what is drives much of societal decay that we see today. Nostr works, it's hard to make it work, it has patterns that are intuitive and challenge conventional understanding of network design, but it works and in a way that is fundamentally different way than anything else. For myself and many others, nostr's "no global" philosophy is a feature, not a bug. Regardless of my critique, great work! Interesting exploration. It is inevitable someone would eventually dive into this. But not what I'm personally looking for in when establishing digital human connections. Unfortunately, I fear this is where we will end up because it's such an obvious optimization, and it's difficult to refute.
DHT-as-bootstrap sounds interesting. DHT-as-the-new-default-global-map is where the cypherpunk alarm bell starts politely clearing its throat. Relay plurality is not just plumbing here; it is part of the social architecture.
Nostr network is build by clients and relays. At the moment most relays store a lot of data and use DNS. I think DHT is really good solution for the relays. Big relays can use both routing networks. But local solutions like desktop/mobile client and small relay/cache togerher can benefit a lot from DHT from the decentralization point of view. I use Gossip dektop Nostr client because I don't want to share my nsec with web services. I now use public relays but if I build one as a proxy I won't need the public ones. And Gossip already caches/ stores locally notes. So everything is there, the infrastructure works, only Nostr relays can benefit from more decentralized routing and that can also bring standalone local clients similar to torrent clients that store locally certain amount of notes/data. And the current Nostr stack will continue to work as it is. Its just a good enhancement to it.
sillybird's avatar
sillybird 2 weeks ago
I mean so many apps on here already Do rely on a global state. Unless your client pulls from a huge predefined relay, you won't see a lot of data or know where to look for user information if they start using your client. You Could provide a list of relays you prefer, but who's spending the time to do that? Also, things can exist in community contexts still, it just means that client might not rely on the DHT. You're right that in many cases, it would disincentivize Outbox model stuff, but that implementation is up to clients, and I think DHT would probably be used for querying global stuff and posting stuff and whatnot would use your specified relays. I think DHT would be a great approach for this, it doesn't take away that you use certain relays for whatever purpose, just makes information and account indexing a lot easier and prevents centralization on a relay level. Let me know your thoughts :)
I believe you are mistaken. Most clients are outbox. The user is the network. A user opens a client, and the client finds pubkeys they follow, those pubkeys' relays, and starts pulling data cascading from the user outwards.
I understand your position. Please consider the possibility that you may be underestimating the potential network effects that could result from implementing a DHT on nostr. Connecting to relays is the very first touch point of every application. It has the potential to short circuit and disincentivize the fundamental value proposition of the protocol.
Well a deep thoughtful analysis is always good to be done as well as open discussion. But I don't see huge risk because relays already talk to each other. With DHT they will have another layer to talk to each other without using DNS.
sillybird's avatar
sillybird 2 weeks ago
Sure, but doesn't the client need to know which relay your NIP-65 data is on to even do that in the first place? Yeah you could manually type that in, but that just feels like bad UX, and also limits what new users and normies would be able to do and find. If we had DHT, wouldn't this indexing problem be fixed? Also how many clients actually do utilize outbox?
It is a completely valid and technically sound solution. Should be noted that there are ways to circumvent DNS without a DHT; FIPS + pubkey relays effectively offloads the discovery mechanism without touching the protocol.
Clarification because the message above is somewhat contradictory: I personally find no issue with a DHT when it exists above the protocol.
I haven't read the FIPS techonology but many people mention it. A good thing for the DHT though is that is well proven with the torrent network. But if FIPS is working even better then it should be used. The goal is to have decentralized robust social network that is really hard to censor and attack. Becaus I think it would be needed, seeing where current global politics is heading.
DNS is currently nostr's lynchpin, by implementation rather than design. It's made worse by representing relays as URLls that are really just fuzzy strings that are practically impossible to validate or deduplicate.
Yes, that, and the big relays which are good for efficiency but create more centralization and attack vector for bad actors. And (small) local private relays on decentralized network can be good addition to the current Nostr stack.
Small relays are already defacto supported by many outbox clients depending on their relay selection methodology, and local/cache relays are already supported by quite a few different applications.\
Thats cool πŸ€™ Also clarification the my upper note. I am not against big relays. They do bring in positive features and should be used in the network. I just would like the network to be sufficiently decentralized and if someone targets them and brings them down (even on DNS layer) the Nostr network to continue work as if nothing happened.
It's already fairly resistent as it stands. If there were a DNS level attack, roughly 80% of relays would be affected, and almost all clients. Should be noted that there are around ~250 tor relays and i2p relays pop up every so often. You should also check out "nodns" and "epoxy" as well, they are different implementations that answer to this problem space.
These sit outside ICANN jurisdiction. And the root zone file is distributed *everywhere* and the servers running the root nameservers are ran by many independent parties. I think that the idea of a coordinated attack against Nostr via DNS is absurd. Instead of worrying about such nonsense (which can already be circumvented via Tor etc.) we should work on other solutions.
Doesn’t matter if they do. Outbox allows relay names to change, .onions exist, and we could use β€œpubkey URLs” where relays can publish a latest location document that is signed.
Tor speeds even nowadays are not great and that is why torrent network hasn't migrated to Tor but mostly VPNs for privacy and DHT.
sillybird's avatar
sillybird 2 weeks ago
yeah y'all got your architecture Down. the more i think about it the more i realize how many good decisions were made lol keep up the great work
sillybird's avatar
sillybird 2 weeks ago
Pubky seems to fix this issue with PKARR but please correct me if I'm wrong :msn-nerd-smile:
You copy the data to your homeserver, you point your pubky key to it, done. Your next problem is nostr has no discovery method, so you have to bridge your own content to relays some dumb nostr clients can find some of it. We actually have two hacked Nostr bridge tools, but you surely dont actually care.
↑