jb55's avatar
jb55 _@jb55.com 8 months ago
damus will stop working when you follow around 1800+ people. the only thing I can think of is some algorithm that looks at all the people who haven't posted in awhile and only probabilistically include them in the filter. The relay pool model (damus) straight up breaks when you follow too many people. The proper fix is have people spread across different relays (outbox), then you can break apart the large query into smaller ones.

Replies (51)

casey's avatar
casey 8 months ago
Any chance we could do like a Poisson distribution to determine the probability and filter those out? Or am I trying to way over complicate this?
jb55's avatar
jb55 _@jb55.com 8 months ago
We’ll just get outbox working and then we’ll see if that helps. @Daniel D’Aquino has been working on getting nostrdb updated and the local relay model working, so hopefully we can start getting that going soon
jb55's avatar
jb55 _@jb55.com 8 months ago
they likely crap out because we do some extra logic: we break large queries in half to get around relay limits, but at some point even that is too much (~2000) we need to start getting smarter. I like the interaction algo idea I talked about a week ago. its like a dynamic contact list. we also need a tool to prune contact lists when they get too big, and a warning for users. at some point even outbox won't be able to handle 50,000+ follows, so we may have to enforce limits in the app...
casey's avatar
casey 8 months ago
Keep rockin guys. Most of this stuff is over my head, but I do work in forecasting and supply chain. Just trying to think about how to apply some of that to the Nostr world.
I guess I can say 150 associations (one interaction per year or a forced interaction like co-workers), but that's the extent of the relationship. I'm probably under 25 significant relationships.
:P's avatar
:P 8 months ago
What about just better relay filtering? Follow lists could be their own relay and could be broken into multiple lists/ relays. Would be awesome if damus ran its own instance of relay.tools and just started spinning up relays left and right.
Seeing nostr facing growing pains kicks like a Bitcoin block size Déjà-vu I'm alright with any algos that make the experience better as long as everyone knows exactly what an algo does and- in some cases - can be enabled/disabled by the user (content filtering algos for example). Algos aren't bad, they're just being used in bad ways elsewhere. Pruning is also a good habit. @YEGHRO made a tool for that and I use it every couple of months to prune dead npubs. Definitely a great tool I'd love to see natively in apps. Keep doing what you're doing man. You'll figure it out! 🍻
I would love a "last time I interacted with" tool that showed dates # numbers of interactions with other npubs.
jb55's avatar
jb55 _@jb55.com 8 months ago
We’re actually doing that now, but for some reason there are still issue we are trying to figure out. At some point it becomes a bandwidth issue regardless
jb55's avatar
jb55 _@jb55.com 8 months ago
Back in 2022 i had the idea of a custom query you would send to relays that was basically “give me all the notes whos authors are on my kind 3” no need to send your entire contact list each time, but its very ad hoc
This might be the current situation, but it doesn't seem fundamental to me. If there's a way to reliably reproduce these issues it should be straightforward to optimize
Sounds like a reason for people to repost, since folks can't follow everyone
Maybe this has changed but I remember seeing a clear difference when going lower than 1K. Suddenly notes weren’t being rejected by relays. This was many others experience as well
Hey jb. Tagged you in a note other day that's related. For some reason the latest people I try to follow get automatically unfollowed a day or so later. Similarly, unfollowing people seems to get undone too. Not trying to harp on my pet issue, just in case you know my issue or want to make a note. Thanks #damus
Maybe bin them by last seen, then check more recent bins more frequently. This ensures that people that are only on a couple times a week get almost the same priority as those who are on daily. What happens if you follow 2k people who post all the time? What about 10k? Damas won't have this problem, but what about specialized clients that follow 100k+? We might need to add separate commands to the protocol to upload filter sets that can be referenced in queries. Or maybe a way to reference a follow list note as a query param.
jb55's avatar
jb55 _@jb55.com 8 months ago
I want feeds for bins by post frequency anyway, I've talked about that a lot and I think it would be pretty cool. just need to calculate post frequency to begin with...
I guess I was thinking of doing the lazy and cheap thing: just use last post time. It's probably good enough for most things.
No I use primal 5% of the time for reasons. I try not to frantically click things there that might conflict with actions I made on Damus (follow/unfollows).
jb55's avatar
jb55 _@jb55.com 8 months ago
do you ever follow/unfollow on primal? that might wipe out your damus actions since they may not be synced with other relays
I thought about that. Possible i did early on, but I avoid it now and still have issue. Example is @npub1c428...yl59 who every 2-3 days I am no longer following. 🤷‍♂️
Yeah, have notified vitor several times, been that way a few months but next release should fix . Not sure if related
hasky's avatar
hasky 8 months ago
Good morning . I can’t imagine follow 1000 people I don’t know . You and the grinder , I know 😁
Oh I didn't really understand rands comment. My guess is I do not. I'll align them better and see what that does. In back of my head I think it could be connected to me following that hashtag (the thing that was crashing amethyst when viewing my profile)
Surprisingly, Looks like I am consistent across two clients I use. I follow something like 925 people, so wanted to trim some so I could add while staying below 1k, but yeah it's screwy for some reason. Not a huge deal as it only seems to effect latest follows like enigma lol image
The primal one is their proprietary caching service. You need it for primal to work. I don’t think you do on Damus though. Might solve your problem. Only trial and error will tell
You don’t need a relay in the relay section for nwc to work. Just the settings in the wallet section. The relay used for nwc should appear there (in the wallet section I mean, in damus at least)