Replies (49)
Can we get rid of g0r3 in nostr with this?
I haven't seen any gore. Use paid and WoT relays to remove all unwanted shit.
If ppl would work as hard to ACTUALLY produce something of value, as they do to Spam/scam 🤷 ...... Think about the possibilities of humanity.
This is the one good thing to come out of spam attacks
we need more WoT filters in clients next :)
@hodlbod do you have an documentation on how your WoT score works in coracle? I've built an easy way for loading the WoT in applesauce and Id like to use the same scoring system as you've used in coracle in noStrudel
yaaaaaaas.
this conversation makes me excited.
What are the chances of integrating
@Vertex DVMs for spam filtering at the client level? I think it's the best thought out solution. Maybe
@Pip the WoT guy can chime in
Oh thank you very much, I'll try that . i don't mind paying, this tech doesn't sell my data like every other free website. Thank you for your response!
GrapeRank is a more sophisticated WoT algo and extensible library than any client is currently using.
Why do we need sophistication, when good enough is always better? Simply because bots and bad actors will always find ways around “hard coded” WoT parameters … whereas individually configured WoT filters shared peer to peer across clients will always be the best performer.
ManiMe
The GrapeRank library runs server side. The reason it takes a few minutes is mostly due to network latency, not calculation time. But no worries, it continues to run in the background regardless of local browser state. This allows an implementer (client) to trigger calculation and check back on the status periodically.
The “default” config used by grapevine.my client ingests and normalizes all follow lists (for 6 degrees out) to establish the network, and also all mutes and reports authored within the same network. I think that’s about 2 million events (no duplicates) retrieved live from the network per calculation, when I run it. These events (ingested by the interpreter module) are not (yet) cached between calculations… so it’s pretty heavy network load.
The library itself is modular, pluggable, and configurable. Modular means any implementer (client or relay) can make use of only some or all of the library modules. Pluggable means that additional event kinds can be added for the interpreter to ingest and normalize (currently only follows, mutes, and reports have plugins made, but this is as simple as a single json file). Configurable means that the behavior of both the interpreter (protocols) and the calculator can be customized per request to allow greater sovereignty for the end user.
https://github.com/Pretty-Good-Freedom-Tech/graperank-nodejs
it feels good to pay for something too.
nostr.wine Relay
The most reliable paid relay on nostr.
is my favorite paid relay. i've been using it for over 2 years now.
Thank you!!
what events does the graperank take? because thats the limitation on the client side. I can only reasonably afford to fetch the kind:3 follows (and maybe mutes) out to 2 levels (friends on friends). any more and I'm overloading the clients network
That relay is lame you can’t even advertise on it
“Out of the box” with no config or plugins, GrapeRank ingests follows 6 levels deep to map the network, and interprets these plus mutes and reports for the calculator. It does this server side. Performance would be greatly improved with addition of a cache, which is one benefit of running server side. Calculated scores, from the context of any npub as “observer”, are saved and retrievable at any time. Adding additional interpreters is dead simple…
GitHub
graperank-nodejs/packages/ptotocols-nostr/index.ts at main · Pretty-Good-Freedom-Tech/graperank-nodejs
Contribute to Pretty-Good-Freedom-Tech/graperank-nodejs development by creating an account on GitHub.
Orly?
@hzrd149 and
@hodlbod, the WoT coracle has a few issues, that will become more and more apparent as nostr grows:
1. each follows/mute count 1. This doesn't consider that there are people that follow 5k others, while most are more picky, nor considers the reputation of the one following. These nuances are reflected by the pagerank formula for instance
https://en.wikipedia.org/wiki/PageRank.
The contribute of one's follows are proportional to its pagerank and inversely proportional to the size of the follow-list. This is true both for global or personalized pagerank.
2. a users has only information on the two-hop network around him. This consists of an average of ~50k people, which is fine for now because the network is small, but won't scale when nostr is at 10M+ users.
Vertex abstracts away all of these complexities, and gives simple to use end-points for many common operations like:
- sorting replies, comments, zaps...
- verify reputation of a target npub
- get recommendations
I am more than happy to have a chat about how give access to these tools guys, just let me know
Great chime in sir :fire:
@hzrd149 just convinced to stop using the Follow concept entirely, lol.
Very curious how pageranking will be done in a more Community centered layer of Nostr, with atomic Label/Badge events.
I agree with your first point, valuing each follow/mute should take into account how many users the user is following. I think ill try to learn the math behind page rank and see if it works any better for me
For 2. I agree this does not scale but I'm trying to find a simple enough solution that a pure (no server) nostr client can implement that will be the most effective. Once I figure out how to implement the "simple" self sovereign method I would love to see how this could be off loaded to a third party server (or even a personal server on a Umbrel)
But for my use case in noStrudel I need the "WoT" to do two things:
1. Sort pubkeys by "rank" (find the real hodlbod test)
2. Filter out new, on-off spam accounts
The effectiveness of these things will vary based on if the user is using a locally calculated WoT or a remote service, but I think thats fine
I agree, local first, then pagerank as progressive enhancement
nope
We pay to avoid advertisement.
Yes. But I will not block people from advertising with posts from their paid npubs unless they spam replies or something.
personalized Pagerank is the best "local", that is as wide as possible, but still relevant.
but yeah it's hard to compute locally.
Would it change if these ranks where computed server-side, but you can do client-side verification that prevent the service provider from lying?
I have a few ideas in mind.
cc
@hzrd149honestly if a community is small enough, Pagerank is not needed because all npubs have been invited.
For open communities, then Pagerank is useful for helping the moderator know which one to kick out automatically
No, because I still want wot numbers if the server goes down
That's true **within** a community.
The questions come when you want to use the Communitues someone is active on as data to derive a notion of **his network** from.
As in:
When he browse to the page of a product / a new community / a podcast show / ....
Do you show the profiles on his network that are part of his network that use that product, are active in that community and are listening to that podcast?
If yes, then you need a way to define that network.
Or for the main Vertex use case too:
For knowing if you are the real Pip, you are now looking at the Web of Follows.
What changes mathematically when we start looking at a Web of Communities?
A web that works with:
- Badges: awarded by Communities, (optionally also accepted by profiles)
- payments (pay to poast, pay for badge, zap the communikey, ....)
- atomic events for reporting and black "listing"
Yes, that'd work for Community Apps.
As the communities' job and effect is that they do most of the spam prevention + user validation.
Page rank is more rarely needed for the individual user and it's fine if servers go down sometimes.
I can fall back on a light local "user network" calculation by using the Communities and Contacts and going max two level deep (ideally just one even, which'd be fine).
okay, so it's more a matter of uptime rather than trust, am I right?
I am thinking about running a second relay so that if the first goes down the second can cover.
isn't it possible to store a cache locally and update it once every unit of time? This would radically lower the uptime requirement because the server now needs only to be online once every hour/day/week...
yeah, I think one level should be stored locally, as it's very easy and useful, and then server for everything else.
It really depends on the usecase, there is no silver bullet.
For example, if you are looking at a product, you could fetch all reviews for that product, and drop reviews from npubs whose pagerank is lower than a threshold. Or you could weight that review using the pagerank (influencers have higher influence, not necessarily a good idea).
Or maybe you can show the average vote of the members of the community, which is more contained. All tradeoffs.
Pagerank can be run on any graph, so one could built it by looking at who replied/zapped/liked to whom, in a given setting.
Also, not all signal should be condensed into a single rank, multiple trust signals should be provided imo. Number of followers, ranks, nip05, badges, all give some clues that your brain can interpret.
It's both uptime and trust, a local fallback will always be necessary. I could run a cache, but it would also be a third party service with questionable uptime, it's just good to have something built into the software itself because it's such an important core function
I'm not too worried about the server lying, since ideally users could choose their provider or maybe run their own.
The way I see it there are two uses for pagerank or "WoT"
1. The client wants to hide content that is outside of the users social network (subjective distance or score)
I'm not sure how this could work without the server sending the whole graph (probably Mb of data) or the client making API requests to check each pubkey (not great for privacy)
For this case I think the simple kind:3 WoT is good enough, at least for now
2. The client wants to sort events or pubkeys by "reputation" or score
Vertex already does this well for user search ( query -> pubkey[] ) but it may be useful to have some kind of generic sorting method for events to allow a client to sort replies ( again probably bad for privacy )
I tend to agree with
@hodlbod. I want my client to work without a dependency on a server. but I'd also like to integrate with some kind of server to improve performance or to get better sorting / filtering results
Random thought. maybe bloom filters could be space efficient way to handle the filtering use case? I don't know much about them but AI says they are used to prove an ID is within a set
I think for 1. and 2. RankProfiles is a solid approach
Rank Profiles
This service sorts a provided list of pubkeys using the specified algorithm.
Example use cases:
Batched ranking of pubkeys
Sorting comments under ...
Let's go through the case with replies:
- user clicks on a note
- client loads all replies for that note
- extract the pubkeys of these replies
- client asks Vertex for the ranks of up-to 1000 pubkeys in a single call, with the user's preferred algorithm
- client hides replies from npubs with rank lower than the user's preferred threshold
- client ranks the other replies by their rank (or a custom formula like rank * zap amount).
It's an interesting idea but I don't see where exactly in the client you would need it.
- for the feed one uses follows
- for the replies, because on the relatively low number one can use the ranks directly (giving the user the ability to set a threshold).
Bloom filters are superior to RankProfiles when you are dealing with +10k pubkeys (not in the follow-list of the user) I think.
Am I missing something?
I am missing something sir. I assume that Coracle stores locally the WoT calculations it computes client-side. Is it true?
If so, why can't it store the ranks provided by Vertex in the same way?
Because the problem is cold start. Someone (new or returning) opening the app for the first time won't have a cache. The cache gets invalidated pretty often too due to limits with web storage
I think your right, the only other one off use case I could think of where the client would need a filter instead of sorting would be if they wanted hide zaps / reactions or browser "global" on a relay. but none of those are things I'm trying to implement :)
yeah but this is true for both your current WoT and the Vertex ranks, no?
The way I imagine it is:
- a user main feed is based on its follow-list, which is pulled from relays
- as the user starts clicking around (search, reading replies...), this calls a service provider like Vertex, and the ranks are stored for later use.
So it would progressively load ranks as needed, not ahead of time.
Believe me I am not trying to push anything, just trying to genuinely understand.
Do u mean zaps and reactions in the users notifications?
I think that too can be done using RankProfiles. It's unlikely one has thousands of notifications, but not impossible if its targeted by an attacker
Yes, but client-calculated wot can be pulled from relays. IOW, vertex is a centralizing factor. It's benign, but still centralizing.
The issue with client-calculated is low quality and suboptimal UX. The problem I reported with profile feeds taking long to show up in Coracle appears to be related, as you see the notes spinner while the "follower" list gets populated. Correct me if I'm wrong.
Having a fallback mechanism sounds like the best of both worlds.
No, that doesn't sound right, wot is calculated concurrently, it shouldn't be holding up feeds (unless too many requests are being sent to a single relay).
> Having a fallback mechanism sounds like the best of both worlds.
Yes, that's what I'm saying
the latest version of wot-relay fixes memory leaks and adds IP rate limiters.
thanks @fiatjaf and @utxo the webmaster 🧑💻 for your work on this and for figuring out why my relay was very angry.
