I'd do it in two rounds. Open up streams for 3 seconds into a map. Aggregate by npub, count how many notes they published, filter out those over X times, refetch the rest for the remainder, with a 10-second timeout.
Login to reply
Replies (3)
Or something like that. @Vitor Pamplona this optimization needs math.
Ideas?
Or do the second fetch with a 100-event limit.
Since you've already filtered-out the frequent-posters, you can hard-limit the remainder without missing anything from anyone.
For most relays this approach works fine, but for some of the larger ones it’s a serious issue. For example, trying to aggregate all events from relay.damus.io over 24 hours. Even with smaller relays, if a user sets a 30-day time limit, you’ll run into the same performance and bandwidth problems.