yeah, they are only reading, not writing, so all that would really be required is to slow them down probably could split the direct followed vs follows of follows into two tiers also so that the second level of depth get less service quality probably maybe should work on it today because this problem with scrapers is fairly annoying it's only nostr WoT spiders at this point, i wish i could just suggest to the spider people to just open a subscription for these and catch all the new stuff live, that would be more efficient for them and less expensive for relay operators at this point it's fairly early in the game for that stuff so maybe some education of WoT devs might help... cc: @Pip the WoT guy there is no point in doing sliding window spidering on events once you have the history, from then on just open a subscriptiion this would be more friendly from the AI spiders if they had a way to just catch new updates, but unless dey pay me! i'm just gonna tarpit them

Replies (1)

Yeah, I don't get the point of spidering data you can just stream. Like, they spider Nostr websites and it's like... just subscribe to the relay. *roll eyes*