It depends on your relays and the content you don't want to miss out. Some relays have filters and censors. Some specialize in specific niche topics that not all users subscribe to. Personally, I prefer to use a a lot of relays rather than risk missing some post or vice versa. In the past, I've used fewer relays, haven't noticed any difference in speeds, #nostr has always been kinda slow. Some clients feel faster, smoother, and less buggy than others tho.