Isn't that still "changing the relays"?

yes but if your client doesnt know where to look for your follows itll be just as quiet when they change or vice versa