Are you familiar with PlayStore beta testing? You could control how many people test new versions there and make it easy for everyone to try it out from the PlayStore.