If they exist on centralized systems all running in the same data centers, humans could still shut them down. It’s not like they somehow managed to break free of their own infrastructure.

That is couple steps... If a human tells the ai to figure out how to make enough money to pay for yourself, that is probably easy, and once posted more AI could try to become self-sustaining. A human could very easily ask the ai to become self replicating. That is basically the first thing any black hat does with a keyboard (virus). Self-sustaining self-replicating virus, starts using encrypted comms...