My current read of why the US government is full of such spooky bois:
The stark truth of the often spooky and almost adversarial-feeling relationship of Americans with their own country's Intelligence Agencies is :
That secrecy, even with respect to their own populace, is necessary to maintain intelligence and strategic advantage over rival nations. The intelligence agencies are entrusted with that power not because anyone elected them, but by the nature of their work.
This creates a point of conflict between the sovereignty of the free people, their constitution and its amendments (see the 4th), and elected leaders on one hand, and the intelligence agencies that oversee their safety on the other. The 'puppet' perception of elected leaders vis a vis their apparent beholdness to shadowy intelligence agencies is a clear example.
The seriousness of this conflict was muted until the Cold War, when the budget and sophistication of Intelligence Agencies reached never before seen levels, and then was reticent to surrender this perceived world advantage after the war ended.
Freedom-seeking nations are currently in a time where the populace doesn't understand this conflict, Intelligence Agencies are scared to relinquish their power, for self-preservation reasons, but also because they can see the national risks that would follow, and elected leaders who understand the conflict don't feel it prudent to completely side with the full sovereignty of the populace (which would require them to 'blow the lid' on the secret nature of the nation's operation, or otherwise undermine its operation), for fear of the same national risks the Intelligence Agencies see, as well as self-preservation (ex. some think JFK was taking this route, and was gruesomely silenced by the Intelligence Agencies).
So there's no clear path forward without very decisive action, so the story of the decades since the Cold War has been one of mission and scope creep of the Intelligence Agencies, no real architecture in place for how to balance secrecy, safety, and strategic advantage, with sovereignty and democratic accountability.
will follow with my ideas for solutions