James Madison actually said...
If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary.
Context
On the necessity of checks and balances in government.
02/07/1788