James Madison actually said...
In a society under the forms of which the stronger faction can readily unite and oppress the weaker, anarchy may as truly be said to reign as in a state of nature.
Context
Discussing the dangers of majority rule.
02/07/1788