James Madison actually said...
The diversity in the faculties of men from which the rights of property originate is not less an insuperable obstacle to a uniformity of interests.
Context
Discussing the link between human diversity and property rights.
11/21/1787