Franklin Roosevelt actually said...
As it was so well said by James Madison, 'We hold it for a fundamental and inalienable truth that religion and the manner of discharging it can be directed only by reason and conviction, not by force or violence.'
Context
Letter on Policy regarding Religious Activities in Mexico
11/12/1935