William Taft actually said...
We speak with great satisfaction of the fact that our ancestors—and I claim New England ancestry—came to this country in order to establish freedom of religion.
Context
Address at the Two Hundred and Fiftieth Anniversary of the Founding of Norwich, Connecticut
07/04/1909