William Taft actually said...
The truth is, in those days such a thing as freedom of religion was not understood.
Context
Address at the Two Hundred and Fiftieth Anniversary of the Founding of Norwich, Connecticut
07/04/1909