William Taft actually said...
Let us leave this beautiful city of the national dead, therefore, with the deepest gratitude to the men whose valorous deeds we celebrate.
Context
Memorial Day Address at Arlington Cemetery
05/29/1911