William McKinley actually said...
The Union is once more the common altar of our love and loyalty, our devotion and sacrifice.
Context
Speech Before the Legislature in Joint Assembly at the State Capitol in Atlanta, Georgia
12/13/1898