Rutherford Hayes actually said...
The policy of the United States, steadily adhered to from the adoption of the Constitution, has been to avoid the creation of a national debt.
Context
Third Annual Message
11/30/1879