Herbert Hoover actually said...
There is only one course of sound fiscal policy, and that is to reduce the expenditure of the Government to the last cent consonant with the obligations of the Government.
Context
Statement on Government Expenditures.
10/15/1931