Theodore Roosevelt actually said...
Constant care should be taken to guard against unnecessary increase of the ordinary expenses of government.
Context
Fourth Annual Message
12/05/1904
Fourth Annual Message
12/05/1904