Theodore Roosevelt actually said...
Shame on the man of cultivated taste who permits refinement to develop into a fastidiousness that unfits him for doing the rough work of a workaday world.
Context
Address at the Sorbonne in Paris, France: Citizenship in a Republic
04/22/1910