Theodore Roosevelt actually said...
There has never yet been a man in our history who led a life of ease whose name is worth remembering.
Context
Remarks at the State Fair Grounds in Concord, New Hampshire
08/27/1902