Harry Truman actually said...
Whenever a holiday falls on a Sunday, Federal offices and establishments shall be closed to public business on the following Monday.
Context
Executive Order 10358—Observance of Holidays by Government Agencies
06/08/1952