Theodore Roosevelt actually said...
When misconduct is committed in the view and presence of the President or head of executive department removal may be made summarily and without notice.
Context
Executive Order
11/16/1905