Andrew Johnson actually said...
As the nature of government requires the power of removal, it was maintained that it should be exercised in this way by the hand capable of exerting itself with effect.
Context
Veto Message
03/01/1867