Andrew Johnson actually said...

I submit to Congress whether this measure is not in its whole character, scope, and object without precedent and without authority.

I have carefully examined the bill 'to regulate the tenure of certain civil offices.'

The power of removal was incident to that duty, and might often be requisite to fulfill it.

I transmit herewith a communication from the Secretary of the Navy,