Andrew Johnson actually said...

To pronounce the supreme lawmaking power of an established state illegal is to say that law itself is unlawful.

I am unable to give it my assent, for reasons so grave that I hope a statement of them may have some influence on the minds of the patriotic...

The power of removal was incident to that duty, and might often be requisite to fulfill it.

I submit to Congress whether this measure is not in its whole character, scope, and object without precedent and without authority.