Andrew Johnson actually said...
He who is opposed to the restoration of this government and the reunion of the States, is as great a traitor as Jeff. Davis or Wendell Phillips.
Context
Remarks in Cleveland, Ohio
09/03/1866