Andrew Johnson actually said...
I would therefore recommend such an amendment of the Constitution as may remove all intermediate agency in the election of the President and Vice-President.
Context
Special Message
07/17/1868