Lyndon Johnson actually said...
It is just as ridiculous to charge a man to vote, and therefore, on that basis, it is in violation of the Constitution.
Context
Remarks to Principal Field Officers of the Internal Revenue Service.
02/28/1966