John Kennedy actually said...
If there are going to be inequalities, it should be on the grounds of their ability and dedication, not on the grounds of their color.
Context
Speech of Senator John F. Kennedy, Fieldhouse, University of Wisconsin, Madison, WI
10/22/1960