Ronald Reagan actually said...
When I took the oath of office, I pledged loyalty to only one special interest group-'We the people.'
Context
Address Before a Joint Session of the Congress on the Program for Economic Recovery
04/27/1981