Lyndon Johnson actually said...
For while many men may appreciate the scope of your work, only one man--the President himself--can fully appreciate its impact on the Office itself.
Context
Letter Accepting Resignation of Theodore C. Sorensen as Special Counsel to the President.
01/14/1964