James Madison actually said...
Whereas advantage has been taken of the loss of a fort more immediately guarding the neighboring town of Alexandria to place the town within the range of a naval force too long and too much in the habit of abusing its superiority wherever it can be applied.
Context
Proclamation—Calling All Citizens to Unite in Defense of the District of Columbia
08/31/1814