James Buchanan actually said...
It can not legitimately resort to force without the direct authority of Congress, except in resisting and repelling hostile attacks.
Context
Special Message
02/17/1859
Special Message
02/17/1859