Millard Fillmore actually said...
No individuals have a right to hazard the peace of the country or to violate its laws upon vague notions of altering or reforming governments in other states.
Context
Second Annual Message
12/01/1851