Abraham Lincoln actually said...
Those only will be brought from the island who desire to return, and their effects will be brought with them.
Context
Executive Order—Regarding the Colonization Experiment
01/31/1864
Executive Order—Regarding the Colonization Experiment
01/31/1864