Many antislavery leaders argued that if legislators would not end slavery, God would punish the entire country. One said that slavery was “a solemn mockery” and “insult to God” and that “the national sanction and encouragement [of slavery] ought to be considered as justly exposing us to the displeasure and vengeance of Him, who is equally Lord of all.”15
The editor of the Connecticut Courant newspaper howled that “the existence of slavery may be viewed as one forcible cause of a final separation of the United States” and called slavery “extreme wickedness.”16
Yet, at the same time, columns ran in newspapers, both in the North and in the South, stating that a study of slavery in ancient Rome and Greece showed that bondage did not make those nations weaker; it made them stronger. Southern congressmen extolled it in their speeches and, if they didn't directly applaud it, warned others that southerners believed in it. “I will tell the truth,” said one southern lawmaker. “A large majority of the people in the southern states do not consider slavery as even an evil.”17
When confronted directly with slavery, Madison often retreated from his position of a slave owner to that of a libertarian, or so he thought. When he went home to Virginia at the end of a term in the Continental Congress, in 1783, he sold his slave Billy, who had lived with him for several years. Under Pennsylvania law, Billy would then be free after seven years. Madison wrote his father that Billy had become “too thoroughly tainted to be a fit companion for fellow slaves in Virga.,” and that he was “covering that liberty for which we have paid the price of so much blood, and had proclaimed so often to be right & worthy the pursuit, of every human being.”18
He pushed his “back to Africa” theory from his early twenties until his death. None of his slaves wanted to go to Africa, though. They were Americans and wanted to remain here. British writer Harriet Martineau was surprised when Madison told her that the slaves were opposed to going to Africa. “He accounted for his selling his slaves by mentioning their horror of going to Liberia, a horror which he admitted to be prevalent among the blacks,” she wrote.19
Those slaves who remained with him all of his life, such as Paul Jennings, liked him. Jennings wrote that he was close to Madison and never saw the president lose his temper toward any slave on the plantation or in the White House. But many of Madison's slaves, whom he thought loved him and his wife, were very happy to leave him when sold to someone else. He noted that in letters to friends. He sold several to a county neighbor to pay a debt in 1819. “I am persuaded [he] will do better by them than I can, and to whom they gladly consent to be transferred,” he wrote Edward Coles.20
Madison's views on slavery were never seriously challenged. Life rolled on in both slave states and free states, speeches were made and speeches were ignored, tobacco sold and profits appreciated. Then, in 1820, came the storm over the Missouri Compromise. Following heated debates, as a solution to the entry of more states into the Union and as a way to quell a brewing social and political storm, Congress decided to admit Missouri as a slave state and, to balance the number of free and slave states, to admit Maine, which had broken free of Massachusetts, as a free state.
Madison agreed with those who wanted Missouri admitted as a slave state, but for an odd reason. He told friends that Missouri would not be invaded by a large number of slaveholders with hundreds of slaves, but probably by hundreds of families each with just a few slaves. Those slaves, he believed, would have better lives until the day, up ahead, when slavery was ended in the country. He stuck to his views against a tidal wave of antislavery invective against the Missouri Compromise by hordes of northerners who were growing increasingly unhappy about the power of slavery in the growth of America.21 The arguments over the Missouri Compromise showed, too, that Virginia's slaveholders and political leaders, such as Madison, were not really in favor of eliminating slavery after all; they just wanted to delay it, hoping the antislavery movement would die. It would not.
Madison was always afraid, like many, of what Jefferson said in a wonderful metaphor about slavery: holding the wolf by the ears and being afraid to let him go.
Madison had written the marquis de Lafayette in 1820, five years before his arrival in America for a much-heralded tour, of the “dreadful fruitfulness of the original sin of the African trade” and fended off attacks by Lafayette when he toured Montpelier later. Madison continually told Lafayette that he wanted to free his slaves, but he kept turning his back on all opportunities to do so. Fanny Wright, Lafayette's feminist companion on his 1825 tour, later wrote Madison and said she wanted to set up a farming collective made up of freed slaves. The bondsmen and bondswomen would work for a certain amount of time to pay off the price of their freedom and, at the same time, be taught how to read and write and run a farm. After a certain point in time, they would be freed and given farmland. She was very confident the collective would work. Madison sneered at her. He told her discipline could not be maintained on such a farm and that “the prospect of emancipation at a future day will sufficiently overcome the natural and habitual repugnance to labor.”22
Madison's alternative idea, hatched in 1819, was to institute a gradual system of emancipation so that plantations would not be left workerless when manumission was accomplished. He was not certain how to do it, but he told friends it was the best plan. Part of it was to get the federal and state governments to sell off millions of acres of public property and use the revenue to pay slaveholders for their slaves and to grant slaves their freedom. The slaveholders, with all that money, could then begin a new system of paid-labor farming and hire back tens of thousands of black and white paid laborers.
He estimated that all of the money needed to free the slaves, some $600 million, could be raised with the sale of 200 million acres, or just one third of total public lands, at a very reasonable $3 per acre. It seemed like a good solution for everybody. “And to what object so good, so great, so glorious could that peculiar fund of wealth be appropriated?” he asked.23
The president never seemed to think through his abolition plans. He insisted on gradual emancipation for blacks, who insisted with equal fervor on immediate freedom. He was convinced that his plan would work because the need for slaves had declined since farming, in general, had declined. He never injected the cotton industry, starting to boom in the late 1820s, into this equation; planters needed more, not fewer, slaves.
One thing he did do, though, and constantly bragged about, was to buy and sell slaves with contracts that called for their emancipation after a few years, usually five or seven. These agreements enabled him to free slaves on a delayed basis. In 1804, Madison purchased a slave named David for $400. “It being understood,” his wife wrote, “that at the expiration of five years he is to become free & that in the meantime Mr. M. is to be his owner.” Dolley did the same. In 1810, Francis Scott Key worked out an arrangement in which Dolley would loan her black freedman servant Joe $200 with which he could use as an advance to purchase the freedom of his wife from Key. The husband and wife would then work for a certain number of years as paid workers and pay off the rest of the price of the wife. Dolley agreed. The Madisons based these contract ideas on established British slavery views from earlier times, that slaves could be considered like indentured servants and bonded for a specific period of time. This way, in a few years, Madison could actually “free” his slaves, or so he argued.
To their credit, the Madisons also tried to hire freed blacks when they could, rather than buy more slaves. In 1808, friend John Tayloe found two coachmen in Richmond, both freed blacks, for the Madisons. One was hired. Three other freed blacks were hired to work in the gardens of Montpelier.24
Madison was named the president of the American Colonization Society in 1833, a few years before his death and, even though ill during most of his term as leader of the group, he helped them gain publicity and raise money. He lobbied for the group, via letters, with Congress. Visitors told him that his plan to have all northerners pay half the cost of
transportation to Africa would never work. They didn't own slaves; why should they be burdened by the cost? He waved them all off. Since northern merchant ships had brought the slaves to America, northerners were responsible for paying the cost of sending them back, he insisted, completely blind to reality. The slaves had to go to Africa, he argued. They could not remain in America because they would be unable to intermingle with the whites, despite the fact that tens of thousands of them freed after the revolution in the north had done just that, and successfully, even though it was painful.
Madison paid no attention to charges that the colonization movement had made little progress under his presidency and during his membership. The slave movement in America was now huge and slavers threatened to take their bondsmen and bondswomen into the new territories and, in time, get those territories made into slave states. After all, over the last several decades, the thirteen original colonies had added eleven more states, with more states in the Louisiana Territory probable within the next ten or twenty years. There would soon be one million slaves in America and the “peculiar institution” was a hot issue in each state, yet all Madison worried about was putting a few hundred slaves on ships to send back to Africa.
He ignored any and all other solutions for the freedom of the slaves. One of the best sent to him was from his own former secretary, Edward Coles. The visionary Coles moved from Virginia to Illinois, a free territory, so that he could free his slaves and start his own life all over again in a state not crippled by slavery. Coles freed his slaves on the way to Illinois, gave them farmland, and showed them how to live on it as free men and how to cultivate it. He remained with them for several months, providing instructions so that they could make it on their own as freemen farmers. It worked.
Madison congratulated Coles, writing him, though, that his plan only succeeded because of his help. He said that “with the habits of the slave, and without the instruction, the property, or the employments of a freeman, the manumitted blacks, instead of deriving advantage from the partial benevolence of their masters, furnish arguments against the general efforts in their behalf.”25
Coles had acted swiftly and successfully; Madison never did. He continued to complain about slavery and yet continued to do nothing to end it.
As the slavery issue grew in intensity through the 1830s, politicians in South Carolina began to suggest that their state, and perhaps other southern states, leave the Union and form their own government in order to protect slavery. This alarmed Madison. He did not believe the pro- and antislavery clashes could actually split the government of the country he had personally designed at the Constitutional Convention. He wrote Henry Clay, “What madness in the South to look for greater safety in disunion. It would be worse than jumping out of the frying pan into the fire; it would be jumping into the fire for fear of the frying pan.”26
Madison seems to have trusted his slaves even though a slave, with assistance from two others, murdered his grandfather Ambrose Madison in 1732. One of the slaves was hanged for the slaying and the other two received twenty-nine lashes for conspiring with the killer.27
The most slaves Madison ever owned at once was 108 in 1802. He sold many off in the 1820s when the economy went bad. Madison did not rid himself of one or two, either, but sold them in large bunches. In October 1834, as the leaves added a magnificent hue to the surroundings at Montpelier, he sold sixteen more to W. H. Taylor. He sold entire families and couples, together. By 1834, his scant farming income had to be made up somewhere and the president found some salvation in unloading his slaves. He did not free them, but sold them, thinking that in letting them go somewhere else he had, in a sense, liberated them, but he only liberated them from Montpelier. He sold nearly two thirds of his slaves between 1802 and 1834, but his revenue from all of that traffic still left him near the bankruptcy line. He had profited from slavery, but now, with the value of slaves much lower due to the general farming recession in the South, Madison was actually losing money on his investment in slaves. The president continued to sell off hundreds of acres of land during that time to pay his bills and the debts of his son and others. Madison had no real investments from which to get cash and had no involvement in nonslave farming to earn money. He had become a victim of the slaver economy and did not have a solution to his problems, which continued to mount as he aged. The Madisons realized, too, that by selling their hardest-working slaves in order to obtain funds to pay off their son's debts, they were now left with those who were too young or too old to work very hard, and their farm business fell off considerably.
He insisted that churches had convinced slaveholders to treat their labor force with more leniencies and this made the lives of the South's hundreds of thousands of slaves better. Masters now made their slaves Christians and took them to church. Others often brought doctors and nurses to their plantations on regular visits to care for their slaves. Some spent extra money on clothing for slaves, let them sell crafts they made at city and village markets, gave them guns for hunting, and wrote passes so they could visit friends and family on other plantations.
Madison had told overseer Mordecai Collins always to make certain that there was an extra cow on hand which could be milked by the slaves for their tables. They also had to have extra food kept in special barrels to be eaten at their leisure. The overseer was advised “to treat the Negroes with all the humanity & kindness consistent with their necessary subordination and work.”28
Madison often scolded slaves but never permitted overseers to whip them. He wrote his father once that a slave who had misbehaved on a trip with him to Fredericksburg was given some “serious reprehensions and threats from me, which have never lost their effect.”29
His slaves appreciated his attitude. One, Paul Jennings, who was with him from 1810 until his death, said that “whenever any slaves were reported to him as stealing or ‘cutting up’ badly, he would send for them and admonish them privately, and never mortify them by doing it before others, they generally served him faithfully. He was temperate in his habits.”30
But why didn't he free his slaves and start a new life as a farmer with paid labor? One answer might have come from Thomas Clarkson, the leading British abolitionist during Madison's retirement, whose efforts to end the slave trade in British-held territories had failed. Clarkson wrote that abolitionists believed that in England, as well as in America, the end of the African slave trade would mean better treatment for slaves on American plantations and eventual freedom, but that did not happen. “We did not sufficiently take into account the effect of unlimited power on the human mind. No man likes to part with power and the more unbounded it is, the less he likes to part with it.”31 Was that the view of Madison, not only a man of power on the plantation and in Orange County but also formerly the most powerful man in America?
His wife, who said she was as opposed to slavery as the president, continued to utilize her domestic slave staff to provide labor for her lavish parties, as she had done in Washington. She organized all of the female and male servants, gave them instructions before each soiree, and oversaw their work. She had them live in slave cabins within fifty yards of the mansion so they could be called upon whenever she needed them. Each morning, she met with a slave “manager” and went over the day's work in the mansion with her. Throughout the day, Dolley supervised the work of the slaves and gave them keys to buildings and cabinets that she kept on a large keychain tied around her waist. She told all of her visitors how much she deplored slavery and yet, like her husband, used her slaves to provide her with a comfortable life.
Dolley kept her relations with her domestics very quiet. She wrote no more than a few lines about her slaves in her hundreds of letters to friends and rarely talked about them, except to remind one and all that the Madisons’ slaves were never whipped. Like her husband, she spent her adult life bemoaning slavery and yet did nothing to free the slaves or help the antislavery crusade. Privately, she complained bitterly about her slaves, especially in her ret
irement, when she thought slaves took advantage of her age and her husband's infirmity. In the summer of 1818, for instance, she complained to her sister Anna that her most trusted slave, Sukey, was annoying her constantly. “Sukey has made so many deprivations on everything, in every part of the house that I sent her to Black Meadow [one of their farms] last week but find it terribly inconvenienced to do with her & suppose I shall take her back again, as I feel too old to undertake to bring up another,” she said, and then, heatedly, she added “so I must even let her steal from me to keep from labor myself, more than my strength will permit, I would buy a paid but good ones are rare & as high as $8 & $900s.”32
She never looked at her slaves as real people. In 1822, she wrote her sister Anna about a business transaction involving a number of items and listed them as “furniture, the negro girl, cow,” with no differentiation between the three. Whenever she referred to guests who brought slaves with them, she added them to the entourage, like horses. In 1806, she talked of “Mrs. Randolph and suite stayed until yesterday morning. I sent Kitt with their Negroes.”33
James and Dolley Madison Page 44
