James and Dolley Madison

by Bruce Chadwick

  Dolley had a dim view of slaves as workers and once described the hands who worked in her orchards as “lazy women.” She wrote several people that her slaves simply would not work without an overseer pushing them to finish their chores each day. Yet, at the same time, she always insisted that the overseers who worked for her did not beat her slaves and treated them well. “As to overseers, I'm glad you refused the first two, Burnly and Shackleford. No whipper of negroes should ever have our people or any others,” she said.34

  Payne Todd was callous toward his slaves and often used them as barter in business transactions or to settle a debt. He had dinner with a man in 1843 and tried to sell him two teenaged male slaves to help satisfy his mother's debts, something his mother did not know about. The man, who did not trust Todd, said the offer seemed “suspicious” and paid no attention to it.35

  Dolley, like her husband, knew all about the various slave insurrections that had taken place in Virginia. Slave Gabriel Prosser led a revolt in 1800 in Richmond. The hundreds of insurgent slaves planned not only to capture or kill a number of white merchants and planters in the state capital but also to kidnap the Madisons’ friend, Governor James Monroe. Friends in Richmond told them all about it. In 1811, five hundred slaves rebelled in Louisiana, and one hundred were killed in battles with whites. Later, in 1811, slave Denmark Vesey planned a large insurrection in Charleston, but it was scuttled and he was hanged. In 1831, there was an even larger slave rebellion, this one led by Nat Turner, in which fifty-one whites were murdered, many in Virginia. The whites throughout the state were badly shaken by it. There is no way to tell what Dolley really thought, but in the letters that do exist, she paid little serious attention to it. “I am quiet, knowing little about it and that I cannot help myself if I am in danger. I hope we will all be on our guard after this,” she wrote a friend about Vesey's rebellion.36

  Madison's dogged insistence on keeping slaves, despite his moral abhorrence of the practice, helped to bring about the tragic economic downfall of his family. He spent so much money on buying slaves and on slave upkeep that he had none left for farm equipment or even for the basic upkeep of his mansion and other buildings at Montpelier. When Charles Ingersoll visited him in 1836, he was astonished at how run-down the entire plantation seemed.

  Madison, and his friends throughout Virginia, held onto slavery as the foundation of their economy, constantly ignoring paid-labor plans suggested by others, the pleas of northern congressmen, abolitionist leaders, civic organizers, and ministers. They would not let go. In the North, the Industrial Revolution began to flourish in the early 1830s with the success of the Erie Canal and others, the growth of the cities, and the explosion of manufacturing. Its nonslave society flourished. In the South, slavery was an albatross for all. It held them back from exploring manufacturing and shipping and the use of their extensive land holdings for anything other than slave-driven tobacco, sugar, and cotton production. Controversies over slavery divided neighbors and friends, and it was under attack from churches following the Great Awakening. The Madisons never let go of slavery, and, in the end, it brought about their ruin, and the ruin of many. Ellen Coolidge, a relative of Thomas Jefferson, whose slave plantation was bankrupt at his death, said that northerners were “at least a century in advance of us in all the arts and embellishments of life.” She added that “the canker of slavery…eats into [Southern] hearts, and diseases the whole body by this ulcer at its core.”37

  Madison knew that. Near the end of his life he was visited by Charles Ingersoll and told him that his farms were no longer productive. “Mr. Madison told me that ever since his Presidency he has been obliged to live beyond his means selling off some of his capital continually, and that he is now in debt. He spoke often and anxiously of slave property as the worst possible for profit.” Ingersoll and Madison discussed southern versus northern farming. “When I mentioned Mr. [Richard] Rush's productive farm of ten acres, near Philadelphia, he said he had no doubt it was more profitable than his with two thousand.”38

  Slavery helped to bring about the near collapse of the plantation. So had the farming recession that set in during the 1820s (just after the national recession of 1819), his imbalance of too old and too young workers, and, as the years went by, his lack of travel, increased isolation, and distance from the commercial world.

  Near the end of his life, Madison poured out his bitterness over slavery to Harriet Martineau at Montpelier amid hours of conversation. Slavery was a stain on both Virginia and America, he said. “He acknowledged without limitation of hesitation, all the evils with which it had ever been charged,” she wrote. Madison told her slavery had debauched slave girls who became mothers as young teenagers with white or black fathers for their children. Blacks were treated terribly all over America, north as well as south. White women treated their domestics as children, he said. Slaves in the lands opened up by the Louisiana Purchase were treated as “beasts.” The masters, who should have accumulated wealth from their free slave labor, instead “lived in a state of perpetual suspicion, fear and anger.” He lamented, too that in eighteen years, the American Colonization Society had removed fewer than two thousand slaves while the overall slave population of the United States increased by more than sixty thousand and, in the 1830s, was growing ever more rapidly (it would reach four million by 1860).39

  The success and profitability of slavery had given James Madison the freedom and time to write the Constitution, serve in Congress, and run the country as president, but in the end that same slavery brought about personal gloom and economic ruin for him and his entire family. In his last years, he knew that Montpelier was going bankrupt, that he had no cash reserves, and that, when he died, his wife would be left with a huge debt. She would be in ruin and slavery was the cause.

  In June 1836, a fragile James Madison turned to his wife, Dolley, nearly always at his side, and told her that he was on the “descending” side of life and that his death was not too far in the future. He was right. Dolley, frantic that he had become so depressed and weak, had summoned a doctor from Baltimore to see her husband, but the physician told her there was nothing he could do for the president; he was simply slipping away. Madison knew he had little time left. “I have outlived myself,” he wrote a friend.1

  He had outlived an era of enormous change in America, some of it due to his work as secretary of state and president and the rest of it due to the progress of the private sector, unimpeded by presidential or congressional restrictions during the administrations of Jefferson or Madison. Their long-held belief in a national government that functioned without interfering with the general economy had brought about significant progress in the country. The war caused imports to drop and homegrown goods to increase in quantity and price. The war had not only brought about a huge American navy to patrol the seas, but a much larger, and successful, merchant class of shipping. Those ships now hunted not only fish in the Chesapeake Bay but whales in the far reaches of the Pacific Ocean. The spinning machine, cording machine, steam engine, automatic milling machinery, the lathe, interchangeable parts in machines, and the assembly line in factories had all created a quick shift from a farm economy, when Washington's first term began, to the Industrial Revolution, at full throttle when Madison died in 1836. For example, the boom drove the sales of factory-made woolens from $4 million in 1810 to nearly $20 million in 1815. The number of spindles in cotton miles in that era increased sixteen fold.

  Madison, a farmer himself, acknowledged to all that manufacturing gains in America, particularly in the North, had been exceptional, and that the slave-driven plantation system was failing, even at Montpelier. It was possible, he said, and other Republican leaders agreed, that the old agricultural South and industrial North were going to merge, aided by national transportation gains, led by the brand-new railroads, which would bring about a whole new American society. America, in Madison's public life, had become a much larger nation, but that increase was due to the waves of immigrants that arr
ived daily. Madison, who was opposed to the Alien Act of 1798, had not only welcomed them with open arms, but, as secretary of state and president, and in retirement, applauded their arrival. Cities had risen dramatically in his lifetime. The population of Washington had quadrupled since he arrived, as had the population of Richmond; New York City had exploded from a mere ten thousand residents during the revolution to over three hundred thousand at the time his wife, Dolley, died in 1849.2

  He was fading. In his last days, James Madison looked out from the second-floor terraces of Montpelier at a whole new America, a whole new world. It had been a time of whirlwind change and unparalleled drama, and he had steered the United States through it, and steered it well.

  By the mid-1830s, the president had stopped talking about politics to visitors and correspondents. “The political agitations are only known to us through the newspapers. They exhibit a complexity and confusion in the state of parties and in the prospect before us, which are impenetrable to those, in retired situations, and not a little so, perhaps, to those on the principal theater,” said Dolley, adding that her husband also did not want to express his views because of the “further developments” that would bring in the simmering state of national politics.3

  Everything that Madison did now took far longer and drained his strength even more. The last actual piece of work he finished was to write a thank-you note to a man who had sent him his manuscript on a piece depicting the life of Jefferson. Madison took a very long time to write the short note, and his barely legible signature slipped off the side of the page.

  He had been ill for several years by the summer of 1836. Dolley kept relatives and friends up-to-date with his ailments and her worries for him. In 1832, she told a friend that “he has not left the house. A painful and diffusive rheumatism has confined and reduced him very much and for some days past. He has been suffering…we trust [that] will yield to the medicine.” In the spring of 1836, she wrote her sister Lucy that “my dear husband has been unusually sick for some days and is at present unable to write, or even to exert his thoughts without oppressive fatigue.” Many times she would just tell people that “my husband has been often sick of late.”4

  The president told his visitors to pull their chairs very close to the couch on which he reclined so that he could hear them. He had been unable to get out of bed or off of his couch for the last six months. Visitors were told to speak loudly and carry most of the conversation because Madison was always so tired. Still, he impressed all. “His mind was bright and with his numerous visitors he talked with as much animation and strength of voice as I ever heard him in his best days,” said slave Paul Jennings, who attended him daily in his final years.5

  Dolley cared for and nursed him constantly. A few years earlier, she wrote a friend of her need to take care of the president in a note that showed her love. “I hope, with great care, to carry my husband through this trying season of chills to that which generally renovates him. He is in better health at this moment than he was a month ago, though still feeble and confined to his room, where my time is spent in the usual efforts to console and amuse him,” she wrote.6

  No one knew how long he would hang on in the early summer of 1836. Several friends had suggested that he take new medicines, stimulants, to stay alive so that he could die on the Fourth of July in 1826, the sixtieth anniversary of the signing of the Declaration of Independence, as Jefferson and Adams had died on the fiftieth anniversary. Madison scoffed and told them he would go when he went. Just a few weeks prior to his death, he left a short memorial to his country. The president wrote “the advice nearest to my heart and deepest in my convictions is that the Union of the States be cherished and perpetuated. Let the open enemy to it be regarded as a Pandora with her box opened; and the disguised one, as the Serpent creeping with his deadly wiles into paradise.”7

  On June 28, his slave Jennings shaved him and helped him dress. Slave Sukey, as old as the president, served him breakfast in a chair in his back bedroom on the first floor. He could see the sun rays washing the backyard of the house outside his window. Sukey said later that he had trouble swallowing and she had asked him why. “It's nothing more than a change of mind, my dear,” he said, and then his head dropped to his chest, his breathing stopped, and he died.8

  His death crushed his wife, who just two weeks before told a friend who asked about the president's life, “I can only feel that he was good and perfect in all these [areas] and that nothing short of true religion can make a man perfect. He was accomplished in literature and science as his writing manifest.”9

  While he was buried at Montpelier, and mourned by his relatives, his friends, and his slaves, there was an outpouring of condolences, and veneration, for the president across the United States and in Europe. “The last of the great lights of the revolution…has sunk below the horizons…[and] left a radiance in the firmament,” wrote an editor at the National Intelligencer. Politicians all over America offered thoughtful eulogies of him, John Quincy Adams spent two months working on a two-and-a-half-hour-long eulogy that was delivered in Boston. But perhaps Henry Clay, talking to a friend years earlier, described Madison best. When asked to compare him to Jefferson, Clay said that “Jefferson had the most genius,” but that “Madison most judgment and common sense.” He added that sometimes Jefferson's impulsive behavior led him to “rash and imprudent and impracticable measures, while Madison [was] cool, dispassionate, practical, safe.” Then Clay added that Madison was, “after Washington, our greatest statesman.”10

  Perhaps his slave, Paul Jennings, who had been with him all of his life, put it best: “Mr. Madison, I think, was one of the best men that ever lived.”11

  The governor of Virginia ordered muskets fired all day long on July 1, 1836, to mourn Madison's death and commemorate his life. Throughout Virginia, and in other states, people pledged to wear black mourning bands around the sleeves of their shirts for thirty days. Dolley's friends all made plans to visit her as soon as they heard the tragic news. One of the first was Jefferson's daughter, Martha, who wrote her, “God bless and support you dear friend under your present affliction.” Perhaps the most moving tribute came from a cousin's husband, Andrew Stevenson, who wrote that “his illustrious services & public career, consecrated by the gratitude & love of his countrymen, will hand him down to posterity as one of the first and best of men. Although you have lost one of the best husbands that ever woman had, you should derive comfort in the reflection that you were so long blessed with his society and shared his happiness and glory.”12

  Madison was buried in a field a few hundred yards from his home that was splashed with sunshine each day.

  With her husband gone, Dolley was the proprietor of Montpelier, in charge of the four farms, thirty-two slaves, and a run-down plantation. She was also the sole parent of Payne Todd, now forty-four, whose problems were increasing, instead of decreasing, as he aged.

  Her grief consumed her and Dolley found it difficult to slip out of her malaise. She knew that Madison was fading for over a year, and she was prepared for his end. Her husband had laid out detailed plans for her to run the plantation after his death, three of Madison's clerks who worked on his papers agreed to stay on and help her and she convinced her niece Anna Payne, alcoholic brother John's daughter, to live with her permanently and serve as her secretary when her father sold his farm and moved to Illinois, another victim of bad times in Virginia (John said he also wanted to move to a free state and was bothered by Madison's slave policy13). Dolley went horseback riding every day for exercise and escape; she had letters of condolence from dozens of top federal and state officials, including President Jackson, that encouraged her; and she plunged into the work of selling Madison's papers to a book publisher to further cement his historical position and to make desperately needed money to pay bills. The former First Lady had much to do.

  “She has at present and will have for some months so much important business to give her attention to that I hope when she h
as time to reflect on the past her distress will be so softened as in a measure to pass away,” said her niece Anna.14

  “The important trust sustained me under the heavy pressure of recent loss and formed an oasis in the desert it created in my feelings,” Mrs. Madison wrote Henry Clay in 1837. A month after the president's demise, a sad Dolley wrote Eliza Collins Lee, “Indeed I have been as one in a troubled dream since my irreparable loss of him…. I owe his wishes, that I should be calm, and strive to live long after him.” A friend, Septima Meikleham, visited her after Madison's death and said that “the change was most sad. The house seemed utterly deserted. The great statesman, loving husband, kind master and attentive friend was gone. And we three seemed lost in the great desolate house.” She added that Dolley, aged sixty-eight, was “broken hearted.”15

  Dolly kept busy running Montpelier and was certain that as long as she was occupied she would not miss her husband. She wrote a friend that she “was involved in a variety of businesses, reading, writing, and flying about the house, garden and grove, straining my eyes to the height of my spirits, until they became inflamed, and frightened into idleness and to quietly sitting in drawing room with my kind connections and neighbors, sometimes like the ‘farmeress,’ and often acting the character from my rocking chair.”16

  She decided, probably at the urging of her son, Payne, to revive Madison's tobacco business, which had been stagnant for years. Her problem was that she had only thirty-two laborers, down from a high of 108 in 1802, and they were too old or too young to work very hard. So Dolley, who always professed her hatred of slavery, began to buy slaves with what little money she had to create a larger and sturdier workforce. She did this over the next four years, raising the number of her slaves to 103. By 1840, her tobacco crops consumed most of her time, and she was deeply involved in the farming business. She did all of the contacts with tobacco agents in Virginia, writing one in 1840 that “we have now sent you the last hogshead of tobacco from the last crop with thanks for all your kind attention” and requested fifty pounds of bacon from him “for black people [by return car] to be paid for by the proceeds.” She had advice on farming from neighbors and friends, and people sent her different strains of tobacco seeds from all over America.17