The First Lady, like everyone, did the best she could to protect her husband and family when medical epidemics struck Virginia. Sometimes her exertions did little good, such as in 1831, when cholera took the lives of George and Sam Washington, her sister Lucy's two sons, who were in their early twenties.
In his late seventies, Madison began to show some signs of physical deterioration. His eyesight was not as sharp as it used to be and his hearing was impaired. He walked slower than he had previously. Yet those who met him then thought he was doing far better than they expected in an era when three quarters of men died before the age of fifty. A man who met him in 1829 remarked that he “was in tolerably good health, thin of flesh, rather under the common size and dressed in his customary black, old fashioned clothes. His form [was] erect, his step firm but somewhat slow, [he] walks without a staff, his visage pale and abounding in small wrinkles, his features well-proportioned but not striking, his head bald…his forehead of common size, his brow grey, heavy and projecting, his eyes small and faded, his nose of ordinary size and straight, his mouth rather small.”24
His wife hardly ever left his side, whether to help him with his books, arrange his schedule, care for him when he was sick, or just have an early-morning conversation with him. He told all who visited how much he loved her.25
In retirement, Madison was urged to join the Agricultural Society of Albemarle (County) and soon became its very respected president. He became absorbed in the work of the society and its farmers and even wrote a lengthy pamphlet on farming that was widely read and frequently discussed.
He worked hard on his farm, consulted with anyone interested in his opinion, took care of a very large extended family, visited friends and relatives throughout Virginia, wrote speeches, answered hundreds of letters, cared for his wife, oversaw work crews in the fields at Montpelier, and paid his son's debts. He was always annoyed by people who accused him of relaxing in retirement. The president wrote one man that “it is an error very naturally prevailing that the retirement from public service, of which my case is an example, is a leisure for whatever pursuit might be most inviting. The truth, however, is that I have rarely, during the period of my public life, found my time less at my disposal than since I took my leave of it; nor have I the consolation of finding that as my powers of application necessarily decline, the demands on them proportionately decrease.”26
Throughout his retirement, Madison's advice was constantly sought by newspaper editors and public officials, at both the state and federal levels. He always stood by the Constitution, no matter how many political schemes were hatched. One idea that infuriated him was Jefferson's suggestion that conventions be called to settle disputes between state and federal court systems. Madison castigated Jefferson for his plan and reminded him that the cornerstone of the Constitution was the ability of the US Supreme Court to overrule state courts.
The president quickly became a revered elder statesman. He kept up with all the international and national news by reading newspapers, which he had on subscription, plus magazines and, as always, a torrent of books on history and politics. Friend Richard Rush, now minister to England, sent him copies of all his diplomatic correspondence, as did President Monroe. Madison consulted with both and was deeply involved in decisions to make treaties and choose allies in diplomatic squabbles (he was instrumental in the planning of the Monroe Doctrine).
Madison quickly forgave England for the War of 1812 and urged all to make peace with the British. He became one of history's first proponents of an organization that would serve a purpose akin to the later United Nations, writing in 1820, “were it possible in human contrivance to accelerate the intercourse between every part of the globe that all its inhabitants could be united under the superintending authority of an ecumenical council, how great a portion of human evils would be avoided. Wars, famines, with pestilence as far as the fruit of either, could not exist; taxes to pay for wars, or to provide against them would be needless and the expense and perplexities of local fetters on interchange beneficial to all would no longer oppress the social state.”27
His wife helped in his work as an elder statesman. She constantly wrote the wives of diplomats around the world and invited all to visit them at Montpelier on any trip they took to America. Many did.
Madison avoided politics but complained bitterly about the rise of his former general, Andrew Jackson. He was disgusted with Jackson's uncouth manners and egomaniacal personality, and the crude, rude followers who surrounded him day and night. He was all in favor of the westerners becoming part of the political process, just not those westerners. His wife was aghast when she learned that some of her magnificent red drapes had been ruined at the riotous postinaugural party Jackson threw at the White House.
One of the most rewarding aspects of President Madison's retirement was his appointment to the board of governors of the brand-new University of Virginia, founded by Thomas Jefferson and others and built to challenge northern schools such as Harvard, Yale, and Princeton as a top institution of higher learning. Madison and Jefferson attended a meeting with nineteen other members of the board and voted to merge the proposed state university with tiny Central College, already in Charlottesville. Jefferson, with Madison's assistance, would be in charge of the design. Madison was happy to be on the board because as president he had tried to get a national university built and had always loved higher education. He was also happy to leave Montpelier and spend several days with Jefferson and other old friends.
Planning for the university, and the construction of the first buildings, went well, but the official opening of the new school was delayed due to the late arrival of professors from Europe by ship. Finally, on March 7, 1819, the opening took place with eight professors and sixty-eight students. Jefferson was elected rector, or board chairman. After Jefferson's death, Madison took over as rector. Throughout his years on the board, Madison kept busy planning the expansion of the university, bringing in new professors, and obtaining funding for it. When he died, he left the school money and his entire 4,000-book library, which became the cornerstone of one of the largest libraries in America.
The other public enjoyment Madison had was an invitation to a convention in Richmond in 1829 called to create a new state constitution. It was the first time James and Dolley Madison had left the Orange Court House/Montpelier area in over twelve years, and it was a shock to the president. While his fame was still intact, his political power was not. He arrived as an Orange County delegate and controlled no votes except his own. His views of slavery and its role in the new constitution were jeered at by many. Virginia had changed and so had the country, and Madison knew it.
The Madisons spent three months at the home of cousin Sarah “Sally” Coles Stevenson in Richmond for the duration of the convention. He and Dolley went to luncheons, parties, balls, and receptions and had a marvelous time. The Richmond nightlife was in fine contrast to the contentious days the ex-president spent at the convention, where the proslavery feelings grew hotter and hotter as each day passed. Madison chaired a committee to determine how slaves would be counted in the voting process because the state was in two very separate sections. The westerners, with far fewer slaves than the easterners, wanted a white-only vote clause, but the easterners insisted on a total population, black-and-white, count for the vote. Madison suggested a white-only vote for delegates in the lower house and a full population vote for the upper house. He was castigated by many and was shocked when even accused of treason. He fell back and compromised, offering the traditional three-fifths vote for each slave. That was voted down, and the easterners, through heavy-handed politics, grabbed power and rammed through a proslave population vote so they could control both houses of the legislature.
Madison realized that while he was venerated, he was not followed anymore. He knew, too, that his efforts at age-old compromise, which had always worked, now failed. And finally, he knew that slavery had catapulted forward in the social conscience as an
issue. It was becoming a runaway train.
He had the admiration of all in Richmond, though. At one point, dozens of men gathered around the seventy-eight-year-old former president just to introduce themselves and shake his hand. All told him how they had looked up to him over the years. He and his wife were at the center of every party they attended and Dolley, as always, was the star. The president smiled broadly at all the receptions, but on their way home, Madison told his wife that the political world of Virginia, and the United States, was changing fast. He was not optimistic about the wild change, either, and he saw dark days ahead.
Between 1809, when James Madison took office, and 1817, when he retired, Montpelier took on a new and much grander look than in previous years. Madison had spent much time adding the two wings to give the house expanded size and a very European look. The guest list of presidents, vice president, senators, congressmen, and judges gave the house a history and elegance, an importance, that no other home in America enjoyed, except for Monticello and Mount Vernon. The interior of the home was special. Madison had collected portraits, busts, and sculptures from all over the world and put them on display in the back parlor, main parlor, and dining room. Less important, but still impressive, portraits adorned the walls of other rooms in the home. Foreign visitors, who scoffed at all American mansions as poor copies of European palaces, agreed that the artwork inside the home gave it a veneer few residences in America possessed.
Some foreign visitors turned out to be not so welcome, though, such as the marquis de Lafayette. The fabled Frenchman, an important figure in the American Revolution, decided to make a grand tour of the United States in 1825. His trip was heralded by the press. “Nothing is spoken of in the North but the Marquis; he has even for a while made the people forget the four Presidents,” wrote Phoebe Morris.1
He first went to Monticello to be feted by Jefferson and then to Montpelier to be a guest of the Madisons. They staged a parade for him in Orange Court House, and he was the guest of honor at dinners at their home. His traveling companion was Miss Frances Wright, a feminist and abolitionist. Madison gave Jefferson, Lafayette, and Wright a tour of his plantation, and it was on that tour that Lafayette and Wright scolded Madison over his ownership of slaves. He told Madison that it was not only wrong for anyone to have slaves but a shame to have nearly a hundred, as Madison did. Was not the revolution fought to end Americans’ enslavement to England? How then could Americans still have slaves?
Madison was stung by the rebuke. As always, Dolley said nothing about the Frenchman's criticism. She wrote her brother-in-law that “I never witnessed so much enthusiasm as his appearance occasioned here and at our court house, where hundreds of both sexes collected together to hail & welcome him. He has promised to spend some time with us again, before he leaves the country,” blithely ignoring the dispute over slavery. Later, Wright wrote an appreciative note for the Madisons’ hospitality and included a copy of her abolitionist prospectus, which she said Madison's support would help. “[Although] from the fear of alarming the minds of those who hearing of the plan without understanding its spirit & object must meet it with opposition, it is necessary to proceed with caution,” she said. Madison ignored it.2
Dolley did not lose her good looks over the years, although she had gained twenty pounds or so as First Lady and some said that she looked heavier (“portly,” sneered one woman). The First Lady aged well. Everyone she used to know in Washington was surprised at how young she looked when she turned sixty. Her hair was not yet gray, her step was lively, her smile was wide, her yen for a good party was unmitigated, and her lust for life was unquenchable. She drifted about the house and grounds like a faint breeze, beautiful still. People who met her in her sixties and had not known her before were shocked at her grace and beauty, which had not dimmed a bit with age. And she had the energy and agility of a young schoolgirl. She walked ramrod straight, with her head held high, and she was always the object of much admiration. At parties in Richmond, held in conjunction with the Virginia State Convention, she was greeted as an icon. She wore her traditional turban to parties and dances. Anne Royall, a journalist who met her there, said that she was “tall, young, active and elegant” and possessed “warm affability.” Another remarked that she did not look like Mr. Madison's wife, but like his daughter. Others said she was “as active on her feet as a girl.”3
The former president and his wife seemed to have adjusted well to retirement back at Montpelier. One visitor wrote that they “looked like Adam and Eve in paradise.”4
Dolley spent many days of her retirement acting as a matchmaker for friends and family. She constantly tried to pair up her cousin Edward Coles, who served as Madison's secretary for several years, and a young woman in Virginia. Coles had spent an extraordinary amount of time trying to meet women and then even more time thinking about their suitability—too much time. Dolley wrote him in the autumn of 1819,“I am afraid, dear cousin, that while you and I deliberate who to choose for a wife, we shall lose some of the finest girls now grown.” She ran off a list of lovely, well-connected young women who did not have the time to wait for Coles and married or became engaged to someone else. “Still I have hopes for you,” Dolley finished optimistically, “that your future one may become manifest to reward your merits and long search.”5
Her efforts did not cease with Coles. She tried to marry off her son, Payne, for years, attempting to match him up with young women in Washington and then, when she retired with Madison, throughout Orange County—with no success. She took the two daughters of her sister Anna under her wing after Anna died in 1832. She tried to instruct the girls, through letters and in person when they traveled to Montpelier, on how to meet men and how to determine which were the best men to meet. She did not believe in luck or happenstance. “We have all a great hand in the formation of our own destiny. We must press on that intricate path leading to perfection and happiness by doing all that is good and handsome, but before we can be taken under the silver wing of our rewarding angel,” she wrote her niece Mary.6
She suggested eligible bachelors as if they were targets. Russia's minister Baron Paul Von Krudener was one. “You set your cap or curls at him…. I see no objection to your becoming Baroness de Krudener,” she told Mary Cutts.7
And she warned her girls, loudly, of young men she did not deem suitable. She called one, W. Willis of Orange County, “a good hearted ignoramus.”8
And if all of that was not enough, Dolley wrote their father, Richard Cutts, and encouraged him to get them out of the house to meet as many young men as they could.9
Very few of her “dream” couplings ever succeeded. Worse, Dolley's constant barrage of suggestions and streams of letters with romantic advice angered the young people she was trying to help. She apologized to Mary Cutts. “I asked too many questions in my last, did I not? Yes, but you are very good & amiable my dear to write me as much as you do & I value you accordingly though I may complain a little now and then.”10
Madison met thousands of people in retirement at Montpelier, Richmond, Orange Court House, Monticello, and other places. He regaled them with stories.
“I have always considered Mr. Madison, emphatically, as the sage of his time,” said James Paulding, who first met him in retirement and came to know him well.
He had not perhaps so much genius as Mr. Jefferson, but in my opinion his mind was more consummate and his faculties more nicely balanced than those of his predecessor; who though justly called the Great Apostle of Democracy. I think sometimes [Jefferson] carried his doctrines to the verge of political fanaticism. Mr. Madison had the power of condensing in his speeches & his writing in great perfection, though he did not always exercise it, for such is the appetite of the people of this country for long speeches, and discussions, that they don't like to swallow the truth in an incontrovertible axiom but prefer it strongly diluted with verbiage.11
And Paulding, like all the other visitors, was witness to the magnificent round of parties, b
alls, and barbecues hosted by the former president and First Lady at Montpelier. The whole countryside was alive with activity when they threw open the doors of Montpelier for frolicks that lasted for days. “At these feasts the woods were alive with carriages, horses, servants and children—all went—often more than 100 guests…happy at the prospect of…pleasure and hilarity, the laugh with hearty good will, the jest after the crops, farm topics and politics,” wrote Mary Cutts. They all attended barbecues hosted by others in town or nearby and enjoyed them.12
Few realized the drain on the Madisons that their whirlwind social life caused. The president, who enjoyed fragile health to begin with, was often tired after parties. Dolley was laid low by the social life, too, complaining in the summer of 1833 that “I have been more indisposed than usual this morning. I could not go to breakfast with a party of eight, two ladies among them, on account of illness.”13
In addition to the physical effects of their social life, there was a large drain on the Madisons’ assets throughout his retirement, which was because of the steep debts from family members. The president had routinely loaned money to relatives in his family and Dolley's for years. Sometimes loans were paid back, and sometimes they were not. The president had real problems in the mid-1820s, after the crop depression. He had turned down his brother-in-law Richard Cutts, a land speculator, for a large, $10,000 loan because he did not trust his judgment. Then his son, Payne, had offered to loan Cutts $4,000 if his stepfather put up the remaining $6,000. He did, reluctantly. Cutts lost all the money. Dolley wrote him a sharp letter demanding the money back.
James and Dolley Madison Page 39
