by Lynne Cheney
At the end of the year, when electoral votes were counted, Thomas Jefferson was reelected president of the United States by 162 to 14. It was a time of nearly complete victory that Madison as well as Jefferson must have found thrilling, but like most heady moments this one contained hints of trouble ahead. Some were coming from abroad, where Great Britain, at war once more with France, was in need of naval manpower. With increasing frequency its ships were stopping American vessels and demanding that all on board muster for inspection by a British officer, who would decide which of the sailors would be impressed for service in the Royal Navy. In truth, there were many British deserters serving aboard American ships, where both conditions and pay were better, but the British, not recognizing the right of a citizen to expatriate and adopt another nationality, also seized many who had been born in Britain and become naturalized Americans. In the sweep to man the British fleet, many who had been born in the United States were impressed as well. A man’s fate could depend on how he pronounced the word “peas.” If he said “pase,” odds were he would be seized.55
Closer to home, fissures in the Republican Party were becoming evident. Southern conservatives, particularly in Virginia, were increasingly convinced that the Jefferson administration was too accommodating. It was not carrying on the fight to minimize federal power with sufficient vigor. When the most influential conservatives assigned blame for this drift away from principle, James Madison’s was the name they mentioned first, and by the time Jefferson was sworn in for his second term, they were searching for someone who could keep the secretary of state from becoming his successor.56
Chapter 14
PORTRAITS
“STUART IS ALL THE RAGE,” one woman reported. “He is almost worked to death, and everybody afraid that they will be the last to be finished.” In 1804 and 1805, a steady stream of carriages brought Washington dignitaries and their wives to a two-room studio just off Pennsylvania Avenue where the famed Gilbert Stuart was painting furiously. Among those sitting for him were the secretary of state and Mrs. Madison, he fifty-three and she thirty-five, but in their portraits the age difference is not apparent. Stuart, known for flattering his subjects, overlooked the web of fine wrinkles that was beginning to give Madison’s face a parchment-like quality and perhaps a few other signs of middle age as well. “Quite pretty he has made us,” Dolley Madison declared.1
Stuart’s portrait of Madison captures the “self-possession” that Jefferson, who knew Madison so well, described, but one can also see why people first meeting Madison found him daunting. The man in the black suit, white shirt, and white cravat looking out from the canvas is the man of whom Brissot de Warville wrote after a 1788 introduction, “His look announces a censor.” This is, one imagines, the face Madison presented to young George Tucker, who was disconcerted by his “sternness” when he met Madison at Monroe’s home sometime around 1800. Madison’s expression was probably occasioned in no small part by Stuart himself, who tried to relax his subjects with a steady stream of chatter—a tactic that would probably have made Madison more standoffish. In this 1804 portrait, Stuart caught the public man, who was, his friends agreed, quite different from the private person. Friends saw twinkling eyes and a smile that lit up his face. They heard him joke and laugh and provoke laughter in others. After Tucker had known him for a long time, he wrote, “He had an unfailing good humor and a lively relish for the ludicrous which imprinted everything comic on his memory and thus enabled him to vary and enliven his conversation with an exhaustless fund of anecdote.”2
For her portrait, Dolley wore a cream-colored gown with an extremely low neckline, as was the latest French fashion. This portrait, later displayed in the White House, would be, at least until the time of this book, unmatched among paintings of first ladies for the amount of flesh displayed and the apparent absence of underpinnings. Dolley looks out from the painting unfazed by the exposure, although very self-aware. Looking at the portrait, one understands why her niece Mary Cutts noted that her mouth “was beautiful in shape and expression.” Her upper lip is a perfect Cupid’s bow, and her slight smile wonderfully enigmatic.3
Dolley’s younger sister Anna also went to Stuart’s studio. Both she and her new husband, Congressman Richard Cutts of Massachusetts, a stolid fellow, had their portraits done. In hers, Anna wears a gown cut as low as Dolley’s, though she appears less comfortable in it. By the time he painted Anna Cutts, Stuart might, despite all the décolletage, have grown a little weary of painting women seated in his studio armchair, their hands folded in their laps. The billowing drapery behind Mrs. Cutts forms a profile, one with a prominent nose. Stuart later admitted that the profile was his own, thus acknowledging that he had entertained himself by putting himself into the Cutts portrait.4
The British minister and Mrs. Anthony Merry also visited Stuart. Dolley Madison had called Mrs. Merry, who was widely blamed for being at the bottom of the imbroglio over precedence, a “strange lass,” but her portrait shows a handsome, confident woman. In contrast to the French fashion worn by Dolley Madison and her sister, Elizabeth Merry wears a white chemisette tucked into her bodice so that it rises up to cover her bosom and frame her face.5 Stuart also painted Spanish minister Don Carlos Martínez de Yrujo y Tacón and his wife, the former Sally McKean. The marquis, possibly displaying more arrogance than anyone else Stuart ever painted, has his hand tucked in his jacket in the fashion Napoleon made famous. His wife, who had grown up with Dolley Madison in Philadelphia and remained her good friend, wears a gown in the French style.
The Merry and Yrujo portraits, particularly in the contrasting dress of the wives, could be seen as symbolizing the breach between Britain, which was at war with France, and Spain, which was essentially a French satellite. But on one point the British and Spanish ministers were in accord. They were furious with the United States. Yrujo had taken Merry’s side in the dispute over protocol and even poured oil on the fire with suggestions of events they might boycott. A volatile man, Yrujo was enraged by U.S. claims that West Florida had been part of the Louisiana Purchase, so much so that at one point he stormed into Madison’s State Department office and loudly upbraided the secretary. To protest American policy, he pulled up stakes in Washington and moved to Philadelphia—but only after he and his wife had finished their sittings with Stuart.6
Yrujo was a minor problem for Madison compared with another of Stuart’s subjects: John Randolph of Roanoke, a Virginian who hailed from a plantation called Bizarre. In Stuart’s painting, Randolph looks like a dark-eyed schoolboy, though he is thirty-one. A youthful illness had arrested his sexual development, leaving him beardless and with a high-pitched voice. He was apparently impotent, but woe be it to anyone who tried to embarrass him about it. When one enemy taunted him for being sexually incapable, Randolph responded, “Does the honorable gentleman mean to boast here in this place a superiority over me in those parts of our nature which we partake in common with the brutes? I readily yield it to him. I doubt not his animal propensities or endowments.”7 A brilliant man, Randolph developed a scathing oratorical style, and he used it—and threats of duels—to intimidate, which made him an effective if unloved leader in Congress. As chairman of the Ways and Means Committee, he had at first served the Jefferson administration well and faithfully, but then he turned his vitriol on Secretary of State James Madison.
Madison’s original sin was to serve, together with Albert Gallatin and Levi Lincoln, the attorney general, on a commission to sort out the great Yazoo landgrab. The scandal had begun when the Georgia legislature sold its “western lands”—most of what is today Mississippi and Alabama—for pennies an acre to speculative groups called the Yazoo companies. Georgia voters, realizing they had been swindled, elected a new legislature that repealed the contract, even though in the meantime lands had been sold to third parties who stood to lose from the overturned sale even if they had no part in the corruption. Madison and the other two federal commissioners agreed to a compromise whereby Georgia, i
n return for ceding all the lands between its western border and the Mississippi River to the federal government, would receive $1.25 million. The commissioners thoroughly documented the corruption involved in the original transaction—every member of the legislature who had supported the sale, save one, had been bribed—but recommended nonetheless that five million of the thirty-five million acres deeded to the United States be set aside and used to settle claims arising out of that sale. Some of those settlements would go to Yazoo companies, and although they would get only a small portion of their claims, the commissioners were likely aware that even that would excite controversy. But the Yazoo companies, shady though they might be, had to be reckoned with. They had contracts—which the Constitution forbade states to overturn.8
In Randolph’s eyes, the commissioners’ compromise was an “act of stupendous villainy,” which attempted “under the forms and semblance of law to rob unborn millions of their birthright and inheritance.” With his unerring gift for spotting anything that even vaguely smacked of hypocrisy, he denied that there were innocents involved. Those who had bought land in secondary sales had been gambling, he said. They had been speculators, many of them northern ones, no less, intent on taking advantage of the South.9
Randolph laced his accusations, in the words of Senator Plumer, who watched the House debate, with “allusions to brothel-houses and pig sties,” the kind of language no member wanted directed toward him. With his intimidating tactics, Randolph kept the Yazoo matter from being settled for years. Agreement would be reached only after the Supreme Court ruled in 1810 that the Georgia Repeal Act was unconstitutional. But in the meantime Randolph gained an epithet to hurl at Madison. “Yazoo man,” he called him, ignoring the fact that Secretary of the Treasury Albert Gallatin probably had more to do with the compromise recommendations than did the secretary of state. Randolph’s focus on Madison, the biographer Irving Brant suggested, lies “in the realm of psychiatry.” Jefferson’s biographer Dumas Malone offered a general concurrence: “It would be impossible to give a wholly rational interpretation of the conduct of as neurotic a person as the Majority Leader.”10
But there were ideological differences that Randolph would enthusiastically point out in the years ahead and personality differences as well. Randolph, a most immoderate man, hated what he called Madison’s “cold and insidious moderation.” The impeachment trial of Justice Samuel Chase was a case in point, with Randolph making a deep emotional investment in the outcome while Madison stood safely aloof. Jefferson, having heard of Chase’s delivering a political harangue to a grand jury in Baltimore, privately suggested that Congress do something about the associate justice. A willing Randolph led the way to a House vote of 73 to 32 to impeach Chase, who had a long record of highly partisan and deeply intemperate behavior, and in February 1805, after Jefferson had been reelected but before he was sworn in, Randolph presented the case against Chase at an impeachment trial in the Senate. Vice President Aaron Burr, under indictment in two states for killing Hamilton, presided, and senators seated themselves to his right and left on crimson-covered benches. When Justice Chase appeared, white-haired, red-faced, and suffering from gout, he was directed to a box in front of and to the left of the vice president. Randolph, from a box opposite, set forth articles that related, among other things, to Chase’s “highly indecent” charge to the Baltimore grand jury and the “spirit of persecution and injustice” he had exhibited in the trial of Jefferson’s nemesis James Callender.11
Randolph was a gifted orator, but he was no lawyer, and he was up against some very good ones, including Luther Martin, the famously bibulous and passionate Marylander who had filibustered at the Constitutional Convention. Randolph’s speeches in favor of impeachment reached moments of eloquence, but they failed to refute Martin’s arguments. Indeed, Randolph had an almost impossible case to make. Chase might have been unfit to be a judge, but that did not mean he was guilty of “treason, bribery, or other high crimes and misdemeanors,” the standards set forth for impeachment in the Constitution. At the end, realizing he had lost, Randolph broke down, with “much distortion of face and contortion of body, tears, groans, and sobs,” as Senator John Quincy Adams described it.12 The prosecution managed to get a majority vote in the Senate on three of the eight articles of impeachment, but on no article was there the necessary two-thirds vote to convict.
Chagrined, Randolph rushed over to the House of Representatives, where he denounced Chase and the Senate and proposed an amendment to the Constitution that would allow the president to remove a judge upon a majority vote requesting him to do so by both houses of Congress. Another of the impeachment trial managers proposed an amendment that would allow state legislatures to recall senators and declare their seats vacant. Neither motion had the slightest chance of passing, and later that evening Senator Adams, encountering Secretary of State Madison at a social event, noted that the secretary “appeared much diverted at the petulance of the managers on their disappointment.” Whether word of Madison’s amusement reached Randolph is uncertain, but it would have been crystal clear to him that neither Madison nor the president intended to defend him. When a group sympathetic to the impeachment wrote to Madison to protest that he had not condemned Chase’s acquittal, he replied that it would be improper for cabinet officers “to pronounce for public use their opinion of the issue, however little disposed they may be to reserve.”13 It was a perfectly valid point, but Randolph must have felt, to use modern political parlance, that he had been hung out to dry.
One might have expected Jefferson to be the object of Randolph’s anger. He, after all, had suggested that something be done about Chase, only to cut Randolph loose when the impeachment trial failed to convict. But the president had just overwhelmingly won a second term, and his popularity made him an uninviting target. It was Madison, who had remained invisible as far as the impeachment proceedings were concerned, whom Randolph blamed.14
Particularly after Jefferson let it be known that he had no intention of running for a third term, Randolph and his allies turned their fire on the secretary of state. Was the central government growing too strong at the expense of the states? Blame Madison. Was our foreign policy wrongheaded? Again Madison was to blame. Randolph and his allies cast Madison, as biographer Ralph Ketcham described it, “in the role of the evil genius who debauched the president from his principles.”15 The secretary of state was, without doubt, influential, but even at this early stage in the nation’s history, politicians understood the advantages of blaming what they perceived to be an administration’s shortcomings on someone other than the president. Such an approach managed two things at once: making the person blamed seem threatening while simultaneously making the president look weak.
• • •
IN THE SPRING OF 1805, Dolley Madison developed a painful abscess on her knee that would not heal. She was confined to bed, doctors were called in, and they applied a caustic paste to the abscess to cauterize it. A month later, however, she was still writing to her sister Anna Payne Cutts from bed. The growth had advanced to “a dangerous state,” she told Anna. But then it seemed to improve, and the Madisons hoped to travel to Montpelier in mid-July.16
Instead, they found themselves on the road to Philadelphia so that Dolley’s knee, grown worse again, could be treated by one of the most famous doctors of the day, Philip Syng Physick. Because riding in a carriage was one of the few activities Mrs. Madison could undertake, she enjoyed the journey—until James was stricken. As Dolley described the awful event to her sister Anna, “On our way one night he [was] taken very ill with his old bilious complaint. I thought all was over with me. I could not fly to him and aid him as I used to do—but heaven in its mercy restored him next morning.” Although James had apparently not had a sudden attack in three years, he pretty clearly had one on this trip. Dolley’s language indicates her urgency, even panic, as well as something of his symptoms, with the phrase “restored him” suggesting that he was temporarily not himself�
��as he would not have been had his “intellectual functions” been suspended. The word “bilious” ties this event to others in the past, including the “bilious attack” at the Virginia ratifying convention in 1788, Madison’s concern about “a bilious attack to which I am become very subject” in 1790, his worries about “symptoms of bile” in 1791, and a “slight attack of bile to which my constitution is peculiarly prone” in 1801.17 Referring to bile, as people had done since ancient times in connection with epilepsy, seems to have become a way of talking about his sudden attacks. Friends would know what he meant, and those not well acquainted with him would not be given too much information.
Madison seems to have recovered quickly, and he and Mrs. Madison continued on their way, but once she was settled in Dr. Physick’s care, she started worrying. Three years since his last attack was long enough to have grown complacent, and the shock of what was apparently another filled her with sudden dread. She told a friend that she was “often miserable with fears” for her husband’s health. Both James and their friends in Philadelphia tried to laugh her anxieties away and convince her they were both safe, and after a time she seemed to grow more confident.18
Dr. Physick splinted Mrs. Madison’s leg so that she had to lie flat in bed and applied caustic three times, “a sad thing to feel,” Dolley wrote, “but it does everything that the knife could do.” In the end, although she resisted it, Physick performed minor surgery, and both Madisons settled in, waiting for her to heal.19 Committed as he was to his work, Madison stayed in Philadelphia for three months.
