James and Dolley Madison

by Bruce Chadwick

She feared for the safety of her husband and herself, but not necessarily from the British. Americans, unhappy with the progress of the war, which the press had early on called “Mr. Madison's War,” or lack of progress, had been threatening the Madisons with harm. “Among other exclamations and threats, they say if Mr. M attempts to move from this House, in case of an attack, they will stop him and that he shall fall with it. I am not the least alarmed at these things, but entirely disgusted and determined to stay with him,” wrote Dolley.

  Back then, in late July, she never thought the day would come when she, the First Lady, and her husband, the president, would be forced to leave the White House and flee for their lives. Now, on the hot morning of August 24, that seemed the most likely course of action. That afternoon, Dolley Madison did not even know where the people scheduled to dine with her were or if they were alive. No matter where she looked on the roof with her spyglass, she saw refugees and the landscapes of a war.

  And she worried about her husband. “I am determined not to go myself until I see Mr. Madison safe, so that he can accompany me, as I hear of much hostility towards him. Disaffection stalks around us,” she wrote with a chill that day.

  She had been worried all day. “Since sunrise I have been turning my spyglass in every direction, and watching with unwearied anxiety, hoping to discover the approach of my dear husband and his friends, but, alas, I can descry only groups of military, wandering in all directions.”8

  She should have been worried. Madison had joined his troops in the rural countryside of Maryland and had given his generals a pep talk in order to get them to defend Washington. They were surrounded by several thousand American soldiers, mostly members of militia units, who seemed overwhelmed in their skirmishes with the British army. President Madison was misled by Secretary of War John Armstrong and General William Winder, who assured him that the British had no intention of marching on Washington. Thanks to that view, the governor of Maryland, entrusted by the president to protect his own state, had done little. He also had no intelligence on the British because neither the federal nor the state governments had bothered to set up a spy network. Madison, who rode about and walked through the bustling camp, greeting the soldiers, seemed bolstered by their promises and wrote a hopeful note to his wife.9

  “I have passed the forenoon with the troops who are in high spirits and make a good appearance. The reports as to the enemy have varied every hour. The last and probably truest information is that they are not very strong, and are without cavalry or artillery and of course that they are not in a condition to strike at Washington. It is believed also that they are not about to move from Marlboro,” he said.10

  Madison was placing much of his estimate of the war in Maryland on the word of Secretary Armstrong, a controversial figure throughout the war. Madison's friend and fellow Virginian, Secretary of State James Monroe, who would succeed Madison as president, continually warned Madison that Armstrong was trying to use victories in the war to make himself president, putting the safety of his own men at risk to do so.

  A few hours later, troops brought in two British soldiers who said they were deserters. Madison, taking more of a hands-on role as commander in chief, interrogated the men himself for quite some time. He learned that the British total force was just as large as he had feared and that men were still getting off large boats in the harbor in Maryland. An aide, Major Thomas McKenney, told the president that he now believed that Armstrong and Winder, a former Baltimore lawyer, were wrong. The British planned to attack and capture Washington itself, something everybody thought unthinkable. Madison was too late. All now looked hopeless. His generals told him that the Americans had to pull back from fighting the English in the field because they were poorly prepared, outnumbered, frightened, and, frankly, had no chance. “They will limit themselves to disputing the crossing of the eastern branch at Bladensburg, six miles from the capital,” wrote the French minister that day.11

  Madison, angry and tired from another long day in the saddle, now feared not only for the army but also for his wife, unprotected back in Washington. He sent her a note that she said later was an emergency letter to let her know that the British army was on its way toward her. It was, she said, “alarming.” “He desires I should be ready at a moment's warning to enter my carriage, and leave the city; that the enemy seemed stronger than had been reported and that it might happen that they would reach the city with intention to destroy it,” she wrote.12

  The president ordered his generals to send men to the capital to protect it, and they did, but orders that went through Winder and Armstrong were misdirected and misunderstood. The next day, instead of having a strong fighting force ready to meet the British at the point where local roads would take them into the capital, the American forces were split in half. General Winder had bollixed his orders and that morning, the twenty-fourth, and had placed two armies, each consisting of 2,500 men, five miles apart from each other at river crossings. Both were tired and, at their size, neither were large enough to defend the city against a well-rested and heavily armed British force of nearly five thousand men.

  Just before midnight that night, a worried President Madison received a frantic note from Monroe, whose scouts monitored every move the British made. “The enemy is in full march for Washington,” read Monroe's short note, written shortly after 9 p.m.13

  Monroe had worked hard since his arrival in Washington earlier. He found disorganization wherever he went and rode to the Maryland countryside as quickly as he could to help officers reorganize their troops in order to defend the area against the British. Monroe, who had fought with George Washington's army in the American Revolution, even changed battlefield assignments at the last minute.14

  The next day, on a hot August morning, Madison sent orders for his forces to stop the English troops at the village of Bladensburg. He hoped that if they could do so, the British would give up the quest to seize Washington, turn around, and head back to their ships. No one had much hope that would happen, even the optimistic Madison. He was in charge of amateur soldiers and didn't even have enough of them. “Extensive and pressing calls have been made for militia and we hope they will be prompt,” he wrote Monroe.15

  They were prompt but could not stem the British tide. After an initial bold attempt to hold back the British, the Americans, few with any combat training, turned and fled southward. They fired a few cannon, a few hundred rounds with their muskets and pistols, and that was it. President Madison, using his own spyglass, watched them retreat. Locals joked that it was the “Bladensburg Races.” General Winder winced. “Our force was principally militia and that of the enemy was all regulars and picked men,” Winder told a newspaper editor in an effort to defend himself.16

  The door to Washington was now wide open.

  All of Dolley's friends had left town, accompanied by the troops who were supposed to guard the White House. “My friends and acquaintances are all gone, even Colonel C. with his hundreds, who were stationed in this enclosure,” wrote the First lady as she looked around the White House.17

  Dolley waited, a last sentinel in the White House, as the day progressed and there was no sign of her husband. Had he been captured? Killed? She heard the dull, muffled sounds of booming cannon to the north, and had been informed that the American army had been routed at Bladensburg. The soldiers had fought for a while, then given up and fled. The British were headed toward Washington and moving rapidly. They might be there by nightfall. Rumors flew. Couriers kept telling people to leave town. Friends sent each other messages with slaves and servants, who hurried through the streets of the city with their notes.

  Everything was chaos that afternoon. Secretary Armstrong rode into town. He kept telling Dolley that there was no danger, that the British would not enter the city, but the mayor of Washington, James Blake, rode over to the White House twice to beg her to get out of town before the British arrived. She kept returning to the roof to look out on the desperate scene with her spyglass i
n a last-ditch effort to find her husband. Off in the distance, she could see hundreds of American soldiers fleeing the Bladensburg battle and walking down the roads toward Washington. Far over the treetops, she heard the last low blasts of British and American cannon as the Bladensburg battle died down. The American soldiers that she did see did not encourage her. They showed “a lack of arms or of spirit to fight for their own firesides,” she wrote.18

  She was right. Many of the men who did sign up to fight with the militia had little interest in the war and hated their American compatriots as much as they hated the British. The case of an unruly man dismissed from a Pennsylvania regiment just two weeks later was typical. An officer wrote that “there was a man drummed out of camp for disobedience of orders. He said he did not care a damn for all the officers [and] that he would be damned if he did not shoot some of them.”19

  Hastily written notes from friends and relatives arrived all day. Her sister Anna sent a slave running to the White House with a short letter. “Tell me for God's sake where you are and what [you are] going to do…we can hear nothing but what is horrible there—I know not who to send this to—and will say but little.”20

  Dolley waited, and waited, for her husband. She had enormous confidence that he would return, that he would not be captured, and that, eventually, he would lead America out of this war. People who knew her husband shared that confidence. Former president Jefferson wrote a friend a few months earlier, “I say so with great satisfaction, when I contemplate the person to whom the powers were handed over. I have known [Madison] from 1779, when he first came into the public councils and from three and thirty years trials, I can say conscientiously that I do not know a man in the world of utter integrity more disinterested and devoted to genuine Republicanism, than himself. He may be seconded by others, betrayed by the Hulls and Arnolds of our country, for such there are everywhere. We shall only appreciate his true value when we have to give him up.”21

  Charles Carroll, a wealthy landowner, drove to the White House shortly after 3 p.m., his carriage horses kicking up large clouds of dust from the dirt street, to urge Dolley to leave the town and travel with him to his large estate at Bell Vue outside of the city. Several other men did the same. Everyone seemed to know that the president was with the army and that she was alone at the White House. They worried about her more and more as soldiers fled town and the city was wide open to an attack. Word that the British troops were approaching the city spread like wild fire.

  Dolley told all of the White House staff except a few servants to get out of town and to find safety. A French servant named Jean Pierre Sioussat, a fifteen-year-old slave named Paul Jennings, and Dolley's trusty servant, Sukey, remained with her all day. By late afternoon, in a huge mix-up, there were no troops protecting the White House or the First Lady. Finally, in late afternoon, when there were reports that the British were on the outskirts of the capital, she finished packing. She had been loading boxes for over a day. “I have pressed as many Cabinet papers into trunks as to fill one carriage; our private property must be sacrificed as it is impossible to procure wagons for its transportation,” she wrote her sister Anna.22 She packed up her much-talked-about blazing-red drapes, a copy of the Declaration of Independence, several boxes of important government papers, some silver, a crate of books, and a small clock. She left a lot of government things behind, as well as personal possessions, such as her own wardrobe and the clothing of the president.

  “It would fatigue you to read the list of my losses, or an account of the general dismay, or particular distresses of your acquaintance,” she later wrote to her friend Mrs. Benjamin Latrobe.23

  She and her servants carried everything to the wagon standing in front of the White House. On the way out, she passed the huge, eight-foot-high and five-foot-wide painting of George Washington by famed artist Gilbert Stuart that had hung on the wall of the dining room of the White House for more than twelve years, since Jefferson moved in. The painting had been purchased by the US government in 1800 for $800 and then secured on the wall shortly afterward. She stared at it. When she redecorated the White House in 1809, she had insisted that Washington's portrait be placed on that wall as the centerpiece of the room and that portraits of all succeeding presidents should hang on the wall of the room around it. Washington, she had told her decorators, as not just the centerpiece of the room; he was the centerpiece of the nation.24

  In a flash, she saw the painting as symbolic of all that America stood for—the revolution, independence, equal rights, prosperity, and democracy. That stirring painting of George Washington, the father of the country, of her country, could not fall into the hands of the dastardly British. If they captured George Washington, they captured America. She could not let that happen. She told her servants that they had to take the huge painting with them. They begged her to forget about the painting and leave. There was little time left to save themselves. They reminded her that thousands of British troops were about to stream into the streets of Washington and that her life was in danger.

  And then there was Carroll, there again with his carriages. He fumed at her delay over the painting. “Mr. Carroll has come to hasten my departure and is in a very bad humor because I insist on waiting until the large picture of General Washington is secured and it requires to be unscrewed from the wall,” she wrote. She did not care about anyone; she took her time to get the picture down from the dining-room wall. As she did, precious minutes ticked away and the enemy moved closer and closer.25

  Hatchets were apparently used by the doorman, servant Jean Pierre Sioussat, and one of the gardeners to break the large, heavy wood frame that held it. The portrait was to be rolled up, but then someone warned that doing so would cause the paint to crack. So it was carefully cut out of the frame with sharp knives by Dolley and others. She did not want their haste to ruin the treasure.

  “It is done!” Dolley wrote triumphantly to her sister moments after she put her knife down and helped take the painting down from the wall.

  Dolley and the servants then carried the painting out of the building as passersby shouted at her to flee, that British troops had been seen at the edge of town. She now had a second fear about the painting. It would fall into British hands if she herself was captured. She needed someone else to hide it. Dolley saw two New Yorkers she knew, Robert De Peyster and Jacob Barker, a ship owner and close friend of her husband, respectively, passing the White House on horses and asked them to take the painting and hide it somewhere safe so that the British could not grab it.

  “Save that picture!” yelled Dolley to the two men. “Save that picture if possible. If not possible, destroy it. Under no circumstances allow it to fall in to the hands of the British.”26

  The men took the portrait and headed northwest. “Carried off, upheld whole in the inner wooden frame, beyond Georgetown, the picture was deposited by Barker in a place of safety. The presidential household got the image of the father of his country—by whom its chief city was fixed near his home, and by whose name it was called—was then snatched from the clutch or torch of the barbarian captors,” wrote Charles Ingersoll later.27

  Ironically, Barker found a wagon to bring some of his prized goods out of town and put the painting in the wagon. In letters, diaries, and journals, many Washingtonians later wrote that they saw the painting sticking up from Barker's furniture as he rode slowly out of town with everyone else. It seems that everybody except the British saw the portrait that day.

  A few moments later, James Smith, a freed black man who worked as a courier for the president rode up and saw Dolley in the front yard. “Clear out! Clear out! Secretary Armstrong has ordered a retreat!” he roared.28 Carroll soon returned in his coach, full of nervous family members, and picked up Dolley and her servants. He had her sister Anna and Anna's husband, Richard Cutts, with him. Their driver, Joe Bolden, then headed northwest and sped out of town, through deep, green forests, across Rock Creek, to the village of Georgetown, some five miles away.
Another servant drove a second wagon full of trunks and a bed, the only piece of furniture removed, tied hard to the back of the wagon.

  Dolley had one last mission. She told Frenchman Jean Pierre Sioussat to take her well-known pet macaw bird to Octagon House, a mansion owned by friends of hers, the Tayloes, but today temporarily occupied by French diplomats. She knew the handsome bird, a favorite for White House visitors, would be safe there.29 As they left, Carroll told them that it would be a long journey to Bell Vue; he had reports from servants and friends that the thick string of refugees had clogged up all the roads. Mrs. Madison planned to stay for a short time at Bell Vue and then head south into Virginia, over one of the Potomac River bridges. Carroll told her that the crossing of the river bridges would be very time-consuming because of the crowds of refugees trying to get over them. He heard that families had spent an entire afternoon clambering their way across the bridge, jammed with carts, wagons, carriages, horses, and buggies, all piled high with trunks and boxes.

  Dolley wrote just before she jumped up into Carroll's carriage, “I must leave this house, or the retreating army will make me a prisoner in it by filling up the road I am directed to take. Where I shall be tomorrow, I cannot tell.”30

  The events of the day raged within Dolley Madison for quite a long time. She wrote a friend later that year, just before Christmas, that she had been so depressed over losing the White House that she had not even mourned for the substantial invaluable losses she suffered in destroyed or stolen clothing and jewelry. And she was mad, too. Dolley wrote, “I was free from fear, and willing to remain in the Castle. If I could have had a cannon through every window; but alas, those who should have placed them there fled before me…my whole heart mourned for my country.”31

  The calm, cool behavior of Dolley in the middle of the panic in Washington impressed all and rekindled their admiration for her. “Mrs. Madison commanded the situation with grace and dignity,” wrote Margaret Bayard Smith. “The most valiant soul in the White House, she remained at her post, guarding its treasures, as the President had admonished her to do when he set forth for Bladensburg. Unintimidated by the sight of friends and acquaintances making their escape from the city, of the officials of the State and Treasury Departments withdrawing with valuable papers, or even by the sound of guns, Mrs. Madison calmly awaited the return of her husband.”32