Master of the Mountain: Thomas Jefferson and His Slaves

by Henry Wiencek

  Harriet 1795, died 1797

  Beverly 1798

  unnamed daughter 1799, died 1800

  Harriet 1801

  Madison 1805

  Eston 1808

  Rumors of Jefferson having a mistress at Monticello began to float through political circles.4 In 1800, William Rind of The Virginia Federalist claimed to possess “damning proofs” of an unspecified “depravity” of Jefferson’s. The rumors took a bit more shape in 1801 when Rind’s Washington Federalist, referring obliquely to a “Mr. J.,” reported that a well-known figure had “a number of yellow children and that he is addicted to golden affections.”5 These charges may not have come from thin air. William Rind and his brother had been the wards of Jefferson’s cousin Edmund Randolph; they had spent time around Monticello and might have heard stories of mixed-race children on the mountain.6 In August 1802 a Hudson, New York, journalist, Harry Croswell, wrote in The Wasp that President Jefferson had a “wooly headed concubine.” Croswell hated Jefferson, having been convicted of libel for stating that Jefferson had secretly paid a journalist to attack George Washington and John Adams in print. (The charge was true, but he lost on the prevailing legal ground that truth was no defense against libel.) One of Croswell’s lawyers was none other than Alexander Hamilton, who may have been the source for the tidbit about Jefferson’s “concubine.”7

  The Sally Hemings scandal erupted on a huge scale a month later, when on September 1 the Richmond Recorder, another Federalist paper hostile to the president, printed a claim that Jefferson had an African-American mistress and children by her. The author of the article was James Thomson Callender, a Scottish émigré who had established himself as a political journalist in Philadelphia several years earlier. It was Callender who had taken payments from Jefferson to fund attacks on Federalists. Croswell ended up in court for publishing the truth about the payments; Callender ended up in jail, under the Sedition Act, for writing the articles. Once an ardent supporter of Jefferson’s, Callender turned against him when the president refused to grant him a patronage job in Richmond.

  Callender deployed a distinctive vocabulary and style: “hard-hitting, sarcastic, heavily satirical,” and, on occasion, “deliberately scurrilous,” according to his biographer, Michael Durey. He had the habit of taking “the most extreme position on an issue” and had a mastery of English prose “from which he extracted new forms of invective.” According to Durey, Callender possessed a “misanthropy…so thoroughgoing as to be egalitarian. Neither wealth, nor learning, nor family background could create an elite superior to the mass of mankind. His was the egalitarianism of a common depravity, premised on the belief that no social group had the moral requirements to exercise authority.” He had a “constant preoccupation with the ubiquity of corruption in American political life.”8

  One might expect that a newspaper story that looms so large over American history appeared on the front page under a banner headline, but it was tucked into the middle of page 2, under the innocuous-appearing words:

  The President

  Again.

  It is well known that the man, whom it delighteth the people to honor, keeps, and for many years past has kept, as his concubine, one of his own slaves. Her name is SALLY. The name of her eldest son is TOM. His features are said to bear a striking although sable resemblance to those of the president himself. The boy is ten or twelve years of age. His mother went to France in the same vessel with Mr. Jefferson and his two daughters….

  By this wench Sally, our president has had several children. There is not an individual in the neighbourhood of Charlottesville who does not believe the story, and not a few who know it….

  The AFRICAN VENUS is said to officiate, as housekeeper at Monticello. When Mr. Jefferson has read this article, he will find leisure to estimate how much has been lost or gained by so many unprovoked attacks upon

  J. T. CALLENDER9

  Jefferson did not respond. He had earlier said that by the time he responded to one charge, twenty more would be printed, so the effort was useless. But two weeks after Callender’s first article appeared, Jefferson’s supporter Meriwether Jones fired back in the Richmond Examiner. Jones played down the Hemings allegation, insisting it was all just partisan politics as usual and asserting that the president’s Federalist enemy Chief Justice John Marshall stood “behind the curtain,” while Alexander Hamilton lurked in the distance. A week later Jones offered a more spirited defense of Jefferson, claiming that “not a spot [has] tarnished his widowed character”10 and asserting that any number of white men could have fathered the Hemings children: “In gentlemen’s houses everywhere, we know that the virtue of unfortunate slaves is assailed with impunity…. Is it strange, therefore, that a servant of Mr. Jefferson’s, at a home where so many strangers resort…should have a mulatto child? Certainly not.”11

  In American Sphinx (1996), Joseph Ellis characterized the widowed Jefferson as an asexual man who directed his passions into architecture rather than women. Though Ellis later changed his thinking about the Hemings allegation, his assessment that Jefferson’s “most sensual statements were aimed at beautiful buildings rather than beautiful women” still resonates because it fits so well with the received image of Jefferson as a cerebral, detached gentleman.12 But there are well-documented episodes of Jefferson’s sexual aggressiveness toward a neighbor’s wife. In 1768 an old friend of Jefferson’s, John Walker, asked him to look after his young wife, Betsy, and their infant daughter while Walker took off on a long frontier expedition to negotiate an Indian treaty. Walker had no idea he had invited a viper into his home. Jefferson repeatedly pressed his attentions upon Betsy, who just as repeatedly rebuffed him. With rather astonishing directness, he continued to show his ardor for Mrs. Walker after her husband’s return, indeed while John was just several rooms away.

  When the failed encroachments later became public, Walker wrote out a statement of what had happened, with many details as related by his wife. During a visit the Walkers made to the home of a mutual friend, the ladies retired to bed, leaving the gentlemen to talk, but Jefferson, then a bachelor, “pretended to be sick, complained of a headache & left the gentlemen among whom I was. Instead of going to bed…he stole into my room where my wife was undressing or in bed. He was repulsed with indignation & menaces of alarm & ran off.” Later, after he was married, Jefferson

  yet continued his efforts to destroy my peace…. One particular instance I remember. My old house had a passage upstairs with private rooms on each side & opposite doors…. At one end of the passage was a small room used by my wife as her private apartment. She visited it early & late…. Mr. J’s knowing her custom was found in his shirt ready to seize her on her way from her Chamber—indecent in manner…. All this time I believed him to be my best [friend] & so felt & acted toward him.13

  When all of this got into the newspapers, Jefferson was compelled to admit the truth of the accusations. To deny them would impugn the honor of a white married woman and force a duel with John Walker. The president made his confession in a private letter to Walker, a copy of which he was forced to send to a member of his cabinet, who served as a silent witness of the confession. Jefferson never breathed a word of this admission to his family. When they asked him why they never visited their old friends the Walkers any longer, Jefferson lied, telling them that he and Walker had argued about money.

  Callender printed his exposés with exquisite timing: midterm elections loomed just weeks away in October. He proclaimed that Jefferson had become the Jonah of the Republican Party, and if it did not toss him over the side, the party would be “gone forever.”14 Jefferson had a hard journey from Monticello back to Washington in October, suffering “excessive soreness all over and a deafness and ringing in the head.”15 He attributed his ailments to bad weather on the road, but the incessant ringing in his head may have been the words Sally…Callender…Sally…Callender…Sally…

  Scandal sells. In the fall of 1802, with Callender at the peak of his jour
nalistic form, skillfully piling invective on Jefferson, Federalist newspapers around the country avidly reprinted his attacks, subscriptions to The Recorder soared to a thousand, then to fifteen hundred, and Callender boasted that circulation “has extended from Maine to Georgia, to the remotest corners of the state of New York, to Vincennes, and to Kentucky.” So many ads poured into the office that the weekly Recorder began to publish twice a week.16 But Callender’s fervent wish that the Republicans would toss Jefferson overboard did not see fulfillment. The electorate was not impressed by his allegations, and Republican candidates won handily in the midterm voting.

  Then, within a few months, Callender’s life unraveled. He was savagely beaten, in a cowardly fashion, by James Monroe’s son-in-law George Hay, who came up behind him with a cudgel and struck him half a dozen blows to the head. Hay had his own quarrels with Callender, but the “Dusky Sally” campaign probably accounted for at least one or two of the blows. In the court proceeding that followed, Jefferson’s nephew Peter Carr pledged bond for Hay. A small group of drunken law students invaded the office of The Recorder and threatened to burn it. Fearing assassination, Callender began keeping a gun, drank more heavily than ever, and talked of suicide. One Sunday morning he was seen staggering around Richmond, apparently drunk. Later that day, July 17, 1803, he was found dead in the shallows of the James River, in a spot where the water was only three feet deep. He was forty-five years old and left four sons who had recently journeyed from Philadelphia to Richmond to live with him. Within hours a coroner’s jury convened, examined the body, and pronounced the journalist’s death an accidental drowning, with intoxication the proximate cause.

  Such swift adjudication by the local coroner was not unusual for the time, but the burial showed haste: before the sun set that day, Callender was laid in a grave.17 To add mystery, some months earlier Meriwether Jones had made a strange, obliquely predictive remark: “Oh! could a dose of the James River, like Lethe, have blessed you with forgetfulness.”18 Either Jones could not resist a nasty parting shot at his fallen enemy, or he tried to forestall an investigation into Callender’s mysterious demise. Callender was barely in his grave when Jones wrote in The Examiner that the death had been not an accident but a suicide, “putting a miserable end to a miserable life.”*19

  The extreme notoriety of the Hemings affair generated an enormous amount of publicity, gossip, and speculation. Much of this has come down to us in various forms to create a constantly buzzing background noise of unverifiable data. In 1811 a Vermont schoolteacher, Elijah P. Fletcher, visited Charlottesville and heard “many anecdotes much to [Jefferson’s] disgrace.” One of Jefferson’s local detractors had the nerve to bring Fletcher to Monticello, where the ex-president courteously received them. Fletcher came down from the mountain convinced, as he wrote in a letter, that “the story of Black Sal is no farce—That he cohabits with her and has a number of children by her is a sacred truth,” but Fletcher did not record a single detail of any evidence he may have spotted. Later, an Italian traveler to Charlottesville wrote, “Apropos of negresses, may I be permitted to say that…I was shown a pretty one—although she was no longer young—who had beautified the last days of Jefferson”; but this also is simply local chatter that proves nothing.20

  More compelling are private remarks that were written down by one of Jefferson’s good friends, John Hartwell Cocke, a wealthy planter who aided Jefferson in establishing the University of Virginia. Cocke knew Jefferson and his family well and had visited Monticello. In his diary in 1853, Cocke—a highly religious man who despised slavery and its “corruptions”—wrote about two planters who had children with slave women and decided to send the women and children to free states, one to Ohio, the other to a Northern city. He personally knew of a score of such miscegenation cases in Virginia, he continued, and had no doubt that hundreds more could be found in the state: “Nor is it to be wondered at, when Mr. Jefferson’s notorious example is considered.”

  Cocke made another diary reference to Jefferson several years later: “All Batchelors, or a large majority at least, keep as a substitute for a wife some individual of the[ir] own Slaves. In Virginia this damnable practice prevails as much as any where, and probably more, as Mr. Jefferson’s example can be pleaded for its defense.”21

  Jefferson’s former overseer Edmund Bacon came to his employer’s defense when he gave his long interview in 1862 about his life at Monticello. Bacon mentioned that Jefferson had ordered him to give money to a daughter of Sally Hemings’s and to put her on the stagecoach to Philadelphia and freedom. Bacon allowed that there was a lot of talk in the neighborhood about this quiet manumission, with people saying the young woman, whom Bacon did not name, was Jefferson’s daughter. But Bacon insisted Jefferson was not the father and that he knew this from the evidence of his own eyes: on “many a morning,” he had seen another man, whom he did not name, leaving Hemings’s room.

  When the U.S. census taker came to the home of Sally Hemings’s son Madison in Pike County, Ohio, in 1870, he was stunned to hear Hemings declare his lineage. “This man is the son of Thomas Jefferson!” wrote the census taker on his official return.22 Three years later a local newspaper editor, S. F. Wetmore, interviewed Madison, who referred to Thomas Jefferson as “my father” in a first-person account published as “Life Among the Lowly, No. 1” in the Pike County Republican on March 13, 1873.23 Wetmore subsequently interviewed another former Monticello slave, Israel Gillette Jefferson, who supported Madison Hemings’s claim:

  I also know that his servant, Sally Hemings…was employed as his chamber-maid, and that Mr. Jefferson was on the most intimate terms with her; that, in fact, she was his concubine. This I know from my intimacy with both parties, and when Madison Hemings declares that he is a natural son of Thomas Jefferson…I can as conscientiously confirm this statement as any other fact which I believe from circumstances but do not positively know.

  Some anonymous troublemaker sent a copy of Israel Jefferson’s article to Jeff Randolph, who wrote a lengthy, outraged rebuttal but apparently never sent it to the newspaper; perhaps he had cooled off.24 The statements of Madison Hemings and Israel Gillette Jefferson saw the light only in an obscure Ohio newspaper and were promptly forgotten; they might as well have been dropped down a hole.

  In 1938, Madison Hemings’s granddaughter Nellie E. Jones wrote to Monticello saying that she had spectacles, an inkwell, and a silver buckle that had belonged to Thomas Jefferson and then to Sally Hemings. Though the then curator, Fiske Kimball, expressed interest in examining the items, the foundation’s president, Stuart Gibboney, did not think the matter worth pursuing. Perhaps in an effort to get Gibboney to pay attention to Mrs. Jones, Kimball pointed out her skill at business correspondence: “This very respectable colored woman writes a letter much more intelligently than many of our own race.” Nonetheless, Gibboney told Jones not to bother sending the artifacts, which were sold to a dealer after Mrs. Jones’s death and disappeared.25

  After some seventy-five years in oblivion, the memoir of Madison Hemings surfaced when an Ohio archivist found it in the early 1950s and sent a copy to John Dos Passos, who was then working on a book about Jefferson. Dos Passos circulated it among a small group of Jefferson specialists.26 Madison Hemings’s twenty-two-hundred-word memoir became a central item of evidence in the Hemings affair. In a scholarly article Dumas Malone and Steven H. Hochman conceded that Hemings had spoken sincerely, but they dismissed outright his claim that Jefferson was his father. They described the newspaper editor, Wetmore, as a biased abolitionist, a wily anti-Jeffersonian who had manipulated Hemings—“quite clearly, the story was solicited and published for a propagandist purpose”—but they did not discuss the text in detail.27 In The Jefferson Image in the American Mind (1960), Merrill Peterson disparaged the Hemings memoir as a manifestation of the old “miscegenation legend” engendered by “the hatred of the Federalists” and “the campaign of British critics to lower the prestige of American democracy by toppling its
hero from his pedestal.”28

  Madison’s memoir found one very influential believer. Fawn Brodie turned the Jeffersonian world on its ear in 1974 with the publication of Thomas Jefferson: An Intimate History. The author of three previous biographies, Brodie maintained that Jefferson and Sally Hemings enjoyed an intense, passionate, but necessarily secret love affair for more than thirty years. She took Madison Hemings at his word when he claimed Jefferson was the father of Hemings’s children.29

  While Hemings descendants rejoiced at Brodie’s biography, many scholars denounced it not only for the author’s conclusions but for her method, which rested heavily on psychoanalytic interpretation. Brodie brought smirks to some scholarly faces, for example, when she theorized that Jefferson must have had Hemings on his mind when he toured the fields of France and commented on the “mulatto” color of the soil.30

  To counter the avalanche of negative publicity that engulfed Jefferson after the publication of Brodie’s biography, Dumas Malone dramatically released a long-suppressed document that, he said, proved Jefferson’s innocence; it was a letter Ellen Randolph Coolidge had written to her husband in 1858 asserting that Jefferson’s nephew Samuel Carr had fathered Sally Hemings’s children. Ellen did not want her accusation against her cousin to become public. In the portion of her letter she wished her husband to show around, Ellen vaguely laid the blame for Monticello’s “yellow children” on the plantation’s “Irish workmen” and “dissipated young men in the neighborhood who sought the society of the mulatresses.” And indeed, Ellen’s letter had been kept private by her descendants for more than a century, perhaps because of her somewhat slanderous remark that her cousin Sam Carr was a “keeper of a black seraglio.” Harold Jefferson Coolidge, Jefferson’s great-great-great-grandson, had allowed Brodie to read the letter but to quote only selected passages, and he resented Brodie’s interest in Jefferson’s alleged black progeny: “I am distressed that the subject which seems to interest you most relates to the controversial matter of Mr. Jefferson’s children and I can assure you categorically that this is not a subject which I wish to have raised by making use of quotations from the letters of Ellen Coolidge.”31