America's Secret Aristocracy

by Birmingham, Stephen;

  Robert and Alida Schuyler Van Rensselaer Livingston had four sons: Johannes (or John); Philip; Robert, Jr.; and Gilbert. John Livingston died young, and the remaining three boys were raised in a manner that modern parents would find quite peculiar, though it was standard at the time. Each colonial family was set up very much like a military unit, with supreme high commandership belonging to the oldest male member and authority descending through the other male members based on seniority. Thus, with his older brother’s death, Philip Livingston became his father’s chief adjutant and second in command. Philip had the authority to issue orders and mete out punishments to the next eldest brother, Robert, and so on, though all assignments and decisions had to receive the final approval of the commander in chief, their father. It was not a system designed to create sibling love, or even friendship, among the brothers. On the contrary, competitiveness and rivalry among brothers were encouraged, on the theory that this made everybody work harder for the common family cause. Pitted against each other from the beginning, the Livingston boys, not surprisingly, grew up hating each other.

  The first casualty of such a system was often the youngest son, whose authority in a family seldom exceeded that of a foot soldier and errand boy. Youngest sons—who, under the rules of primogeniture, would inherit nothing from their fathers—were usually left to fend for themselves, and those with any ability at all usually found that the most respectable occupation left open to them was that of a clergyman. Poor Gilbert Livingston, however, trained for nothing except subservience, would become the family’s first black sheep and would be written off with no more than a shrug. Among Gilbert’s several weaknesses was a fondness for gambling, and he became a familiar figure in the gaming establishments around Boston Corners. In due course, Gilbert would spend time in prison for his debts, with none of his family willing to lift a finger to help him. After prison, the rest of Gilbert’s life was spent hiding from his creditors.

  As for Robert, the middle son, he was dreamy and dilettantish. He studied law, and practiced it in a desultory sort of way, but was more interested in the finer things in life—art, music, furniture, and sumptuary affairs. The boys’ father showed little interest in, or use for, Robert, which naturally delighted Philip.

  Philip, his father’s principal son and heir, was early assigned as his father’s chief henchman, a job he seems thoroughly to have enjoyed. By all accounts Philip, who was Sarah Jay’s grandfather, became an even more aggressive landlord and rent collector than his father. Expressing his total disdain for his tenants, he described them as “worse than northern savages” and wrote, “Our people are hoggish and brutish. They must be humbl’d.”

  Leaving the untidy business of collecting rents to Philip, the first lord was able to distance himself from the cash register a bit and devote himself to the more respectable pursuit of a gentleman farmer: developing his lands. Certainly the prospects seemed almost limitless. The last glacier had stripped most of the land of its topsoil, and so it was not particularly suited for agriculture. But there were forests of valuable timber, hills filled with lead and iron ore, and waterfalls that could be used to provide power for the forging and shaping of iron as well as the grinding of grain. There were fields that could be cleared for the raising of chickens, pigs, sheep, and cattle. And finally there was the mighty Hudson River itself—a stream that ran fresh down as far as Poughkeepsie, where it picked up tidal water from the Atlantic—that teemed seasonally with spawning shad, sturgeon, and bass and that provided a natural avenue of navigational trade between upstate New York and one of the greatest harbors in the world.

  This male pecking order, with Robert, Sr., as the absolute boss, son Philip as his right-hand man who did the dirty work, and young Robert as the not very hardworking gentleman lawyer, might have remained permanent if a fortuitous event had not changed everything. All at once, young Robert became a family hero.

  How this happened is the Livingston family’s Indian-down-the-chimney story. One night at the manor house, when Robert’s father was in New York on business and Robert was staying with his mother, he became suspicious of a group of Indians moving through the trees outside in what seemed like a pattern of stealth. Young Robert lowered his lamp and got into bed, but he kept his eyes open. First he heard a noise on the roof above, and presently a pair of bare brown legs appeared through the top of the fireplace. An Indian was lowering himself into the house through the chimney, and no sooner had the entire Indian emerged on the hearth than Robert sprang out of bed, seized his shotgun, and killed the intruder. It was a little rash, perhaps, but it was clear that the Indian was up to no good and was probably an advance man for others who intended to raid the house and massacre its occupants. Robert was hailed for having saved his mother’s life, and as a reward, his father changed his will and left thirteen thousand acres of Livingston Manor to Robert.

  Having his younger brother elevated to hero status and losing a sizable share of his patrimony as a result did not sit well with Philip. The enmity between the brothers deepened.

  Meanwhile, the boys’ father had embarked upon a venture designed not only to make the family even richer but also to add luster—even glory—to the Livingston family name in the first immigrant generation. This involved the first lord’s friend and fellow Scotsman Captain William Kidd. Captain Kidd, it should be remembered, had started his career as a respectable merchant seaman, had gone on to invest in Manhattan real estate, and by the late 1600s lived in a splendid house and was regarded in every way as a proper, even prestigious, member of New York society. It was Robert Livingston’s notion that Captain Kidd should put out to sea again for the purpose of catching and bringing down pirate ships in the Red Sea. Kidd liked the idea, and with Robert Livingston spearheading the venture, a syndicate of investors was put together to finance the enterprise. It all sounded very respectable, even patriotic, and the Crown itself approved, giving Kidd permission to capture not only illegal pirate booty but also to attack and raid any enemy—i.e., French—ships that he might encounter. Naturally, the Crown would accept a percentage of the loot. Captain Kidd set off on this mission in the Adventure Galley in 1696, as his backers rubbed their hands in glee at the prospect of dividing millions in pirate treasure.

  What went wrong has never been entirely clear, but it began to seem that pirate ships were in short supply and that Kidd had decided on his own that rounding them up was not where the real money was. To the dismay of his investors in New York the shocking news came back that Kidd himself had become a pirate. Even worse, he had had the poor judgment to attack and loot at least two legitimate foreign ships that were not French at all but were flying the flags of Great Britain’s allies. The Crown was outraged and set out to capture Kidd and bring him to trial.

  When this was finally accomplished, Kidd added treachery to his miscreancy by testifying against his former backers and financial partners. Piracy pure and simple, he swore, had been the point of the scheme all along, and the whole thing had been masterminded from the beginning by Robert Livingston, first lord of Livingston Manor.

  Of course there will always be the possibility that Kidd was telling the truth. History does not tell us, because Robert Livingston was never formally charged with piracy or brought to trial. And fortunately for Robert Livingston, and unfortunately for Kidd, a document that could possibly have cleared Kidd was inconveniently “lost.” The court did not believe him, and Kidd was hanged in London on May 23, 1701. But, even though Livingston hotly denied Kidd’s charges and was, in a sense, exonerated of them, the embarrassment of the Kidd affair would continue to becloud Robert Livingston’s career with suspicions and rumors for the rest of his life.

  Another rumor would also die hard—that Livingston had managed to slip aboard Kidd’s ship while it was hiding in Long Island Sound and make off with most of Kidd’s stolen treasure, which Livingston had buried here and there about his manor grounds to keep it from his fellow investors. To this day, hopeful souls with metal detectors
can be spotted roaming about Rensselaer and Columbia counties, looking for the buried gold of Captain Kidd. If any was buried there, it has never been found.

  Whispers about piracy would prove useful to Robert Livingston’s growing number of enemies. Because of these insinuations, it was not until many years later, when he was sixty-three, that Robert was rewarded with what nearly every Livingston since has thought of as his natural due: high public office. And even then the post was not as lofty a one as Robert may have thought he deserved. In 1718, he was elected speaker of the New York provincial assembly. But the Captain Kidd “scandal” would be just the first of many that would stain the Livingston family’s reputation for years to come.

  In the meantime, the battle of the Livingston brothers continued. At the heart of it were the unequal shares of land and money the brothers would divide when their father died, but the fight was fought on every conceivable level. And the fact that the boys were required to live in close proximity did not ease the bitter situation. Philip Livingston had married “properly,” to the aristocratic Catrina Van Brugh, of whom his parents heartily approved. He then promptly asserted his masculinity by siring, in short order, six sons.

  Robert, after a series of dilatory romances with young women, all of whom his parents found thoroughly unsuitable—and none of whom lasted long—finally made an even more unsuitable marriage in 1717, to one Mary Howardon, of no family at all. Furthermore, not long after marrying Mary, Robert more or less abandoned her, though their union did produce one child, a son named Robert R. Livingston. (In the family’s habit of doubling names, Robert R.’s full name was Robert Robert Livingston, and it was his son, Robert R., Jr., who was John Jay’s early law partner.) When the boys’ father died in 1728, he must have left the world convinced that Philip’s would be the strong line of Livingstons and Robert’s would be the weak line.

  However, once Robert had received the inheritance that his youthful heroic act had earned him, the tables began to turn. Robert quietly began acquiring more parcels of property, and presently he was building a stately mansion of his own on his land. The brothers continued to quarrel over the most petty issues. When Robert’s mansion was finished—a huge red brick Georgian house on a bluff overlooking a great curve of the Hudson—he wanted to call the estate Callendar after the Scots earls from whom the Livingstons were now grandly claiming descent. Philip, as the new manor lord, refused to let him use this name, claiming it was too lofty for a mere second son. Next, Robert obligingly proposed Ancram, the name of the Scots village where their grandfather had once preached. Once more Philip said no. And so Robert turned from Scots sources to French, with which his bossy brother could find no fault, and named his new place Clermont. It was a name that would go down importantly in American history, and Clermont the place remains today. This Robert Livingston would from then on be known as Robert of Clermont.

  In the meantime, while Robert of Clermont was patiently trying to find a name for his estate that would pass muster with his brother, Philip, second lord of Livingston Manor, had ambitiously embarked on a project of his own, which he hoped would make him enormously rich. This was to mine the iron ore his father knew was lying beneath the manor lands and to build a forge to shape it. More and more of Philip’s time, energy, and money were going into his forge, and yet by 1743 Philip’s profits had been minimal. (It would take the Revolutionary War to make the forge profitable.) Philip was running up huge debts, and his creditors were making impatient sounds. To add to his woes, settlers on his manor lands were not arriving in sufficient numbers to generate the rents he needed to keep the place going. And on top of everything else, the number of tenants who were delinquent in their rents was growing; setting fire to their houses or driving off their cattle did not seem to offer any immediate financial solution to the problem. How galling it must have been to Philip to watch his younger brother next door getting steadily richer, living a life of luxury and ease and hardly working at all while he, the furiously hardworking Philip, was growing poorer by the day! And how very pleased, and even a bit smug, Robert of Clermont must have felt from his side of the fence watching the same thing happen! Was it possible that Robert of Clermont’s line would one day be known as the rich line of Livingstons and Philip’s would become the poor line? It most certainly was.

  Soon the brothers were battling again, over water rights from the stream that divided their two properties. When Philip’s forge and foundry, and workers’ houses surrounding them, were finally completed, he got even with Robert by naming the new little ironworks Ancram, a name that Robert had wanted for his house.

  The Battle of the Livingston Brothers would not end. Instead, it gathered momentum—a momentum that would carry their bitter rivalry on into the next generation as each brother seemed to enter a competition to see who could produce the most successful and distinguished offspring. Philip, with six sons to Robert of Clermont’s one, must have assumed that he handily had the lead. And it is true that at least three of Philip’s sons had major careers in government and politics. Philip’s son William, who was Sarah’s father, of course represented New Jersey in the First and Second Continental Congresses, though he left Philadelphia abruptly in 1776 to avoid voting on the Declaration of Independence, about which he had some Tory misgivings. In 1778, however, he was elected the first governor of the new state of New Jersey and was so popular in that post that he was reelected governor continuously until his death in 1790.

  His older brother, Peter Van Brugh Livingston, was a successful merchant and Whig leader in New York and one of the founders of the College of New Jersey, now Princeton University. William and Peter’s middle brother, Philip Livingston, Jr., was a member of the Continental Congress from 1774 until his death and, with fewer doubts than his brother William, signed the Declaration of Independence, though without the confident flourish of John Hancock.

  And yet it was Robert of Clermont’s one son, Robert Robert, who was able to do more for the family than all of his cousins combined. He was, like his father, a lawyer, but unlike his father, he was a brilliant and ambitious one. In the family, this would lead to his designation as “Judge Livingston.” But even better than that, he married the richest girl in America. What his old uncle Philip must have felt when he witnessed that event can only be imagined.

  She was Margaret Beekman, the eighteen-year-old daughter of Colonel Henry Beekman and the sole heiress to the immense Beekman patent, a quarter-million-acre tract comprising most of what is now New York’s Dutchess County. Margaret Beekman was already a Livingston in a complicated sort of way. Her father was her new husband’s uncle Gilbert Livingston’s brother-in-law. Also, her mother was a Livingston and an heiress as well, the daughter of one of the original Robert Livingston’s nephews, another Robert, who had followed his successful uncle from Scotland. Thus the newlyweds were second cousins once removed, the bride’s mother was the bridegroom’s aunt, and a total of three great fortunes—two from Livingstons and one from the Beekmans—joined hands in holy matrimony in 1744.

  The consanguinity between Henry Beekman’s daughter and Robert of Clermont’s son could have been even closer if love had been allowed to take its course. Originally, Mr. Beekman had wanted to marry Philip and Robert of Clermont’s sister Joanna, but Joanna’s parents had rejected his suit. And so Beekman had countered by marrying Joanna’s cousin Janet and got into the Livingston family anyway, through the back door, as it were. It seems that Robert and Alida Livingston objected to Beekman not because he had no money; he was very rich. But they didn’t care for his looks. Henry Beekman had a prominent birthmark on his face, and the Livingstons branded him “disfigured.”

  Still, galling though it must have been to the jealous Philip, there seemed no questioning the fact that if Judge Robert and Margaret Beekman Livingston had children, as indeed they would, the Robert of Clermont line of Livingston descent would be far richer than that of the poor manor lord.

  But if Robert of Clermont allowed himself to g
loat over having so far outdistanced his older brother, there is no record of it. In any case, Robert of Clermont was far too busy buying the Catskill Mountains.

  10

  Weak Blood

  Among all the first American families, those of Scots descent show a certain dominance—families begotten by big, robust, hard-driving men of the earth or of the sea who showed no shyness about their ability to impregnate their womenfolk with gonglike regularity and to produce, from these couplings, that essential commodity for carrying on their enterprises and their name: sons. One of the most notable of these men was the single progenitor of what was to become the leading family of colonial Virginia, the Randolphs. Like the descendants of Robert Livingston, the descendants of William Randolph today number in the thousands, thanks largely to the fact that William Randolph sired seven sons and two daughters, who gave him a total of thirty-seven grandchildren, a feat that was even more astonishing considering the 50 percent infant mortality rate in the seventeenth century.

  But then we have a mystery. Why is it that, in some families, an ability has been maintained to produce at least one or two men or women of distinction in each succeeding generation, while in others the blood has begun to run so thin that talent—for business or public service or whatever—seems to have all but disappeared? In every family, of course, there is a different answer or cluster of answers, and yet the comparison between the Livingstons of New York and the Randolphs of Virginia is interesting. A Livingston has been mayor of New York City and later served as minister to France and secretary of state in Andrew Jackson’s cabinet. Another Livingston also served as minister to France and had a hand in negotiating the Louisiana Purchase, even though, as we shall see, another Livingston scandal would somewhat taint that enterprise. New York City abounds in Livingston monuments, including B. Altman and Company, the St. Regis Hotel, the Hayden Planetarium, and the buildings that cover three of the four corners at Broad and Wall streets, all of which came from the drawing board of the architect Goodhue Livingston, who also designed the Mellon Bank in Pittsburgh, the Mitsui Bank in Tokyo, and the Palace Hotel in San Francisco. Robert Linlithgow Livingston, Jr., is a vigorous young congressman from Louisiana, while other distinguished Livingstons and Livingston relatives and relatives by marriage today include Carter Brown, director of the National Gallery of Art, S. Dillon Ripley, curator of the Smithsonian Institution, and John Jay Iselin, for many years the head of New York’s Channel 13.