by Mac Griswold
The next stop is a blank wall in the front hall, or so it seems. Andy waits a beat, which allows me to see the outline of a secret jib door cut into the garlanded silk wallpaper. He then creaks open the thick panel to reveal a walk-in safe, a vault that Andy has been told could withstand the hottest flames for six hours. I can barely keep from saying that fire would probably consume this wooden house in minutes, the timbers are so old and dry. The vault ceiling is about seven feet high, and the walls—lined with high dark chests and file cabinets—press in on us from every side. Old trunks crowd the red-tiled floor. By the single sixty-watt bulb, I can pick out one with a curved top painted the same acid blue-green as the parlor undercoat; another is covered in hide. Thousands of documents are housed here in these trunks and many others, in drawers, and in albums in which Andy has flattened some several hundred of his most precious ones, decoding them over countless Sunday mornings. (Andy, quietly determined not to be a churchgoer like his wife, has found a convincing escape.) Other papers have not been read since the original recipients broke the wax seals, or slit the envelopes, and then bundled them up with ribbon or string for reference someday. Someday in 1690, or 1790, or 1890, or today.
Andy opens what he calls “the object case.” Each dark metal shelf is loaded—a framed letter from Thomas Jefferson to Ezra L’Hommedieu about “the Hessian fly,” a crop pest; an antique meerschaum pipe; an Indian treaty of 1654, written on both sides of the parchment paper and signed with the “marks” of many Indian sachems. And there are things that are counted as treasures only by those who have stashed them here: a woman’s long, fat braid of light brown hair, a tarnished metal spirit level barely an inch long with the bubble still intact. Silver candelabra lord it over prosaic metal file cabinets. In glass-front cupboards, display dishes are heaped with brooches, strands of beads, and earrings, a stone block incised with the date 1777, a small yellow brick, sets of porcelain cups, pincushions bristling with pins. As Andy puts each precious artifact into my hands, he tells me its story, making the past as vivid for himself as for me.
We take the heaviest album into the dining room, where there is more light. I watch while Andy quietly smooths out on the mahogany table the original parchment charter for what had once been an 8,000-acre water-bounded domain—the entire island. Blobs of red wax, the personal stamps of the various parties, dot the wide sheet, from which dangles a large, handsome governmental seal. Signed in 1666 by Richard Nicolls, the first governor of the English colony of New York, the document defines the place, “by the Indyans formerly called by the name of Manhansucke Ahaquatzuwamocke and now commonly known as by the name of Shelter Island.” The island, and 435-acre neighboring Robins Island, had previously been conveyed “into the hands of Constant Silvester of the Island of Barbadoes Esqr, and Nathaniell Silvester now Inhabiting and residing in Shelter Island aforesaid Merchant,” and their two other partners in 1651. But Nicolls’s signature now affirms more than mere ownership: “NOW KNOW YEE That … I do hereby … Give Grannt and Confirme … that the said Island & premises, now is, and forever hereafter, shall be … deemed, reputed, taken and held as an absolute Intire, Infranchized Township mannor and place of it selfe in this Government.” (Governor Nicolls and his home government mistakenly imagined that the feudal system of hereditary rights bestowed with the land itself would take root across America.) For the powerful privileges of manorial squires, which included the appointment of a magistrate and exemption from taxes and military levies, the two Silvester (later Sylvester) brothers paid Nicolls a hefty sum, £150 sterling in beef and pork, probably raised on the island. I briefly wonder what happened to the other two partners, then discard the thought—there are too many other questions crowding my mind.
Such as: Why isn’t this incredible document about the very earliest colonial history of this country in a museum or a library? Like the several hundred acres of open land that Andy still owns, reduced from the original 8,000-acre domain, the creased old charter has survived intact for more than three centuries through a combination of design and accident and pure luck. I was brought here by water to be shown an eighteenth-century house and a landscape seemingly unchanged by time; I am now about to discover a great deal more. Before my eyes, the written history of this refined eighteenth-century house has now been pushed back about a hundred years to a highly experimental period of European settlement. Andy Fiske carefully slides the charter back into its heavy red leather housing.
In the west parlor, we gaze at a gorgeous nineteenth-century French panoramic wallpaper: snowy peaks, garden terrace balustrades topped with overflowing urns of flowers, and peacocks. I sign the guestbook and turn to admire the mantel. To the left of it is a closed door with two oval holes cut in the transom panel above. Walking toward the door, Andy says, “This leads to the slave staircase,” as if it were the most natural thing in the world to have a slave staircase in your early Georgian house on Long Island. A breeze suddenly blows through the window that overlooks the creek, lifting the organdy curtains like a breath. The openings in the transom become eyeholes in a mask, studying us.
The staircase behind this door spirals up to the attic, Andy remarks, pointing out how narrow and high the steps are. But nobody walks up this pinched back staircase any longer: the steps are blocked by a collection of small vases and dishes, mostly crystal and metal, which glitter in the light from a glassed side entry. Later I can’t help but notice the luxurious treads of the front stairs, whose four-inch risers are so shallow I seem to float effortlessly upward.
It begins to sink in: the “servants” who Andy said built the house were slaves. So were those who lived in the house built by Nathaniel and his wife in the 1650s. In common seventeenth- and eighteenth-century American parlance, slaves were often called servants, as were indentured servants, whose terms did not last for life and whose children would be free. As I listen to Andy’s glancing mention of the staircase, I realize I am face-to-face with slavery in the North.
In 1984, Andy was among the comparatively few who were aware of Northern slavery. His family had lived with that knowledge as his ancestors had lived with their captives. The door to memory had never closed.
Human bones were discovered in 1991 in New York City about a mile and a half north of Wall Street as builders began to clear the ground for another skyscraper. The struggle to stop construction occupied many column inches in every newspaper. Archaeologists who excavated only a portion of the area estimate that as many as ten thousand people may have been interred there. During a long century (the burial ground was opened in the late 1600s and closed in 1794), burials may have stretched for miles beyond the original five- to six-acre plot. By 2003, 419 human remains exhumed from what was identified as the old “Negro Burial Ground” had been carefully analyzed by Howard University. Nearly half of them were those of children under twelve. Placed in wooden coffins hand-carved in Ghana, once the grisly center of the West African slave trade, all 419 were reburied at the site and mourned publicly by some ten thousand visitors. The discovery of the African Burial Ground brought with it the dawning recognition that wealthy colonial society in the North—epitomized perfectly by the Sylvesters’ lucrative Long Island plantation—was initially shaped by generations of captive people, until 1827, when slavery was finally abolished in New York State.
Did slaves live in the attic of this house? Andy assures me they did. I pose my questions politely and ask whether I might see it. To avoid the knickknacks on the lowest flight of the slave staircase, we take a different route to reach the second floor. In the top hall hangs a photograph of James Russell Lowell, a committed abolitionist, his snowy beard spread comfortably over his waistcoat. He looks at me. What does Mr. Lowell think about slaves at the manor, I wonder. And why is he here?
On we go, Andy and I, through a bedroom door that leads to the top flight of the slave staircase. Climbing the steep steps makes my calves ache. Or maybe it is just that I can’t yet believe who climbed them. The attic is full of racks of cl
othes, colored lithographs in chipped but handsome frames, a tiger-skin rug complete with snarling head, a beautiful train set, hatboxes, and dozens of flowered china chamber pots. Andy says the slaves may have slept on the boards that rest on joists above our heads. Two crude ladders lie on the floor. Are they proof that somebody climbed up there? The spaces around the two massive chimneystacks are the warmest places in the house, Andy adds.
Andy is not unsympathetic to my interest in the slaves, nor is he apologetic about them. They are part of his history, his life, what roots him to this land. He is proud to have preserved all this and to know all he does about it. Standing in the dusty summer heat, he tells me about the earliest slaves, twenty-four people who are listed as property of Nathaniel Sylvester or of his partners, in Nathaniel’s 1680 will. But Andy is more interested in pointing out the detailed images of ships, probably scratched with a penknife, in the whitewashed sheathing inside the attic dormers. A sailor himself, Andy admires the knowledge of hulls and rigging displayed by the unknown carver. One dormer looks out over Gardiners Creek. Who wanted to sail away from here?
Burying Grounds
Now Andy wants to lead me outdoors on a walk to what his family calls the Quaker Graveyard. We leave the house by the front door. As we turn to follow a deeply cut turf track that runs south through some woods, Andy pauses, looks back, and points to a tall stone standing in the lawn near the giant boxwoods. “The First House stood there,” he says with certainty. Well, of course, I say to myself, Nathaniel Sylvester, the hardy seafarer and merchant who came to the island in 1651 and died here in 1680, couldn’t have lived in this fine Georgian structure. It will turn out that Nathaniel’s dwelling was more than the standard, modest, one-over-one First Period house that still can be seen occasionally in New England. Much later, I will discover it was described as “the late mansion house of Captain Nathaniell Sylvester” and stood surrounded by a throng of outbuildings. But Andy is moving on into the woods. So I leave my phantom Jacobean house, its mullioned windows glittering in the sun, to follow him. We emerge at the top of the creek, where a low metal railing encloses a small cemetery. From here we can look back toward the bridge that my friend and I had rowed under.
From what I have seen this morning, I get an uneasy sense of a history that has been prettified and mummified in family stories like the ones I just heard from Andy. This graveyard memorializes Nathaniel Sylvester and his clan as protectors of oppressed Quakers. Only later will I discover that, contrary to what Andy had told me, they were in fact Quakers themselves—and slaveholders, the ultimate contradiction. The top of an imposing table monument erected in 1884 bears a family coat of arms and a list of the manor’s proprietors—which, to my surprise, acknowledges the island’s original inhabitants, the Manhansetts. Scattered around the monument are a few gravestones. Some are carved with the winged skull that reminded seventeenth-century believers of where they came from and where they were headed, dust to dust. But Nathaniel and Grizzell’s remains are not among them. More questions …
We leave the graveyard via a bridle trail cut through a grove of tall white pines. The path is carpeted with soft reddish-brown needles; we make no sound as we walk. We skirt a freshwater swamp edged with pepperidge trees and fragrant summersweet. The fine south front of the manor house appears from time to time through the trees and across a wide meadow.
The penumbra of beauty and power that vibrates around this house bears little relation to its compact size. The south front measures only forty-two feet across. From this angle, so different from my first water view of it, the serene shingled mass and ample hip roof are perfectly framed by the two mighty chimneys just behind the ridge. Why does this place create such a profound impression? Partly because it has all apparently survived—house, garden, landscape, papers, stories, and people still around to tell those stories. In this day of You Are Here restorations and re-creations, it’s a potent mix. And partly it’s a matter of perfect proportions. Early Georgian architectural harmony and symmetry triumphed with the first flowering of the British Empire. The sense of domination over the physical world, the aura of special grace, and the confidence of the Enlightenment are twined with that empire, and with this structure. More than anything else, however, my sense of this place comes from my visit to the attic with Andy, the stories he has told me, and my queasy sense of more to come. And all this is made increasingly extraordinary because we are in a Long Island enclave where ostentatious beachfront houses on tiny parcels of land that cost millions of dollars are built, torn down, rebuilt, and flipped every year.
The unknown people who lived here and helped create this place step forward when Andy and I reach what he says some call the slave graveyard, others the Indian burying ground. As I drove in, I had noticed an old gray fence line, but missed a big boulder lying closer to the main drive. Now I see an inscription carved on the flat side: “Burying Ground of the Colored People of the Manor since 1651.” There are no gravestones in this graveyard under the pines. Andy tells me that more than two hundred people may be buried here. I don’t ask where are the records, why are there no markers? I can guess the answer. These are the ones who slipped soundlessly through history.
In later conversations, Andy will expand on what he knows about the manor slaves, acknowledging that the forced labor and hardships of many people helped to make possible the building of this fine yellow house and the fortunes of this family, still in residence after nine generations. For now, all I see is that he keeps the interior of the fenced rectangle comparatively free of underbrush. The fence is in good repair. Andy opens the fine little gate on the northeast side. The metal gate latch clicks crisply behind us. We are back outside.
Reading a Plantation Landscape
Although the lure of the manor was undeniable, it was not until I was writing a book about George Washington that I grasped what I had seen on Shelter Island. Results of documentary and archaeological research at Mount Vernon drove home the obvious: that often when Washington said “I,” he was talking about the slaves who did the work of shaping his immense plantation landscape.
Before the discovery of New York’s African Burial Ground, slavery in the North had generally been considered a passing phenomenon pushed by a pressing need to clear the land, not as a powerful social structure that lasted for more than two hundred years. Northern slavery was largely obliterated from memory because it didn’t serve the North’s version of the Civil War. Although the fight for African American freedom began in New England, the story of race relations in this region was put aside. Slavery became the skeleton in the attic.
New York City slavery comprised only one aspect of the system. By the time Andy’s house was built in the 1730s, Long Islanders owned more human chattel than any other group of colonists in the North. In outlying areas such as Shelter Island, up to half the workforce was enslaved. After the Revolution, in 1799, the new state enacted agonizingly complex graduated manumission laws. New York State’s last slaves were set free on the Fourth of July, 1827, only thirty-four years before the outbreak of the Civil War. Buying and selling continued through the years of graduated manumission: Sylvester Dering, Andy’s forebear and lord of the manor between 1785 and 1820, purchased a “Negro man Joseph” in 1810. In 1821, Dering’s widow would emancipate London, the last of the slaves at the manor. Because there was never a single cash crop nor a united slaveholding oligarchy to point a finger at, the ubiquitous presence of slavery, which shaped New York physically, financially, and socially in profound ways, was obliterated. After the Civil War, the legend of a victorious Abolitionist North expanded. As the struggle for an appropriate monument to the cemetery continued in New York City, and even before my Washington book was published in 1999, I found myself longing to be back in the Sylvester Manor vault.
Andy died of cardiac complications in 1992, at the age of eighty. Alice, in her late seventies by the time I returned in 1996 to do some research for an article about the manor garden, was still wearing a version o
f the remarkable mobcap she had worn when we first met in 1984. It turned out to be her daily attire, one of dozens in chintz or plain fabric made up for her by her dressmaker. She also wore fifties-style plastic poppit beads carefully matched to her blouse and skirt, along with diamonds and gold bracelets. And wrist-length white gloves. People on the island said, “Alice dresses up to go to the post office.” Indeed she did. On summer afternoons, crisply clad in white, she also whacked croquet balls across the part of the garden lawn dedicated to the game. She brushed her hair a hundred strokes every day, as instructed by her mother long ago, and often demonstrated to me how she could put her hands flat on the floor from a standing position. No doubt she owed her flexibility to the many years in the garden that occupied so much of her time and affection, those two acres where she dug and sweated, hollered and gave orders. An autocrat, she was also smart, funny, manipulative, outspoken, observant, and, on occasion, tactful and empathetic. She was very much the lady of the manor, loving both the place and the status it gave her.
When Alice and Andy met, both of them were just emerging from their first marriages. He was land poor, owning little except the island property and a great book collection. To economize, he lived in the back bedrooms of his house, shutting off much of it in winter to save money. Alice, the only heir to a compressed-gas fortune, saved him and his place from penny-pinching, land sales, and slow decay. Still, theirs was a love match and more; the marriage had lasted for fifty-four years.
