My Three Fathers

And the Elegant Deceptions

of My Mother, Susan Mary Alsop

by Bill Patten

PublicAffairs, 400 pp., $27.95

In 1996, at age 47, Bill Patten, son of Susan Mary Alsop, the author and doyenne of the great age of Georgetown, found out to his shock that he was the son not of his mother’s first husband, Bill Patten Sr., but of Duff Cooper, the British politician and diplomat (and with his wife, Lady Diana, one-half of one of the 20th-century’s most glamorous couples) with whom his mother had had an affair. Understandably poleaxed by this revelation, he embarked on a struggle to regain his bearings, which led him in turn to this book: a personal story, a social history, and a four-part assessment of his mother and the three leading men she encountered: his real father Duff Cooper; his putative father Bill Patten; and her second husband, Joseph Alsop, the well-known (and gay) American journalist, who entered into a chaste marriage with his boyhood friend’s widow, and served as the younger Bill’s stepfather and mentor until he, Alsop, died.

A personal journey, it is also a story of intrigue, deceit, and ambition, set in a background of upper-case History, in which Bill Patten’s real father is the only one he doesn’t perceive as a father; friends cuckold friends, and go on being friendly; and his mother’s second husband is not really her husband, as most people think of the word. The book refers at times to Susan Mary as a “Henry James heroine,” but this understates it: My Three Fathers in itself is three James novels, conflated and crushed into one. At times it is like The Europeans, in which American innocents (Susan Mary and the elder Bill Patten) are charmed and corrupted by scheming aristocrats (Duff and Diana). At other times it is like The Wings of the Dove, in which the elder Bill Patten plays the Milly Theale role of the American innocent doomed to die young who still wins the hearts of those who betray him. And sometimes it is like The Golden Bowl, in which a couple (Joseph Alsop and Susan Mary) wed in the interests of social advancement, and become the premiere host and hostess of John F. Kennedy’s Washington. Detailing it all is the narrator and critic, a detached and somewhat unhappy observer who longs to be in a less complex book altogether and, by its ending, has found his way out.

Susan Mary Jay was born in Rome in 1918 into a family of diplomats that went back to John Jay, the Founder, and had lived in San Salvador, Romania, and Argentina before she was eight. The family had been in Buenos Aires a matter of months before her sister Emily died, an experience from which she and her parents would never recover, and which may have bred in her the reserve, distance, and stoic detachment of which her only son later complained. Brought up in Maine, where her parents retired, she grew up a slender (if not anorexic) young woman, a serious soul and delicate beauty, with small chiseled features and dark chestnut hair. In spite of her looks and interest in style (she worked for Vogue for a few years before she was married), the impression she left on perceptive observers was one of resilience, endurance, and grim resolution.

She was a “tough, appreciative little guest,” Evelyn Waugh wrote to Nancy Mitford after she stayed with his family. “I thought of her as a brave little soldier,” the author confesses. “The woman I knew was a gritty survivor,” Edwin Yoder would write. The survivor and soldier would have her hands full when, in 1939, she married Bill Patten, whose lung problems were a perpetual problem and who also fell somewhat short of the drive and talent to make a name for himself in the larger political world that she craved. Susan Mary saw his limitations, and accepted them, without wholly forgoing her central ambitions. “Bill never had the standing in the world of power that my mother aspired to, but nonetheless she adored him,” the younger Bill tells us, adding that she later told him that, if she only craved power, “I wouldn’t have married your father, whose career was all too clear before we were married, and who was the best and bravest man I know.”

The elder Bill Patten, who had struggled at Groton, held low-pressure, midlevel jobs both before and after his marriage, and might have stayed in them happily if not for a chance meeting in 1944 with Sumner Welles—an old friend of Peter Jay, Susan Mary’s father—who pulled strings to get him posted to Paris as an economic analyst attached to the Foreign Service Auxiliary. And so in April 1945 Bill and Susan Mary landed in Paris, then the teeming hub of postliberation diplomacy, where Duff and Diana held court.

Duff and Diana—the It Couple of the mid-20th century who would have had the scriptwriters at Masterpiece Theatre working nights and on weekends—were then ensconced at the British embassy in the war-ravaged city, and at the height of their glamour and fame. Duff’s great-great-grand-father was King William IV, whose two children by his wife the queen died in infancy, but whose 10 children by his mistress, the Irish actress called “Mrs. Jordan,” did not. The sixth of these, Duff’s great-grandmother, married the eighteenth Earl of Errol, and it was her granddaughter who became Duff’s mother, after his father, a society

physician named Alfred Cooper, rescued her from the social disgrace she had courted when she ran off in succession with two different men. Duff took his brains from his father, but his libido came from his mother’s side of the family, as he evolved into a womanizer on the John F. Kennedy level, known for the scope and variety of the affairs he conducted, even while married (like Kennedy) to a stunning and erudite wife.

Born in 1890, Duff went to Eton and Oxford, joined the Foreign Office in 1916 and the Grenadier Guards the year after, survived trench warfare and came home a hero, and resumed his career, becoming an intimate of Winston Churchill and a hero to the Churchill wing of the Tory party when he resigned his cabinet post in protest over the Munich agreement in 1938. In 1919 he had married Diana Manners, daughter of the Duchess of Rutland—though not of the Duke; her natural father was Henry Cust, a philandering man about Parliament—a free-spirited soul and ravishing beauty who supported him financially in his early political efforts by touring in a play, The Miracle, in Britain and the United States. Duff adored Diana, and was unfaithful to her from the very beginning—a state of affairs from which she first suffered, but came to tolerate, and then to abet, befriending many of her husband’s diversions and consoling them when the relationships came to their end.

“Duff and Diana enjoyed an unusual but functional relationship at the center of an extraordinary world of celebrities, writers, and intellectuals,” writes the younger Bill Patten. When her son once asked her how she endured Duff’s betrayals, she answered quite simply, “They all were the flowers. .  .  .I was the tree.” In time, Susan Mary became one of the flowers and a friend of Diana’s—so close that when Nancy Mitford wrote Don’t Tell Alfred, a satire loosely based on Diana, she wrote in Susan Mary as her abjectly devoted American friend.

As far as he knows, Bill Patten was conceived during a weekend at Ditchley, the country house owned by Sir Ronald Tree (second husband of Susan Mary’s girlhood friend, Marietta) in October 1947 while the elder Bill Patten was in a nursing home in London, two years after Diana had seated Susan Mary next to Duff at an embassy dinner, “an unusual honor for a twenty-seven year-old diplomat’s wife.” The consensus is that Susan Mary had been smitten by Duff—a hero, a writer, an intellect, and her beau ideal of a statesman—while he had been less so, calling her “sweet,” “charming,” and “most attractive,” but also maintaining that “it would be dishonest to pretend that I am madly in love.”

Throughout, Susan Mary and Duff went on adoring Diana, and all three remained devoted to the elder Bill Patten, she writing of Bill’s “unfailing kindness” and Duff of his reluctance to “cloud the happiness of what has seemed to me to be the perfect ménage.” To the younger Bill Patten, who said he would always think of the elder Bill as his father, the most unsettling part was the ongoing friendship, and the acquiescence of the elder Bill in it all.

I feel disappointed that Bill tolerated my mother’s affairs so nobly. .  .  . Although my mother had told Duff .  .  . that her husband was a Puritan who would never forgive her .  .  . it appears that Bill not only forgave [her] but also got along famously with Duff.

At times, he blamed them for exploiting Bill’s weakness; at times, he blamed Bill for accepting it; at times, he respected Bill for his civilized attitude; at times, he wanted him to knock Duff Cooper out. “I feel a messy swell of emotions when I think about Bill’s response,” he confesses. “Sometimes I feel contempt, sometimes admiration. There is no clear or easy answer in my mind as to how Bill ‘should’ have reacted to my mother’s affair with Duff.”

In 1947, the Coopers were recalled to London when Susan Mary was newly pregnant with Bill. “It is hard to know whose departure my mother mourned most, Duff’s or Diana’s,” the author informs us. “She, along with some of Duff’s other mistresses, helped plan a secret farewell party for Diana before the Coopers left the Embassy in 1947, though there is no record of whether she joined her fellow mistresses in performing a unicorn ballet.” When Duff died suddenly on New Year’s Day 1954, en route to Jamaica on a cruise with Diana, Susan Mary left at once to comfort the widow. She spent the next six years devotedly nursing her husband, whose many health problems continued to worsen. Her love for Diana went on unabated, as she visited her in London 20 years later, spending her time, in the absence of servants, “scrubbing & cleaning & carrying trays up four flights.” The elder Bill Patten died in 1960, and one of the first letters of condolence came from Joe Alsop, his school friend and roommate. And the third and last of the fathers-who-were-not-quite-fathers entered the younger Bill Patten’s life.

Like Duff Cooper (and like Susan Mary), Joseph Wright Alsop V had an illustrious forebear, an addiction to power, a love of politics, the arts, and good living, and a desire to shine in the world. His maternal grandmother was a sister of Theodore Roosevelt, and as a young newspaperman in the Washington of the mid-1930s, he had been one of the few to be welcome both at the White House of cousins Franklin and Eleanor and at the mansion on Massachusetts Avenue where cousin Alice (Roosevelt Longworth) held court. After the war, he and his young brother Stewart joined forces in a widely read political column and he had emerged as a social, as well as a policy, arbiter, famous for the quality of the wine, food, and conversation served to a carefully chosen and powerful audience at his Georgetown residence. All he lacked was a hostess as refined as he was, and the death of his old friend presented a new opportunity. And so in a chapter coldly called “Nabbing My Mother” Patten describes a strange courtship that ended in a still stranger marriage, which nonetheless would last more than 12 years.

“Joe’s improbable campaign to nab my mother began with his condolence letter after Bill’s .  .  . death,” the author informs us. “Within a month .  .  . he was writing regularly. He spent two weeks in France in May 1960. .  .  . it was the beginning of a nine-month campaign.” Correctly, Bill ties this to the concurrent campaign of Joe’s friend John F. Kennedy to get into the White House, which was one of the lures Alsop held out to his bride. Apparently to her surprise, Alsop also informed her he was homosexual and that the marriage would thus be platonic, but that the merger would open up to both parties new social vistas that neither could reach on his or her own. At first, Susan Mary rejected the offer, but time (and Kennedy’s victory) may have altered her outlook. Bill Patten had died in March 1960; Kennedy won in November; and on February 19, 1961, Susan Mary married her husband’s old friend. Soon after, Bill and his sister were moved into Joe’s cinderblock mansion on Dumbarton Avenue in Georgetown, and the next phase in the story began.

Kennedy ended his first day as president by knocking on Alsop’s door around two in the morning, a symbol of Joe’s new importance in Washington, which would last something less than three years. There were visits of the Alsops to the White House for large and small dinners, and less frequent visits of the Kennedys to their house for dinners, including one in October 1962, at which Kennedy conducted talks with Russian experts during the Cuban missile

crisis. After the White House itself, and the Robert Kennedy house in McLean, Virginia, the Alsop abode was the leading social attraction in Washington, the center for people of power and influence, where substance and style converged. Susan Mary and Joe were filling the roles once filled by Duff and Diana in postwar Paris, and in 1963, Diana came to Washington where she was introduced to the Kennedys at the Alsops, and then asked to dine at the White House itself. In the high days of Camelot, Patten recalls a lunch with Alice Longworth and Truman Capote “laughing mischievously about how President Kennedy had gotten their friend Marina Agnelli to swim nude with him in the White House pool.” Other times they “joined the president and Jackie at the White House for a cozy Sunday evening dinner of fresh caviar and bottles of Dom Perignon.” And then “they dined with the president and Jackie shortly before they left for Texas,” where Jackie showed them the pink suit with blue piping she was planning to wear.

After Texas, of course, everything changed. The world that Joe promised her began to unravel, and so did the marriage, as Joe—distraught at his loss and events in the world and the country—took out his frustrations on her. He became truculent, drunk, and often abusive: Dinners turned into ordeals and arguments, as he harangued guests, and often their hostess, on the errors of their opinions and ways. Susan Mary moved out and into the Watergate (and five years later, into her mother’s big house in Georgetown) where, approaching 60, she began a new life as a writer, producing over the next few decades hundreds of essays and four books. This helped her reinstitute contact with Joe—now writing books in his own retirement—and the two resumed their former alliance, serving as co-hosts at each other’s dinners, and going out as a couple. Always, she remained plugged-in and connected, befriending Nancy Reagan in the 1980s, leaving her son amazed at her power to adapt to sudden and social changes. When she died in 2004, Bill would be surprised at the extent to which she had become a Washington legend, of sorts; but by that time he had long left her orbit and dwelt in another world.

Bill Patten writes his memoir through the prism of things that he learned only later, and how this may have changed his impressions of what he remembered can never be known. What is certain, however, is that as soon as he could, he had begun to distance himself from the high-powered and glittering worlds of access and privilege his mother and “fathers” had relished and craved. Step by step, he began edging himself out of their universe: “Launched by Joe,” as he put it, to enter his universe, he began his business life in the mid-1970s in a posh bank in Boston, wearing suits to the office and lunching in a Beacon Street club. But in another year he had taken a grittier job in real estate renovation, which led by degrees into moving to Maine. By 1977 he was living year ’round on an island and by 1979 was running a small weekly newspaper, locked into life in a small rural hamlet, a very long distance from Georgetown, or France. Doubtless his mother was mystified, but it was exactly the things that annoyed his mother that now appealed to him: “A friend .  .  . noted that I looked for what he called the ‘rawness’ in life,” Patten tells us, saying he loved his neighbors’ “rough honesty and lack of concern with appearance,” which he found such a contrast to his mother’s exquisite, mannered ménage.

On top of this, Joe’s efforts to network on his behalf had been counterproductive: “Joe’s intense supervision effectively helped undermine my own sense of career ambition. It was clear to me that I could not compete in the realm of giants that used to sit around Joe’s garden room.” What struck him as authentic looked to his mother like downward mobility, and he cites innumerable small instances that led him to believe, perhaps correctly, that she looked down upon him and his doings. Her friends were cabinet ministers in the British and American governments; his were small businessmen, teachers, and laborers who had never heard of Duff Cooper. He cringed when he visited his mother in Washington and she asked whom he wanted to see there—exposing that he knew, by her lights, no interesting or powerful people. Unlike Joe, he writes, she “made me feel like I had failed” by following paths other than those she had taken in life.

By her lights, however, it was soon to get worse. In 1994, he enrolled in a seminary, served for three years as the minister of a very small rural congregation, and from that moved on to an adjacent vocation: counseling violent men. He spends time now “in a state prison in Shirley .  .  . a maximum security prison [that] houses a large number of ‘lifers,’ ” talking to society’s outcasts who are guilty of terrible crimes. “It was a merciful thing that Joe was dead,” he writes of his decision to enter the ministry, adding, “I didn’t enroll in order to make him turn over in his grave nor consciously to make my mother uncomfortable,” but there can be no doubt that it was an act of rejection. As he would write later:

The straightforward talk of the men I work with in prisons satisfies some urge inside of me, some visceral response to the half-truths of my mother, and, to a lesser degree, of Joe.

It is a strange life for a son of Susan Mary Jay Patten Alsop (and of Duff Cooper), who was conceived in a great country house by a king’s great-great-grandson, and had been brought up in Paris and Georgetown among (in the words of the Alsops’ biographer Robert Merry) people who “took on the world.” And it is a strange book he has written, too, for several things don’t seem to add up. It would make sense for somebody gliding along easily in life to be shocked into rebellion by the Duff Cooper revelation, but he had begun his rebellion years earlier. It would make sense for someone who found out that he had a new, different father to find in that father’s life solutions to things that had seemed inexplicable. But in this case, that just deepens the mystery: The younger Bill Patten makes more sense as the son of the gentle and quiet Bill Patten the elder than he does as the son of the restless and worldly Duff Cooper. The author seems also to conflate intrigue and deception with power-seeking and privilege. But many small towns have their secrets, too: He mentions that his plainspoken friends in Maine were indifferent to Duff because they resented his privilege; but Duff and Joe and, in her later years, Susan Mary, were hardworking, diligent people who might easily have risen on merit alone.

In her late seventies, Susan Mary developed a drinking problem and was packed off to a clinic, in the course of which her secret about her son’s biological father would at last be forced out. En route to the plane, she had gone through her messages, which included calls from Mrs. Colin Powell and Mrs. Yitzhak Rabin. Her son notes this wryly, as if her real addiction had been to people in power, which he considers a fraudulent and unworthy enterprise. Better to tend one’s own garden, he seems to be saying, than to aspire to power and changing the world. But of course, the world and the garden are not unconnected, and the motives of those who dwell in the world are not always vainglorious. And it is the Powells and Rabins of the world—not to say the Duff Coopers—who make it possible for the ministers, farmers, and part-time reformers to tend to their gardens in peace.

Noemie Emery is a contributing editor to The Weekly Standard and columnist for the Washington Examiner.

Next Page