Carl Colby Sought His Famous Father and Found Himself

CIA director William Colby (right) testifying before Congress.

The poster for The Man Nobody Knew.

This Thursday evening, an exclusive group of Washington players will gather at a private and mysterious downtown club to listen to the memories of a man whose late father was infinitely private and, for almost his entire adult life, a mystery to him. The exclusive group is the Empire Salon, the mysterious location is the Alibi Club, and the son is Carl Colby. Who is the father? He’s William E. Colby, former career intelligence officer and onetime CIA director, who died almost 15 years ago in an apparent canoeing accident. Carl has taken his father’s intriguing, enigmatic, and often confounding life and made it into a very personal documentary, The Man Nobody Knew.

Its subtitle could be: “A film that transformed the filmmaker, at 60, from a man into a boy.” This is how Carl describes what the film means to him, as it developed from a historical film into one that delves deeply into his family’s private story. “I needed closure,” he says. “I was growing up during the making of the film. At the end, the boy is gone and the man shows up. Me.”

Five years ago, Matt Damon and Angelina Jolie starred in a movie called The Good Shepherd, a fictional story that traced the birth and formative years of the Central Intelligence Agency. Damon played a dedicated operative who in his private life was elusive to his children and seemingly disinterested in his marriage. The Man Nobody Knew is almost a parallel story, but is true life—practically a home movie about what it’s like to be in a CIA family. Carl calls it a “quintessential Washington story, a spy story.”

Carl appears in the film only as narrator. The stars, beyond his father, are the people who speak to the camera and who weave the tale: his mother, Barbara Colby, and a gallery of Washington figures who are connected in one way or another to the intelligence community. They include former CIA directors James R. Schlesinger and William Webster; former White House national security advisers Brent Scowcroft, Robert McFarlane, and Zbigniew Brzezinski; veteran OSS and CIA officers Donald Gregg, Fisher Howe, Rufus Phillips, Hugh Montgomery, and Hugh Tovar; former KGB chief Oleg Kalugin; former senator Bob Kerry; and journalists and authors such as Bob Woodward, Walter Pincus, Tim Weiner, and Seymour Hersh. There’s a lot of gray hair, but also a wealth of juicy and knowledgeable spy talk.

Though “agency” families live among us, particularly in the McLean, Va., suburbs adjacent to CIA headquarters, Washingtonians may know them without being aware of that fact—or ever finding out. It is an opaque world, though less so than when Carl Colby was growing up.

President Gerald Ford presents Colby with the National Security Medal in 1976.

When did he realize his dad wasn’t like all the other dads? He remembers vividly. It was 1961 in Saigon. He was 10, standing on a diving board, and “my best friend said, ‘Your father’s a spy.’” Carl thought William was a diplomat, an easy cover to explain the family’s nomadic forays to political hot spots like postwar Rome and early-war Saigon. “That night I went back and asked my father. He looked at me with a wan smile and said, ‘Let’s keep that our little secret,’” Carl says. “We didn’t talk about it in the family. I didn’t discuss it with my sisters or mother. But it explained more what he was doing, why he was talking to different people, sometimes slipping away from family picnics. I became protective of him, because he was tasked with something special. He wasn’t just a businessman; he was on a mission.”

Growing up CIA meant Carl saw the world from a strategic point of view, and sometimes, as in Saigon, watched war exploding literally outside the front door of their home. (Their next-door neighbor was “family friend” South Vietnamese President Ngo Dinh Diem.) It would have made sense to live in a state of high anxiety due to his father’s work, but Carl says his father’s cool assuaged any fear.

“His work did not scare me,” he says. “Frankly, he was so capable. He operated at a lower boiling point than you and I.” Carl recalls a “pitched battle in front of our house. My father stuffed us under the staircase and went upstairs to work the phone, probably calling Washington. I walked up the stairs to see what he was doing. He told me to ‘get down,’ and just then a 50-caliber round took out half the room.” Did the elder Colby panic? No, Carl says. “He told me this was pretty serious and I better go back under the stairs.”

The film covers the high points of William Colby’s career, including winning a Silver Star for parachuting behind enemy lines in World War II, being recruited by the CIA and serving under cover in Rome, overseeing the controversial Phoenix Program in Vietnam, and returning to the US to become director of Central Intelligence, appointed by President Richard Nixon in 1973. It was not an easy time to hold that job. The wake of Watergate led to Congressional investigations into CIA covert operations, known as the “family jewels” and presumed to be kept secret for all time. Congress, the media, and the American public would have none of that, and Colby—taking the moral high ground—testified openly about what had been hidden for so long. It was the first time a public light was shone inside the intelligence community, and the blowback was fierce. President Gerald Ford and others forced Colby’s resignation. His son calls him a “whipping boy” for CIA indiscretions.

“He was embittered by his experience with Ford,” Carl says. “They threw him out because he didn’t lie to Congress.”

This only added more stress to an already strained home life. Barbara Colby is blunt in the film about her husband asking for a divorce. She replies, “But we’re Catholic. Catholics don’t divorce.” That didn’t seem to matter. Carl believes his parents “drifted apart. In the beginning, it was fun to pretend, ‘Who are we tonight?’ as they played their roles in their cover life. But later it deteriorated into, ‘It’s none of your business what I do.’ Most marriages do not survive if one of [the partners] is always pretending.” Colby divorced Barbara, remarried, and started a new life. Sally Shelton Colby was his wife until his death, but she is not in the film.

CIA director William Colby (right) testifying before Congress.

I interviewed her,” Carl says. “I wanted insight into his character that she might provide. She was married to him for a good 13 years. I wanted insight into his death, his state of mind.” But he found the interview unenlightening. “I have a cordial relationship with her, but she comes into the story seven years after he resigned. She’s a footnote to my story.”

Also not in the film is President George H.W. Bush, Colby’s successor at the CIA. Did he refuse to be interviewed? “That was out there dangling but never came to fruition,” Carl says. “I didn’t get a firm yes or no. Bush is very complicated.”

The Man Nobody Knew has garnered a pile of good reviews. The Los Angeles Times referred to it as a “remarkable feat of personalized biography.” The Village Voice called it “compelling,” and the Wall Street Journal declared it to be “powerful. Shattering.” It also serves as a valentine from son to mother; Carl credits Barbara with giving texture and revelation to the film through her several interviews with him. In her mid-80s, she comes across as sharp in her recollections and at peace with the journey.

It is a strong film, and will have a special significance for Washington audiences, who generally have a more nuanced understanding of history’s intersection with espionage—though anyone who loves a spy thriller will be enriched. And the choice of the Alibi Club as the venue for discussing the film is apt, as the townhouse at 1806 I Street is itself practically undercover.

The quintessential Washington men’s club, it was formed in the late 19th century as a place for the city’s elite to hide away with an “alibi.” It hasn’t changed much. There are 50 members, and no new member can be tapped to join until an existing member has died. The roster includes both former presidents Bush, as well as assorted senators, diplomats, Supreme Court justices, generals, and well-connected businessmen. Carl Colby says, however, that his father was never a member. He joined the Cosmos Club, but his real club was “the agency.”

William Colby disappeared in late April 1996. An adept sailor, he’d taken a canoe out on the water from his and Sally’s weekend home on Cobb Island in southern Maryland. And then he didn’t return. Given his professional past, speculations of foul play instantly arose.

“He had enemies,” Carl admits, “but, no, I never really thought there was much in that.” He said he knew when his father went missing that he was dead. “I didn’t see it adding up to foul play.” When his body was found, the medical examiner said it looked like drowning after a heart attack or stroke. Carl had last talked to his father two weeks before, a conversation about family history that prompted Carl to write him an eight-page letter.

“His wife called after he died and said, ‘Oh, he got your letter. He was really moved.’” Carl says he thought, Why did I have to hear that from her and not him? “But I knew the answer. I wouldn’t have heard it from him.”

Carol Ross Joynt