Robert Caro: a life with LBJ and the pursuit of power

It all seems terribly wrong, but Robert Caro is a little envious. After spending the better part of his adult life chronicling the rise to power of Lyndon Johnson – dedicating more time to researching the Texan's climb to the White House than Johnson actually lived it, and publishing a volume about once a decade to ever greater acclaim, with the final instalment yet to come – Caro feels he missed out.

Most reporters would envy him the time to research and the space to write the 1.5m words he's already dedicated to his subject, and this is even before he tackles the bulk of a presidency that did more for the rights of America's black minority than any since the civil war and before it was swamped by lies and deceit over Vietnam.

But Caro, in the jacket and tie he dresses in each working day to overcome what he calls his inherent laziness, muses over how great it would have been to be a newspaper foreign correspondent. A close friend is Joseph Lelyveld, the renowned reporter who wrote one of the finest exposés of apartheid South Africa, Move Your Shadow, before going on to edit the New York Times. "Whenever I talk to him, I think: boy, did I miss out," says Caro in a bold New York accent. "Covering the whole world, getting to see the whole world. Because you don't get to know a place unless you work in it."

The place that Caro has worked for decades is a mystery to most people. He has probably done as much as any writer of this age to explain the world of power.

In his decades of exploration, Caro has all but gone broke, uprooted his wife Ina from New York to live for years in the vast, isolated Texas Hill Country, stalked the corridors of Congress and endured bitter denunciations for his all too human revelations of how Johnson built and wielded power.

The pay-off has been four magnificent volumes, collectively known as The Years of Lyndon Johnson, that paint a picture of a political colossus who propelled himself from soul-destroying poverty to the presidency via a grip on the US Senate that hadn't been matched for a century beforehand or the half-century since. The latest, The Passage of Power, has Johnson languishing in the vice presidency at the beginning of the 60s under the shadow of John F Kennedy, a man whose political skills and ruthlessness he badly underestimated, until Lee Harvey Oswald's bullets delivered up the White House and the opportunity to finally use power the way he wanted.

With each instalment, Caro's reputation has grown to such an extent that at a Washington talk this month some of those who've paid for tickets make a show of bowing down as he walks into the room. It's a gesture he's none too comfortable with.

Caro at heart sees himself as a reporter, not a historian. For 36 years he has dug deep into Johnson's life in search not of the man for the sake of it, but for an understanding of how he wielded power so effectively. "I don't think of my books as being biographies. I never had any interest in doing a book just to write the life of a great man. I had zero interest in that," he says. "My interest is in power. How power works."

Caro is 76 now. When he began researching The Years of Lyndon Johnson the former president had been dead little more than three years and still occupied a central place in the American consciousness. The Vietnam war, which Johnson escalated to the point where it ripped his country apart, had ended in humiliation only a year earlier with the fall of Saigon. The struggle for civil rights had widened with the rise of militant black power groups. Johnson was despised on the right and left by the time he was driven from office in 1968.

Attitudes toward him have mellowed to the point that well-known Vietnam draft dodger Bill Clinton, in reviewing Caro's latest volume in the New York Times, could write that he found plenty to admire in LBJ and never hated him the way that many of his generation did.

But even as Johnson receded into history, Caro's unflagging enthusiasm for his subject was fed by a craving to understand how this brutish, bullying, often racist man struggled out of the grip of rural Texas. Along the way he manipulated a school election by blackmailing a girl into dropping out of the race and stealing a congressional race, and then he went on to take control of a moribund Senate and oversee passage of the first civil rights legislation since the civil war.

"I'm fascinated by Johnson. If you don't like me you say I'm obsessed," Caro says. "We're taught Lord Acton's axiom: all power corrupts, absolute power corrupts absolutely. I believed that when I started these books, but I don't believe it's always true any more. Power doesn't always corrupt. Power can cleanse. What I believe is always true about power is that power always reveals. When you have enough power to do what you always wanted to do, then you see what the guy always wanted to do."

Early Days

Caro's first book, The Power Broker, laid bare Robert Moses, the legendary urban planner who for decades wielded more power in New York than any elected politician. Caro paints a picture of a man who used his control of public bodies to mobilise banks, unions, the press and the church to take on the city's mayors and the state's governors as he remade New York with public works, including bulldozing roads through longstanding neighbourhoods, forcing hundreds of thousands of people from their homes. The account won Caro his first Pulitzer prize. Obama has described "just being mesmerised" by the book and said it helped shape how he thinks about politics.

Caro came at Moses as a young reporter who didn't understand how the city he lived in worked. He was working for Newsday, a Long Island paper, and was assigned to write about Moses's plans for a new bridge that was widely regarded as a very bad idea. Caro wrote a series of articles exposing the shortcomings of the scheme. He found many politicians agreed with him, but when the crunch came Moses got his way. The New York state legislature voted overwhelmingly in favour.

Caro calls that moment transformational because he realised he didn't understand how it could have happened. "I started writing The Power Broker because I was a reporter and I felt I wasn't explaining how power worked, urban power in New York City. Neither did anybody else. There was nothing to read on it. Here was this guy who was never elected to anything and he had more power than politicians, so I started that book as an examination of that. I was trying to find out where he got his power and how he used it to shape New York. There's not enough understanding of the realities of power. In a democracy, supposedly we hold power by what we do at the ballot box, so therefore the more we know about political power the better our choices should be and the better, in theory, our democracy should be."

The Moses book was supposed to take less than a year to write but ended up consuming seven and left Caro and his wife broke. The couple sold their Long Island home and moved with their young son to a flat in Brooklyn. Ina took up a teaching job to pay the bills. Johnson seemed a natural next stop. His power was evident but the hows and whys were not.

Johnson grew up in the Texas Hill Country watching his father rise and then fall, dragging the family back into grinding poverty. "I remember saying to Ina I didn't think we'd have to live in the Hill Country. There are, like, seven books all with stories of his youth, so I thought I could just do it with some interviews to get some background and colour. I didn't think it would take any time," he says.

Then Caro went to Texas. "I started interviewing these people and driving around the Hill Country and I said to Ina: 'I don't understand these people and I don't understand Lyndon Johnson. So we have to move there.' Ina, of course, as she always does, said: 'Sure.' We lived there for most of three years. The first time I went to Johnson City I was able to see all the kids who went to high school with him. Then you talk to his relatives and the unanimity of their belief – Sam Houston, the brother, saying the most important thing for Lyndon was not to be like Daddy because Daddy failed and Daddy was a laughing stock and Lyndon Johnson had this horrible childhood. I think everything was very rooted in that."

The work paid off. That first book, The Path to Power, was greeted as a revelation not only for its insights into the true nature of Johnson but for its transformation of the staid form of political biography. It also shocked by laying bare Johnson's brutish, bullying, coarse ways. Some of his old allies recoiled. The Johnson presidential library refused to stock the book for years. Johnson's widow, Lady Bird, had granted a lot of access at the beginning, but at some point, and for reasons Caro does not fully understand, she suddenly cut it off. Johnson's daughters refused to see him at all. "The family is so hostile to me," he says. "Daughters are devoted to their parents. I had to write things that it's painful for them to read."

Caro has lived LBJ for so long now that he fears he's become a Johnson bore. Anecdotes pour out. At one point he apologises for talking about the man so much, but then remembers that's why we're here. He says that when he's interviewing sources for his books he writes "SU" on his notepad, shorthand for shut up, to remind himself to stop talking. It's during one of those pauses that Caro leaps in with a question. What, he asks, do I think of America?

I say I've been surprised at the dysfunction of the government, the gridlock in Congress, but that the really surprising thing is how many Americans like it that way. It's a fundamentally different view of government from in Europe, where people may be cynical about politicians but still look to them to improve their lives and protect them from other forces, such as rampant capitalism. In the US there's a rising tide of people who believe the very existence of government is the source of the country's ills. Even the poor will tell you that if they could just get the government off their backs they'd be free and prosperous.

"That's a product of the 60s," says Caro. "If you look at America on 22 November 1963 [the day of Kennedy's assassination] it's a very different place than the America you're describing. That's when Johnson becomes president. Five years later he leaves the presidency. America has changed into basically what you are talking about. Everyone thinks that distrust of government started under Nixon. But that's not true. It started under Johnson. It started with Vietnam and the 'credibility gap'. There used to be this feeling under Eisenhower and Kennedy and Roosevelt and Truman that government was a solution. Trust in the presidency fell precipitously under Johnson – real lows. And it's never come back. It's a trend that, if you're liberal, is really discouraging."

That's one legacy of a president who, despite winning an election in 1964 with the largest margin in US history, managed to divide the country in so many ways even he recognised he was unelectable again. But there's another legacy – the one that really interests Caro in his latest volume.

Johnson had been immensely powerful as leader of the Senate. Caro moved his family to Washington, attended just about every Senate committee meeting for two years and interviewed every elected member in order to understand how Johnson took control of a body that was seen as all but moribund and turned it into a centre of power for the first time since the 1850s. "The Senate was this mess until 1955. Lyndon Johnson becomes majority leader for six years until he becomes vice president. For six years the Senate becomes the centre of governmental ingenuity, energy and creativity in Washington. It basically writes its own bills. The civil rights bill is sent over by [President] Eisenhower, but Johnson rewrites it into his own bill. It passes minimum wage, disability, housing. Johnson leaves. From the minute he's gone it becomes the same mess it is today. For half a century it's been the same mess. The only time the Senate worked like the founding fathers intended it to work, as both a check on government but also a creative force in government, was the six years he was majority leader. It can be done. It just takes a type of political legislative genius to do it."

The political genius, Caro concluded, lay partly in Johnson's ability to take arcane rules and make them work to build control and loyalties. But for all his political brilliance, he miscalculated in 1960 in underestimating John Kennedy's hunger for the presidency, and in leaving his own run for the White House too late. Johnson then had to decide whether to accept Kennedy's offer of the vice presidency.

He made a calculation which was to have a chilling outcome. The historical record showed he stood more chance of getting into the White House by Kennedy dying in office than he did by trying to run again in another eight years. Johnson also lived by the phrase "Power is where power goes", meaning he thought he could continue to wield the control he had in the Senate from the vice president's office. Kennedy quickly killed any such ambition.

But then Johnson was propelled into the White House and the man revealed himself for what he was. Kennedy had been struggling to get civil rights legislation through Congress. "What does Johnson do?" asks Caro. "He becomes president and everyone is saying to him – he's been president for four days, and these guys are all sitting around the table drafting his first speech and they're all saying: 'Don't use up your capital on civil rights; you're going to antagonise the southerners, they control everything – it's a noble cause but it's a lost cause.' Johnson says: 'What the hell's the presidency for then?'"

Johnson picked up the civil rights bill and within days he was working Congress once again, and it passed. "The civil rights bill was going nowhere. It was dead. Johnson comes in – and I just marvel – this is miraculous the way he picks up this bill in a matter of days and gets it moving towards passage. It's a use of political power such as you seldom see."

Absolute power had revealed Johnson as genuinely compassionate on the issue of civil rights and, later, in combating poverty with the Great Society legislation. This was the man forged by Hill Country, not the millionaire lifestyle of the Kennedys, a man who was so poor he could not afford to finish college and took a job teaching Mexican-American children to speak English – the only teacher they ever had who really cared about them.

Not that Caro has any illusions about the other side of a man who bullied and blackmailed his way to power. "My opinion of him hasn't changed. He really had compassion, he really wanted to help. But whenever ambition collided with compassion it was the ambition that won," he says.

All this raises the interesting question of whether, if Kennedy had lived and Johnson had failed to become president, a civil rights law would have been passed. It's a wrenching question for Caro, who deeply admires Kennedy. Caro points up Kennedy's cool handling of the Cuban missile crisis as evidence of his mastery of foreign affairs as he stands up to the pressure from the hawks in his own cabinet, the generals and Johnson, who want to attack Cuba.

"You cannot overrate the greatness of Jack Kennedy in the inspirational aspects and the foreign policy aspects of the presidency," Caro says. "However, you can't say the same about his domestic accomplishments. Would these acts have become law? Not just the civil rights and voting rights, but Medicare, Medicaid, all these education bills? There's a real doubt in my mind about whether they would have become law without Lyndon Johnson."

So where does that place this man? Is he among the greatest of presidents, whatever his shortcomings? "I certainly don't think that, because his presidency did not end in triumph. It ended in Vietnam with him lying about it, and his constant misleading of the American people about Vietnam and the feeling Americans had deep down inside that we were doing something to be ashamed of," he says. "But he was one of the most significant presidents in American history. His was a watershed presidency, and I use that in the precise meaning of the term. A watershed is the top of a mountain divide, and on one side of it the waters run one way and on the other side they run this way. During his presidency the waters started running a different way and they haven't really started coming back yet."

But there's one more thing for which Caro thinks perhaps Johnson carries some of the credit. Without the civil rights and voting rights legislation, it's very possible the White House would have a different occupant today.

"Lyndon Johnson passes the civil rights act in 1964 and the voting rights act in 1965. At that time blacks voted in such small percentages in this country, it was a different atmosphere. In 2008 Obama was elected. That was 43 years. It's the blink of history," he says.

Caro gets a lot of questions about Obama at public events. Is he another Kennedy – all inspiration and no delivery? Some say Obama came to power with little understanding of how to work the political system or that he was too much of a consensus politician to get the better of a highly partisan Congress where Republicans are more interested in sinking his presidency than doing what's right for the country. "I don't quite agree with it, even if I'm the only one who feels this way," he says. "I think he has had real accomplishments that no one gives him credit for."

Caro's talking about healthcare – the landmark reform whose fate is in the hands of a politicised and politically divided supreme court, and where the balance of power leans towards the conservatives. "When I think about healthcare, I really think of something Lyndon Johnson said when he was passing the first civil rights bill, the 1957 civil rights bill when he was still in the Senate. He has this crude phrase: 'Once you break the virginity it's easier next time.' Then he also said in a more polished way: 'Once you pass it it's easier to go back and fix it.' I keep thinking that we have a healthcare bill. It's done something. It's put 30 million people on health insurance who didn't have it before. It's prevented companies from denying people with pre-existing conditions. It's done something. It's not a perfect bill. But it's there. It will be easier to go back and fix. Congress has stopped him in many things, but he has achieved others. It seems to me he doesn't get nearly enough credit." After all, Obama doesn't have a Lyndon Johnson to crack open the Senate.

There's another question Caro faces at almost every public function. Sometimes it's circumspect, sometimes direct. But it boils down to the fact that Caro is 76 now and if he takes another decade to write the fifth and final volume – the drama of LBJ's presidency – we might never see it. It's in Caro's will that if he dies before it's finished, the work is to be abandoned. No one else is to complete it.

So how long? "I've done most of the research on the last volume. I think I can do this in a short period of time. I'm estimating three years," he says. "But why would you believe me?"