Behind a black and shiny door in west London, but one with no number on it, there lives the man with a chequered past as a prime minister of the United Kingdom. Whatever he achieved, he will be remembered for taking the country out of the European Union, in the same way that Anthony Eden is judged on one big failure – Suez.

A week before we meet at this terraced house, I spend ten hours reading Cameron’s hugely anticipated memoir, For the Record. On every page, I hear his good-humoured, pragmatic, occasionally exasperated voice. This afternoon, it is, nevertheless, faintly surprising to see Cameron in the flesh three years on from his evaporation from public life after Britain resisted his arguments and voted out.

Broken? He looks almost exactly the same. In his navy polo shirt and chinos, he is, perhaps, a little slimmer, his hold-your-glance blue eyes a little more prone to wateriness. Fit, though. He runs, walks, plays tennis and climbs mountains, despite being “strapped up in every direction – knees, back, elbows”. Today he has just flown back home from a family holiday on Jura in the Hebrides, where he has been enjoying the sea. “I like swimming in cold water,” he says. It makes a change, I suppose, from hot.

The Rage Outside the Door

We settle down by the garden doors of his sitting room. When I carelessly place the cup of PG Tips he has made me on a side table, he slides a coaster beneath it. Contra his reputation as a chillaxer, he notices detail. As PM, he got up at 5.30 or 5.45 every morning to prepare for meetings. As for being an essay-crisis prime minister, the book tells us he rarely even worked in the evenings at Oxford.

I praise For the Record with genuine enthusiasm. It is tastily candid about his own mistakes. His regrets over the European negotiation and the referendum campaign fill two chapters. He comes across as likeable, human. He admits to getting “off his head” on dope at Eton and smoking it again, later on, with his wife, Samantha, and her friends. (He refuses to tell me whether he tried cocaine.)

He seems relieved I like this book. However, I say. What I don’t get is a sense that he understands just how angry people are with him.

He refuses to tell me whether he tried cocaine.

He takes a moment to recover from this.

“Yup. Well. I tried to address that. I know a lot of people are. Some people will never forgive me for holding a referendum. Others for losing it. There are, of course, all those who wanted a referendum and wanted to leave who are glad that a promise was made and a promise was kept. You get lots of different reactions. What I tried to do in the book was explain why I felt this was inevitable. This issue needed to be addressed and I thought a referendum was coming, so better to try to get some reforms we needed and have a referendum. But I accept that, you know, that effort failed. I do understand some people are very angry because they didn’t want to leave the EU. Neither did I.”

Do people shout at him?

“People say all sorts of things.”

Shout them?

“I’ve had some robust exchanges.”

It is worse, though, than people just being angry, I say. I first interviewed Cameron on a train when he was running for Tory leader in 2005. An academic came up and called him a “disgrace” for the cuts in education. When he left, Cameron mused, “I think we’ll put him down as an undecided.” The anger was historical, Thatcher era. Cameron’s attempt to detoxify the Tories and charm the country had a chance.

When Britain Was Great

The next time I saw him was in 2012 when I was chasing the Olympic torch for The Times and torch and PM both bowled up at Blenheim Palace. That summer became synonymous with a peak of British optimism. Today he insists the spirit of the torch can be rekindled. He will even amaze me by suggesting that another referendum might be the way forward. Well, I count myself an optimist, too, but for me it feels as if we live in a country broken by the first one.

“I understand that, but the country was divided whether we should be in the EU before the referendum. Some passionately wanted to leave; some passionately wanted to stay. Some were very angry that they’d been promised referendums and never had them delivered.”

But this feels like the worst of times. “I totally recognize the uncertainty has been painful and difficult. It’s been difficult for all sorts of people in all sorts of walks of life.”

How hard has it been for him?

“I think about this every day. Every single day I think about it, the referendum and the fact that we lost and the consequences and the things that could have been done differently, and I worry desperately about what is going to happen next. I think we can get to a situation where we leave but we are friends, neighbors and partners. We can get there, but I would love to fast-forward to that moment because it’s painful for the country and it’s painful to watch.”

Does he find sleeping hard?

“I worry about it a lot.”

“Every single day I think about the referendum and the fact that we lost.”

I explain that I am nagging away at how he feels because people say he came from a privileged home, was the product of a happy marriage. Although he suffered the terrible tragedy of a disabled son who died, he is famously happily married himself. He is wealthy. He is an Old Etonian. These people think that for him Brexit is water off a duck’s back.

“I hope the book goes some way to addressing that and, also, the idea that upon losing I just disappeared and swanned off. I thought I would be in Downing Street for a three to four-month period, helping with the transition to a new prime minister, and then suddenly that all collapsed and I was out within a matter of days. I totally understand how that impression caught on.”

And then, when he announced his accelerated departure date after Theresa May was anointed his successor, he was recorded humming as he turned back to No 10.

“That was simply because I thought, ‘The door’s not going to open,’ and I was trying to calm myself down because there had been moments where it didn’t open.”

It happened once after he made a statement on the fall of President Mubarak in Egypt.

“It was a perennial worry. But I wasn’t happy to leave. I was miserable about giving up the job I loved and working for the country I loved.”

Was he depressed? Would he say that?

“Yes, hugely depressed about it.”

Clinically depressed?

“If you’re asking, ‘Am I on medication?’, no, I’m not. Well, I am on medication because I put my back out.”

“I was miserable about giving up the job I loved.”

Does he feel guilty?

“Look, having a referendum was not a decision that I took in any way lightly. I get very frustrated when I read that a referendum was held because of the results of the 2014 European election [which was bad for the Tories, great for Ukip]. It’s simply not true. The referendum was announced a year beforehand [in 2013] and I thought about it more than any other decision I took, because I knew it was an enormous decision. But it seemed to me that there was a genuine problem between Britain and the EU with the eurozone crisis and the development of the euro that needed fixing.

“There was also – I don’t deny it for a second – a huge political pressure to have a referendum, partly because we’d had treaty after treaty and promise after promise, and this issue was not going to go away.”

Euro-Skepticism Sets In

Even before the centralizing Jean-Claude Juncker was elected president of the European Commission, over Cameron’s opposition, in 2014, the die was cast. In 2011, Cameron vetoed a change to the EU treaty designed to help tackle the eurozone crisis, and the Europeans ignored him, carried on without the UK. “That’s what the renegotiations were all about.”

Yet if he had known what we now know – He would have found a way out, wouldn’t he?

“I still come to the same conclusion: that we were going to have a referendum.”

Could he not have renegotiated without threatening a referendum?

“I don’t think we would have got anywhere. I think the referendum was both necessary to achieve anything in the renegotiation and necessary in and of itself, because it was coming. Politicians couldn’t keep saying, ‘We’ll have a referendum at some stage but not now.’”

Two things emerge from the book about the negotiations. One is that Cameron was, all along, much more of a Eurosceptic than we recognized. The other is that, although he was angry that the EU did not concede more, he was pleased to come back with a deal that exempted Britain from the EU’s founding ambition of forging “ever closer union”, limited welfare payments to European migrant workers and let Britain off funding bailouts to eurozone countries.

“One of my biggest mistakes was I let expectations about what could be achieved get far too high,” he says. In consequence, when he returned with the deal, the press “dumped” on it.

“I let expectations about what could be achieved get far too high.”

Yet two months before his deal was finalised, he wrote Christmas cards to his European counterparts assuring them he was a winner who would keep Britain in Europe – or so the story goes. “I don’t think I put it like that,” he assures me. Tomas Prouza, the Czech Republic’s European affairs secretary at the time, nevertheless told a BBC documentary this year that Cameron was “basking” in his 2015 election win and had said to him, “I am a lucky man: I know how to win.”

Which, it turned out, he did not. The morning after the vote, he rang President Obama and Europe’s leaders repeating to each a short speech of regret that was not quite an apology. He had had a strategy to keep Britain in, and it didn’t work. “I am sorry,” he told them.

And Then There Is Boris Johnson

The 2016 referendum campaign accounts for by far the most anxious and vituperative chapter in the memoirs. I ask why Remain lost. “We ended up with very strong technical and economic arguments and the opposition had a very powerful emotional argument. The issue of immigration plus that emotional argument was a winning combination. The argument about control, it resonated with people, and when you asked them, ‘Well, what is it we’re going to control?’ it was this issue of immigration.”

Some said he should have tried an emotional argument himself.

“Well, when I tried to make the argument, which I believe very strongly, that the EU has helped to foster positive relations between countries who weren’t always positively inclined towards each other, it was written up as ‘Cameron predicts World War Three’. It just didn’t work.”

Does he buy the theory that the Leave campaign won it with voodoo social media arts that found voters who had never voted before?

“It’s true that the reason, or one of the reasons, the opinion polls got it wrong was that they usually look at people who say they’re going to vote and look at their voting history and if they’re classic non-voters, they tend to cross them out. This time non-voters voted. Although I was actually very nervous throughout the campaign.”

He didn’t enjoy it, did he?

“I loved the explaining and arguing and that side of politics, persuasion, but then, as it went on, I just felt more and more bogged down. It turned into this terrible Tory psychodrama and I couldn’t seem to get through. What Boris was doing was more exciting than the issues I was trying to get across. I felt like I was in a sort of quagmire by the end.”

He writes that Johnson behaved “appallingly.”

“Boris had never argued for leaving the EU, right? I suppose some people would say all is fair in love and war and political campaigns. I thought there were places Conservatives wouldn’t go against each other. And they did.”

He writes that Johnson behaved “appallingly.”

Let us not omit Boris’s inaccurate claim that we send the EU £350 million a week that could be spent on the NHS.

“I’m afraid it is a real problem in politics – and there is no real answer to this. If you’re having a row about your issue, you’re winning, even if the numbers are wrong. Something I got wrong was that the latent Leaver gene in the Conservatives was much stronger [than I thought]. There were lots of people who had never expressed the view of wanting to leave the EU and then suddenly decided they absolutely did want to.”

And so to Boris Johnson. The book explains that Johnson made him laugh but Cameron did not always trust him. Does he trust him now?

“He was a capable mayor. He was easy to work with. I thought he was very talented. I wanted him in my government. We’ve had issues. Even before Brexit there were sometimes tensions and disagreements but, on the whole, we’ve got on well. Look, he’s got a very clear strategy and plan. It’s, you know, not the approach that I would have taken, but I want him to succeed.”

So when Johnson said that the chances of a no-deal Brexit under him were “a million to one against”, did he trust him on that? “I think the odds are a little bit off – but, as I said, he has a plan which he is pursuing.”

In the book he calculates that the clinching argument for Johnson’s decision to jump for Brexit was probably personal ambition. “I think he was genuinely torn, but I came to the conclusion in the end that it was too tempting not to run that campaign and go that way.”

Did Johnson expect to win the referendum? “Boris thought he was going to lose.”

We saw that on his face during the speech he made on the morning of Cameron’s resignation. He looked shell-shocked, didn’t he? “Well, I was so shell-shocked I didn’t really recognise the symptoms, but when he made the decision to back Leave he said, ‘Brexit will be crushed.’”

“Boris thought he was going to lose.”

Cameron had explained to Johnson that he had a good chance of succeeding him as prime minister and that, come further treaty change, he could win more opt-outs for Britain. In “a rational world”, Cameron thought, that would be the better option for him. He also made it clear that if Johnson backed Remain, he would give him a top-five cabinet job, hinting at defence secretary. Johnson resisted that bribe and another, perhaps decisive, nail was banged into the coffin of Remain. On the day Cameron resigned, Johnson sent a text apologising for not being “in touch”. He could not, he wrote, think what to say.

Theresa May’s Futile Struggle

Cameron’s immediate successor was not Johnson, however. Of May’s long struggle, Cameron praises her “phenomenal” work. It is clear, however, he did not admire her belligerent negotiating stance. He would have favored, rather, a “partnership Brexit”, by which he seems to mean co-operating rather than battling with Europe and letting parliament, not one party, own the exodus.

“I remember frequently texting her about the frustration of getting a deal and then seeing Brexiteers vote it down at the risk of the whole project they had devoted themselves to. Maddening. And infuriating. There’s an argument that Brexit is just impossible to deliver and no one could have done, and there’s an argument that, well, wrong choices were made. This is somewhere in between.”

So what happens next? A no-deal Brexit?

“I think it is a bad outcome. I very much hope it doesn’t happen. I don’t think it should be pursued.”

A second referendum?

“I don’t think you can rule it out because we’re stuck.”

Two weeks later, Boris Johnson prorogues parliament. A week after that, he loses his majority, the crucial no-deal vote in parliament, denies the whip to those who defied him and attempts to call an election. Cameron writes to me explaining what he thinks of the prime minister now.

“Of course, as a new prime minister, I wished Boris well. I wanted him to get a deal from the EU that would have passed in the House of Commons. If that was to happen, I would have been elated. But clearly, while he started out down that road, the strategy has morphed into something quite different. Taking the whip from hard-working Conservative MPs and sharp practices using prorogation of Parliament have rebounded. I didn’t support either of those things. Neither do I think a no-deal Brexit is a good idea.”

More Time with the Family

His oldest daughter, Nancy, a climate change activist, is now 15. Her sister, Florence, is nine. Their son, Elwen, was born in 2006. His wife Samantha runs an upmarket clothing brand. He works for a UK-China investment fund and, now, at an American-based artificial intelligence company. It should be said that he also works for several charitable or not-for-profit organisations, including Alzheimer’s Research UK and the former US secretary of state John Kerry’s clean oceans campaign. The profits from For the Record are destined for charities.

Looking down at me from a shelf behind him is a photograph of a little boy, Ivan, the son who died aged 6 in 2009 from complications of epilepsy and cerebral palsy.

“Florence never knew him, but we talk about him a lot. Nancy does more than Elwen, because Elwen remembers him but, you know, he was very young. Whereas Nancy really remembers. He’s buried at Chadlington [a church near their Oxfordshire home]. We go and see him and give him an update.”

On a shelf behind him is a photograph of a little boy, Ivan, the son who died aged 6 in 2009.

I say that if people expect him to be broken by Brexit, they forget he got through Ivan’s death.

“I don’t think you ever really get over it,” he says. “Some people say to me, ‘I’m very glad you took him out for walks and you took him out in the public and you didn’t hide him away.’ I’d never thought about that because, of course, I wouldn’t hide him away. He’s my son. I love him.”

The first lines of Cameron’s obituaries will record that he lost the Brexit referendum. I ask, had it swung in his favour, how they might have read. He talks instead about his “great sadness” at “the path not taken”, the one that would have allowed a flourishing Britain the best of both worlds, in Europe but not part of its grand plan.

“It pains me what has happened and the fact that we lost and the mistakes I made, and all the rest of it, but it also pains me because I think the option of staying on a reform basis would have been a great boon for Britain – not just an economic boon. We would have done something that other European countries hadn’t, which is gone back to our people and said, ‘We know you’re not happy with this. We’re not happy either. We want to change it and here are some changes,’ and then reaffirm our membership on that basis and say to the people, ‘This isn’t the end of the road. There’ll be more of this because you are the masters. We’re the servants. We’re listening to what needs to be done.’”

On that might-have-been for what might have been one of Britain’s better prime ministers, we finish. He leads me out to the street. He has not done an interview in a long while. He’s exhausted, he says. As I walk back to the Tube, I recall my train ride with the Tories’ great young hope 14 years ago. After he had explained his plans for Britain, I turned off my tape recorder and muttered that it all sounded great, but that it would all go wrong in the end. “Of course,” he acknowledged sunnily. It took 11 years, and how. When Downing Street’s heavy black door finally closed on David Cameron in the summer of 2016, the moment did not just define his premiership. It redefined Britain.