He'll readily admit that at his age - he turned 86 last week - names don't come as quickly to him anymore. But don't be fooled: he still remembers a thing or two about a thing or two, and if you give him enough time, the forgotten name comes back to him as well.
"I don't take pleasure out of getting old," he says, puffing on his cigar. "It's a very difficult thing."
Because you're the only one left of your generation? "No, because I have a terrific desire to do things, and I'm not capable anymore. I would love to run for the next elections, and have five more years. First of all to repair the mistakes that have been made the last four years, and to do a few other things that I still have in mind. I was very sorry not to be re-elected."
His career as mayor started in 1965 "very unexpectedly. I ran because I wanted to show solidarity with [David] Ben-Gurion, not in order to become mayor. I didn't think I had a chance to run against Mapai - then Mapai was still powerful."
He had worked for Ben-Gurion, first at the Jewish Agency in the 1940s and later as director-general of the Prime Minister's Office, which is where Kollek honed his skills at fund-raising. "I collected money to pay all the bills for Ben-Gurion for books - and they were expensive. Whenever he went abroad he went to bookshops, and then I had to cover the expenses... never anything personal, but books, that was his special thing."
Kollek sometimes stares off into space while he reminisces, and you can see it in his eyes that he is picturing the scene as it happened, whether it's the 1948 war or the 1967 war, exactly as if it were yesterday.
"During the War of Independence, Ben-Gurion constantly wanted to attack Jerusalem. Several times during the war he suggested it, that we should attack and take Jerusalem. It would have saved us all the arguments now if he would have done so."
Although Ben-Gurion lost that argument - "in spite of his reputation, he was very democratic" - to a decision by the executive of the Jewish Agency, which Kollek refers to as the "the government en route," Ben-Gurion won another one: to make Jerusalem the seat of government.
"There was an argument about where to have the capital," Kollek recalls. "Haifa was more in the center of the country, if you discounted the Negev, which was empty. Tel Aviv had all the prerogatives of Tel Aviv. I'm not certain that if Ben-Gurion, who had a historic sense, wouldn't have insisted that Jerusalem would have become the capital. So here you have a very decisive decision."
Nineteen years later, and it was Kollek who was faced with historic decisions.
"When the city was united, I had already been mayor for over a year. I was extremely worried how this would hold together, suddenly 70,000 Arabs [were part of city] - I started a great number of things at that time. The biggest one - at least in theory and to a very great extent in practice - was that we gave the Arabs equal rights. We gave free access to the holy places. We decided, partly under the influence of the rabbis, to give them the Temple Mount. The rabbis were for it because it was forbidden for a Jew to go up there.
"In 1948 there were approximately 70,000 Jews and 70,000 Arabs in the two parts of the city, when the city was divided. In '67' the population was the same on the Arab side, and we had increased by threefold. Since then the Arabs have increased at a rate similar to the Jews."
WHILE faced in 1967 with the practical problems of the city, there were also the political issues. It was a euphoric time, certainly, with the liberation of the Old City; but was unification good for peace between Arabs and Jews?
Again, Kollek remembers his mentor, Ben-Gurion, who by then had retired to his home in Sde Boker. "We went to the Western Wall with Ben-Gurion on the 11th of June. On the way we collected a number of people - a small British delegation with Lord Rothschild at the head; we met the commander of the air force on the street, so he joined us; and others, too. We went through Jaffa Gate - the war wasn't entirely over, we still heard shots in Jerusalem as we walked down there. He didn't say anything at the Wall, [but] you could see how moved he was.
"I had been through the whole city the day before, I had walked in behind Dayan, Narkiss and Rabin, the famous picture. I was in the line behind them - the first line certainly belonged to the army - and I had seen out of every window these white shmattes hanging out, and I felt this was too big a victory. I had very nebulous ideas about it, but it was very clear that this kind of victory wouldn't bring us only good.
"And Ben-Gurion expressed it much clearer. We went back to our house, we were about 20. Everybody said, 'now after this great disaster the Arabs will make peace.' That was the occasion when Ben-Gurion said, 'the Arabs won't make peace, they cannot make peace because of that disaster, they are after all a great nation, with millions of people. What we should do is decide now to give them everything back. What we can't give back is Jerusalem because of the history of Jerusalem, and maybe here or there we have to make some change in the border, but give it back to them now.'
"He was the only one, no one agreed with him. And I think we are now coming around to that exactly. Imagine what trouble we would have saved ourselves if Ben-Gurion had been prime minister [in 1967] and we had done that. Maybe we would have peace now, maybe not, I don't know. Maybe the Arabs would have become only too certain that they can win in a war, and would have started again and again, as they did eventually. But we would have saved ourselves some war, we would have saved ourselves a great deal of misery... also a great deal of joy, and our feeling of pride, but still... I don't know..."
He says the biggest change in the city since the Six Day War is the feeling of the people, and not necessarily for the better. "I think the '67 war was a disaster because it changed our ego - it made us so certain of everything, so absolutely..."Arrogant? "Yes." At that time, Kollek says, "There were some Jews who came and said, 'Look, let's destroy the mosques [Dome of the Rock and Al-Aksa] now'. That I didn't agree to this may have been a good decision..."
LOOKING back on his time as mayor, Kollek says "the two greatest successes in the city are the zoo and the museum. When we started the museum, everyone told us - everybody, Pinchas Sapir, Golda Meir - 'wait a little, you have so many other problems, now you want to build a museum? We'll solve all our problems and then we'll build a museum.'
"I had a different consideration. I was at the embassy in Washington for two years, and I had tried to ask people to give some works of art to the museum, which was then at Bezalel [in town.] Practically everybody refused. People who have collections are as attached to their pieces of art as they are to their children, and they want them - like their children should be married to a good person, they want [their art] in a nice surrounding with air-conditioning and everything else.
"And I saw that we wouldn't get anything if we didn't start a museum building. So it started partly because of this, and partly because agriculture changed, from a tiny little plow to a machine-driven plow." The new machinery allowed archeologists to plow deeper and find more relics, but the problem was how to care for those artifacts so they wouldn't disintegrate.
"You have to properly take care of it in a museum. So the museum started with the idea of an archeology museum on the one hand, and with getting good pieces of art on the other.
"I started an exhibition of 'gifts promised,' and we sent them back afterwards. Now they are coming back to the museum as people pass away, as permanent gifts to the museum."
As an example, "We are now getting a Pissarro which is the best Pissarro ever did. There are two of them that he did, one of them is in the Hermitage and this was in a private collection. We waited for it for 40 years, or 30 years, but we knew we would get it. Every time I visited New York I went to visit it," he says with a laugh.
Kollek says he has no regrets over the things he did as mayor, only the things he didn't get to accomplish.
"I think we should have built more kindergartens, and more schools." Asked how he'd like to be remembered, Kollek says "Not at all," but he understands his place in history is secure. "You have always to be in the right place at the right time. Yes, I was, and I made use of it." Perhaps that's Kollek's legacy: being in the right place at the right time - with the right stuff.