When I was about nine, my father took me out of school for the afternoon, since this was the days before Labour's mad attempts to fine parents for truancy, and we went to a movie.

I don't think I was technically old enough to get into the show, so I was pretty incoherent with excitement before it began. It involved semi-naked women. It had baddies of intergalactic badness. It had the triumph of the British secret services. In fact, it had everything that a nine-year-old could desire, and by the end I was white and rigid with happiness. I was shaken. I was stirred. I had witnessed Sean Connery in Diamonds Are Forever, and I knew the kind of guy I wanted to be.

No, not Blofeld, you idiot. I wanted to be Bond. "Morning, Moneypenny," I sometimes say to my secretary, frisbeeing my hat at the hat-stand, and the good Batley-born girl pretends to see the joke. Call it prepubescent; call it preposterous; call it preparatory school. But I would be amazed if most of the red-blooded males reading this column, and indeed the red-blooded females, did not share my occasional urge to get up from their chair, sidle to the door, whip out the old Walther PPK and say pow in a loud voice.

You can therefore imagine my feelings the other night when I found myself sitting next to Sir Sean Connery, as he now rightly is.

We were in the green room, waiting to go on Parkinson and, as I looked at that rugged profile, like some Grampian tor, I wondered what on Earth I was doing. I mean, I could see why Connery was on the show. He had seen off Oddjob. He had grappled with Pussy Galore. He had been equally good as the father of Indiana Jones and the renegade captain of a Russian submarine.

But why was I there? It is true I had a book to plug (geddit now), but I could tell that Parky wasn't really interested in that. Parky had a theme in his head, and it was the contrast between your columnist, whose upbringing was one of blissful bourgeois tranquillity, and the tough early years of his other guests. Ricky Tomlinson, the actor, had been banged up during the building labourers' strike. Sean had begun as a milkman.

Wasn't it true, said Parky, looking at me in that lethal, gentle, bag-eyed way, that I had been really rather spoilt? Weren't chaps like me irrelevant these days?

His view seemed to be that people who had been to public school were a kind of evolutionary relic, the pointless memento of some age before meritocracy. If we were geological features, we would be ox-bow lakes. If we were parts of human anatomy, we would be the appendix.

And of course I had to admit that he was in many ways right. Unlike Parky, my father was not a miner. We did not live in a hole in the road, or eat cornflakes packets for breakfast, or get up before we went to bed. Parky was making a point about image, and ever since I have been wondering about this question, as I brood on the Tory party, and its continuing struggle to modernise itself for government.

The BBC asked some students to name the animal they identified with the Tory party, and the answers were frankly disappointing. Some said a snake, on the grounds that the plan to scrap top-up fees also involves a commitment to reduce the numbers going into higher education. Some said a kind of spavined giraffe. Some said a tortoise.

No one thought the party was some cool and groovy animal like a bounding cougar, or a panther, let alone a lion. And I have to ask myself, after my experience on Parkinson, whether I, Johnson, am helping things or making matters worse.

Last night I was speaking to a group of Tory women, and the charming lady chairman rather put me off in mid-flow. "I'll tell you what's wrong with the Tory party," she said. "You are!" It seemed that she meant that, as a public-school-educated man in a grey suit, I was not only off-putting to the havering voter. I was also hogging a seat that might be given to a woman, or a member of an ethnic minority.

It was no use pointing out that the Tories have just recruited a black and a gay candidate to stand, respectively, in Windsor and Norfolk North. It was no use saying that I am all in favour of encouraging female candidates, even to the point - yes, it's worth a look - of all-women shortlists.

I was the problem, she told me, and I was so cast down that after a while I decided to fight back. Look here, I said: you say the Tory party is sexist. But which was the first great Western power to have a female prime minister, and which party produced her? And anyway, isn't it frankly patronising and silly to keep saying that we must modernise by "reaching out" to this or that minority?

The Tory party is there to speak for all humanity, for everybody who believes in the elementary principles of conservatism: that if you let people get on with their lives, and develop their potential, you will generate the wealth society will always need to pay for the poorest and neediest.

And anyway, if you look at the career of Sir Sean Connery, he is a shining example of what this party is all about. He may claim publicly to be nothing but an SNP man. But he is an ardent reader of a magazine whose name I will not mention, but which will be familiar by now. Yes, my guess is that Bond is a natural conservative. Not the fictional Bond, who went to Eton, but the real one, the self-made one. And with Bond on our side, who can lose?

• Boris Johnson is MP for Henley and editor of The Spectator.

Information appearing on telegraph.co.uk is the copyright of Telegraph Group Limited and must not be reproduced in any medium without licence. For the full copyright statement see Copyright