DESMOND GUINNESS – scion of the brewery clan, friend of Mick Jagger and Paul Getty, author of Dublin, A Grand Tour and founder of the Irish Georgian Society – really wanted a Palladian house with rococo plasterwork in England. He scoured the countryside, was gazumped on a haunted house which is now the home of Prince Michael of Kent, and ended up with Leixlip, one of the first castles built by the Normans in Ireland.

In 1958, Desmond and his first wife, the late Princess Marie-Gabrielle of Urach-Wurtemberg, moved into Leixlip with 540 books, a mattress and a gun. The couple were known as part of the jeunesse dore. ‘Absolute rubbish,’ says Desmond, a gentle charmer in a Fair Isle waistcoat, who has the bluest blue eyes, wispy white hair and a seductive voice. ‘We were interested in architecture, engravers and silversmiths.’ But he loves beautiful women, doesn’t he? ‘Mmm, I certainly do.’ (Reflective pause.) ‘Well, that’s a conversation stopper.’

The castle had mod cons like windows with stone mullions (added in the 17th century) and 19th century battlements. When the couple moved in, it had baths, but they were in the fields being used as water troughs, and wiring so unsafe the electricity board refused to reconnect the power. But the interior had been elegantly remodelled in Georgian times.

They set about decorating it and buying at country sales. Desmond bid 10 shillings for an unremarkable kitchen table in one crowded auction. ‘It turned out to be the most amazing carved Irish mahogany with a black marble top and has since appeared in several publications.’

Desmond now lives in Leixlip with Penny, his wife of 10 years and a former lover of Lucian Freud; Eileen, his cook of 31 years, who is ever ready for impromptu dinners for 40; and the ghost of a multi-headed hound whose heads appear at the windows. ‘I’ve often undone the shutters thinking I’m going to meet him. I will one day, I’m sure.’ Desmond’s son Patrick and daughter Marina, who has three children by different fathers, live nearby. The day of our interview the castle has been taken over by Michael York, Jacqueline Bisset, Edward Fox and a film crew working on September. ‘The house is pretending to be a dour Scottish castle.’ Workers are busy repainting the ‘set’. And a coach of Americans are coming for a slide show on Irish architecture. ‘They wanted to know what to wear … can you imagine?’

We’re talking in Desmond’s favourite room, the sitting room, in the castellated 14th century tower with its 4ft-thick walls. There are invitations from the Ambassador of Italy and Marianne Faithfull and samples of Pierre Frey material scattered on the table.

It’s the only room light enough to read in without electric lights and the wooden stove burns in here all winter. Like all castles it’s draughty. One guest who called for more blankets in the night was given a pile that included a flag. He woke up frozen, draped in the flag, and thought he’d died. The radiators in this room are hidden in whiskey barrels. Charmingly tatty and unmatching curtains at the windows do their best to keep out the cold.

Desmond affects, amusingly, not to know how many rooms there are. ‘I’m not really sure. But there are 11 bathrooms,’ he says. The hall, paved with terracotta tiles and white flagstones, is being painted yellow by the film people. The dining room is navy with a jolly chinoiserie tapestry and jasmine growing up the 1765 Gothic revival windows. ‘We used to light the candles in here and Marianne Faithfull, a great, great friend who lives next door (five miles away), used to sing in front of the fireplace. She was such a dazzling beauty. You couldn’t possibly resist her.’

The library has an 18th century gilded plaster frieze and French engravings pasted on the walls. Nearby is the drawing room with its Irish mahogany table carved with shells, Savonnerie carpet and Newbridge dolls’ house. William Acton’s drawings of the Mitford sisters adorn the walls. But today it’s full of packing cases and film people.

Upstairs is the chapel. ‘We’ve never prayed in it. It’s never been anything but a passageway for us.’ It has a little pony cart in it in which Marina used to drive as a child. ‘One day I went to an antiques shop and saw it. I thought it looked familiar. It used to live in the stables and had been stolen.’

Under the window in the room known as King John’s bedroom, there’s a copper Victorian bath tub in which Mick Jagger photographed Jerry Hall. The pictures were stolen and later appeared in a British newspaper. ‘My idea,’ says Desmond, ‘had been to hide the tub in the armoire, so people could take a bath in the cupboard.’ The room has a four-poster state bed from Stowe and white lines chalked on the floor from when it was used as a squash court. The Guinnesses’ bedroom is cosy, with a portrait of his mother. ‘By John.’ (Augustus, that is.) His mother is, of course, Diana Mitford who was once married to Sir Oswald Mosley. Beyond this, there’s yet another guest bedroom, hidden away, with French scenic wallpaper of 1822 with views of Italy. ‘They used to sell this wallpaper by saying, ‘Teach your children geography without leaving the home.” There appears to be a different book by a Mitford or Guinness by every bed.

It is at Leixlip that they have an annual Georgian cricket match. And where, as a girl, when Marina wanted to join a travelling circus, Desmond invited a circus to pitch its tents on his land. Whereupon a dromedary embarked on an intimate relationship with their cow.