Having pulled in vain on the front-door bell of Nevill Holt, Leicestershire, and with no sign of another entrance along its vast, sprawling facade, I resort to the mobile phone and Klaus, the house manager, appears. It’s nearly lunchtime on a Saturday, but “the boss has not quite recovered from last night” and hasn’t made it downstairs yet.

The boss in question is David Ross, 41, and I am not the only one with reason to be grateful for the invention of the mobile. Ross is deputy chairman of Carphone Warehouse, one of Europe’s largest phone retailers, which he set up in 1989 with Charles Dunstone, an old friend from Uppingham school. With an estimated fortune of £562m, Ross is number 113 on this year’s Sunday Times Rich List, and bought Nevill Holt, parts of which date back to the 13th century, in 2000. Its last incarnation was as a boys’ prep school, and it was on the market for £1.1m, though Ross is coy about saying how much he paid for it.

Ross works in London during the week but uses Nevill Holt at weekends, and also travels up for the shooting season and in July when the Hampshire-based Grange Park Opera holds performances there. The 30,000sq ft house sits on top of a gentle hill not far from his alma mater, with huge views over the Welland Valley. It is a gloriously eccentric architectural assortment, with castellations, turrets, cloisters and wings added to the original 13th-century great hall over the years. It was home to the Cunard family from 1876 to 1912; Nancy Cunard, the writer and society hostess, was born there in 1896.

Ross eventually appears, dressed in jeans and a snazzy T-shirt, decorated with a pair of sequined revolvers, that he has thrown on to greet us. Saffron Aldridge, the society girl and model, who is staying for the weekend, joins us for a while before wandering off to sun herself on a bench.

The last thing that most people — especially those, such as Ross, who rarely give interviews — would want to do after a night of partying is to be bossed around by a photographer, but he is gracious.

He balks, however, at posing in the gold Land Rover, which is parked in front of the house and was used by Ranulph Fiennes on one of his Arctic expeditions — it’s taking the image of the blinging lord of the manor a step too far.

Ross has spent five years restoring the Grade I-listed property back to a private home, “ripping out blackboards, school rooms, dormitories and boys’ lavatories”, he says. He used Simon Morray-Jones, the architect who worked on Babington House, the fashionable hotel and club in Somerset, and the interiors are a similar concoction of contemporary grooviness and the original fabric of the house, which was revealed when the stud walls and hanging ceilings of the 90 or so school rooms were removed.

Rather than go for period pastiche, “it was key to maintain the historic fabric but to use modern lighting and heating and everything else to make it a house for the 21st century”, says Ross. “I wanted to make it comfortable and functional and also to look as if it is relevant today and not a product of the past.”

The same approach is adopted in the garden, although it soon becomes obvious that Ross is not the man to ask about horticultural detail. While he may have the Midas touch when it comes to mobiles, he is first to admit that his fingers are far from green. He’s a generous boss, however, giving Jane Rogers, his head gardener, a free rein.

“Jane doesn’t come and tell me how to sell mobile phones, and I don’t tell her how to run the garden. Fair deal,” he says. “I think you should stick to what you are good at in life. I am very lucky to have a fantastic gardener, and I wouldn’t want to get in her way.” Rogers, a Kiwi who has been working for Ross for a year, beams with pride. “Thank you, David, I appreciate that.”

“It’s the most beautiful space,” says Rogers, admiringly, as we look back over the wide, long lawns to the fairy-tale amalgam of architectural allsorts at the back of the house. A large cedar of Lebanon dominates the lower lawn and, to the sides, wide borders that run the length of the lawn are filled with colourful herbaceous plants. These have replaced island beds, which were dotted along the lawns and broke up the space.

Together with Rupert Golby, a Chelsea gold medal-winning designer, Rogers has been working on upgrading the hard landscaping such as paths and paving stones to high-quality materials, and to get some structure back into the garden. “Everything was tiddly,” she says. “When I came here, there was nothing taller than 2ft. We need to lift and add a bit of wow.”

Surprisingly for a gardener with a big budget and a hands-off boss, she has done this by moving plants around the garden and growing from seed in the kitchen garden’s glasshouses rather than buying in masses of new stock. “We’ve grown thousands of plants,” she says. Taller plants such as cardoons, alliums, Madonna lilies, Crambe cordifolia and bronze fennel now provide the “oomph” Rogers and Golby are after.

Explore Property

• Buying & Selling

• New Homes

• Overseas

• Interiors

• Investment

• Gardens

• Cool In Your Code

• Specials

• First-time buyer

Times recommends

• Second homes: credit crunch casualty or money-spinner?
• How I built my own sliding house
• Top 10 incentives for house buyers