The 1970s was the golden age of wife-swapping and barn conversions. As an adolescent living at home in a nondescript early 1960s four-bedroom detached home, I was amazed when my friend Nick’s parents bought a barn, shortly before swapping wives with a couple who lived in a modernist box on stilts.

The idea of living in a converted agricultural building was frankly almost perverted then, and those who did so were artists or musicians or illiterate cowhands. But these strange homes held a kind of glamorous, theatrical fascination. I don’t exactly remember a wagon wheel pinned to the outside of Nick’s parents’ place, because as a late teenager I spent more time in the pub than in anybody’s house and the architectural language of the two is a bit blurred. But I do remember interior stone walls pointed in dark grey cement and then lacquered with polyurethane varnish. I even helped varnish one in 1976, to my shame.

Wife-swapping may be a thing of the past, but tacky barn conversions are still big. Too big. Heritage Counts, English Heritage’s 2005 report on the state of the historic environment, came out two weeks ago and in it, EH’s chief executive, Simon Thurley, warns that we are facing the extinction of the barn as a recognisable species in Britain’s countryside. Alongside the extinction of the dormouse and common sparrow. Oh, and farming itself.

The disappearance of traditional barns and farming are, of course, connected, as even a cowhand can figure out. I came to own a barn as a result of this trend (on our farm in Somerset, which is a sort of extinct farm but with decorative sheep). Unfortunately, it was in the mid-1990s, by which time I was a fully paid-up member of the Society for the Protection of Ancient Buildings, a sort of provisional wing of the conservation movement, so I was nervous about “conversion” and “modernisation”. Moreover, we’d spent every last bean on the place, so there was no way I could afford to convert the barn into anything other than the lowly, light industrial shed it is.

And, to boot, there’s not a bat’s chance in a Rentokil factory of getting planning permission to convert for residential use here anyway, because my local authority has the worthy and noble policy of conversion from farms into factories and offices, not houses. This is based on the not unreasonable idea that converting a barn into a house involves all kinds of interventions that destroy any integrity the building had, while everybody knows that office workers are more than happy to answer the phone while sitting on an earthen floor in a through-draught threshing barn.

This principle of non-residential change has underwritten policy on barn conversions for 15 years now, but it’s not ubiquitous, nor is it foolproof. A rural economy will never sustain the same number of business farms as there were once agricultural farms; and not all barns and sheds readily convert to human use: we have a double row of low pigsties that would readily make a fresh-air creche for small children, but that’s the only use I can think of for them. So across the country, permissions continue to be granted to convert farm buildings into houses and farmyards into driveways.

I can’t bring myself to like what has been happening, because our agricultural buildings represent the largest remaining body of unloved but historically important buildings — a forgotten reservoir of vernacular history that is evaporating before our eyes. And it upsets me that too much emphasis has been placed on protecting and listing housing in this country, whereas so few farms are paid any attention.

There are about half a million traditional farm buildings in Britain — the same number as all of the country’s listed buildings put together — but only 7% of them are listed. Moreover, half of Britain’s conservation officers say that dereliction and destruction of agricultural buildings is the single biggest problem they face. Just to illustrate the seriousness of the situation, by extrapolating from a field survey in Hertfordshire, English Heritage has calculated that by 2020, every single timber farm building there (and, we could conclude, in Britain) will have been converted, demolished or collapsed from neglect.

But it’s not just barns that are vulnerable. Many old farmhouses are not listed, and I’m often baffled when I see what people (some of them friends) do to these charming, ancient buildings after suddenly coming into money.

Good repair and conversion of an old house or barn or chicken shed ought to be done slowly and carefully, and it’s often best executed by owners who also spend their money slowly and carefully. It should start with the premise that the character of an old building is a delicate thing, a subtle skein woven from thousands of tiny details and irregularities, worn surfaces and visibly layered histories.

Don’t be fooled. There is no such thing as “soul” in a building. The whole idea that ancient houses somehow reverberate to their own stories and historical inhabitants is rubbish. What we perceive as “character” is simply that accrual of detail and layers, making it very easy to destroy mindlessly the character of an old place. You just have to wield some blockwork and concrete and, hey presto, there go 1,800 tiny details. Fake flagstones and plasterboard will destroy another 500. UPVC windows are like Domestos: they will kill 100% of known historical details. What I don’t understand, though, is why people buy a charming old place only to destroy the thing they loved most about it.

Because so many agricultural buildings remain unlisted, local authority conservation officers can’t guide or enforce any kind of conservation measures. If you live in an unlisted old house, you’re probably delighted to hear that. But English Heritage is furious, especially about barns, because they don’t adapt very well. “We would rather see barns remain derelict than done up the wrong way,” it says. “Poor barn conversions are causing a visual rape of the countryside. They are a real cancer on the landscape.”

Well, that’s true. There are a lot of them in our shires, and their open siting in the countryside gives them undue prominence, so that people driving out for a Sunday post-prandial assume this is how all barns should look. Which is apparently not like a barn at all, but like Mrs Tiggywinkle’s cottage, with a threshing door stuck in the middle of one wall, a carriage lamp and lots of cream gravel where once there stood a mire of excrement and mud.

Explore Property

• Buying & Selling

• New Homes

• Overseas

• Interiors

• Investment

• Gardens

• Cool In Your Code

• Specials

• First-time buyer

Times recommends

• Second homes: credit crunch casualty or money-spinner?
• How I built my own sliding house
• Top 10 incentives for house buyers