Murder, drugs and guns were just some of the terms used by people who tried to warn us against buying a brownstone townhouse in Bedford Stuyvesant, Brooklyn. But the abundance of exquisite and unspoilt late Victorian architecture, its proximity to Manhattan and the bargain prices on offer in the supposed gangster’s paradise they call Bed-Stuy were enough for my fiancée Alida and I to ignore the nay sayers for long enough to discover that they were quite wrong.

And thank God we did. For the price of a tiny bedsit in Manhattan — a small island popular with holidaymakers some 12 minutes by subway from Bed-Stuy — the intrepid house hunter can still snap up four storeys of Italianate brownstone grandeur with a front and back garden for around $600,000 (£343,000).

Alida and I are not the only ones who have discovered Bed-Stuy. On every block around here there are huge skips filled with the debris of a house clearance — the property speculator’s hallmark. Celebrities such as Heath Ledger have been spotted house-hunting near by, while 5,000 people a day visit brownstoner.com, a website dedicated to the pursuit of finding the renovator’s dream house in one of Brooklyn’s forgotten neighbourhoods. But Bed-Stuy is more than just an incredible investment opportunity for an adventurous property speculator: it is one of the most fascinating neighbourhoods of New York, steeped in history and close to Manhattan’s sleepless streets, yet no tourist ever sets foot here.

Our house is a rare beauty, even by Bed-Stuy standards, and its story is the story of the neighbourhood itself. But before I tell it I should satisfy your curiosity about the murder, drugs, and guns that I mentioned earlier. Bed-Stuy was tarnished with a bad reputation in the 1970s and 1980s, when New York City was lost to a savage recession and a brutal crime wave. Heroin and crack were rife, particularly in Brooklyn, and parts of Bed- Stuy were among the city’s worst neighbourhoods.

But the nearby fashionable Williamsburg, the affluent Stuyvesant Heights, and artistic Bushwick shared all of those attributes in equal measure with its neighbours. It is also important to understand that many of Brooklyn’s neighbourhoods are divided along racial as well as geographic lines. Williamsburg is dominated by Hasidic Jews, while Bay Ridge is an Irish and Italian working-class area. Bed-Stuy has for the past 50 or 60 years been a neighbourhood inhabited largely by black professionals. African-American doctors, lawyers and bankers have made the splendid brownstones of Bed-Stuy their family homes for generations.

These family links survived the downturn of the Seventies and Eighties and many houses are still owned by the children and grandchildren of the professionals who bought them in the Fifties and Sixties. It was their belief in the power of property ownership to anchor a family’s fortunes that saved Bed-Stuy from destruction in the face of urban decay. But our house, and others like it near by, was built many years earlier by a different ethnic group. Brooklyn, as its name suggests, was built alongside an important watercourse and was therefore very attractive to German immigrants seeking to replicate the great breweries of Bavaria.

America’s first industrialised breweries were built in Brooklyn — 48 of them in all — and they made their owners very wealthy indeed. Maria Bauermeister was the daughter of one such brewery owner, and it was for her that our brownstone was decked out with an opulent interior in 1906, although the house had been built in 1899. The ornate Victorian fireplaces that we use today were chosen for her. So, too, were the intricately forged cast-iron stove in the dining room and the marble and oak built-in ice box alongside it. Heavy oak “pocket doors” disappear into the lath and plaster walls as they slide open to reveal a Germanic filigree panel between them at ceiling level.

Outside, the familiar Brooklyn stoop of 12 carved stone steps is flanked by heavy cast-iron banisters, while a neat row of spiked black railings marks our boundary to the north. After Bauermeister’s death in 1973, the house was left to her companion, Willie Mae Dixon. But Dixon shuttered the place up, opening it only a few weeks a year, according to our neighbours.

Eventually Willie Mae left the house to her own daughter, who lived in it for only a couple of years before selling it to the person from whom we bought it. It is a remarkably short lineage of just four previous owners that has ensured that this 107-year-old palace has remained in such remarkable condition. There were 6,000 such brownstones built at the same time as ours in Bed-Stuy, but many of them have been pulled down to make way for shops, railway lines, schools and apartment blocks.

For all the grandeur of its original detail, our house still needs a lot of work to bring it back to its former glory. And so, too, does Bed-Stuy. Serious crime rates have dropped by more than 65 per cent since 1993, but there are still robberies, burglaries and the occasional shooting. And as with every inner-city area, there are some streets it is not sensible to walk down late at night.

But then some things are worth putting up with for a really nice original fireplace.

• James Doran is Wall Street Correspondent of The Times

  For $600,000

• SoHo: $625,000, King Street, studio apartment, 530 sq ft, in a 19th-century converted schoolhouse. Includes bicycle

  storage space. For sale by Corcoran.

• Greenwich Village: $609,000, East 9th Street, four-room, one-bed apartment with small bathroom and kitchen, about 700 sq ft, for sale by Corcoran (www.corcoran.com).

Times recommends

• Second homes: credit crunch casualty or money-spinner?
• How I built my own sliding house
• Top 10 incentives for house buyers