We are in the 1920s, and confidence bordering on the insane has seized the bosses of Britain’s electric companies. One day they will be nationalised, but first they are to build the biggest power station in Europe, and nothing is too ambitious. The architect is to be a famous name, and the interior will be so fine that the engineers wear slippers.

Now fast forward to 2007. The power station is a derelict shell. A one-bed flat in the area can command up to half a million pounds, and yet the world’s biggest brick building stands empty – and has been decaying for nearly 20 years.

This is Battersea power station. It is so vast you could drop in St Paul’s Cathedral. And it is surrounded by 32 acres of prime London riverfront, all unused. The site changed hands recently for £400m, but locals are sceptical that anything much is going to happen. So what is it about Ugly Betty Battersea that keeps her in this state? Every instinct suggests an intriguing story.

The trouble with Betty is that she blows up in people’s faces. She cusses those who think they can direct her destiny. John Broome, the first developer to enter the tomb of the power plant, booked Margaret Thatcher to open a theme park there at 2.30pm on May 21, 1990. But his project ran into the ground so fast that he is remembered locally as something of a dreamer. After him came speculators from the Far East, who spoke with passion about their first property on this side of the planet. In 13 years, all they managed to redevelop were press releases. And now, 24 years after the power station was decommissioned, we have new players. The latest developers are Dublin’s Johnny Ronan and Richard Barrett, two tough operators in an Irish property boom that is reaching out and across the world. And still the landmark is crumbling.

Right now, it looks as though one of Britain’s most popular buildings is to have its chimneys replaced by replicas, before views of its famous silhouette are blocked by jumbo-sized flats and offices. Do the developers want to destroy Ugly Betty so that they can build luxury flats without the inconvenience of restoring a national landmark? Or does Betty somehow bring down each visionary who tries to save her? This is not just a property whodunnit. The hole in London’s heart raises bigger questions.

We live in an age that rightly believes in free markets, and does not trust official bodies. But there are limits to what private enterprise can do, and saving Battersea power station might cross them. Can capitalism actually deliver a project on this scale? If so, why hasn’t it? Or is this simply an old-fashioned tale in which private greed meets public incompetence, and developers are allowed to do nothing until they are ready to make a fortune? Place your bets.

To some, she was a very ugly Betty indeed. At first it was feared that smoke from the new power station would ruin the paintings in the Tate Gallery across the river, and many Londoners were shocked at the rising of this brutal block in their midst. She began in 1933 as an upturned two-pin plug, before two more chimneys made her into an upturned table. At her peak in the 1950s she was burning 10,000 tons of coal a week, and sucking up 340m gallons of water a day from the Thames. Sir Giles Gilbert Scott’s building even caused the BBC to delay the launch of BBC2 in 1964, when a fire at Battersea caused power failures throughout the capital.

This was heavy engineering at its best, the urban equivalent of battleships, steam trains and ocean liners. As early as 1939, a survey of celebrities named her the second most popular modern building. She was “the cathedral of the electrons”, in H J Massingham’s phrase, with fine architectural detail forming a stark overall shape. Station A – the half of the power station built first – has mighty bronze doors bearing godlike figures depicting Power and Energy, not to mention a marble turbine hall and the art-deco control room with its parquet floor.

By the 1980s, when Britain was not such a confident place, the effect was just astounding. Going onto the roof in 1983, the journalist Felix Barker, an architect’s son, was amazed to find decorative detail breaking up the vast brick walls, and darted crenellations in the upper parapet. “Up there,” he wrote, “you don’t think of that wretched inverted-table image: you’re reminded of the magnificent brick cathedral of Albi in the south of France.” And all this architectural subtlety was for the benefit of one humble man on the roof who, in the early days, kept a lookout for weather changes that might mean a sudden demand for more electricity.

Even if all we saw was the exterior, Britain came to view Battersea with affection. The station made the cover of Pink Floyd’s 1977 album Animals, with an inflatable pink pig floating between the chimneys, supposedly as a “capitalist pig” watching over industry. In those pre-Photoshop days, the pink pig was made by the Zeppelin company and tied to a chimney for the pictures. But it broke free of its moorings and rose into the Heathrow flight path, to the incredulity of pilots in approaching planes. The runaway farm animal was escorted by police helicopters before drifting to the ground in Kent, and when the photography was rearranged, snipers were on hand to shoot it down if it made another escape.

By 1977 Battersea was already a place of legends, but even as Pink Floyd pressed its album, people knew the ageing power station would close. The Central Electricity Generating Board (CEGB), which had the task of keeping the lights on in England and Wales, wanted to tear down Scott’s building and redevelop the site. But they couldn’t, because in 1980 it was listed. So they held a competition instead.

A committee of worthies invited candidates to find a new use for London’s most famous industrial landmark, and picked a proposal to turn it into an indoor Alton Towers. This, at the time, seemed an excellent idea, even if some in Battersea expressed doubts. (How were 2m visitors a year going to get there? And where would they park?) For a city of 7m, 1980s London did not cater that well for families. On a wet and windswept weekend, the choice was to queue in the rain outside Madame Tussauds or to jostle among other damp families in a museum.

And here was John Broome, a handsome 6ft 3in entrepreneur who seemed to make the prime minister go weak at the knees, pledging a soaring hall with five tiers of restaurants and entertainment. In a white helmet marked PM, Mrs Thatcher tripped ecstatically through the Battersea mud with Broome, and promised to come back two years later, at 2.30pm on that May day in 1990, to perform the opening.

This was a new flowering of capitalism. The prime minister praised a “wonderful example of private enterprise and local government working hand in hand for the benefit of Britain” before congratulating Broome on the venture’s “touch of pure genius”. In the words of Timon and Pumbaa in the children’s cartoon, what could possibly go wrong?

In the event, the only people there on May 21 were Alf Dubs, the Labour MP for Battersea, and a modest group of campaigners and conservationists, who performed the mock opening of an empty building that had no roof and was already derelict. If Britain at the time of Betty’s construction was confident, then Britain today is best described as naive. Every turn in the Battersea saga has been helped along by naivety.

Tragically for Broome, recession brought down the curtain. Perhaps this should not have been a surprise. Broome was 46 and had always been, er, optimistic. Missing from his presentations to bankers was the fact that as a public schoolboy of 16, he had bid at auction for a £2,400 house when he had £122 in the Post Office and it was illegal for under-21s to own property. By his own account, his parents saved the family from disgrace by creating a trust to buy the building. (“But I paid back the money,” he points out. “I honoured my word.”)

As far as Betty’s custodians were concerned, he did have a credible track record: he started the 1980s boom in theme parks after marrying into the family that owned Alton Towers, where he introduced the corkscrew rollercoaster. But if the CEGB had carried out a more rigorous “due diligence” before handing her over in 1987, the clues to a somewhat cavalier business style were there. They could not be expected to know that in 1973, his wedding invitations took their theme from James Bond. But they might have noticed that in 1980 he did not ask for planning consent for the Alton Towers corkscrew until eight months after building it, and that his property company, JLB Investments (Chester) Ltd, ended the year to March 1987 with a deficit of £274,971.

At Battersea, the projected costs under this colourful figure soared from £127m to £229m, beyond what even the plausible Broome could borrow, and in 1990 he sold Alton Towers to repay some of his debt. The London theme-park project, for which Betty had more or less been given away – Broome paid the CEGB £1.5m for her – was dead. It was not entirely his fault. The curse of the power station meant that she turned out to have her feet in London clay, which everyone knew, but without proper foundations, which nobody realised. This month, explaining himself for the first time, Broome presented the timing of his exit as a great achievement.

“I relinquished myself of all big projects before the recession struck in all its intensity,” he said. “To get out of it without being rolled into the bottom of a 10m-degree volcano was an almighty success, certainly equal to the success of carrying on to a successful scheme.” The entrepreneur left the power station with no roof, no west wall and no coherent future. And so started a journey in which new proposals would concentrate less and less on reusing the power station and more and more on the potential for hotels, flats and offices on the extra land around the site. Enter Victor Hwang.

If everything at Battersea happened slowly, it was never slower than under the four Hwang brothers, who bought Broome’s £70m mortgage at Battersea for a reported £10m, then repossessed the site. It took them three years to secure legal ownership. It took nine more to buy the new parcels of land they wanted. The Hwangs are a Taiwanese family who built 980 apartments in 18 towers in Hong Kong, and run hotels and fast ferries there. In Battersea, Victor Hwang was the dominant brother.

If Broome’s flaw is an ability to be surprised by reality, Hwang’s is to be emotionally engaged. “Any suggestion that Victor Hwang went to Battersea simply to rip off a great landmark by letting it rot is completely wrong,” says Steven Norris, who met him in 1993 because he was John Major’s minister of transport in London. “Victor lived, ate and breathed the project seven days a week. His passion for it never dimmed.”

According to Norris, the “quixotic, charismatic, occasionally infuriating, often inspirational” Hwang was unable to stand back from our Betty. He wanted nothing less than to shift London’s centre of gravity from the West End to south of the river. It didn’t bother him that the site was next to a waste-transfer station on one side and Battersea Dogs & Cats Home on the other. It was only a mile, as the pigeon flies, from Buckingham Palace, and it was to become the smartest shopping destination in Europe. Every luxury global brand would be there, creating a new Bond Street and King’s Road in one. But this doesn’t do it justice. It’s hard for mere words to capture what Hwang intended.

He wanted hotels, which he designed down to the furniture in the bathrooms. There would be a conference centre twice the size of Westminster’s QE2 Centre. At the heart of the power station would be a 2,000-seat permanent home for Cirque du Soleil, the Canadian acrobats whose performances in an invented language, Cirquish, have held audiences rapt across the world. The theatre would have a huge, transparent back wall so that visitors could watch rehearsals during the day; then they might head upwards to the “Covent Garden in the air”, with terraces from which you would gaze into the sunset over the tree tops of Battersea Park. Wandering routes would take you to the top of old Betty, as if you were on a pilgrimage to Mount Fuji. At the top of one chimney would be the world’s most intimate restaurant, with a single round table for two. Also planned at various times were a glass ballroom, a sports complex, a village for the film-making industry, a 16-screen cinema, a new footbridge over the Thames, a railway link to Victoria and, almost as an afterthought, a few hundred apartments. It is a vision that comes somewhere between inspired and insane and is entirely in the spirit in which the London Power Company erected Battersea power station in the first place.

But Hwang was too involved. “He constantly changed his mind as a new idea consumed him,” says Norris. “Months of work, and millions of pounds, would be written off in a single day as the plans changed.” In the end, human frailty had more to do with his failure than the curse of Battersea, and when, late last year, Hwang won planning consent for a less visionary, dog’s-dinner version of his scheme, with 1.46m square feet of entertainment, leisure and shopping within the power station as well as two hotels, 750 flats and 750,000 square feet of offices and exhibition space, he knew he wasn’t going to carry it out. As soon as consent was granted, he sold the site for £400m. A colleague says he is “pretty broken-hearted”.

Man buys land for £10m and sells for £400m. Makes profit of £390m for doing nothing, right? This is the easy version of the Hwang story, but it is not certain that he made any money at all. He claims to have spent £200m in Battersea; where the wonga went is hard to see, as very little of it was lavished on poor old Betty, who came out of Hwang’s 13-year dream in the same state as she entered it. But that does not mean Hwang is telling stories. It is possible that, with his extra land purchases, professional fees, and the interest on his debt, he did lay out £200m, and the £400m he received is not what it seems. Some £150m of it is in loan notes, which he can convert into the shares of Battersea’s new owners. So in cash terms he comes out no more than £40m ahead, which in Betty’s world is breaking even.

To local people – especially poorer local people, who live in Battersea and once worked there, making cement or candles or beer on its industrial riverfront – it’s all very surreal. “What I find offensive is, if I don’t pay my council tax on time, it’s a criminal offence. But they let this lot get away with millions,” says June Hautot, who would love to see the site offer something for local children.

“We had a couple of years when we were hopeful that this was going to be the saviour of our beloved building,” says Brian Barnes, a local artist. “But we got fed up with inactivity, and now we’re in the same position again.”

They believed in Hwang. Wandsworth council passed his plans, and English Heritage, which gets £129m of public funds to protect historic buildings, failed to issue repairs notices and let itself be persuaded that the station’s chimneys need to be demolished and rebuilt. Were they naive? If so, Gordon Brown should join them. In March last year he used the power station to launch a white paper on further education, and was photographed there wearing a hard hat and an unusual smile. He praised a scheme to train local people in construction skills “including heritage bricklaying”, as the press release said, “which will be in heavy demand on site when the work starts on the restoration and preservation of the iconic Power Station building this summer”. Of course, the summer came and went, the work failed to start, the 3,000 jobs forecast in the press release have not been created, and old Betty entered her 18th winter with no west wall and her boiler house open to the skies.

It needn’t be like this. All the way through this story, there has been an alternative vision for Battersea. Compared with the drama of the all-or-nothing masterplans of a John Broome or Victor Hwang, the alternative might strike you as unambitious, even dreary. But the odd thing is, we know it would work, because it has been tried already in the Yorkshire town of Halifax.

Dean Clough is a vast Victorian complex built for a carpet manufacturer called Crossley. By 1860 it was the town’s biggest employer, covering 18 acres – three times the size of the Battersea building – and providing work for 5,000 people. As the textiles industry contracted, operations were scaled down and in 1983, the same year as Betty was decommissioned, the mills closed.

No rock group has flown a pig over Dean Clough. It closed amid high unemployment in northern England, and its prospects looked doomed. Yet today it is home to a theatre, 145 businesses, cafes, restaurants, conference facilities, an art gallery, the BBC, the NHS, a hotel… and its 870,000 square feet of floor space employs nearly 3,500 people.

This is the achievement of the late Jonathan Silver and his friend Sir Ernest Hall, a Lancashire-born pianist and philanthropist, who bought Dean Clough with two others. “Oh,” says Hall, a little misleadingly, “I gave some free space to some artists, and opened a cafe.” But then he attracted businesses to his little community in the vast factory, and the crucial difference is that he did not attempt everything in one go, with the only outcomes being glory or disaster. So, over the same time frame as the failures in south London, Dean Clough has become a thriving model for using industrial buildings in the post-industrial age.

Capitalism will only tolerate so many dreamers before it finds a realist to take their place. And after the setbacks of the Broome and Hwang years, a new era is starting. The Irish have arrived. Betty’s new custodians are Johnny Ronan and Richard Barrett, the main players behind a public company called Real Estate Opportunities, which is listed in London and Dublin but domiciled in Jersey. The £400m purchase was approved at a meeting of REO shareholders held in Jersey between Christmas and New Year.

These are men of international ambition who, in contrast with the publicity-loving Broome, don’t give interviews. In 2005 their private company, Treasury Holdings – which owns 58% of REO – announced what was said to be the largest single property development ever undertaken by an Irish company: to create 18,000 homes and three town centres in a joint venture in China. They are also responsible for Spencer Dock, a 52-acre site in Dublin.

Barrett, a poetry-reading barrister, spends most of his time in Shanghai, overseeing the pair’s efforts to create the most successful western property company in China. “Johnny and I were very lucky to find in each other equal ambition, complementary talents and a shared vision,” he says. A leading figure in the Irish National Trust has described the pair as “gratuitously truculent”.

The bearded Ronan, the son of a Tipperary pig farmer who went into property, trained as an accountant before he and Barrett set up Treasury Holdings in 1989. Comfortable with displays of wealth, he is among Mercedes’s few customers for its Maybach limousine, which he uses as an office, and is thought to have spent £300,000 on a 21st-birthday party for his daughter Zoe.

The men’s first act has been to dump Hwang’s masterplan and to appoint a big name, the Uruguayan architect Rafael Vinoly, to come up with a new one by this autumn. Believe it or not, there is hope.

Until now, the story of Battersea has been that whatever the developers decide, the authorities tend to go along with. They don’t have an alternative vision. But the new owners are starting again. With Vinoly’s feet barely under the table, a director of Treasury Holdings told this magazine that the idea of Battersea as an international leisure destination is being abandoned, and instead it is to be “a new urban district of London”. Ronan and Barrett imagine a place for Londoners “to live, work and play”. In other words, they are likely to drop anything that is unrealistic and increase the number of flats and offices to whatever they can get away with. But the hope lies in this: unlike Broome, who ran theme parks, or the Hwangs, whose empire extends to hotels, ferries, oil and gas, these are pure property developers. The chances are, they will get the job done.

The new masterplan will have a balance between homes and offices, so people can live within walking distance of work if they want to, and affordable housing “so it’s a mixed community, not just a ghetto”. The power station is not for any soaring scheme but for “the normal stuff that people do at weekends: culture, entertainment, shopping”, says the director, Rob Davies. “Think of Covent Garden. It’s a bit touristy, but people feel good there.” This will be a place for Londoners to linger, he adds. “The site is big enough to create its own character.”

Whatever they decide, they may be helped by fortunate connections. Initially it is Wandsworth council, which has been Conservative-controlled since 1978, that decides if the revised £2 billion scheme goes ahead. Among the names on Ronan and Barrett’s payroll is Richard Tracey, the chairman of Wandsworth’s Conservatives, who is “community relations manager” at the power station. He was brought in as a consultant three years ago by Victor Hwang’s company, Parkview International, and kept on by Ronan and Barrett when Treasury Holdings took over.

Tracey’s wife, Kathy, is a senior Conservative councillor. She is a member of the cabinet that directs Wandsworth policy, and sits with the council’s leader and deputy leader on its executive committee. Parkview also found a place for Ian Thompson, Wandsworth’s former borough planner, who became a part-time “planning consultant” on a reported £75,000 two months after leaving his public post. But he has not been kept on by Ronan and Barrett.

Kathy Tracey does not disclose her husband’s job on the register of interests, and indeed is not required to; there is no suggestion that she or her husband has acted improperly. Richard Tracey, an open and genial man, says he accepted the consultancy “because I have lived in Wandsworth for more than 30 years, and it’s a matter of great interest to me”. Even so, this is one of Europe’s biggest projects

and the facts belong in the public domain. Wandsworth has been kind to the developers, as Hwang acknowledges. “We’ve appreciated the unwavering support of Wandsworth council, who have seen us through in excess of 50 submissions since 1996,” he says.

There is also the bigger question of whether public assets sold for one purpose should be used for another. Right back at the beginning of the story, Alf Dubs, then Labour MP for Battersea, wrote to the CEGB to ask what safeguards there would be if the winning competition entry failed to proceed. The obvious risk was that if the indoor Alton Towers foundered, and the power station ended up in such a state that it was demolished, a large tract of riverfront would become very valuable. The CEGB replied that, apart from Wandsworth council’s planning controls, no safeguards were considered necessary. You’d think this might be a question for MPs on the Public Accounts Committee; after all, if public assets that are more or less given away end up part of a £2 billion property development, that’s of public interest. But the committee has never looked into the sale.

We are where we are. Two tough Irishmen have Betty in their power, and history suggests that whatever they decide to do with her, Wandsworth and English Heritage will co-operate. Their most recent favour was a legal agreement defining how much work needs to be done at the power station before other parts of the £2 billion project can go ahead. The developers got what they wanted.

You can understand that. Everyone wants to secure Betty’s future, and one way is to help the developer. But what a contrast this implies between the can-do spirit with which she was built, and the dismal expectations now.

“The failure in Battersea is total,” says Keith Garner, an architect and conservationist who researched the power station for his dissertation in 1993. “There are no jobs, no houses, no businesses, no employment, no general regeneration of the area, no tax revenue. Everyone is failing to do their job. The politicians, the local authority, civil servants – even the press has failed to report this in a proper way. The developers don’t develop; the architects just do paper plans. There’s a multiple car crash of failure and incompetence.”

We, the public, have an interest in this, and we need to send a message to all of them. Sites as good as this come up in central London once or twice in a century. It was Ronan and Barrett’s decision to risk £400m in Battersea, and in business there are no one-way bets. So they should tell their man Vinoly to come up with something worth the wait – something real that London can be proud of. Because if you don’t make it good, you never know, the planners might just turn the new scheme down. After all the acquiescence and incompetence, this may seem as likely as a pig flying over Kent. But sometimes, just sometimes, it is how capitalism works.

Times recommends

• Second homes: credit crunch casualty or money-spinner?
• How I built my own sliding house
• Top 10 incentives for house buyers