The big fish tell a big story. On one wall is a 15ft black marlin, nearly 90 stone of fighting fish, caught off Australia by Lee Marvin. On the opposite wall is a blue marlin, slighter but still an impressive 43 stone, caught by the late Hollywood superstar’s widow, Pamela, the girl he allowed to get away when they were both young and foolish, but reeled in a quarter of a century later.

Now, 20 years after the death of the gravel-voiced star of The Dirty Dozen and Point Blank, Pamela, 77, is putting the house in the foothills of Tucson, Arizona, where they found both love and peace, on the market for £3m.

The prize-winning marlin dominate the media room that Marvin designed after buying the sprawling, ranch-style home in 1975 for nearly £1m. They are unlikely to stay when Pamela sells up in order to spend more time at her other home, in Manhattan. She says that they may be donated to Australian maritime museums near the place where they were caught.

The tall brunette will be holding on to her most prized possessions, such as her late husband’s Oscar for the comedy western Cat Ballou, where, as a spectacularly drunken gunslinger, he learnt to ride a horse sideways. And the gold record for Wand’rin’ Star, his croaky ballad from Paint Your Wagon, an unlikely No 1 hit in Britain in 1970. Yet she is painfully aware that she will be leaving behind a family home shaped by one of Hollywood’s coolest and most idiosyncratic actors.

The couple, who married in 1970, had been living in Malibu, west of Los Angeles. Marvin was a marine during the second world war, winning a Purple Heart after he was wounded during battle in the Pacific, and the ocean was always where he felt most at peace. But Malibu, full of sex, drugs and rock’n’roll, was too close to Hollywood for his peace of mind.

“We talked about getting out with Clint Eastwood, who recommended his area in Carmel, up near San Francisco,” says Pamela. “We went up there, but it was foggy in June, and we did not want that. So we kept looking until Lee made this film called Pocket Money in Arizona.

“It was so wild and open, it was like the ocean, in a way, which is what we were looking for. Yet it was only a hour’s flying from Los Angeles.”

Tucson’s desert isolation has never kept it off the celebrity map. Clark Gable had a house there in the 1930s, when the city was an overnight train journey from Los Angeles. Paul McCartney, who gave Tucson a mention in Get Back, maintains the family ranch on the east side of town where his first wife, Linda, died in 1998. The actress Bo Derek owns a ranch in the hills and the singer Linda Ronstadt lives in town, where her brother was once chief of police. Calexico, a leading group in the alt-country music scene, are based in Tucson.

It was not just the desert that lured the Marvins, but the house itself. Designed by the architect Josias Joesler in 1936, it was set on 12 acres, and large enough for the couple and eight children – four from each of their previous marriages. “It was built to last for ever, and after LA we loved that,” says Pamela. “It had thick adobe [unfired clay brick] walls, which keep it cool in the summer and warm in the winter. Oh, yes, it was built to last.”

The couple bought the house for about $400,000 – about £167,000 at 1975 exchange rates – partially funding the purchase with Marvin’s fee for the film Shout at the Devil, in which he starred with Roger Moore, and which came out the following year. “I was happy with the house, but Lee wanted to change it all – there were lots of bathrooms, but they were small and needed to be updated. The kitchen, too,” Pamela says. “He designed all the ironwork and fixed up things from the local thrift stores. Every one of the six bedrooms has his mark on it. And Lee picked out the dark-blue tiles for the swimming pool because they reminded him of the sea. I remember him getting down there with the guys, with a six-pack of beer, and fixing the tiles. He loved working with his hands.”

Apart from the kids, there also had to be room for fish. The big fish. “Lee introduced me to game-fishing, first off Hawaii, then Australia,” Pamela says. “I think he was a little upset when I was the first to get a world record, for catching a 607lb blue marlin off Hawaii with light tackle. Then Lee caught a black marlin weighing 1,232lb off Australia. We had to build a media room, with a big-screen television and stuff, just for the marlin. This is where we spent our quiet evenings.”

Today, the two champion fish hang opposite each other in the room, along with the huge hooks the couple used for fishing. Below Pamela’s fish is a sofa, draped with American and Australian flags. In between rests a floppy old fold of sailcloth – Marvin’s beloved fishing hat.

It was a final marker of a romance so unlikely, even Hollywood would think twice about filming it. The couple’s road to Tucson began 30 summers earlier, just after the end of the second world war, at an idyllic pool outside Woodstock, then a country town north of New York.

“I was 15 years old, and loved to cycle into the woods to meet up with my friends at the local swimming hole,” recalls Pamela, a vigorous and colourful great-grandmother who has a New Yorker’s sense of irony leavened with a laconic western drawl. “And one day, standing in the water, tall, amazingly handsome, charismatic, with a booming voice, was Lee. Oh my God, I could not take my eyes off him – that was it for me.”

Marvin was six years older than the lovestruck teen, and was a descendant of America’s first president, George Washington. He had already had a taste of acting – starring in a US Marine short about leadership under fire, in which he was cast perfectly. Although he had the acting bug, Marvin went to work as a plumber’s assistant and, in one of the oddest breaks in show-business history, was asked, while repairing a lavatory in a local community theatre, to stand in for an actor who had fallen ill. The couple’s all-too-brief relationship ended when Lee went to New York to pursue theatre work, then, in the early 1950s, moved to Hollywood.

“Oh, yes, I understood,” Pamela says. “He left me high and dry to go off to work. I could not blame him, he needed to get out of town, but... I carried on with my own life in Woodstock, watched him rise in the movies, but never thought we would become involved again.

“Then I got this call in New York, where I was working in a news clippings agency – Lee asking to see me the same night I was due to go on a date with this other man. My sister persuaded me to turn Lee down, and I kept the date with the man I later married and had four kids with.” Yet throughout the 1950s and 1960s, as she happily raised her children, Pamela was aware of a gravelly voice in her ear, from movie screens or television chat shows. “He did come back to Woodstock occasionally. Our families knew each other, so it was inevitable we would bump into each other,” she says. “I interviewed him for a local radio show I hosted. But we were both married, carrying on with our lives, just friends until, well... ”

In 1970, Lee – who, like Pamela, was divorced, and had yet to become embroiled in what was to become a landmark “palimony” case with ex-girlfriend Michelle Triola – phoned his former sweetheart at home. “That was the night our lives changed. He said he was coming to Woodstock to see me. I packed the children off to spend the night at my mother’s and baked an apple pie.

“He arrived in this big car with a suit-case that he rolled into the living room. Then he said, after all those years, ‘Okay, let’s go. I’ve come to get you – we’re getting married.’ And that was it. After that night, we never parted. The desert home was our refuge – we had visitors, but Lee did not enjoy parties. We enjoyed the wildness of the area instead.”

So, why leave a house that holds such memories? “It’s too big for me, my granddaughter and great-granddaughter,” Pamela says. “And there is upkeep. I have also inherited a place in New York, where I now spend six months of the year, but that gift means taxes. And I want to move while I am on my own two feet, so it’s time to say goodbye to all this.”

The house has been put on the market by Russell Long, a Tucson-based agent (00 1 520 529 1116, www.longrealty.com). Despite the dark US property market, which has hit Tucson moderately hard, Long expects to sell to a well-heeled buyer seeking both Hollywood history and an authentic touch of Americana.

“This is Tucson, one of the last places in America where you can still taste the Old West,” he says. “There is magic in the air, and a home like this, with its wildlife, sunsets and romantic history – well, there is no place like it anywhere else in the world.”

Times recommends

• Second homes: credit crunch casualty or money-spinner?
• How I built my own sliding house
• Top 10 incentives for house buyers