PHNOM PENH, Feb 10 (Reuters) - The UN’s shock decision to abandon negotiations with Cambodia on a long-awaited Khmer Rouge tribunal has raised doubts that credible justice will ever be done for the 1.7 million victims of the "killing fields".

Rights activists, political analysts and diplomats are now wondering whether the UN can be persuaded to reopen talks on international participation or leave Prime Minister Hun Sen to carry through on threats to hold his own tribunal.

The UN’s chief legal counsel Hans Corell announced on Friday the world body was ending talks with Cambodia on setting up a special court after concluding that, as currently envisaged by Phnom Penh, it could not guarantee its impartiality.

The decision followed years of difficult dialogue with a government that has appeared reluctant to prosecute former Khmer Rouge leaders blamed for an estimated 1.7 million deaths in a catastrophic agrarian revolution from 1975-1979.

Government officials expressed shock at the UN move and said a trial would go ahead with or without UN involvement.

"It does encourage people to react in kind and do something - maybe move faster toward a trial without the U.N.," said a source close to the negotiations.

Cabinet Minister Sok An who heads the government’s task force on setting up a Khmer Rouge tribunal, said Cambodia would keep the door open for a U.N. return.

But government spokesman Khieu Kanharith said it had two other options: a trial without international involvement, or one involving judges and financial help from individual countries.

He said the government wanted a trial before general elections in 2003. "Now we are deciding what to do, and awaiting reactions from different U.N. countries," he said.

Some analysts are pessimistic about the possibility of the United Nations being persuaded to rejoin the process and said without its participation, Cambodia would target only selected Khmer Rouge leaders and leave untouched former members of the regime now holding positions of power in the government.

While governments including France and the United States have urged the United Nations to reconsider its decision, some diplomats say it would be very hard for the world body to make an about face after issuing such a firm statement.

"There will be no trial except a show trial," said Lao Mong Hay, director of the Khmer Institute of Democracy. "There will be no credibility and it won’t extend to other Khmer Rouge leaders in the government.

"Many Cambodians, like myself, don’t think our rulers are serious about a trial, firstly because some of them might be implicated," Lao Mong Hay said.

Several government ministers have Khmer Rouge backgrounds, including Prime Minister Hun Sen, who was a junior military commander before defecting to the Vietnamese, who overthrew the radical Maoists in 1979.

The government wants a trial limited to about 10 former leaders of the regime, most of whom live in quiet retirement in an enclave on the Thai border. But the United Nations has refused to be bound by such a limit.

A key sticking point has been over former Khmer Rouge foreign minister Ieng Sary, who was granted a royal pardon in 1996 and now lives in Phnom Penh.

If the government holds its own trial, it will likely start with the only two Khmer Rouge figures currently in detention.

They are former military commander Ta Mok, known as "the butcher", and Kang Kek Ieu or "Duch" - the chief of the group’s main political prison, where 16,000 people were tortured then sent to their deaths.

Ta Mok and Duch were arrested in 1999 and detained under special legislation outlawing the Khmer Rouge. Unless legislation is passed to extend their detention both must be released in coming months.

Rights activist Kek Galabru, president of the Cambodian League for the Promotion and Defense of Human Rights (LICADHO), said a trial without the United Nations would be neither impartial or credible.

"The UN should not abandon Cambodia. Yugoslavia, East Timor and Rwanda were able to have a war crimes tribunal - why not Cambodia?

"How can we explain to younger generations that they must respect the law if a group of fanatics responsible for the deaths of two million people are still free?"

An Asian diplomat said despite public support for international involvement, Hun Sen could manage a massive coup ahead of next year’s elections by holding a limited tribunal and plausibly blaming the UN for abandoning the process.

He noted that Hun Sen was currently riding high on self-confidence after sweeping local elections held last week, the first commune-level polls held since independence in 1953.

"Hun Sen is a good strategist. I’m sure he’s planned for this," the diplomat said.