David Cameron has admitted defeat after the government lost a 10-year battle with the Guardian to keep secret a “particularly frank” cache of lobbying letters written by Prince Charles.
Following a supreme court ruling on Thursday that 27 letters between the heir to the throne and ministers must finally be published under the Freedom of Information Act, the “deeply disappointed” prime minister has told aides to prepare their release.
The judges decided by five to two that the government had acted illegally when it vetoed the publication of Charles’s ministerial correspondence, the release of which had previously been approved by a lower court.
Related: Supreme court clears way for release of secret Prince Charles letters
It will be the first time that any of Charles’s “black spider memos” have been made public through official channels, and their release is expected to reveal how the heir to the throne attempted to influence government policies in private correspondence.
The cabinet had warned that the disclosure could potentially damage Charles’s political neutrality and “seriously undermine” his ability to fulfil his duties if he becomes king.
The prime minister said: “This is a disappointing judgment and we will now consider how to release these letters. This is about the principle that senior members of the royal family are able to express their views to government confidentially. I think most people would agree this is fair enough.”
No timetable has been set for the release of the letters and there is likely to be legal argument about the redaction of parts of the letters over as-yet-undefined privacy concerns. However, Cameron’s official spokesman made clear there would be no more appeals against the judgment.
The letters are between Charles and ministers in seven Whitehall departments, including those responsible for health, farming, and planning – all areas where Charles has previously lobbied politicians on controversial issues, such as the availability of complementary medicine through the NHS, organic farming and architecture. They were sent in 2004 and 2005.
They are said by the former attorney general Dominic Grieve to contain Charles’s “particularly frank” views on a range of issues.
Paul Flynn MP, a Labour member of the Commons political and constitutional reform committee, said the ruling could increase public demand for the crown to be passed straight to Prince William when the Queen dies.
“This opens up a much bigger issue,” he said. “If there are serious questions about the suitability of Prince Charles as a monarch, there could be a question in the public mind about whether to skip a generation. The attorney general already said the main justification for keeping the letters secret was they would hinder Charles’s ability to be a successful monarch.”
The supreme court ruling does not mean all of Prince Charles’s correspondence with ministers will now be made available using the Freedom of Information Act. In 2010, the government tightened the rules so that there is an absolute exemption prohibiting the release of any correspondence between public authorities and the monarch or heir to the throne.
The anti-monarchy campaign group Republic predicted that publication might swell support for their cause by revealing Charles “as a serious political force rather than as apolitical and harmless”.
The ruling on the letters was handed down by Lord Neuberger, president of the supreme court. The seven members of the court who passed judgment were the last of 16 judges in total who have considered the Guardian’s request to see the letters, which was first submitted in April 2005.
The freedom of information tribunal in 2012 finally ordered their release, only to be vetoed by Grieve, then the attorney general. That sparked more legal wrangling, which led to a court of appeal victory for the Guardian, which was again challenged by the government.
Neuberger said on Thursday: “We dismiss the attorney general’s appeal, and the decision of the upper tribunal that the advocacy correspondence should be disclosed under the Freedom of Information Act … stands.”
He added it was not reasonable for Grieve to have issued the veto “simply because, on the same facts and admittedly reasonably, he takes a different view from that adopted by a court of record after a full public oral hearing”.
Related: Repercussions of the release of Prince Charles’s ‘black spider memo’ letters | Letters from Michael Rosenthal, Professor Kevin Dunion, Penelope Gibbs and others
A spokeswoman for the prince said: “Clarence House is disappointed that the principle of privacy has not been upheld.” A royal aide added that Clarence House was relatively relaxed about the imminent publication of the letters themselves. The palace is understood to be more concerned that the broader principle of privacy in communications between the Prince of Wales and ministers has not been upheld.
Margaret Beckett, Labour’s environment secretary at the time the letters were written, welcomed the ruling. She said: “If views are being expressed and influence being exerted, then there is a certain amount of right for the public to know.”
Downing Street said it is also now considering introducing new legislation which will reaffirm ministers’ power to veto the release of information under the act.
“Our FoI [freedom of information] laws specifically include the option of a governmental veto, which we exercised in this case for a reason,” Cameron said. “If the legislation does not make parliament’s intentions for the veto clear enough, then we will need to make it clearer.”
Maurice Frankel, director of the Campaign for Freedom of Information, said any move to strengthen the veto would be very unwelcome. “It gives ministers the opportunity to overturn the wheelbarrow every time they don’t like a decision,” he said.