Will AI destroy civil service impartiality and make bureaucracy unbreakable?
Westminster warned that tech concentration will bake immovable ideology into the systems that power government.
Debates over the impartiality of the civil service have raged ever since a permanent class of bureaucrats was first anointed in the UK almost two centuries ago.
But the widespread adoption of AI tools threatens to end the argument forever by baking bias and unchallengeable ideology into the critical systems of government.
During a recent House of Lords debate on Civil Service impartiality, tech entrepreneur and GOV.UK co-founder Baroness Martha Lane Fox explicitly described the threat AI poses to Britain's public sector, warning the tech is "changing what we mean by an impartial state".
She said: "There used to be a reasonably clear idea of where policy stopped and delivery began. Ministers made decisions; the Civil Service implemented them.
"Technology is making that distinction much less clear, because the design of a digital system is itself a policy choice."
The Baroness added: "A biased decision no longer requires a biased person. The problem might sit in the data or in software bought from a supplier.
"It might be an assumption that nobody has noticed. It might be built into a system, and it can be repeated across thousands of cases."
The unbearable bias of being
The example Lane Fox gave to illustrate her argument is fraud. When systems are built to identify suspicious claims, someone must first decide what those claims look like and how to identify them.
"If technology helps determine access to a public service, somebody decides what information is important, who is important and who is not," Lane Fox added.
"Those choices may be buried inside software but they can have a profound effect on an individual life."
Once a judgment is embedded in systems and software, it can be applied consistently across thousands or millions of cases, while making it much harder for citizens, civil servants and even ministers to see exactly how the judgment was made, let alone how to challenge it.
READ MORE: TalkTalk collapse could pose a "genuine risk to life", government warns
AI is a different kind of risk. Institutions like public bodies already reward conformity. When ideologically biased creators program and train AI tools, then roll them out across government, those bureaucrats' prejudices could end up baked into every process artificial intelligence is involved in.
On a small scale, that could mean government communications which are next to useless because they intone endless nothings in grey corporate-inspired newspeak.
At worst, decisions and policies could be auto-generated at machine speed, with no human in the loop to insert balance.
Disaster down under
We've already seen the risks of blind automation play out in real time during Australia’s Robodebt scandal.
In 2016, Canberra vowed to recoup at least A$3 billion from welfare recipients, using automated income calculations to assess claimants' earnings.
The system failed to account properly for fluctuating income and was abandoned four years later, with the government forced to refund A$721m across 470,000 unlawful debts affecting around 373,000 people.
Thankfully, our wise leaders are already waking up to the risk of algorithmic bias.
On the same day as Lane Fox’s speech, the Council of Europe warned that AI in public administration is already shaping access to rights and services, creating risks of discrimination, opacity and unequal access to redress, which is a bureaucrat's way of saying we, the unwashed, may lose our ability to challenge unfair decisions.
READ MORE: Critical UK subsea power cables suffer "multiple exposures", National Grid reveals
The EU has already classified AI used to decide access to essential public benefits and services as high risk, requiring public bodies to assess potential impacts on fundamental rights under the AI Act.
In the Netherlands, MPs backed a motion requiring citizens to be told when an automated risk assessment contributed to a government decision, while a separate bid to force disclosure of algorithms operating without meaningful human intervention narrowly failed by 73 votes to 75 in the Dutch Parliament.
Additionally, Canada has confirmed that its automated systems used in core functions like immigration processing cannot refuse applications or even recommend refusal, which must remain a human decision.
And New Zealand’s Algorithm Charter requires government agencies to retain human oversight and provide a way for people to challenge decisions informed by algorithms.
One vendor to rule them all
All of which sounds great, but may not necessarily be enough to respond to the scale of the problem, particularly because technology concentration is already a problem for government, which relies heavily on Big Tech tools.
More than half of cloud-hosting spend on G-Cloud is going to AWS alone. MPs also recently warned that Britain’s reliance on a small number of US-based digital providers represents a “strategic and economic vulnerability”.
Britain is also racing towards full bore AI government.
More than 20,000 government employees participated in a large-scale Copilot trial organized by the Government Digital Service. We also revealed that the Crown Prosecution Service prompted "legal Russian roulette" warnings when it tested the AI tech last year.
Civil servants taking part in the wider Government Digital Service trial used generative AI for tasks including drafting documents, summarizing meetings and handling administrative work.
The government said workers saved an average of 26 minutes a day, equivalent to almost two working weeks each year.
READ MORE: Corporate bureaucracy could give AI attackers an advantage over defenders, NCSC warns
Government analysis published this year estimates that AI and automation could eventually generate £23.7 billion of benefits across the wider public sector, including healthcare, education and policing.
Ministers are now examining what that means for the machinery of government itself.
Responding to the debate, Baroness Twycross said a major Civil Service review will consider the impact of AI and technology alongside Whitehall's structure and operations, with a report due before summer 2027.
The real danger is not that AI will make bureaucracy biased. Bureaucracy has managed that perfectly well on its own for centuries because conformists tend to do well in large organizations.
It is hardly surprising that the public sector tends towards one side of a debate or another.
More worrying is the possibility that AI could make those biases harder to see, harder to challenge and almost impossible to get rid of.
And if you think it's hard to sack a useless civil servant, just try reforming the technology that underpins the public sector, where major government ICT projects take a mind-numbing average of 10.7 years to "deliver".
With a postman like that, who needs mail thieves?
