Fear of AI concentrates power too

Fear of AI can be excellent business. If, in the name of protecting ourselves, we end up handing a few companies the power to decide who may develop this technology, we will have turned safety into a barrier to competition. That is the risk of regulatory capture that worries me, particularly after the controversy of the past two weeks over AI safety and its potential for apocalypse: that those already inside manage to shut the door on everyone else, and that we thank them for it.

That can happen even when the fear is sincere. Jacob Coxon, a young British mathematician and former OpenAI and Anthropic researcher whose statements about the lack of safety in frontier models triggered the recent controversy, left Anthropic before his shares vested. Evan Hubinger, who leads alignment stress-testing at Anthropic and previously worked as a researcher at OpenAI, estimated the risk of AI-driven extinction within a decade at more than 10%: a personal judgement, not a scientific measurement. His more uncomfortable warning was something else: they still had no plan for controlling a superintelligence, nor were they clearly on course to develop one.

Taking that seriously may require slowing or halting specific developments. But such a restraint needs public criteria, independent review and conditions for lifting it. A laboratory’s fear should not give it authority over its competitors.

Dario Amodei’s proposal, the most significant of recent weeks, calls for moderating the pace, bringing in external evaluators and permitting certain safety discussions through a limited exemption from competition rules. Some of these ideas are reasonable, but who will choose those evaluators — some of his early suggestions are companies with strong ties to Anthropic that appear far from neutral — who will check their independence, and how will a company that does not yet exist enter the market? If the leaders set requirements that only they can afford, safety ends up protecting their business.

David Sacks, the White House’s former AI lead and current co-chair of the science and technology council advising Trump, responded with an uncomfortable objection: they can slow down without waiting for their preferred regulation to be approved. We should listen to him with the same critical spirit we apply to the laboratories. Trump insists on beating China, while Beijing rejects alarmism and champions development and safety. A slowdown observed by only some participants could redistribute power without resolving the risks.

There are different ways to organise that oversight. Musk proposes testing between competitors; Zuckerberg favours independent evaluators and responsibility on the part of each laboratory. He has even said that he delayed the release of his own models when they were not yet ready for public use. Altman has also promised evaluators access comparable to that of employees. What matters will be what they can check and to whom they are accountable. Altman fears both losing control of AI and too much power becoming concentrated in too few hands. Let us take that second fear very seriously, even when it makes the person expressing it uncomfortable.

Jensen Huang, Nvidia’s founder and CEO, has taken the objection further. In July, he warned that some companies might seek government regulations that gave them an advantage. This very weekend, he rejected apocalyptic predictions about AI without denying the existence of real safety problems: ‘we must advance as fast as we can, but no faster than we should’. His proposal is to regulate real, verifiable harms, rather than merely hypothetical risks, and to apply existing laws first. His argument also deserves a cautious hearing: Nvidia is one of the largest economic beneficiaries of an AI race that does not stop.

In Europe, Mistral accuses its rivals of promoting regulation that favours them, as Reuters reported on 18 September. The criticism deserves attention, although Mistral, like all the others, is also defending its business.

Yann LeCun, a Turing Award winner and one of the pioneers of deep learning, compares the diversity of open AI with pluralism in the press. That strikes me as a good starting point for Europe: public computing capacity, open models and evaluations accessible to universities and small businesses. We can set requirements according to capabilities and risks, and help people meet them.

The grandmother receiving a fake video call and the small business facing an attempt to steal its data also need powerful defences. Criminals already use unrestricted models and abuse commercial services. Open source can make protection cheaper, even if it also facilitates attacks. Those capabilities need to be evaluated without reserving the best defence for those who can pay enormous sums for it.

That is why the theatre of fear worries me so much. Yoda — forgive the Star Wars reference — warned us that it leads to the dark side. We do not need to travel to a distant galaxy to understand the point: a frightened society may accept restrictions that it would question in a calmer frame of mind. Naval Ravikant, entrepreneur, investor and one of Silicon Valley’s gurus, compared AI to fire or a nuclear weapon: depending on its nature, we will want everyone to have it, or nobody to be able to have it. I am inclined to think it clearly resembles fire much more closely. It is certainly not just a weapon, but something that can do a great deal of good, while we should also remember the analogy’s limits.

And when considering who gains from fear, we should also ask who may lose from waiting. We do not already live in a perfect world worth freezing and preserving exactly as it is. Suffering already exists: Buddhism speaks of dukkha, suffering and dissatisfaction; the Christian tradition speaks of a ‘vale of tears’, although its more recent documents have increasingly defended joy, particularly under Pope Francis. UN estimates for 2023 amount to some 170,000 deaths a day. We do not know how many thousands of deaths AI will be able to prevent each year; we do know that the wait takes place within real people’s lives.

Curing diseases, slowing ageing, understanding the brain or securing clean energy through systems that also help halt climate change deserve effort. To give just one example, AI has already helped control plasma in fusion experiments, although that is not yet a power station. More abundant energy could also ease disputes and wars over resources, which we see in the news every day. Those benefits are not guaranteed, but nor should we give up pursuing them. Even Amodei shares that ambition; the debate is also about who governs the path towards it.

The concentration of power that would allow the market to be closed off could also leave those benefits in very few hands. That is why I believe societies and governments need to begin serious work now on policies that adapt to the speed of technological progress. This includes beginning to adjust taxation to the rapid AI take-off we seem already to be entering, and creating universal basic incomes that provide security during the transition. Someone who gets up at five in the morning to go to a job they hate has different worries and urgencies from those who enjoy power and prestige and have far stronger incentives to preserve the status quo. The potential post-scarcity society we might reach in a few years or decades, after what looks set to be a complex and difficult transition, will remain an empty promise if some elites confuse protecting their position with protecting everyone.

Fear can have another, less obvious consequence: distancing us from the tool itself and making us use it less well. A society that sees AI primarily as a threat will have less incentive to get to know it, experiment with it and learn its limits. That will not necessarily make us safer. AI literacy is associated with lower anxiety and a greater willingness to use it judiciously; in organisations, moreover, when its use is viewed with fear or stigma, it can be driven underground into shadow AI, beyond security controls. This is an important paradox: the more powerful this technology becomes, the more we need citizens to know how to use it, when to trust it and when not to. The aim should be informed caution, not a society so frightened of AI that it gives up learning to live with it.

I want AI that is safe, accessible and capable of reducing inequality. With independent testing, liability for harm and proportionate controls. We must develop it that way, but also, as far as possible, not a day later than necessary. Because an unjustified delay can also prolong avoidable suffering. Prudence must answer for what it prevents, as well as for what it permits.

Fernando Nieto Lobato

Director of Digital Innovation at Institución Educativa ALEPH and editor of the estrategIA newsletter

English text alternative for the infographic¶

How to make AI safe without handing its future to a few. Fear can be sincere and still favour barriers to competition and distance us from the tool.

What fear can cause

• Accepting restrictions. A frightened society may accept rules it would question in calmer circumstances. If only the leaders can comply, the market closes.
• Learning less. Seeing AI primarily as a threat can distance us from getting to know it, using it and testing its limits.
• Concealing its use. In organisations, fear and stigma can lead to AI being used outside security controls.

Informed caution. Understanding capabilities and limits so that we can decide when to trust AI and when not to.

‘we must advance as fast as we can, but no faster than we should’ — Jensen Huang

Taking risks seriously

• Independent evaluation. Check who chooses the evaluators, what they can examine and to whom they are accountable.
• Restraints with conditions. Halt specific developments when necessary, with public criteria, independent review and conditions for lifting the restrictions.
• Liability for harm. Require controls proportionate to capabilities and risks.

The article’s proposals

• Access and competition. Public compute, open models and evaluations accessible to universities and small businesses. Openness can make defence cheaper, but can also facilitate attacks.
• Shared benefits. Adapt taxation and create universal basic incomes that provide security during the transition.

Who loses from waiting. Future health and energy benefits are not guaranteed. An unjustified delay can prolong avoidable suffering.

Prudence must answer for what it prevents, as well as for what it permits. — estrategIA

Translation note: Huang’s quotation in the text and infographic is translated from the Spanish original; its exact original English wording has not been independently established here. Relative dates, estimates, policy arguments and attributed statements retain the context of the article’s original publication on 23 September 2026. The reported extinction estimate remains explicitly a personal judgement, not a scientific measurement.