people

Nick Bostrom

Philosopher warning that superintelligence may become the central existential risk and governance challenge

Nick Bostrom
https://www.theguardian.com/technology/2016/jun/12/nick-bostrom-artificial-intelligence-machine

Nick Bostrom is a philosopher and researcher best known for putting the idea of existential risk and artificial superintelligence into a rigorous public frame. He was a founder of the Future of Humanity Institute at Oxford and has written extensively on long-term risk, governance, and the possibility that advanced AI could outstrip human values or human control Nick Bostrom — official website, Nick Bostrom — Wikipedia.

His most influential work centers on the idea that increasingly capable AI may not simply automate tasks but eventually produce systems with cognitive advantages far beyond those of humans. In his book Superintelligence, Bostrom argues that a superintelligent system could pursue goals with extraordinary strategic power, and that without careful alignment and governance the result could be catastrophic. He also developed the concepts of instrumental convergence and orthogonality to explain why highly capable systems may not be benign even if they are narrowly tasked Superintelligence: Paths, Dangers, Strategies, Nick Bostrom — official website.

Bostrom's concerns are not limited to science-fiction scenarios. He has emphasized existential risk as a category of threats that could permanently curtail the future of civilization, and he has argued that AI is one of the most consequential technologies in that class. In the paper "The Vulnerable World Hypothesis," he proposed that some technologies may be inherently destabilizing because they reduce the margin for error in global systems; in this framing, AI is not simply another tool but a political and civilizational risk multiplier The Vulnerable World Hypothesis, Existential Risk Prevention as Global Priority.

He also frames the problem as one of governance, not just technology. Bostrom has argued that international coordination, safety research, and careful deployment decisions matter because the stakes are large and the dynamics are competitive. This is a recurring theme in his public work: advanced AI could produce extraordinary benefits but also enable concentration of power and strategic failures that are hard to reverse once they emerge TED talk: What happens when our computers get smarter than we are?, The New Yorker: The Doomsday Invention.

For readers focused on AI risk, Bostrom's importance is straightforward: he made the idea of a powerful, transformative intelligence a serious public and policy question. His work is part of the intellectual foundation for current conversations about superintelligence, alignment, and catastrophic misuse. Even where people disagree on the precise probabilities or timelines, his arguments are notable for turning a speculative future into an immediate governance problem Nick Bostrom — official website, TED talk: What happens when our computers get smarter than we are?, The New Yorker: The Doomsday Invention.

Sources

#ai-risk#existential-risk#superintelligence