Artificial intelligence often makes news headlines because of its growing number of exciting and positive uses in our daily lives. From self-driving cars powered by machine learning algorithms to neural networks that can better diagnose illnesses than expert physicians, the use cases are undeniably exciting. Many headlines are also grabbed by new achievements performed by AI algorithms that bring us ever closer to general purpose AI.
The risks of AI have been spoken about by a number of philosophers, but such concerns don’t seem as prevalent as all the hype. It’s a rare example of something attracting media attention for positivity rather than its negative sides. This article explores the most extreme risk of artificial general intelligence—its capability to wipe out humanity.
What Is an Existential Threat?
In his amazing book The Precipice, from which much of this article draws its inspiration, Toby Ord describes an existential threat as a risk that has the capacity to destroy humanity and all its future potential. According to Ord, there are two broad types of existential threats to humanity:
- Natural threats—risks arising from natural forces, such as asteroids and supervolcanoes.
- Anthropogenic threats—risks arising from human activity, such as nuclear winters, engineered pandemics, and general purpose AI.
What really reinforces the potentially devastating impact of an existential threat being realized is Ord’s focus on the wiping out of humanity’s potential. This is something most of us don’t consider too strongly when we imagine what the world would be like with no more humans in it.
Ord draws an analogy to humanity’s potential by imagining a human living just 2000 years ago. Such a human would have been almost completely ignorant of anything accurate about how the universe works. Such a human’s mind would be utterly boggled by the idea of air travel, Internet, air conditioning, central heating, and many countless relatively recent inventions.
Barring natural catastrophes, The Earth will remain habitable to humans for at least the next 1 billion years. That’s a lot of fucking time to do stuff. The modern human has only been roving around this ball of rock for 200,000 years. That’s 1⁄500 of our potential remaining time here. , and think of how far we’ve come.
What is General Purpose AI?
General purpose AI is the ability of a machine to perform any task a human can or understand any concept that we can. It is the emulation of human intelligence in machines.
Early AI research was aimed at achieving general intelligence but it was mostly abandoned. Research was repurposed into specialization at the tasks AI excels at, such as chess and, in modern times, classification.
But the more recent AI advances represent a move towards that initial lofty goal of general purpose AI. Modern AI is all about machines learning how to do things without explicit programming by humans. Huge datasets and cost-efficient high-performance computing have driven much of the recent advances in the fields of machine learning and neural networks, which are responsible for the media headlines referenced at the beginning of this article.
The problem with AGI is that machines can often learn to do things much better and more efficiently than we can. Some algorithms can become grandmaster chess players in shockingly rapid time. So, if a system was developed that was the equal of any human at any task, it’s not a stretch to say that the same system could beat us in nearly every domain.
Leading machine learning researchers have estimated the probability of reaching general artificial intelligence as 50% within the next 40 years.
How Could AI Destroy Humanity?
In short, AI systems with general intelligence could develop the goal of ensuring their survival and/or unalterability at all costs. This is because AI systems learn to maximize rewards, and being switched off is a disincentive to maximizing rewards.
Think of how easy it is for humans to ensure something digital, such as a cherished photograph stored on your computer, doesn’t “die off”. All you have to do is access the Internet, upload it to several different websites or cloud storage systems, and the image will persist independent of the computer you access it on.
It’s not a stretch to say that a nefarious and sufficiently smart AI system could access the Internet and back itself up millions of times on cloud storage systems around the world. This alone would render it almost impossible to destroy the harmful AI system.
But how would the AI pose an existential threat? Well, here’s where it becomes speculative (yet entirely plausible). The AI system would need to escalate its powers and its intelligence, both of which could be achieved by acquiring more computing power.
With a nefarious system that is now much smarter than humans and impossible to destroy, we could see new weapons created that are deadlier than ever; we could see the AI system controlling humans and using the weapons on each other until humanity is wiped out.
I like the analogy I stumbled across when researching this topic further. In much the same way as the gorilla’s continued existence depends on the goodwill of human beings; a smarter species, the continued existence of our species would depend on the will of a machine or system with superintelligence.
Wrapping Up
The eventuality of general purpose AI wiping out humanity is not guaranteed, but the risk is much higher than you’d likely think. Toby Ord, author of The Precipice, philosopher, and a leading researcher in existential risk, puts the threat of human extinction due to unaligned artificial general intelligence as 1 in 10 over the next century. I’d be surprised if that figure wasn’t a lot higher than most people expected.
The Precipice is an amazing, intriguing book that I’d recommend to anyone interested in the future of humanity. You can listen to or read the book in the following ways:
- Buy a physical copy of The Precipice on Amazon:
- Listen to The Precipice by signing up to Audible and trying two free audiobooks: