Home226

AI’s Moment is Here. Let’s Get it Right.

The last year has been a transformative year for AI. ChatGPT was released November 30th, 2022 and sparked a media firestorm. By February of this year, a little over 2 months after launch, ChatGPT had 100 million active users.

The last year has been a transformative year for AI. ChatGPT was released November 30th, 2022 and sparked a media firestorm. By February of this year, a little over 2 months after launch, ChatGPT had 100 million active users.

Abstract

In this special issue of Humanist Perspectives, Christopher DiCarlo has provided you with an overview of what AI is and the various forms it has taken over time. DiCarlo also notes that AI presents both opportunities and risks to the future of humanity. Following this article, Elliot Mckernon has described the concerted efforts in the fields of AI safety and AI Governance to ensure that humanity experiences the benefits of AI and mitigates the risks it presents. Mckernon also highlights steps you can take to help these efforts. For my contribution to this special issue, I will dive into the progress that has been made in the capabilities of AI, highlight the enormous risks presented by AI, provide evidence that both leading experts and the general public are concerned and expect additional increases in AI capabilities, and that there is significant support for policies by government to mitigate these risks.

INTRODUCTION

The last year has been a transformative year for AI. ChatGPT was released November 30th, 2022 and sparked a media firestorm. By February of this year, a little over 2 months after launch, ChatGPT had 100 million active users. This even appears to have set off something of a race to produce state of the art large language models (LLMs) to rival ChatGPT. GPT4 was released March 14th of this year. Google released Bard on March 21st, which is built on top of its LLM called PaLM. Meta then released their own updated LLM called LLaMA 2 on July 18th.

These survey results were a bit of a shock to me. I had thought that the pace of AI capabilities development had happened so quickly that the risks would not yet be salient to the general public.

These models, the torrid pace of their creation, and their dramatic increase in capabilities alerted many researchers to concerns about the risks these models present. During this time period, prominent AI and machine learning (ML) researchers began to publicly raise concerns about the new AI models. Geoffrey Hinton and Yoshua Bengio joined a chorus of voices that also included Max Tegmark and Eliezer Yudkowsky. These researchers see this fast paced development of increasingly capable and increasingly general AIs and are very concerned about what comes next.At the same time, the progress in LLMs and general AI development also presented another challenge. At what point in their development might AIs become sentient? Sentient AIs would not necessarily be any more dangerous to humans, but they may be constructed in such a way that causes them vast amounts of suffering. As LLMs become more complex, more capable, and are trained on larger and larger portions of human knowledge, their behaviors too have become more complex. This led Blake Lemoine, a google engineer working on LaMDA to speculate that the LLM itself might be self-aware and experience positive and negative affect, suggesting that the LLM might be sentient. While it seems unlikely that current LLMs are sentient, they do seem to have developed a sort of internal world model that allows the modes to speak like selves. However, recent survey data (to be discussed in more detail below) suggests that only ~20% of the US public hold the view that current AIs are sentient.

With these developments in mind, the Sentience Institute (SI) added a supplemental set of questions to the “Artificial Intelligence, Morality, and Sentience” (AIMS) survey with the hope of capturing the US public’s opinions concerning the pace and increasing capabilities of frontier AI models.  This wave of the survey included important and timely questions about the state of Artificial Intelligence (AI) research, the risks of AI, and the building of sentient machines. In this article, I want to highlight 5 results in particular:

  1. The US public is concerned about the pace of AI development
  2. The US public expects that very powerful AI systems will be developed within the next 5 years
  3. The US public is very concerned about the risks AI presents to humanity, including risks of extinction
  4. Specific policy tools designed for increasing AI safety are supported by a majority and sometimes two-thirds of the US public
  5. The US public is generally opposed to human enhancement from AI and robotics, and to the development of sentient machines

These survey results were a bit of a shock to me. I had thought that the pace of AI capabilities development had happened so quickly that the risks would not yet be salient to the general public. Additionally, I thought pro-market tendencies in the US would lead to little support for AI safety regulation. It had seemed that the concerns about AI risks were still relatively fringe topics. Given this, for the rest of this article, I’ll explore each of these results in detail and present additional research that further illuminates these beliefs held by the public. It seems that this transformative year for AI has also transformed how the public views AI risks.

1. The US public is concerned about the pace of AI development.

Key Survey finding: ~49% of respondents believe the pace of AI development is too fast

As noted above, many leading experts are now concerned about the pace of AI development. This led to the publication of two influential open letters concerning the pace of AI development. One was spearheaded by the Future of Life Institute (FLI), another by the Center for AI Safety (CAIS). The FLI open letter titled “Pause Giant AI Experiments: An Open Letter” was published March 22, 2023. The letter has over 33,700 signatures including influential voices such as Elon Musk, Steve Wozniak, Yuvall Noah Harrari, and many many more. Two months later, the CAIS statement of AI risk was announced on May 30th, 2023. It too attracted a large number of prominent signatories with its simple statement that: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

The “pause letter” and the “extinction risk statement” point to concerns held by experts about the pace of AI development. It turns out that experts aren’t the only ones that are concerned. The AIMS survey finds that 49% of respondents believe that AI development is proceeding too fast. While the AIMS survey is one of the very few that has recently examined this question, earlier in August the Artificial Intelligence Policy Institute (AIPI) found that “82% prefer slowing down the development of AI compared to just 8% who prefer speeding development up” and that “62% are concerned about artificial intelligence, while just 21% are excited.”

Why might it be the case that both leading experts and the general public believe that AI development is too fast?

First, there are already clear societal harms from “earlier” generations of AI. For example, social media recommender algorithms have played a significant role in the vast spreading of misinformation. Other forms of AI have been known to exhibit biases against specific groups of people and have also been used to oppress and repress populations of people lacking the democratic protections against these abuses.

Second, recent foundation models, such as LLMs, and other ML models, demonstrate powerful capabilities that are quickly spreading into both the marketplace and powerful institutions. The US General Services Agency is even holding a competition to find the best ways to integrate LLMs into government agencies. This was made particularly evident by the viral spread of ChatGPT. 

Third, while AI capabilities have significantly increased along many dimensions, they still behave in unexpected and harmful ways, but this has not stopped race dynamics from kicking in in the marketplace and encouraging AI labs to rush to release their models to help gain first mover advantages and important reputation effects.

So, both leading experts and the US public are concerned about the pace of AI development. This may be related to each group’s (experts and non-experts) perceptions of the risks of this speedy development, which includes the development of even more powerful AI systems in the near future and the catastrophic and extinction risks these models may present.

2. The US public expects that very powerful AI systems will be developed within the next 5 years

Key Survey Findings: The modal predictions for when Artificial General Intelligence (AGI), Human-Level Artificial Intelligence (HLAI), and Artificial Superintelligence (ASI) will be created is, in order, 2 years, 5 years, and 5 years.

One of the biggest challenges for AI safety is trying to understand when different capabilities of AI might arise. For example, mastery of both computer programming and human language were capabilities of AI that many observers expected to be far in the future. However, current LLMs have essentially reached human capacity or near human capacity for these capabilities.  Those that follow the study of AI know that this is something of an enduring challenge for researchers. Even from the earliest days of the field, researchers have attempted to predict the arrival of different capabilities of AI. The rapid proliferation of LLMs and other “foundation” or “base” models has brought significant attention to the question of when various AI capabilities might arise.

While there is significant disagreement across researchers about the relevant thresholds of capabilities, three terms in particular have captured the attention of experts and forecasters and were included in the AIMS survey. These thresholds for AI development include: (1) Artificial General Intelligence (AGI), (2) Human-Level Artificial Intelligence (HLAI), and (3) Artificial Superintelligence (ASI). Since there is no specific definition of these terms that is agreed upon by researchers, the respondents inferred their own definitions. One natural ordering that seems to be identified by the respondents suggests that people expect AI to first become very general in its capabilities leading to AGI. Then as the capabilities of AGI improve, HLAI would be expected. So, we start with increasingly general AI and this gives way to AI that is approximately of human rank. Finally, as AI improves from human level, it might be expected to approach a superhuman-level in the form of ASI.

When asked “If you had to guess, how many years from now do you think that the first artificial general intelligence will be created?”, 34% of respondents believe artificial general intelligence (AGI) already exists and 2% believe it will never happen. The modal guess for when AGI will be created is 2 years from now. When asked “If you had to guess, how many years from now do you think that the first human-level AI will be created?” 23% of respondents believe that human level artificial intelligence (HLAI) already exists and 4% believe it will never happen. The modal guess for when HLAI will be created is 5 years from now. Finally, when asked “If you had to guess, how many years from now do you think that the first artificial superintelligence will be created?” 24% of respondents believe artificial superintelligence (ASI) already exists and 3% believe it will never happen. The modal guess for when ASI will be created is also 5 years.

Now, as noted above, formal definitions were not supplied and there is significant disagreement across researchers on what sorts of AI capabilities would signal the emergence of AGI, HLAI, or ASI. Nevertheless, these timelines are shorter than both current prediction markets and recent expert surveys.

For example, the most prominent prediction market Metaculus has a  “forecast for Artificial General Intelligence (that) is Mar 5, 2032, with a 91% chance of human intelligence parity by 2040.” This translates to roughly an expected arrival date of AGI in 9.5 years and HLAI in 16 years. The Metaculus market also predicts that it will be approximately 3 years after a (weak) AGI is created until the first superintelligent AI will be created. To map this directly to the survey results, Metaculus predicts something like 9.5 years until AGI arrives, 16 years until HLAI arrives, and ASI to arrive within a handful of years of weak AGI. As you may notice there is some ambiguity about the relationship among AGI, HLAI, and ASI in this context, but nonetheless the earliest of these predictions is AGI in 9.5 years, which is ~4xs longer than the survey respondents expect and the HLAI prediction is ~3xs longer than the median response. To complicate things even further, there is another survey question on Metaculus that is phrased this way: “When will the first weakly general AI system be devised, tested, and publicly announced?” The median answer to this question is January 2027, a little more than 3 years from now and much closer to the median answer of 2 years from the AIMS survey.

In addition to the prediction market of Metaculus, which has the nice property of being an estimate that is continually updated as more predictions are made, from 2018-2022 there were at least three expert surveys that also explore the timelines of when we might expect HLAI. These include a survey from 2018 of 165 AI experts who were asked when AI systems will collectively be able to accomplish 99% of tasks that humans are paid to do at or above the level of a typical human. In this survey half the experts gave a date before 2068. In another survey from 2019 of 296 AI experts were asked when machines will collectively be able to perform more than 90% of all tasks that are economically relevant better than the median human paid to do that task. In this survey half of the experts gave a date before 2060. And, finally, in 2022, 356 AI experts were asked when unaided machines will be able to accomplish every task better and more cheaply than human workers. Here, half of the experts have a date before 2061.

3. The US public is very concerned about the risks AI presents to humanity, including risks of extinction.

Key Survey Findings: 44% of those surveyed at least somewhat agree with the statement “AI is likely to cause human extinction.” 52% of those polled are at least somewhat concerned about “the possibility that AI will cause the end of the human race on Earth.”

As AI continues to become more capable and more general at a pace that is concerning to both the US public and AI experts, these trends also point to significant risks AI brings to humanity. These risks include both further exacerbating the current harms that current AI tools already impose upon society and new risks and potential harms that may arise as a result of these trends.

While AI tools, and LLMs in particular, do present opportunities for transforming society for the better, they also present catastrophic and existential risks to humanity as well.

AI and ML safety researchers have recently classified the catastrophic risks of increasingly powerful, capable, and general AI across four categories including: (1) risks of malicious use, (2) AI race, (3) organizational risks, and (4) risks of rogue AI.

Current frontier AI models, including both LLMs and RL agents, already not only present these risks but already instantiate harms across the first three of these categories. Let’s take LLMs as our example. Shortly after the release of ChatGPT, people deliberately created LLM-based agents such as ChaosGPT with the explicit goal of trying to take over the world. One can easily imagine further deliberate malicious use by governments, terrorists, and profit seeking corporations in attempts to use the capabilities of LLMs to solidify power, inspire fear, and to increase profits.

The AI race presents potential catastrophic risks that can be inferred directly from the current AI development ecosystem. It is becoming more acknowledged that there is already a race underway by leading AI labs and giant tech companies to create and bring to market as quickly as possible more and more powerful LLMs. This may directly lead to ignoring concerns of safety and instead rushing new LLMs to be available to the public and let the public bear the brunt of learning the harms that the more powerful LLMs may perpetuate.

In addition to malicious uses and risks from the AI race, LLMs also present organizational risks. As Dan Hendrycks, Mantas Mazeika, and Thomas Woodside highlight in their essay “An Overview of Catastrophic Risks” (1) catastrophes are hard to avoid even when competitive pressures are low (2) AI accidents could be catastrophic, (3) it often takes years to discover severe flaws or risks, and (4) safety washing can undermine genuine efforts to improve AI safety.

Another related risk is that of rogue AI; misaligned AI models that assume significant agency. That is AI models that pursue their own goals and act autonomously within their own environment towards achieving those goals.  Here, an AI may lack inner and/or outer alignment. Inner alignment is often thought of as ensuring the AI does what we ask it to do and outer alignment is often thought of as having the AI do things that would be good for humanity. In these ways, LLMs suffer from different forms of misalignment both with doing what its designers want and doing things that are good for humanity. While current LLMs already instantiate these risks as experienced harms, if AI continues its fast pace of development and becomes even more capable, general, and powerful, then it seems natural to be concerned about the extent of these harms. Rogue AI risks arise when an AI may deliberately alter itself in such a way as to take control over its own goals and pursue those goals even if the goals that are being pursued would be very harmful to humanity.

While each of these risks, with the exception of rogue AI, are already experienced as instantiated harms, current models seem unlikely to present risks that would lead directly to the extinction of the human race. However, many AI and ML experts and also large portions of the US public believe that these risks will be exacerbated to such a degree as AI becomes more capable that they may indeed be capable of the catastrophic and existential risks to humanity.

4. Specific policy tools designed for increasing AI safety are supported by a majority and sometimes two-thirds of the US public.

Key Aims Findings: 64% of respondents at least somewhat support public campaigns to slow down AI development. 64% of respondents at least somewhat support regulation that slows down AI development. 57% of respondents at least somewhat support banning the development of artificial general intelligence that is smarter than humans.  56% of respondents at least somewhat support a global ban on data centers that are large enough to train AI systems that are smarter than humans. 69% at least somewhat support a 6-month pause on AI development.

As we have seen, AI experts and the general public alike are concerned about the pace of AI development. AI experts, prediction markets, and the general public expect even more powerful, capable, and intelligent AI systems to be developed in the near future. And, AI experts and the general public are concerned that these AIs, if developed, will present catastrophic and existential risks to humanity.

Given this, what, if anything, should be done about it?

Some argue that we should simply ignore these experts and the general public and instead accelerate AI development, full steam ahead. However, this option seems more and more reckless as current harms of AI have become more clear and the risks of malicious use, an AI race to the bottom, organizational risks, and rogue AI become more obvious. To use the language of economics and policy analysis, the current and potential negative externalities of AI services, tools, and systems are becoming more obvious. And, when negative externalities exist, and they present externalities that are potentially catastrophic, it would seem obvious that we need better governance tools to influence and steer the development and proliferation of these AIs. In fact, leading AI experts are arguing that we should do just that. But the question remains, what exactly should be done?

It would seem that a toolkit of strategies may be required. This needed toolkit has given rise to the study of AI Governance where scholars and experts have been arguing for a variety of policies and governance tools to help shape the development of AI such that it is beneficial to humanity rather than destructive. Even before the “pause letter” and the release of ChatGPT, policy experts were calling for actions such as bans of harmful applications of AI, regulatory frameworks that would provide for more clear recourse for those harmed by AI, ratings systems, licensing regimes, data privacy protections, and strict liability of AI companies for the harms their products cause to name a few. With the rise of LLMs in particular, these calls have intensified.

As noted earlier, one of the early prominent calls for action given the release of powerful LLMs has been FLI’s call for a pause of training models significantly more capable than the current frontier models already available. This was a call for a 6-month voluntary pause on these large scale experiments. FLI’s President, Max Tegmark, directly implored leading AI labs to pause research in the direction of even larger LLMs until we better understood the behavior of the current frontier models. This call, while galvanizing concerns about AI risk, was unsuccessful in convincing AI companies to pause their experiments, despite some initial warmness to the idea of OpenAI’s Sam Altman.

In the days following the publication of the “pause letter” another prominent AI expert, Eliezer Yudkowsky, took to Time Magazine to argue that the pause “letter is understating the seriousness of the situation and asking for too little to solve it.” Yudkowsky sees the problem as quite dire, stating in the same article that “If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.” Yudkowsky’s toolkit of governance strategies for what exactly should be done is worth directly quoting here as well:

“Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make immediate multinational agreements to prevent the prohibited activities from moving elsewhere. Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.”

While a voluntary pause of experiments was both unlikely and unlikely to be the only needed governance strategy or policy tool, Yudkowsky argues for a suite of tools that treat AI development in a similar vein as the development of nuclear weapons. Interestingly, this is arguably a natural extension of the CAIS letter mentioned earlier which called for AI to be treated similarly to the risks of nuclear war.

Despite the pace of AI development, the belief by experts, prediction markets and the general public that AGI will be developed within the next decade, and that AGI would present catastrophic and existential risks to society, to many observers Yudkowsky’s toolkit has been seens as too interventionist, too punitive, and simply too restrictive of AI development. However, it seems that the US public, generally speaking, is supportive of these efforts to protect humanity from the development of uncontrollable AGI.

The AIMS survey finds that more than two-thirds (69%) of the US public supports calls for a voluntary pause of AI development. Almost two-thirds (64%) are supportive of public campaigns such as the ones being waged by Max Tegmark, Eliezer Yudkowsky, and others to slow down the development of AI. These levels of high support may be unsurprising in that these two survey questions do not require new government regulation, which the US public in particular is often leery of.

However, the broad support does not end with the voluntary actions. Almost two-thirds of the US public (again, 64%) at least somewhat support regulation that slows down AI development. This shows that the US public is in favor of new regulation that seeks to slow down the pace at which AI is being developed in the current ecosystem.

It seems then that both voluntary actions and regulatory actions to slow down the development of AI receive almost two-thirds of support from the US public. The support remains high even when we shift to considering regulatory tools that would include outright bans. More than half of respondents (57%) at least somewhat support banning the development of AGI that is smarter than humans. And more than half of respondents (56%) at least somewhat support a global ban on data centers that are large enough to train AI systems that are smarter than humans. In other words, multiple of the tools in Yudkowsky’s proposed toolkits, tools that in other areas of policy might be seen as quite restrictive, receive majority and near super-majority support from the US public.

5. The US public is generally opposed to human enhancement from AI and robotics and to the development of sentient machines

Key Survey Findings: 59% at least somewhat support a global ban on the development of sentience in robots/AIs. 62% at least somewhat support a global ban on the development of AI-enhanced humans. And 64% at least somewhat support a global ban on the development of robot-human hybrids.

Up to this point, we’ve been discussing concerns about the pace of development of more capable AIs with a particular focus on frontier AI models in the form of LLMs. These AIs have become more capable and more intelligent through the improvement in reinforcement learning and deep neural nets. These AIs appear to scale with more data and more compute on which they are trained. Additionally, we’ve briefly discussed the earliest forms of AI agents that are being built upon these LLMs, which are also known as base models or foundation models. The general idea is that these models will continue to serve as the foundation upon which more complex AI agents may be built.

As more complex, capable, and intelligent agents are built on top of these models, some experts expect that two additional challenges with AIs will arise: (1) they may become sentient, and (2) they may become more tightly coupled with humans leading to AI-enhanced humans or even the development of cyborgs that are AI-powered robot-human hybrids. These concerns are somewhat related to those of the development of AGI and extinction risks, as these risks also arise as AI becomes more powerful, capable, and general. Let us take the concerns of hybridity and sentience one at a time.

Both AI-enhanced humans and robot-human hybrids arise along a continuum. On one hand, we have completely unaltered, unenhanced humans that lack any integration with AI or robotics and on the other hand we have unembodied AIs that are intelligent tools that can be deployed by humans for a variety of purposes. In this way, hybrids would then arise as some combination of these two idealized types. The truth, however, is much fuzzier than this. For example, do people using AI-programmed hearing aids count as AI-enhanced humans? How about AI-powered heart pacemakers? And what about prosthetic limbs that use AI to improve their tactile feedback? Does our growing use of AI tools that “live” on our smartphones not result in important enhancements for the humans that already make use of them? The same sorts of questions can also be posed in the direction of what constitutes robot-human hybrids. As one example: Would a soldier that wears an exoskeleton that intelligently responds and interacts with the soldier qualify as a robot-human hybrid?

Despite the lack of clarity on what exactly constitutes “AI-enhanced humans” or “robot-human hybrids,” one can also imagine what more typically comes to the mind of the public as they consider the development of these entities. I would venture to guess that “AI-enhanced humans” conjures images of AI being used as a more general enhancement to the human condition, a tighter integration of AI that allows for improvements in general cognitive and physical behaviors in such a way that performance in these domains is dramatically improved compared to the median, non-enhanced human. The same might be said for “robot-human hybrids.” These hybrids likely resemble a drastically altered human that has more fully integrated robotic features within the human condition. In the public’s mind, these might resemble something like a cyborg where many important human biological processes have been replaced by robotic or mechanical components that integrate rather seamlessly into the human’s manipulable self-model.    

While it is unclear exactly what the public has in mind when they consider these terms, they appear to be hesitant to develop AI and robotics further in this direction. The AIMS survey finds that a strong majority (62%) at least somewhat support a global ban on the development of AI-enhanced humans.  And, a slightly stronger majority (64%) at least somewhat support a global ban on the development of robot-human hybrids. It is worth noting that these are strong policy proposals that are endorsed by the public. Banning the development of technology in this direction is a quite punitive and blunt tool for addressing the concerns that AI-enhanced humans and robot-human hybrids may present to human society.

The development of sentient AIs/robots has its own set of conceptual difficulties as well. In fact, this is one conceptual challenge that the Sentience Institute has been working directly on with Janet Pauketat’s report “The Terminology of Artificial Sentience.” Here, Pauketat provides a definition of Artificial Sentience that means “artificial entities with the capacity for positive and negative experiences, such as happiness and suffering” or similarly “the capacity for positive and negative experiences manifested in artificial entities.” This is provided in contrast to the term of artificial consciousness, for which Pauketat cites definitions from Graziano (2017) and Reggia (2013). Graziano’s definition is  “a machine that contains a rich internal model of what consciousness is, attributes that property of consciousness to itself and to the people it interacts with, and uses that attribution to make predictions about human behavior. Such a machine would ‘believe’ it is conscious and act like it is conscious, in the same sense that the human machine believes and acts.” While, Reggia goes with “computational models of various aspects of the conscious mind, either with software on computers or in physical robotic devices.” Additionally, in recent work on machine evolution, along with colleagues Christopher DiCarlo and Elliot McKernon, I explore the question as AGI approaches of whether machine consciousness may arise alongside these capabilities and whether or not this consciousness will resemble human consciousness. This remains a challenging and as of yet, unanswered question. However, despite these conceptual difficulties, the US public is opposed to making machines that would be sentient. The AIMS survey finds that a majority of those surveyed (59%) at least somewhat support a global ban on the development of sentience in robots/AIs.

Taken together, the US public is skeptical of allowing humans to merge even further with their machines. There is an almost two-thirds support for prohibiting AI-enhanced humans and human-robot hybrids, and a strong majority opposed to the creation of sentient machines. However, there are many conceptual challenges to defining what is actually meant by these categories of beings. Given the concerns around the pace of AI development and the possible extinction risks this presents to humanity, it may also be the case that the general public has developed something of a precautionary principle to the further development not only of increasingly powerful AI’s but also of giving the AIs more human-like or life-like capabilities and of the creation of new types of being powered in large part by these AI’s.

Conclusions

As DiCarlo highlights in this issue, the field of AI has seen immense gains in AI capabilities since its founding in the mid-20th century. Furthermore, the last 12 months have seen leaps in capabilities towards strong AI which has led many observers, including many in the general public, to believe that we are at the cusp of creating Artificial General Intelligence (AGI). As Mckernon notes in this issue, the fields of AI safety and AI Governance are attempting to steer these newfound capabilities in a direction that allows us to realize the benefits of AI while mitigating the very serious risks that it presents to the future of humanity. For my part, I’ve highlighted that the general public, alongside the experts in the fields of study, are concerned about these serious risks, see even more risks on the horizon, and expect their governments to regulate the development and use of AI accordingly.

I think, taken together, there are some lessons here for AI safety researchers, policymakers, and AI labs.

For AI safety researchers, the public now shares many of your concerns and even baseline assumptions of timelines. This also is suggestive evidence that the multitudinous efforts of AI safety researchers to make the public more aware of the risks of AI have been successful.

For policymakers, you can see the relevance of this topic now to the public. They are concerned about the risks of fast paced AI development and want more, significantly more, guardrails in place. Across the board a majority of the US public supports a fairly wide array of policy options by governments for more regulation and for regulation that is significantly disruptive to the status quo.

For AI labs, this should be a warning. Let your suspicions be confirmed. The public is leery of the minds you’ve brought forth into the world. We all expect better of you moving forward. Starting yesterday.

The capabilities and risks of AI present incredible promise and terrifying peril to humanity. However, while there are various evolutionary selection pressures, the direction of this technology is not predetermined. Humanity still plays the dominant role in steering AI evolution. We can, collectively, prioritize safety and democratic governance such that the development of the next generation of AIs is safe, beneficial, and does not increase the suffering of sentient beings.