AGI Open Letter
From AGIRI.org
(DRAFT)
To whom it may concern,
Throughout the past half-century, the field of Artificial Intelligence (AI) has been progressing steadily. Today, AI systems perform or assist with a variety of tasks, such as delivering more effective Web search, scheduling and planning military and industrial operations, diagnosing diseases, and analyzing voluminous quantities of data for phenomena as varied as credit card fraud to scientific anomalies.
As progress continues, the problem-solving abilities of AI systems will approach and then surpass the brightest human minds. These systems will acquire the ability to improve their own programming without human assistance, giving rise to more capable versions more quickly than a human team could have done directly. Such a system would increase in its capabilities, over time, from those of an individual human, to those of a small company, to a large corporation, ultimately to rival nations and the human race as a whole.
The creation of such entities should be approached with considerable forethought. To distinguish between narrow-purpose AI and AI designed for general intelligence and self-improvement, the term "Artificial General Intelligence" (AGI) has been coined.
AGI is not a precise term, but there is a qualitative difference between the task-specific narrow-AI systems the AI field has produced to date, and the AGI systems it will produce in the (perhaps not too distant) future. These will possess a level of mental generality and flexibility at least equaling that of the human mind, and a capacity for volume and detail of work exceeding it tremendously. AGI is inclusive of the notion of "Human-Level AI" that has gained currency recently, but is more general in that it explicitly encompasses the possibility of powerful AGI systems that are architected fundamentally differently than human mind/brains, with different strengths and weaknesses, and eventually, potentially far greater intelligence.
The prospect and implications of such powerful machine intelligence have been called "Singularity" by futurists, by analogy to singularities in mathematical physics, particularly black holes. This analogy invokes the concept of the "event horizon" which shields the singularity from outside observation. This point of view is informed by the notion that it is difficult for humans to predict what will be done by intellects which exceed our own.
Other futurists have referred to the same prospect as the "Intellectual Revolution," by analogy to the Industrial Revolution. Although this appears at first sight a more pedestrian prediction, giving as it does a model in terms of which some of the outcomes can be imagined, a sober analysis reminds us that the original Industrial Revolution did indeed constitute a major watershed in human affairs. The character of the human condition following such a revolution may well depend critically on the nature of the machines that enable it.
The ultimate nature of these machines is highly uncertain today, as is the nature of the social and economic context in which they will operate as the future unfolds. Potentially, AGIs will be created in many nations for many purposes, including military and corporate ones. AGI creators hoping for a positive influence on the future must consider a wide variety of scientific, engineering, social, cultural and physical factors.
It is our position that avoidance of AGI is unrealistic, and indeed undesirable. Humanity has much to gain from better problem solving, deeper insights, new points of view, and tireless helpers. As AI researchers and futurists, it is our responsibility to do as much as we can to ensure such a positive outcome.
We do not expect to confront these questions in the distant future. While the result of any research is difficult to predict with high confidence, we consider it very plausible that advanced AGI may be arriving in the next decades, not centuries. Some of the creators of the first AGI may well be among the researchers and students of today. The policies and precedents we set in the present will have influence over what will happen in the future. The potential benefits of AGI vastly exceed those of any other technology. We must do our best to ensure they are realized and made accessible to all.
We believe the time has come for AI researchers and engineers to work together with society at large to craft a roadmap for the development of this technology to bring about a desirable and beneficial future.
Signatories TBA
- Signatures From Leading AI Researchers
- Signatures From Additional Leading Scientists
- Signatures From Additional Supporters
(Origins of the letter found here and contributers include: Nate Allen, Michael Anissimov, Tyler Emerson, Jay Fox, Ben Goertzel, J. Storrs Hall, Bruce Klein, Richard Leis, Steve Omohundro, Carl Shulman, Pei Wang and Brian Wowk)
