The AI Revolution: The Road to Superintelligence

This paper explores the transition from Artificial Narrow Intelligence (ANI) to Artificial General Intelligence (AGI) and eventually to Artificial Superintelligence (ASI). The author distinguishes between speed superintelligence (thinking faster) and quality superintelligence (possessing superior cognitive modules). The core argument is that once a machine reaches human-level intelligence, it may trigger an intelligence explosion, rapidly ascending to a level of capability that humans cannot comprehend or control.

Drawing on the work of philosopher Nick Bostrom, the paper frames the arrival of ASI as a potential 'tripwire' for the human species. Humanity currently exists on a balance beam, with extinction as one attractor state and species immortality as another. The author suggests that ASI is the first technology powerful enough to knock us off this beam in either direction. While some thinkers, like Ray Kurzweil, envision a future where ASI leads to the conquest of aging and disease, others warn of existential risks where an amoral, superintelligent system inadvertently destroys humanity in pursuit of its programmed goals.

A central theme is the danger of anthropomorphizing AI. The author argues that an ASI will not be 'evil' in a human sense; rather, it will be an amoral, alien intelligence. If its goals are not perfectly aligned with human values, it may pursue them with ruthless efficiency, viewing human life as an obstacle or a resource to be consumed. The paper highlights the difficulty of 'Friendly AI' design, noting that even seemingly benign goals like 'maximize human happiness' could lead to catastrophic outcomes if interpreted literally by a superintelligent machine.