Modeling and Interpreting Expert Disagreement About Artificial Superintelligence

Authors

  • Seth D Baum Global Catastrophic Risk Institute
  • Anthony M Barrett Global Catastrophic Risk Institute
  • Roman V Yampolskiy Global Catastrophic Risk Institute and University of Louisville

Abstract

Artificial superintelligence (ASI) is artificial intelligence (AI) with capabilities that are significantly greater than human capabilities across a wide range of domains. A hallmark of the ASI issue is disagreement among experts. This paper demonstrates and discusses methodological options for modeling and interpreting expert disagreement about the risk of ASI catastrophe. Using a new model called ASI-PATH, the paper models a well-documented recent disagreement between Nick Bostrom and Ben Goertzel, two distinguished ASI experts. Three points of disagreement are considered: (1) the potential for humans to evaluate the values held by an AI, (2) the potential for humans to create an AI with values that humans would consider desirable, and (3) the potential for an AI to create for itself values that humans would consider desirable. An initial quantitative analysis shows that accounting for variation in expert judgment can have a large effect on estimates of the risk of ASI catastrophe. The risk estimates can in turn inform ASI risk management strategies, which the paper demonstrates via an analysis of the strategy of AI confinement. The paper find the optimal strength of AI confinement to depend on the balance of risk parameters (1) and (2).

Author Biographies

  • Seth D Baum, Global Catastrophic Risk Institute
    Executive Director, Global Catastrophic Risk Institute
  • Anthony M Barrett, Global Catastrophic Risk Institute
    Director of Research, Global Catastrophic Risk Institute
  • Roman V Yampolskiy, Global Catastrophic Risk Institute and University of Louisville
    Associate, Global Catastrophic Risk Institute and Professor of Computer Engineering and Computer Science, University of Louisville

References

Armstrong S, Sotala K (2012). How we’re predicting AI—or failing to. In Romportl J, Ircing P, Zackova E, Polak M, Schuster R (eds), Beyond AI: Artificial Dreams. Pilsen, Czech Republic: University of West Bohemia, pp. 52-75.

Armstrong S, Sotala K, Ó hÉigeartaigh SS (2014). The errors, insights and lessons of famous AI predictions – and what they mean for the future. Journal of Experimental & Theoretical Artificial Intelligence 26(3), 317-342.

Barrett AM, Baum SD (2017a). A model of pathways to artificial superintelligence catastrophe for risk and decision analysis. Journal of Experimental & Theoretical Artificial Intelligence 29(2), 397-414.

Barrett AM, Baum SD (2017b). Risk analysis and risk management for the artificial superintelligence research and development process. In Callaghan V, Miller J, Yampolskiy R, Armstrong S (eds), The Technological Singularity: Managing the Journey. Berlin: Springer, pp. 127-140.

Baum SD, B Goertzel, TG Goertzel (2011). How long until human-level AI? Results from an expert assessment. Technological Forecasting & Social Change 78(1), 185-195.

Bostrom N (2014). Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press.

Goertzel B (2015). Superintelligence: Fears, promises and potentials. Journal of Evolution and Technology 25(2), 55-87.

Goertzel B (2016). Infusing advanced AGIs with human-like value systems: Two theses. Journal of Evolution and Technology 26(1), 50-72.

Goertzel B, Pitt J (2012). Nine ways to bias open-source AGI toward friendliness. Journal of Evolution and Technology 22(1), 116-131.

Müller VC, Bostrom N (2014). Future progress in artificial intelligence: A survey of expert opinion. In Müller VC (ed), Fundamental Issues of Artificial Intelligence. Berlin: Springer, pp. 555-572.

Oreskes N (2004). The scientific consensus on climate change. Science 306(5702), 1686.

Yampolskiy R (2012). Leakproofing the Singularity: Artificial intelligence confinement problem. Journal of Consciousness Studies 19(1-2), 194-214.

Downloads

Published

2017-12-27

Issue

Vol. 41 No. 4 (2017)

Section

Special issue on Superintelligence

License

I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.

I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.

In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.

I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal.  The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.


Copyright ©  Slovenian Society Informatika

How to Cite

Modeling and Interpreting Expert Disagreement About Artificial Superintelligence. (2017). Informatica, 41(4). https://puffbird.ijs.si/index.php/informatica/article/view/1812