An AGI Modifying Its Utility Function in Violation of the Orthogonality Thesis

James D. Miller, Roman Yampolskiy and Olle H\"aggstr\"om

Papers from arXiv.org

Abstract: An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the orthogonality thesis since the modifications would be influenced by the AGI's intelligence. AGIs in highly competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory.

Date: 2020-01
New Economics Papers: this item is included in nep-upt
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://arxiv.org/pdf/2003.00812 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2003.00812

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().