Genetic Programming and Evolvable Machines最新文献

英文中文

Reward tampering and evolutionary computation: a study of concrete AI-safety problems using evolutionary algorithms 奖励篡改和进化计算:使用进化算法研究具体的人工智能安全问题

3区计算机科学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Genetic Programming and Evolvable Machines

Pub Date : 2023-09-19 DOI: 10.1007/s10710-023-09456-0

Mathias K. Nilsen, Tønnes F. Nygaard, Kai Olav Ellefsen

Abstract Reward tampering is a problem that will impact the trustworthiness of the powerful AI systems of the future. Reward Tampering describes the problem where AI agents bypass their intended objective, enabling unintended and potentially harmful behaviours. This paper investigates whether the creative potential of evolutionary algorithms could help ensure trustworthy solutions when facing this problem. The reason why evolutionary algorithms may help combat reward tampering is that they are able to find a diverse collection of different solutions to a problem within a single run, aiding the search for desirable solutions. Four different evolutionary algorithms were deployed in tasks illustrating the problem of reward tampering. The algorithms were designed with varying degrees of human expertise, measuring how human guidance influences the ability to discover trustworthy solutions. The results indicate that the algorithms’ ability to find and preserve trustworthy solutions is very dependent on preserving diversity during the search. Algorithms searching for behavioural diversity showed to be the most effective against reward tampering. Human expertise also showed to improve the certainty and quality of safe solutions, but even with only a minimal degree of human expertise, domain-independent diversity management was found to discover safe solutions.

奖励篡改是一个影响未来强大人工智能系统可信度的问题。奖励篡改(Reward tamering)描述的是人工智能代理绕过其预期目标，导致意外和潜在有害行为的问题。本文研究了进化算法的创造潜力是否有助于在面对这一问题时确保可靠的解决方案。进化算法可能有助于对抗奖励篡改的原因是，它们能够在一次运行中找到一个问题的不同解决方案的不同集合，帮助寻找理想的解决方案。在演示奖励篡改问题的任务中部署了四种不同的进化算法。这些算法是根据不同程度的人类专业知识设计的，衡量人类指导如何影响发现可靠解决方案的能力。结果表明，算法寻找和保留可信解的能力在很大程度上依赖于在搜索过程中保持多样性。搜索行为多样性的算法被证明是对抗奖励篡改最有效的方法。人类专业知识也显示出提高安全解决方案的确定性和质量，但即使只有最低程度的人类专业知识，也发现领域独立的多样性管理可以发现安全解决方案。

{"title":"Reward tampering and evolutionary computation: a study of concrete AI-safety problems using evolutionary algorithms","authors":"Mathias K. Nilsen, Tønnes F. Nygaard, Kai Olav Ellefsen","doi":"10.1007/s10710-023-09456-0","DOIUrl":"https://doi.org/10.1007/s10710-023-09456-0","url":null,"abstract":"Abstract Reward tampering is a problem that will impact the trustworthiness of the powerful AI systems of the future. Reward Tampering describes the problem where AI agents bypass their intended objective, enabling unintended and potentially harmful behaviours. This paper investigates whether the creative potential of evolutionary algorithms could help ensure trustworthy solutions when facing this problem. The reason why evolutionary algorithms may help combat reward tampering is that they are able to find a diverse collection of different solutions to a problem within a single run, aiding the search for desirable solutions. Four different evolutionary algorithms were deployed in tasks illustrating the problem of reward tampering. The algorithms were designed with varying degrees of human expertise, measuring how human guidance influences the ability to discover trustworthy solutions. The results indicate that the algorithms’ ability to find and preserve trustworthy solutions is very dependent on preserving diversity during the search. Algorithms searching for behavioural diversity showed to be the most effective against reward tampering. Human expertise also showed to improve the certainty and quality of safe solutions, but even with only a minimal degree of human expertise, domain-independent diversity management was found to discover safe solutions.","PeriodicalId":50424,"journal":{"name":"Genetic Programming and Evolvable Machines","volume":"99 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135061119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}