Ritesh Noothigattu, Djallel Bouneffouf, Nicholas Mattei, Rachita Chandra, Piyush Madan, Kush R. Varshney, Murray Campbell, Moninder Singh, F. Rossi
Autonomous cyber-physical agents play an increasingly large role in our lives. To ensure that they behave in ways aligned with the values of society, we must develop techniques that allow these agents to not only maximize their reward in an environment, but also to learn and follow the implicit constraints of society. We detail a novel approach that uses inverse reinforcement learning to learn a set of unspecified constraints from demonstrations and reinforcement learning to learn to maximize environmental rewards. A contextual bandit-based orchestrator then picks between the two policies: constraint-based and environment reward-based. The contextual bandit orchestrator allows the agent to mix policies in novel ways, taking the best actions from either a reward-maximizing or constrained policy. In addition, the orchestrator is transparent on which policy is being employed at each time step. We test our algorithms using Pac-Man and show that the agent is able to learn to act optimally, act within the demonstrated constraints, and mix these two functions in complex ways.
{"title":"Teaching AI Agents Ethical Values Using Reinforcement Learning and Policy Orchestration","authors":"Ritesh Noothigattu, Djallel Bouneffouf, Nicholas Mattei, Rachita Chandra, Piyush Madan, Kush R. Varshney, Murray Campbell, Moninder Singh, F. Rossi","doi":"10.24963/ijcai.2019/891","DOIUrl":"https://doi.org/10.24963/ijcai.2019/891","url":null,"abstract":"Autonomous cyber-physical agents play an increasingly large role in our lives. To ensure that they behave in ways aligned with the values of society, we must develop techniques that allow these agents to not only maximize their reward in an environment, but also to learn and follow the implicit constraints of society. \u0000We detail a novel approach that uses inverse reinforcement learning to learn a set of unspecified constraints from demonstrations and reinforcement learning to learn to maximize environmental rewards. A contextual bandit-based orchestrator then picks between the two policies: constraint-based and environment reward-based. The contextual bandit orchestrator allows the agent to mix policies in novel ways, taking the best actions from either a reward-maximizing or constrained policy. In addition, the orchestrator is transparent on which policy is being employed at each time step. We test our algorithms using Pac-Man and show that the agent is able to learn to act optimally, act within the demonstrated constraints, and mix these two functions in complex ways.","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"16 1","pages":"2:1-2:9"},"PeriodicalIF":0.0,"publicationDate":"2019-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90635414","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A device for use in association with a carburetor of the type used for supplying fuel at the intake manifold of a gasoline internal combustion engine has an annular chamber at the exit port of the carburetor, adapted to collect uncarbureted raw gasoline particles or globules for drainage through an external vaporizing system under the influence of the vacuum created at the intake manifold during operation of the engine for reintroduction as vaporized fuel at the intake manifold, thereby converting raw gasoline, otherwise wastefully fed to the engine, into efficiently combustible fuel.
{"title":"Message from the Vice President, Systems, IBM Research Division","authors":"T. Agerwala","doi":"10.1147/rd.521.a","DOIUrl":"https://doi.org/10.1147/rd.521.a","url":null,"abstract":"A device for use in association with a carburetor of the type used for supplying fuel at the intake manifold of a gasoline internal combustion engine has an annular chamber at the exit port of the carburetor, adapted to collect uncarbureted raw gasoline particles or globules for drainage through an external vaporizing system under the influence of the vacuum created at the intake manifold during operation of the engine for reintroduction as vaporized fuel at the intake manifold, thereby converting raw gasoline, otherwise wastefully fed to the engine, into efficiently combustible fuel.","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"160 1","pages":"2-2"},"PeriodicalIF":0.0,"publicationDate":"2008-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75131347","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Benner, Mike Ignatowski, J. Kash, D. Kuchta, M. Ritter
Optical fiber links have become ubiquitous for links at the metropolitan and wide area distance scales, and have become common alternatives to electrical links in local area networks and cluster networks. As optical technology improves and link frequencies continue to increase, optical links will be increasingly considered for shorter, higher-bandwidth links such as I/O, memory, and system bus links. For these links closer to processors, issues such as packaging, power dissipation, and components cost assume increasing importance along with link bandwidth and link distance. Also, as optical links move steadily closer to the processors, we may see significant differences in how servers, particularly high-end servers, are designed and packaged to exploit the unique characteristics of optical interconnects. This paper reviews the various levels of a server interconnect hierarchy and the current status of optical interconnect technology for these different levels. The potential impacts of optical interconnect technology on future server designs are also reviewed.
{"title":"Exploitation of optical interconnects in future server architectures","authors":"A. Benner, Mike Ignatowski, J. Kash, D. Kuchta, M. Ritter","doi":"10.1147/rd.494.0755/","DOIUrl":"https://doi.org/10.1147/rd.494.0755/","url":null,"abstract":"Optical fiber links have become ubiquitous for links at the metropolitan and wide area distance scales, and have become common alternatives to electrical links in local area networks and cluster networks. As optical technology improves and link frequencies continue to increase, optical links will be increasingly considered for shorter, higher-bandwidth links such as I/O, memory, and system bus links. For these links closer to processors, issues such as packaging, power dissipation, and components cost assume increasing importance along with link bandwidth and link distance. Also, as optical links move steadily closer to the processors, we may see significant differences in how servers, particularly high-end servers, are designed and packaged to exploit the unique characteristics of optical interconnects. This paper reviews the various levels of a server interconnect hierarchy and the current status of optical interconnect technology for these different levels. The potential impacts of optical interconnect technology on future server designs are also reviewed.","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"91 1","pages":"755-776"},"PeriodicalIF":0.0,"publicationDate":"2005-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77345357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Vice President, BladeCenter Development, IBM Systems and Technology Group","authors":"Douglas A. Balog","doi":"10.1147/rd.496.a","DOIUrl":"https://doi.org/10.1147/rd.496.a","url":null,"abstract":"","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"48 5 1","pages":"806-"},"PeriodicalIF":0.0,"publicationDate":"2005-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76174135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Vice President, Systems Hardware Development, IBM Systems and Technology Group","authors":"M. Desens","doi":"10.1147/rd.483.a","DOIUrl":"https://doi.org/10.1147/rd.483.a","url":null,"abstract":"","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"59 1","pages":"292-"},"PeriodicalIF":0.0,"publicationDate":"2004-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87046138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Senior Vice President, IBM Research Division","authors":"P. Horn","doi":"10.1147/rd.485.a","DOIUrl":"https://doi.org/10.1147/rd.485.a","url":null,"abstract":"","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"44 1","pages":"596-"},"PeriodicalIF":0.0,"publicationDate":"2004-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"90224825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Vice President for Technology and Chief Technologist, IBM Technology Group","authors":"B. Meyerson","doi":"10.1147/rd.472.a","DOIUrl":"https://doi.org/10.1147/rd.472.a","url":null,"abstract":"","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"37 1","pages":"98-"},"PeriodicalIF":0.0,"publicationDate":"2003-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75094103","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Message from the Director, Austin Research Laboratory, IBM Research Division","authors":"M. Rosenfield","doi":"10.1147/rd.475.a","DOIUrl":"https://doi.org/10.1147/rd.475.a","url":null,"abstract":"","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"1 1","pages":"520-"},"PeriodicalIF":0.0,"publicationDate":"2003-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89977164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
During the past decade, interconnects have replaced transistors as the dominant determiner of chip performance. To sustain the historical rate of advance in performance, monolithic interconnect technology has rapidly evolved to keep pace with advances in transistor density and performance. New and radically different interconnect technologies will become increasingly important to future gigascale microsystems.
{"title":"Interconnect Opportunities for Gigascale Integration","authors":"J. Meindl","doi":"10.1109/MM.2003.1209464","DOIUrl":"https://doi.org/10.1109/MM.2003.1209464","url":null,"abstract":"During the past decade, interconnects have replaced transistors as the dominant determiner of chip performance. To sustain the historical rate of advance in performance, monolithic interconnect technology has rapidly evolved to keep pace with advances in transistor density and performance. New and radically different interconnect technologies will become increasingly important to future gigascale microsystems.","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"26 1","pages":"245-264"},"PeriodicalIF":0.0,"publicationDate":"2002-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75949458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper describes an access function subsystem embedded in portions of an experimental microcode compiler which was built and used during 1973--6 using the IBM PL/I optimizing compiler under VM/370 and CMS. The use of the access function subsystem in this context was itself an experiment, performed by a group for all of whom PL/I was a new language and VM/370 a new operating system. The implementation of the subsystem was done strictly within the confines of the PL/I language. The basic objectives were ease of use, provision of a focal point for global storage management, extensive run-time validity checking with appropriate diagnostics, and data protection. Beyond satisfying these objectives, the subsystem proved more valuable than anticipated due to positive contributions made to debugging code in the VM/370 interactive development environment.
{"title":"Experience with access functions in an experimental compiler","authors":"F. Ris","doi":"10.1145/1096464.1096465","DOIUrl":"https://doi.org/10.1145/1096464.1096465","url":null,"abstract":"This paper describes an access function subsystem embedded in portions of an experimental microcode compiler which was built and used during 1973--6 using the IBM PL/I optimizing compiler under VM/370 and CMS. The use of the access function subsystem in this context was itself an experiment, performed by a group for all of whom PL/I was a new language and VM/370 a new operating system. The implementation of the subsystem was done strictly within the confines of the PL/I language. The basic objectives were ease of use, provision of a focal point for global storage management, extensive run-time validity checking with appropriate diagnostics, and data protection. Beyond satisfying these objectives, the subsystem proved more valuable than anticipated due to positive contributions made to debugging code in the VM/370 interactive development environment.","PeriodicalId":13051,"journal":{"name":"IBM J. Res. Dev.","volume":"28 1","pages":"40-51"},"PeriodicalIF":0.0,"publicationDate":"1984-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87097408","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：