Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma
The growing shift towards a Smart Grid involves integrating numerous new digital energy solutions into the energy ecosystems to address problems arising from the transition to carbon neutrality, particularly in linking the electricity and transportation sectors. Yet, this shift brings challenges due to mass electric vehicle adoption and the lack of methods to adequately assess various EV charging algorithms and their ecosystem impacts. This paper introduces a multi-agent based simulation model, validated through a case study of a Danish radial distribution network serving 126 households. The study reveals that traditional charging leads to grid overload by 2031 at 67% EV penetration, while decentralized strategies like Real-Time Pricing could cause overloads as early as 2028. The developed multi-agent based simulation demonstrates its ability to offer detailed, hourly analysis of future load profiles in distribution grids, and therefore, can be applied to other prospective scenarios in similar energy systems.
{"title":"Multi-Agent Based Simulation for Decentralized Electric Vehicle Charging Strategies and their Impacts","authors":"Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma","doi":"arxiv-2408.10790","DOIUrl":"https://doi.org/arxiv-2408.10790","url":null,"abstract":"The growing shift towards a Smart Grid involves integrating numerous new\u0000digital energy solutions into the energy ecosystems to address problems arising\u0000from the transition to carbon neutrality, particularly in linking the\u0000electricity and transportation sectors. Yet, this shift brings challenges due\u0000to mass electric vehicle adoption and the lack of methods to adequately assess\u0000various EV charging algorithms and their ecosystem impacts. This paper\u0000introduces a multi-agent based simulation model, validated through a case study\u0000of a Danish radial distribution network serving 126 households. The study\u0000reveals that traditional charging leads to grid overload by 2031 at 67% EV\u0000penetration, while decentralized strategies like Real-Time Pricing could cause\u0000overloads as early as 2028. The developed multi-agent based simulation\u0000demonstrates its ability to offer detailed, hourly analysis of future load\u0000profiles in distribution grids, and therefore, can be applied to other\u0000prospective scenarios in similar energy systems.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"34 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Hoon Yi
Due to emergent capabilities, large language models (LLMs) have been utilized as language-based agents to perform a variety of tasks and make decisions with an increasing degree of autonomy. These autonomous agents can understand high-level instructions, interact with their environments, and execute complex tasks using a selection of tools available to them. As the capabilities of the agents expand, ensuring their safety and trustworthiness becomes more imperative. In this study, we introduce the Athena framework which leverages the concept of verbal contrastive learning where past safe and unsafe trajectories are used as in-context (contrastive) examples to guide the agent towards safety while fulfilling a given task. The framework also incorporates a critiquing mechanism to guide the agent to prevent risky actions at every step. Furthermore, due to the lack of existing benchmarks on the safety reasoning ability of LLM-based agents, we curate a set of 80 toolkits across 8 categories with 180 scenarios to provide a safety evaluation benchmark. Our experimental evaluation, with both closed- and open-source LLMs, indicates verbal contrastive learning and interaction-level critiquing improve the safety rate significantly.
{"title":"Athena: Safe Autonomous Agents with Verbal Contrastive Learning","authors":"Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Hoon Yi","doi":"arxiv-2408.11021","DOIUrl":"https://doi.org/arxiv-2408.11021","url":null,"abstract":"Due to emergent capabilities, large language models (LLMs) have been utilized\u0000as language-based agents to perform a variety of tasks and make decisions with\u0000an increasing degree of autonomy. These autonomous agents can understand\u0000high-level instructions, interact with their environments, and execute complex\u0000tasks using a selection of tools available to them. As the capabilities of the\u0000agents expand, ensuring their safety and trustworthiness becomes more\u0000imperative. In this study, we introduce the Athena framework which leverages\u0000the concept of verbal contrastive learning where past safe and unsafe\u0000trajectories are used as in-context (contrastive) examples to guide the agent\u0000towards safety while fulfilling a given task. The framework also incorporates a\u0000critiquing mechanism to guide the agent to prevent risky actions at every step.\u0000Furthermore, due to the lack of existing benchmarks on the safety reasoning\u0000ability of LLM-based agents, we curate a set of 80 toolkits across 8 categories\u0000with 180 scenarios to provide a safety evaluation benchmark. Our experimental\u0000evaluation, with both closed- and open-source LLMs, indicates verbal\u0000contrastive learning and interaction-level critiquing improve the safety rate\u0000significantly.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"10 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma
This paper addresses the critical integration of electric vehicles (EVs) into the electricity grid, which is essential for achieving carbon neutrality by 2050. The rapid increase in EV adoption poses significant challenges to the existing grid infrastructure, particularly in managing the increasing electricity demand and mitigating the risk of grid overloads. Centralized EV charging strategies are investigated due to their potential to optimize grid stability and efficiency, compared to decentralized approaches that may exacerbate grid stress. Utilizing a multi-agent based simulation model, the study provides a realistic representation of the electric vehicle home charging ecosystem in a case study of Strib, Denmark. The findings show that the Earliest-deadline-first and Round Robin perform best with 100% EV adoption in terms of EV user satisfaction. The simulation considers a realistic adoption curve, EV charging strategies, EV models, and driving patterns to capture the full ecosystem dynamics over a long-term period with high resolution (hourly). Additionally, the study offers detailed load profiles for future distribution grids, demonstrating how centralized charging strategies can efficiently manage grid loads and prevent overloads.
{"title":"Multi-Agent Based Simulation for Investigating Centralized Charging Strategies and their Impact on Electric Vehicle Home Charging Ecosystem","authors":"Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma","doi":"arxiv-2408.10773","DOIUrl":"https://doi.org/arxiv-2408.10773","url":null,"abstract":"This paper addresses the critical integration of electric vehicles (EVs) into\u0000the electricity grid, which is essential for achieving carbon neutrality by\u00002050. The rapid increase in EV adoption poses significant challenges to the\u0000existing grid infrastructure, particularly in managing the increasing\u0000electricity demand and mitigating the risk of grid overloads. Centralized EV\u0000charging strategies are investigated due to their potential to optimize grid\u0000stability and efficiency, compared to decentralized approaches that may\u0000exacerbate grid stress. Utilizing a multi-agent based simulation model, the\u0000study provides a realistic representation of the electric vehicle home charging\u0000ecosystem in a case study of Strib, Denmark. The findings show that the\u0000Earliest-deadline-first and Round Robin perform best with 100% EV adoption in\u0000terms of EV user satisfaction. The simulation considers a realistic adoption\u0000curve, EV charging strategies, EV models, and driving patterns to capture the\u0000full ecosystem dynamics over a long-term period with high resolution (hourly).\u0000Additionally, the study offers detailed load profiles for future distribution\u0000grids, demonstrating how centralized charging strategies can efficiently manage\u0000grid loads and prevent overloads.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"113 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hanjun Choi, Hyunsung Kim, Minho Lee, Chang-Jo Kim, Jinsung Yoon, Sang-Ki Ko
Many spatiotemporal domains handle multi-agent trajectory data, but in real-world scenarios, collected trajectory data are often partially missing due to various reasons. While existing approaches demonstrate good performance in trajectory imputation, they face challenges in capturing the complex dynamics and interactions between agents due to a lack of physical constraints that govern realistic trajectories, leading to suboptimal results. To address this issue, the paper proposes a Derivative-Based Hybrid Prediction (DBHP) framework that can effectively impute multiple agents' missing trajectories. First, a neural network equipped with Set Transformers produces a naive prediction of missing trajectories while satisfying the permutation-equivariance in terms of the order of input agents. Then, the framework makes alternative predictions leveraging velocity and acceleration information and combines all the predictions with properly determined weights to provide final imputed trajectories. In this way, our proposed framework not only accurately predicts position, velocity, and acceleration values but also enforces the physical relationship between them, eventually improving both the accuracy and naturalness of the predicted trajectories. Accordingly, the experiment results about imputing player trajectories in team sports show that our framework significantly outperforms existing imputation baselines.
{"title":"DBHP: Trajectory Imputation in Multi-Agent Sports Using Derivative-Based Hybrid Prediction","authors":"Hanjun Choi, Hyunsung Kim, Minho Lee, Chang-Jo Kim, Jinsung Yoon, Sang-Ki Ko","doi":"arxiv-2408.10878","DOIUrl":"https://doi.org/arxiv-2408.10878","url":null,"abstract":"Many spatiotemporal domains handle multi-agent trajectory data, but in\u0000real-world scenarios, collected trajectory data are often partially missing due\u0000to various reasons. While existing approaches demonstrate good performance in\u0000trajectory imputation, they face challenges in capturing the complex dynamics\u0000and interactions between agents due to a lack of physical constraints that\u0000govern realistic trajectories, leading to suboptimal results. To address this\u0000issue, the paper proposes a Derivative-Based Hybrid Prediction (DBHP) framework\u0000that can effectively impute multiple agents' missing trajectories. First, a\u0000neural network equipped with Set Transformers produces a naive prediction of\u0000missing trajectories while satisfying the permutation-equivariance in terms of\u0000the order of input agents. Then, the framework makes alternative predictions\u0000leveraging velocity and acceleration information and combines all the\u0000predictions with properly determined weights to provide final imputed\u0000trajectories. In this way, our proposed framework not only accurately predicts\u0000position, velocity, and acceleration values but also enforces the physical\u0000relationship between them, eventually improving both the accuracy and\u0000naturalness of the predicted trajectories. Accordingly, the experiment results\u0000about imputing player trajectories in team sports show that our framework\u0000significantly outperforms existing imputation baselines.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma
Power-to-Hydrogen is crucial for the renewable energy transition, yet existing literature lacks business models for the significant excess heat it generates. This study addresses this by evaluating three models for selling electrolyzer-generated heat to district heating grids: constant, flexible, and renewable-source hydrogen production, with and without heat sales. Using agent-based modeling and multi-criteria decision-making methods (VIKOR, TOPSIS, PROMETHEE), it finds that selling excess heat can cut hydrogen production costs by 5.6%. The optimal model operates flexibly with electricity spot prices, includes heat sales, and maintains a hydrogen price of 3.3 EUR/kg. Environmentally, hydrogen production from grid electricity could emit up to 13,783.8 tons of CO2 over four years from 2023. The best economic and environmental model uses renewable sources and sells heat at 3.5 EUR/kg
{"title":"Multi-agent based modeling for investigating excess heat utilization from electrolyzer production to district heating network","authors":"Kristoffer Christensen, Bo Nørregaard Jørgensen, Zheng Grace Ma","doi":"arxiv-2408.10783","DOIUrl":"https://doi.org/arxiv-2408.10783","url":null,"abstract":"Power-to-Hydrogen is crucial for the renewable energy transition, yet\u0000existing literature lacks business models for the significant excess heat it\u0000generates. This study addresses this by evaluating three models for selling\u0000electrolyzer-generated heat to district heating grids: constant, flexible, and\u0000renewable-source hydrogen production, with and without heat sales. Using\u0000agent-based modeling and multi-criteria decision-making methods (VIKOR, TOPSIS,\u0000PROMETHEE), it finds that selling excess heat can cut hydrogen production costs\u0000by 5.6%. The optimal model operates flexibly with electricity spot prices,\u0000includes heat sales, and maintains a hydrogen price of 3.3 EUR/kg.\u0000Environmentally, hydrogen production from grid electricity could emit up to\u000013,783.8 tons of CO2 over four years from 2023. The best economic and\u0000environmental model uses renewable sources and sells heat at 3.5 EUR/kg","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"37 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190539","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The electrification of the transportation and heating sector, the so-called sector coupling, is one of the core elements to achieve independence from fossil fuels. As it highly affects the electricity demand, especially on the local level, the integrated modeling and simulation of all sectors is a promising approach for analyzing design decisions or complex control strategies. This paper analyzes the increase in electricity demand resulting from sector coupling, mainly due to integrating electric vehicles into urban energy systems. Therefore, we utilize a digital twin of an existing local energy system and extend it with a mobility simulation model to evaluate the impact of electric vehicles on the distribution grid level. Our findings indicate a significant rise in annual electricity consumption attributed to electric vehicles, with home charging alone resulting in a 78% increase. However, we demonstrate that integrating photovoltaic and battery energy storage systems can effectively mitigate this rise.
{"title":"Analyzing the Impact of Electric Vehicles on Local Energy Systems using Digital Twins","authors":"Daniel René Bayer, Marco Pruckner","doi":"arxiv-2408.10763","DOIUrl":"https://doi.org/arxiv-2408.10763","url":null,"abstract":"The electrification of the transportation and heating sector, the so-called\u0000sector coupling, is one of the core elements to achieve independence from\u0000fossil fuels. As it highly affects the electricity demand, especially on the\u0000local level, the integrated modeling and simulation of all sectors is a\u0000promising approach for analyzing design decisions or complex control\u0000strategies. This paper analyzes the increase in electricity demand resulting\u0000from sector coupling, mainly due to integrating electric vehicles into urban\u0000energy systems. Therefore, we utilize a digital twin of an existing local\u0000energy system and extend it with a mobility simulation model to evaluate the\u0000impact of electric vehicles on the distribution grid level. Our findings\u0000indicate a significant rise in annual electricity consumption attributed to\u0000electric vehicles, with home charging alone resulting in a 78% increase.\u0000However, we demonstrate that integrating photovoltaic and battery energy\u0000storage systems can effectively mitigate this rise.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"19 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mechanism design is a well-established game-theoretic paradigm for designing games to achieve desired outcomes. This paper addresses a closely related but distinct concept, equilibrium design. Unlike mechanism design, the designer's authority in equilibrium design is more constrained; she can only modify the incentive structures in a given game to achieve certain outcomes without the ability to create the game from scratch. We study the problem of equilibrium design using dynamic incentive structures, known as reward machines. We use weighted concurrent game structures for the game model, with goals (for the players and the designer) defined as mean-payoff objectives. We show how reward machines can be used to represent dynamic incentives that allocate rewards in a manner that optimises the designer's goal. We also introduce the main decision problem within our framework, the payoff improvement problem. This problem essentially asks whether there exists a dynamic incentive (represented by some reward machine) that can improve the designer's payoff by more than a given threshold value. We present two variants of the problem: strong and weak. We demonstrate that both can be solved in polynomial time using a Turing machine equipped with an NP oracle. Furthermore, we also establish that these variants are either NP-hard or coNP-hard. Finally, we show how to synthesise the corresponding reward machine if it exists.
{"title":"Synthesis of Reward Machines for Multi-Agent Equilibrium Design (Full Version)","authors":"Muhammad Najib, Giuseppe Perelli","doi":"arxiv-2408.10074","DOIUrl":"https://doi.org/arxiv-2408.10074","url":null,"abstract":"Mechanism design is a well-established game-theoretic paradigm for designing\u0000games to achieve desired outcomes. This paper addresses a closely related but\u0000distinct concept, equilibrium design. Unlike mechanism design, the designer's\u0000authority in equilibrium design is more constrained; she can only modify the\u0000incentive structures in a given game to achieve certain outcomes without the\u0000ability to create the game from scratch. We study the problem of equilibrium\u0000design using dynamic incentive structures, known as reward machines. We use\u0000weighted concurrent game structures for the game model, with goals (for the\u0000players and the designer) defined as mean-payoff objectives. We show how reward\u0000machines can be used to represent dynamic incentives that allocate rewards in a\u0000manner that optimises the designer's goal. We also introduce the main decision\u0000problem within our framework, the payoff improvement problem. This problem\u0000essentially asks whether there exists a dynamic incentive (represented by some\u0000reward machine) that can improve the designer's payoff by more than a given\u0000threshold value. We present two variants of the problem: strong and weak. We\u0000demonstrate that both can be solved in polynomial time using a Turing machine\u0000equipped with an NP oracle. Furthermore, we also establish that these variants\u0000are either NP-hard or coNP-hard. Finally, we show how to synthesise the\u0000corresponding reward machine if it exists.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"2011 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qian Wang, Tianyu Wang, Qinbin Li, Jingsheng Liang, Bingsheng He
With the emergence of large language models (LLMs), LLM-powered multi-agent systems (LLM-MA systems) have been proposed to tackle real-world tasks. However, their agents mostly follow predefined Standard Operating Procedures (SOPs) that remain unchanged across the whole interaction, lacking autonomy and scalability. Additionally, current solutions often overlook the necessity for effective agent cooperation. To address the above limitations, we propose MegaAgent, a practical framework designed for autonomous cooperation in large-scale LLM Agent systems. MegaAgent leverages the autonomy of agents to dynamically generate agents based on task requirements, incorporating features such as automatically dividing tasks, systematic planning and monitoring of agent activities, and managing concurrent operations. In addition, MegaAgent is designed with a hierarchical structure and employs system-level parallelism to enhance performance and boost communication. We demonstrate the effectiveness of MegaAgent through Gobang game development, showing that it outperforms popular LLM-MA systems; and national policy simulation, demonstrating its high autonomy and potential to rapidly scale up to 590 agents while ensuring effective cooperation among them. Our results indicate that MegaAgent is the first autonomous large-scale LLM-MA system with no pre-defined SOPs, high effectiveness and scalability, paving the way for further research in this field. Our code is at https://anonymous.4open.science/r/MegaAgent-81F3.
{"title":"MegaAgent: A Practical Framework for Autonomous Cooperation in Large-Scale LLM Agent Systems","authors":"Qian Wang, Tianyu Wang, Qinbin Li, Jingsheng Liang, Bingsheng He","doi":"arxiv-2408.09955","DOIUrl":"https://doi.org/arxiv-2408.09955","url":null,"abstract":"With the emergence of large language models (LLMs), LLM-powered multi-agent\u0000systems (LLM-MA systems) have been proposed to tackle real-world tasks.\u0000However, their agents mostly follow predefined Standard Operating Procedures\u0000(SOPs) that remain unchanged across the whole interaction, lacking autonomy and\u0000scalability. Additionally, current solutions often overlook the necessity for\u0000effective agent cooperation. To address the above limitations, we propose\u0000MegaAgent, a practical framework designed for autonomous cooperation in\u0000large-scale LLM Agent systems. MegaAgent leverages the autonomy of agents to\u0000dynamically generate agents based on task requirements, incorporating features\u0000such as automatically dividing tasks, systematic planning and monitoring of\u0000agent activities, and managing concurrent operations. In addition, MegaAgent is\u0000designed with a hierarchical structure and employs system-level parallelism to\u0000enhance performance and boost communication. We demonstrate the effectiveness\u0000of MegaAgent through Gobang game development, showing that it outperforms\u0000popular LLM-MA systems; and national policy simulation, demonstrating its high\u0000autonomy and potential to rapidly scale up to 590 agents while ensuring\u0000effective cooperation among them. Our results indicate that MegaAgent is the\u0000first autonomous large-scale LLM-MA system with no pre-defined SOPs, high\u0000effectiveness and scalability, paving the way for further research in this\u0000field. Our code is at https://anonymous.4open.science/r/MegaAgent-81F3.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"43 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David Molina Concha, Kyeonghyeon Park, Hyun-Rok Lee, Taesik Lee, Chi-Guhn Lee
We introduce a novel problem setting for algorithmic contract design, named the principal-MARL contract design problem. This setting extends traditional contract design to account for dynamic and stochastic environments using Markov Games and Multi-Agent Reinforcement Learning. To tackle this problem, we propose a Multi-Objective Bayesian Optimization (MOBO) framework named Constrained Pareto Maximum Entropy Search (cPMES). Our approach integrates MOBO and MARL to explore the highly constrained contract design space, identifying promising incentive and recruitment decisions. cPMES transforms the principal-MARL contract design problem into an unconstrained multi-objective problem, leveraging the probability of feasibility as part of the objectives and ensuring promising designs predicted on the feasibility border are included in the Pareto front. By focusing the entropy prediction on designs within the Pareto set, cPMES mitigates the risk of the search strategy being overwhelmed by entropy from constraints. We demonstrate the effectiveness of cPMES through extensive benchmark studies in synthetic and simulated environments, showing its ability to find feasible contract designs that maximize the principal's objectives. Additionally, we provide theoretical support with a sub-linear regret bound concerning the number of iterations.
{"title":"Algorithmic Contract Design with Reinforcement Learning Agents","authors":"David Molina Concha, Kyeonghyeon Park, Hyun-Rok Lee, Taesik Lee, Chi-Guhn Lee","doi":"arxiv-2408.09686","DOIUrl":"https://doi.org/arxiv-2408.09686","url":null,"abstract":"We introduce a novel problem setting for algorithmic contract design, named\u0000the principal-MARL contract design problem. This setting extends traditional\u0000contract design to account for dynamic and stochastic environments using Markov\u0000Games and Multi-Agent Reinforcement Learning. To tackle this problem, we\u0000propose a Multi-Objective Bayesian Optimization (MOBO) framework named\u0000Constrained Pareto Maximum Entropy Search (cPMES). Our approach integrates MOBO\u0000and MARL to explore the highly constrained contract design space, identifying\u0000promising incentive and recruitment decisions. cPMES transforms the\u0000principal-MARL contract design problem into an unconstrained multi-objective\u0000problem, leveraging the probability of feasibility as part of the objectives\u0000and ensuring promising designs predicted on the feasibility border are included\u0000in the Pareto front. By focusing the entropy prediction on designs within the\u0000Pareto set, cPMES mitigates the risk of the search strategy being overwhelmed\u0000by entropy from constraints. We demonstrate the effectiveness of cPMES through\u0000extensive benchmark studies in synthetic and simulated environments, showing\u0000its ability to find feasible contract designs that maximize the principal's\u0000objectives. Additionally, we provide theoretical support with a sub-linear\u0000regret bound concerning the number of iterations.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"63-65 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190418","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In the context of increasingly complex environmental challenges, effective pollution control mechanisms are crucial. By extending the state of the art auction mechanisms, we aim to develop an efficient approach for allocating pollution abatement resources in a multi-pollutant setting with pollutants affecting each other's reduction costs. We modify the Combinatorial Multi-Round Ascending Auction for the auction of escape permits of pollutants with co-dependent reduction processes, specifically, greenhouse gas emissions and nutrient runoff in Finnish agriculture. We show the significant advantages of this mechanism in pollution control through experiments on the bid prices and amount of escape permits sold in multiple auction simulations.
{"title":"Auctioning Escape Permits for Multiple Correlated Pollutants Using CMRA","authors":"Keshav Goyal, Sooraj Sathish, Shrisha Rao","doi":"arxiv-2408.10148","DOIUrl":"https://doi.org/arxiv-2408.10148","url":null,"abstract":"In the context of increasingly complex environmental challenges, effective\u0000pollution control mechanisms are crucial. By extending the state of the art\u0000auction mechanisms, we aim to develop an efficient approach for allocating\u0000pollution abatement resources in a multi-pollutant setting with pollutants\u0000affecting each other's reduction costs. We modify the Combinatorial Multi-Round\u0000Ascending Auction for the auction of escape permits of pollutants with\u0000co-dependent reduction processes, specifically, greenhouse gas emissions and\u0000nutrient runoff in Finnish agriculture. We show the significant advantages of\u0000this mechanism in pollution control through experiments on the bid prices and\u0000amount of escape permits sold in multiple auction simulations.","PeriodicalId":501315,"journal":{"name":"arXiv - CS - Multiagent Systems","volume":"19 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142190422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：