In this poster, we will present new tool, HyPLC [30], which implements the translation of discrete control code of verified hybrid program models to PLC controller code and, vice versa, the translation of existing PLC code into the discrete control actions for a hybrid program given an additional input of the continuous dynamics of the system to be verified. This approach allows for the generation of real controller code while preserving, by compilation, the correctness of a valid and verified hybrid program. PLCs are common cyber-physical interfaces for safety-critical industrial control applications, and HyPLC serves as a pragmatic tool for bridging formal verification of complex cyber-physical systems at the algorithmic level of hybrid programs with the execution layer of concrete PLC implementations. This poster will also discuss future directions of HyPLC such as support of multiple tasks (and, by extension, multiple PLCs), interval arithmetic, as well as security analyses.
{"title":"Toward multi-task support and security analyses in PLC program translation for verification: poster abstract","authors":"Luis Garcia, Stefan Mitsch, André Platzer","doi":"10.1145/3302509.3313335","DOIUrl":"https://doi.org/10.1145/3302509.3313335","url":null,"abstract":"In this poster, we will present new tool, HyPLC [30], which implements the translation of discrete control code of verified hybrid program models to PLC controller code and, vice versa, the translation of existing PLC code into the discrete control actions for a hybrid program given an additional input of the continuous dynamics of the system to be verified. This approach allows for the generation of real controller code while preserving, by compilation, the correctness of a valid and verified hybrid program. PLCs are common cyber-physical interfaces for safety-critical industrial control applications, and HyPLC serves as a pragmatic tool for bridging formal verification of complex cyber-physical systems at the algorithmic level of hybrid programs with the execution layer of concrete PLC implementations. This poster will also discuss future directions of HyPLC such as support of multiple tasks (and, by extension, multiple PLCs), interval arithmetic, as well as security analyses.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132109859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recent radio frequency (RF) sensing techniques use a network of RF sensors to detect and locate people that do not carry any devices and can operate in non line-of-sight environments. Model-based device-free RF sensing systems use statistical models to quantify human presence and motion based on the received RF signal measurements. However, such methods often require the fine tuning of multiple model-dependent parameters in order to achieve sub meter accuracy. In this work, we propose to use deep neural networks together with visual tracking systems to effectively generate training data so as to learn a general model. Our method can automatically produce human motion and occupancy images from RF sensor network measurements without the need for manual RF model parameter tuning.
{"title":"Deep intelligent network for device-free people tracking: WIP abstract","authors":"Yang Zhao, Ming-Ching Chang, P. Tu","doi":"10.1145/3302509.3313312","DOIUrl":"https://doi.org/10.1145/3302509.3313312","url":null,"abstract":"Recent radio frequency (RF) sensing techniques use a network of RF sensors to detect and locate people that do not carry any devices and can operate in non line-of-sight environments. Model-based device-free RF sensing systems use statistical models to quantify human presence and motion based on the received RF signal measurements. However, such methods often require the fine tuning of multiple model-dependent parameters in order to achieve sub meter accuracy. In this work, we propose to use deep neural networks together with visual tracking systems to effectively generate training data so as to learn a general model. Our method can automatically produce human motion and occupancy images from RF sensor network measurements without the need for manual RF model parameter tuning.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"109 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121994525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kuk Jin Jang, Y. Pant, Bo Zhang, James Weimer, R. Mangharam
Medical cyber-physical systems, such as the implantable cardioverter defibrillator (ICD), require evaluation of safety and efficacy in the context of a patient population in a clinical trial. Advances in computer modeling and simulation allow for generation of a simulated cohort or virtual cohort which mimics a patient population and can be used as a source of prior information. A major obstacle to acceptance of simulation results as a source of prior information is the lack of a framework for explicitly modeling sources of uncertainty in simulation results and quantifying the effect on trial outcomes. In this work, we formulate the Computer-Aided Clinical Trial (CACT) within a Bayesian statistical framework allowing explicit modeling of assumptions and utilization of simulation results at all stages of a clinical trial. To quantify the robustness of the CACT outcome with respect to a simulation assumption, we define δ-robustness as the minimum perturbation of the base prior distribution resulting in a change of the CACT outcome and provide a method to estimate the δ-robustness. We demonstrate the utility of the framework and how the results of δ-robustness evaluation can be utilized at various stages of a clinical trial through an application to the Rhythm ID Goes Head-to-head Trial (RIGHT), which was a comparative evaluation of the safety and efficacy of specific software algorithms across different implantable cardiac devices. Finally, we introduce a hardware interface that allows for direct interaction with the physical device in order to validate and confirm the results of a CACT for implantable cardiac devices.
医疗信息物理系统，如植入式心律转复除颤器(ICD)，需要在临床试验中对患者群体进行安全性和有效性评估。计算机建模和仿真的进步允许生成模拟患者群体的模拟队列或虚拟队列，并可作为先验信息的来源。接受模拟结果作为先验信息来源的主要障碍是缺乏明确建模模拟结果不确定性来源和量化对试验结果影响的框架。在这项工作中，我们在贝叶斯统计框架内制定了计算机辅助临床试验(CACT)，允许在临床试验的所有阶段对假设和模拟结果进行明确建模。为了量化相对于模拟假设的CACT结果的稳健性，我们将δ-鲁棒性定义为导致CACT结果变化的基本先验分布的最小扰动，并提供了一种估计δ-鲁棒性的方法。我们展示了该框架的实用性，以及δ-稳健性评估的结果如何通过应用于Rhythm ID Goes Head-to-head试验(右)在临床试验的各个阶段得到利用，该试验是对不同植入式心脏装置的特定软件算法的安全性和有效性的比较评估。最后，我们介绍了一个硬件接口，允许与物理设备直接交互，以验证和确认植入式心脏设备的CACT结果。
{"title":"Robustness evaluation of computer-aided clinical trials for medical devices","authors":"Kuk Jin Jang, Y. Pant, Bo Zhang, James Weimer, R. Mangharam","doi":"10.1145/3302509.3311058","DOIUrl":"https://doi.org/10.1145/3302509.3311058","url":null,"abstract":"Medical cyber-physical systems, such as the implantable cardioverter defibrillator (ICD), require evaluation of safety and efficacy in the context of a patient population in a clinical trial. Advances in computer modeling and simulation allow for generation of a simulated cohort or virtual cohort which mimics a patient population and can be used as a source of prior information. A major obstacle to acceptance of simulation results as a source of prior information is the lack of a framework for explicitly modeling sources of uncertainty in simulation results and quantifying the effect on trial outcomes. In this work, we formulate the Computer-Aided Clinical Trial (CACT) within a Bayesian statistical framework allowing explicit modeling of assumptions and utilization of simulation results at all stages of a clinical trial. To quantify the robustness of the CACT outcome with respect to a simulation assumption, we define δ-robustness as the minimum perturbation of the base prior distribution resulting in a change of the CACT outcome and provide a method to estimate the δ-robustness. We demonstrate the utility of the framework and how the results of δ-robustness evaluation can be utilized at various stages of a clinical trial through an application to the Rhythm ID Goes Head-to-head Trial (RIGHT), which was a comparative evaluation of the safety and efficacy of specific software algorithms across different implantable cardiac devices. Finally, we introduce a hardware interface that allows for direct interaction with the physical device in order to validate and confirm the results of a CACT for implantable cardiac devices.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116226975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Geoffrey Pettet, Ayan Mukhopadhyay, Chinmaya Samal, A. Dubey, Yevgeniy Vorobeychik
This work presents a dashboard tool that helps emergency responders analyze and manage spatial-temporal incidents like crime and traffic accidents. It uses state-of-the-art statistical models to learn incident probabilities based on factors such as prior incidents, time and weather. The dashboard can then present historic and predicted incident distributions. It also allows responders to analyze how moving or adding depots (stations for emergency responders) affects average response times, and can make dispatching recommendations based on heuristics. Broadly, it is a one-stop tool that helps responders visualize historical data as well as plan for and respond to incidents.
{"title":"Incident management and analysis dashboard for fire departments: ICCPS demo","authors":"Geoffrey Pettet, Ayan Mukhopadhyay, Chinmaya Samal, A. Dubey, Yevgeniy Vorobeychik","doi":"10.1145/3302509.3313329","DOIUrl":"https://doi.org/10.1145/3302509.3313329","url":null,"abstract":"This work presents a dashboard tool that helps emergency responders analyze and manage spatial-temporal incidents like crime and traffic accidents. It uses state-of-the-art statistical models to learn incident probabilities based on factors such as prior incidents, time and weather. The dashboard can then present historic and predicted incident distributions. It also allows responders to analyze how moving or adding depots (stations for emergency responders) affects average response times, and can make dispatching recommendations based on heuristics. Broadly, it is a one-stop tool that helps responders visualize historical data as well as plan for and respond to incidents.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131329716","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we demonstrate the feasibility of smart malware that advances state-of-the-art attacks by (i) indirectly attacking a computing infrastructure through a cyber-physical system (CPS) that manages the environment in which the computing enterprise operates, (ii) disguising its malicious actions as accidental failures, and (iii) self-learning attack strategies from cyber-physical system measurement data. We address all aspects of the malware, including the construction of the self-learning malware and the launch of a failure injection attack. We validate the attacks in a data-driven CPS simulation environment developed as part of this study.
{"title":"Availability attacks on computing systems through alteration of environmental control: smart malware approach","authors":"Key-whan Chung, Z. Kalbarczyk, R. Iyer","doi":"10.1145/3302509.3311041","DOIUrl":"https://doi.org/10.1145/3302509.3311041","url":null,"abstract":"In this paper, we demonstrate the feasibility of smart malware that advances state-of-the-art attacks by (i) indirectly attacking a computing infrastructure through a cyber-physical system (CPS) that manages the environment in which the computing enterprise operates, (ii) disguising its malicious actions as accidental failures, and (iii) self-learning attack strategies from cyber-physical system measurement data. We address all aspects of the malware, including the construction of the self-learning malware and the launch of a failure injection attack. We validate the attacks in a data-driven CPS simulation environment developed as part of this study.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128561474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dohwan Kim, Yuchang Won, Seunghyeon Kim, Y. Eun, Kyung-Joon Park, K. Johansson
Design of wireless control systems has been extensively studied, which is one of the fundamental issues in cyber-physical systems. In this paper, we empirically investigate heterogeneous sampling rate assignment with a testbed when multiple physical systems are controlled through an IEEE 802.11 network. Among the critical design variables in wireless control systems, we focus on the sampling rates because they are always key control knobs regardless of network protocols. There has been little experimental research on heterogeneous sampling rate optimization for IEEE 802.11 wireless control systems, where the sampling rates of each control loop may have different values. We first formulate the co-design problem in an optimization framework with respect to the heterogeneous sampling rates by explicitly taking into account the relations of the sampling rates with the control cost, network energy consumption, and network delay. We further relax the problem as convex optimization, which is provably solved in polynomial time. Our empirical study ensures that the approximate solution is tightly close to the original optimum. To validate the proposed optimization framework, we build a disk-levitation tube testbed, which wirelessly controls the height of 20 disks at the same time. Our empirical study confirms that our optimization formulation is highly effective in practice.
{"title":"Sampling rate optimization for IEEE 802.11 wireless control systems","authors":"Dohwan Kim, Yuchang Won, Seunghyeon Kim, Y. Eun, Kyung-Joon Park, K. Johansson","doi":"10.1145/3302509.3311045","DOIUrl":"https://doi.org/10.1145/3302509.3311045","url":null,"abstract":"Design of wireless control systems has been extensively studied, which is one of the fundamental issues in cyber-physical systems. In this paper, we empirically investigate heterogeneous sampling rate assignment with a testbed when multiple physical systems are controlled through an IEEE 802.11 network. Among the critical design variables in wireless control systems, we focus on the sampling rates because they are always key control knobs regardless of network protocols. There has been little experimental research on heterogeneous sampling rate optimization for IEEE 802.11 wireless control systems, where the sampling rates of each control loop may have different values. We first formulate the co-design problem in an optimization framework with respect to the heterogeneous sampling rates by explicitly taking into account the relations of the sampling rates with the control cost, network energy consumption, and network delay. We further relax the problem as convex optimization, which is provably solved in polynomial time. Our empirical study ensures that the approximate solution is tightly close to the original optimum. To validate the proposed optimization framework, we build a disk-levitation tube testbed, which wirelessly controls the height of 20 disks at the same time. Our empirical study confirms that our optimization formulation is highly effective in practice.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"149 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126915877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
George Gunter, Yanbing Wang, Derek Gloudemans, Raphael E. Stern, D. Work, M. D. Monache, R. Bhadani, Matt Bunting, Roman L. Lysecky, J. Sprinkle, Benjamin Seibold, B. Piccoli
Adaptive cruise control (ACC) is the first wave of vehicle automation that will reach the mainstream. It has been shown in [3] that automation of a small fraction of vehicles in traffic (e.g., 5%) can change the emergent properties of the flow, for example by dissipating phantom jams. Substantial theoretical and experimental underpinnings of vehicle automation and platooning were established in the from the USDOT Automated Highway System effort [1]. However, it is not yet clear whether the ACC vehicles that are currently commercially available will dampen or amplify phantom jams.
{"title":"String stability of commercial adaptive cruise control vehicles: WIP abstract","authors":"George Gunter, Yanbing Wang, Derek Gloudemans, Raphael E. Stern, D. Work, M. D. Monache, R. Bhadani, Matt Bunting, Roman L. Lysecky, J. Sprinkle, Benjamin Seibold, B. Piccoli","doi":"10.1145/3302509.3313325","DOIUrl":"https://doi.org/10.1145/3302509.3313325","url":null,"abstract":"Adaptive cruise control (ACC) is the first wave of vehicle automation that will reach the mainstream. It has been shown in [3] that automation of a small fraction of vehicles in traffic (e.g., 5%) can change the emergent properties of the flow, for example by dissipating phantom jams. Substantial theoretical and experimental underpinnings of vehicle automation and platooning were established in the from the USDOT Automated Highway System effort [1]. However, it is not yet clear whether the ACC vehicles that are currently commercially available will dampen or amplify phantom jams.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133657889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Taxi drivers face to poor working conditions such as low wages and long working hours. To improve the conditions, a taxi dispatch system is very important. F. Miao et al. proposed a method for calculating an optimal taxi cruising path based on the model predictive control and the new parameter of supply demand ratio [1]. T. Ushio et al. proposed a method for calculating an optimal taxi cruising path based on a mixed logical dynamical model [2]. X. Chen et al. proposed a hierarchical taxi-dispatch system consisting of global and regional decision-makers [3]. The global decision-maker calculates the optimal taxi dispatch in a longer time period, while the regional ones determine schedules of taxis in the corresponding regions in a shorter time period. However, these previous mechanisms have lower scalability and real-time properties. To tackle these problems, we propose a novel hierarchical support system for a taxi dispatch system. In our proposal, a macro level component calculates the optimal taxi movement by using a taxi flow model, whereas micro level components determine the optimal taxi cruising in the corresponding regions. Both of them are conducted in the same time period.
{"title":"Hierarchical taxi dispatch system with local coordination among micro-level components: WIP abstract","authors":"Sota Takashima, Naomi Kuze, T. Ushio","doi":"10.1145/3302509.3313314","DOIUrl":"https://doi.org/10.1145/3302509.3313314","url":null,"abstract":"Taxi drivers face to poor working conditions such as low wages and long working hours. To improve the conditions, a taxi dispatch system is very important. F. Miao et al. proposed a method for calculating an optimal taxi cruising path based on the model predictive control and the new parameter of supply demand ratio [1]. T. Ushio et al. proposed a method for calculating an optimal taxi cruising path based on a mixed logical dynamical model [2]. X. Chen et al. proposed a hierarchical taxi-dispatch system consisting of global and regional decision-makers [3]. The global decision-maker calculates the optimal taxi dispatch in a longer time period, while the regional ones determine schedules of taxis in the corresponding regions in a shorter time period. However, these previous mechanisms have lower scalability and real-time properties. To tackle these problems, we propose a novel hierarchical support system for a taxi dispatch system. In our proposal, a macro level component calculates the optimal taxi movement by using a taxi flow model, whereas micro level components determine the optimal taxi cruising in the corresponding regions. Both of them are conducted in the same time period.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134280828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qitong Gao, Davood Hajinezhad, Yan Zhang, Y. Kantaros, M. Zavlanos
In this paper, we propose a model-free reinforcement learning method to synthesize control policies for mobile robots modeled as Markov Decision Process (MDP) with unknown transition probabilities that satisfy Linear Temporal Logic (LTL) specifications. Specifically, we develop a reduced variance deep Q-Learning technique that relies on Neural Networks (NN) to approximate the state-action values of the MDP and employs a reward function that depends on the accepting condition of the Deterministic Rabin Automaton (DRA) that captures the LTL specification. The key idea is to convert the deep Q-Learning problem into a nonconvex max-min optimization problem with a finite-sum structure, and develop an Arrow-Hurwicz-Uzawa type stochastic reduced variance algorithm with constant stepsize to solve it. Unlike Stochastic Gradient Descent (SGD) methods that are often used in deep reinforcement learning, our method can estimate the gradients of an unknown loss function more accurately and can improve the stability of the training process. Moreover, our method does not require learning the transition probabilities in the MDP, constructing a product MDP, or computing Accepting Maximal End Components (AMECs). This allows the robot to learn an optimal policy even if the environment cannot be modeled accurately or if AMECs do not exist. In the latter case, the resulting control policies minimize the frequency with which the system enters bad states in the DRA that violate the task specifications. To the best of our knowledge, this is the first model-free deep reinforcement learning algorithm that can synthesize policies that maximize the probability of satisfying an LTL specification even if AMECs do not exist. Rigorous convergence analysis and rate of convergence are provided for the proposed algorithm as well as numerical experiments that validate our method.
{"title":"Reduced variance deep reinforcement learning with temporal logic specifications","authors":"Qitong Gao, Davood Hajinezhad, Yan Zhang, Y. Kantaros, M. Zavlanos","doi":"10.1145/3302509.3311053","DOIUrl":"https://doi.org/10.1145/3302509.3311053","url":null,"abstract":"In this paper, we propose a model-free reinforcement learning method to synthesize control policies for mobile robots modeled as Markov Decision Process (MDP) with unknown transition probabilities that satisfy Linear Temporal Logic (LTL) specifications. Specifically, we develop a reduced variance deep Q-Learning technique that relies on Neural Networks (NN) to approximate the state-action values of the MDP and employs a reward function that depends on the accepting condition of the Deterministic Rabin Automaton (DRA) that captures the LTL specification. The key idea is to convert the deep Q-Learning problem into a nonconvex max-min optimization problem with a finite-sum structure, and develop an Arrow-Hurwicz-Uzawa type stochastic reduced variance algorithm with constant stepsize to solve it. Unlike Stochastic Gradient Descent (SGD) methods that are often used in deep reinforcement learning, our method can estimate the gradients of an unknown loss function more accurately and can improve the stability of the training process. Moreover, our method does not require learning the transition probabilities in the MDP, constructing a product MDP, or computing Accepting Maximal End Components (AMECs). This allows the robot to learn an optimal policy even if the environment cannot be modeled accurately or if AMECs do not exist. In the latter case, the resulting control policies minimize the frequency with which the system enters bad states in the DRA that violate the task specifications. To the best of our knowledge, this is the first model-free deep reinforcement learning algorithm that can synthesize policies that maximize the probability of satisfying an LTL specification even if AMECs do not exist. Rigorous convergence analysis and rate of convergence are provided for the proposed algorithm as well as numerical experiments that validate our method.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"2 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123562373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we aim to optimize the process of Connected and Automated Vehicles (CAVs) merging at a traffic intersection while guaranteeing the state, control and safety constraints. We decompose the task of automatic merging for all the CAVs in a control zone around a merging point into same-lane safety constraints and different-lane safe merging, and implement these requirements using control barrier functions (CBFs). We consider two main objectives. First, to minimize travel time, we make the CAVs reach the road maximum speed with exponentially stabilizing control Lyapunov functions (CLF). Second, we penalize energy consumption as a cost in an optimization problem. We then decompose the merging problem into decentralized subproblems formulated as a sequence of quadratic programs (QP), which are solved in real time. Our simulations and comparisons show that the method proposed here outperforms ad hoc controllers used in traffic system simulators and provides comparable results to the optimal control solution of the merging problem in earlier work.
{"title":"Decentralized merging control in traffic networks: a control barrier function approach","authors":"Wei Xiao, C. Belta, C. Cassandras","doi":"10.1145/3302509.3311054","DOIUrl":"https://doi.org/10.1145/3302509.3311054","url":null,"abstract":"In this paper, we aim to optimize the process of Connected and Automated Vehicles (CAVs) merging at a traffic intersection while guaranteeing the state, control and safety constraints. We decompose the task of automatic merging for all the CAVs in a control zone around a merging point into same-lane safety constraints and different-lane safe merging, and implement these requirements using control barrier functions (CBFs). We consider two main objectives. First, to minimize travel time, we make the CAVs reach the road maximum speed with exponentially stabilizing control Lyapunov functions (CLF). Second, we penalize energy consumption as a cost in an optimization problem. We then decompose the merging problem into decentralized subproblems formulated as a sequence of quadratic programs (QP), which are solved in real time. Our simulations and comparisons show that the method proposed here outperforms ad hoc controllers used in traffic system simulators and provides comparable results to the optimal control solution of the merging problem in earlier work.","PeriodicalId":413733,"journal":{"name":"Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125093106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：