Motivated by online advertising applications, we consider a version of the classical multi-armed bandit problem where there is a cost associated with pulling each arm, and a corresponding budget which limits the number of times that an arm can be pulled. We derive regret bounds on the expected reward in such a bandit problem using a modification of the well-known upper confidence bound algorithm UCB1.
{"title":"Bandits with budgets","authors":"Richard Combes, Chong Jiang, R. Srikant","doi":"10.1145/2796314.2745847","DOIUrl":"https://doi.org/10.1145/2796314.2745847","url":null,"abstract":"Motivated by online advertising applications, we consider a version of the classical multi-armed bandit problem where there is a cost associated with pulling each arm, and a corresponding budget which limits the number of times that an arm can be pulled. We derive regret bounds on the expected reward in such a bandit problem using a modification of the well-known upper confidence bound algorithm UCB1.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-06-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122490189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2015-05-26DOI: 10.1109/ICRA.2015.7140035
Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Chris Amato, J. How
Markov decision processes (MDPs) are often used to model sequential decision problems involving uncertainty under the assumption of centralized control. However, many large, distributed systems do not permit centralized control due to communication limitations (such as cost, latency or corruption). This paper surveys recent work on decentralized control of MDPs in which control of each agent depends on a partial view of the world. We focus on a general framework where there may be uncertainty about the state of the environment, represented as a decentralized partially observable MDP (Dec-POMDP), but consider a number of subclasses with different assumptions about uncertainty and agent independence. In these models, a shared objective function is used, but plans of action must be based on a partial view of the environment. We describe the frameworks, along with the complexity of optimal control and important properties. We also provide an overview of exact and approximate solution methods as well as relevant applications. This survey provides an introduction to what has become an active area of research on these models and their solutions.
{"title":"Decentralized control of partially observable Markov decision processes","authors":"Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Chris Amato, J. How","doi":"10.1109/ICRA.2015.7140035","DOIUrl":"https://doi.org/10.1109/ICRA.2015.7140035","url":null,"abstract":"Markov decision processes (MDPs) are often used to model sequential decision problems involving uncertainty under the assumption of centralized control. However, many large, distributed systems do not permit centralized control due to communication limitations (such as cost, latency or corruption). This paper surveys recent work on decentralized control of MDPs in which control of each agent depends on a partial view of the world. We focus on a general framework where there may be uncertainty about the state of the environment, represented as a decentralized partially observable MDP (Dec-POMDP), but consider a number of subclasses with different assumptions about uncertainty and agent independence. In these models, a shared objective function is used, but plans of action must be based on a partial view of the environment. We describe the frameworks, along with the complexity of optimal control and important properties. We also provide an overview of exact and approximate solution methods as well as relevant applications. This survey provides an introduction to what has become an active area of research on these models and their solutions.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127791332","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-20DOI: 10.1109/CDC.2013.6760101
R. Castro, M. Tanelli, R. Araújo, S. Savaresi
The coordinated control of vehicle actuators is gaining more importance as new platforms are becoming available, with chassis endowed with many different actuators that may help controlling the vehicle motion. Further, in-wheel motors (IWMs) allow using a single system to apply both positive and negative torques at the wheels, which can be actuated independently one from the other. In electric vehicles (EVs), moreover, such a freedom in the actuation mechanisms opens the way to the combined optimization of performance and energy consumption issues. In this paper, the problem of torque allocation for maximizing the vehicle performance in EVs is addressed. The proposed strategy is compared against a benchmark, a-causal optimal solution showing that only a negligible loss of performance is experienced.
{"title":"Torque allocation in electric vehicles with in-wheel motors: A performance-oriented approach","authors":"R. Castro, M. Tanelli, R. Araújo, S. Savaresi","doi":"10.1109/CDC.2013.6760101","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760101","url":null,"abstract":"The coordinated control of vehicle actuators is gaining more importance as new platforms are becoming available, with chassis endowed with many different actuators that may help controlling the vehicle motion. Further, in-wheel motors (IWMs) allow using a single system to apply both positive and negative torques at the wheels, which can be actuated independently one from the other. In electric vehicles (EVs), moreover, such a freedom in the actuation mechanisms opens the way to the combined optimization of performance and energy consumption issues. In this paper, the problem of torque allocation for maximizing the vehicle performance in EVs is addressed. The proposed strategy is compared against a benchmark, a-causal optimal solution showing that only a negligible loss of performance is experienced.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115123017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-13DOI: 10.1109/CDC.2013.6759928
B. Houska, M. E. Villanueva, B. Chachuat
This paper presents a novel algorithm for bounding the reachable set of parametric nonlinear differential equations. This algorithm is based on a first-discretize-then-bound approach to enclose the reachable set via propagation of a Taylor model with ellipsoidal remainder, and it accounts for truncation errors that are inherent to the discretization. In contrast to existing algorithms that proceed in two phases-an a priori enclosure phase, followed by a tightening phase-the proposed algorithm first predicts a continuous-time enclosure and then seeks a maximal step-size for which validity of the predicted enclosure can be established. It is shown that this reversed approach leads to a natural step-size control mechanism, which no longer relies on the availability of an a priori enclosure. Also described in the paper is an open-source implementation of the algorithm in ACADO Toolkit. A simple numerical case study is presented to illustrate the performance and stability of the algorithm.
{"title":"A validated integration algorithm for nonlinear ODEs using Taylor models and ellipsoidal calculus","authors":"B. Houska, M. E. Villanueva, B. Chachuat","doi":"10.1109/CDC.2013.6759928","DOIUrl":"https://doi.org/10.1109/CDC.2013.6759928","url":null,"abstract":"This paper presents a novel algorithm for bounding the reachable set of parametric nonlinear differential equations. This algorithm is based on a first-discretize-then-bound approach to enclose the reachable set via propagation of a Taylor model with ellipsoidal remainder, and it accounts for truncation errors that are inherent to the discretization. In contrast to existing algorithms that proceed in two phases-an a priori enclosure phase, followed by a tightening phase-the proposed algorithm first predicts a continuous-time enclosure and then seeks a maximal step-size for which validity of the predicted enclosure can be established. It is shown that this reversed approach leads to a natural step-size control mechanism, which no longer relies on the availability of an a priori enclosure. Also described in the paper is an open-source implementation of the algorithm in ACADO Toolkit. A simple numerical case study is presented to illustrate the performance and stability of the algorithm.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"269 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122075883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-11DOI: 10.1109/CDC.2013.6760170
Edward N. Hartley, J. Maciejowski
A reconfigurable field-programmable gate array (FPGA)-based predictive controller based on Nesterov's fast gradient method is designed using Simulink and converted to VHDL using Mathworks' HDL Coder. The implementation is verified by application to a spacecraft rendezvous and capture scenario, with communication between the FPGA and a simulation of the relative dynamics occuring over Ethernet. For a problem with 120 decision variables and 240 constraints, computation times of 0.95 ms are achieved with a clock rate of 50 MHz, corresponding to a speed up of more than 2000 over running the algorithm directly on a MicroBlaze microprocessor implemented on the same FPGA.
{"title":"Graphical FPGA design for a predictive controller with application to spacecraft rendezvous","authors":"Edward N. Hartley, J. Maciejowski","doi":"10.1109/CDC.2013.6760170","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760170","url":null,"abstract":"A reconfigurable field-programmable gate array (FPGA)-based predictive controller based on Nesterov's fast gradient method is designed using Simulink and converted to VHDL using Mathworks' HDL Coder. The implementation is verified by application to a spacecraft rendezvous and capture scenario, with communication between the FPGA and a simulation of the relative dynamics occuring over Ethernet. For a problem with 120 decision variables and 240 constraints, computation times of 0.95 ms are achieved with a clock rate of 50 MHz, corresponding to a speed up of more than 2000 over running the algorithm directly on a MicroBlaze microprocessor implemented on the same FPGA.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134118282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-10DOI: 10.1109/CDC.2013.6760648
D. Pucci, T. Hamel, P. Morin, C. Samson
The paper contributes towards the development of a unified control approach for longitudinal aircraft dynamics. It states conditions that allow to adapt the control strategies developed for orientation-independent external forces to the orientation-dependent case. The control strategy presented here is a step to the automatic monitoring of the flight transitions between hovering and cruising for convertible aerial vehicles.
{"title":"Nonlinear control of aerial vehicles subjected to aerodynamic forces","authors":"D. Pucci, T. Hamel, P. Morin, C. Samson","doi":"10.1109/CDC.2013.6760648","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760648","url":null,"abstract":"The paper contributes towards the development of a unified control approach for longitudinal aircraft dynamics. It states conditions that allow to adapt the control strategies developed for orientation-independent external forces to the orientation-dependent case. The control strategy presented here is a step to the automatic monitoring of the flight transitions between hovering and cruising for convertible aerial vehicles.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"117 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122124472","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-10DOI: 10.1109/CDC.2013.6760675
Lina Ye, P. Dague, Farid Nouioua
Predictability is an important system property that determines with certainty the future occurrence of a fault based on a model of the system and a sequence of observations. The existing works dealt with predictability analysis of discrete-event systems in the centralized way. To deal with this important problem in a more efficient way, in this paper, we first propose a new centralized polynomial algorithm, which is inspired from twin plant method for diagnosability checking and more importantly, is adaptable to a distributed framework. Then we show how to extend this algorithm to a distributed one, based on local structure. We first obtain the original predictability information from the faulty component, and then check its consistency in the whole system to decide predictability from a global point of view. In this way, we avoid constructing global structure and thus greatly reduce the search space.
{"title":"Predictability analysis of distributed discrete event systems","authors":"Lina Ye, P. Dague, Farid Nouioua","doi":"10.1109/CDC.2013.6760675","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760675","url":null,"abstract":"Predictability is an important system property that determines with certainty the future occurrence of a fault based on a model of the system and a sequence of observations. The existing works dealt with predictability analysis of discrete-event systems in the centralized way. To deal with this important problem in a more efficient way, in this paper, we first propose a new centralized polynomial algorithm, which is inspired from twin plant method for diagnosability checking and more importantly, is adaptable to a distributed framework. Then we show how to extend this algorithm to a distributed one, based on local structure. We first obtain the original predictability information from the faulty component, and then check its consistency in the whole system to decide predictability from a global point of view. In this way, we avoid constructing global structure and thus greatly reduce the search space.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114717485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-10DOI: 10.1109/CDC.2013.6760933
Stefano Falasca, Massimiliano Gamba, L. Greco, A. Chaillet, A. Bicchi
The possibility to remotely control a plant through a communication network offers strong advantages in terms of installation, flexibility, and maintenance. These advantages come at the price of specific challenges due to the format of the transmitted data, the network nodes scheduling, and the inherent delays. To address these obstacles, we have recently developed a control methodology that exploits the packetized nature of the transmitted data by relying on a model-based prediction of the control signal to be applied. However, this methodology was limited to static controllers. Dynamic control laws indeed require a more careful synchronization between the plant and its remote model. This paper aims at filling this gap by exploiting the transmission of the measurements history, rather than their instantaneous value only.
{"title":"Packet-based dynamic control for nonlinear networked systems","authors":"Stefano Falasca, Massimiliano Gamba, L. Greco, A. Chaillet, A. Bicchi","doi":"10.1109/CDC.2013.6760933","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760933","url":null,"abstract":"The possibility to remotely control a plant through a communication network offers strong advantages in terms of installation, flexibility, and maintenance. These advantages come at the price of specific challenges due to the format of the transmitted data, the network nodes scheduling, and the inherent delays. To address these obstacles, we have recently developed a control methodology that exploits the packetized nature of the transmitted data by relying on a model-based prediction of the control signal to be applied. However, this methodology was limited to static controllers. Dynamic control laws indeed require a more careful synchronization between the plant and its remote model. This paper aims at filling this gap by exploiting the transmission of the measurements history, rather than their instantaneous value only.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127771791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-10DOI: 10.1109/CDC.2013.6760400
C. Cheah, Xiang Li, Xiao Yan, Dong Sun, H. C. Liaw
Optical tweezers are one of the most common and useful tools in non-contact cell manipulation. While several control methods have been developed for cell manipulation using optical tweezers, the control input is commonly treated as the position of the laser beam and open-loop controllers are designed to move the laser source. Investigating the interaction between the robotic manipulator of laser source and biological cells can help us gain understanding into the dynamic manipulation problem using optical tweezers. However, the interaction between the cell dynamics and the manipulator dynamics leads to a fourth-order overall dynamics, and the use of high-order derivatives of the state variables is usually required in the overall control input. In this paper, a simple setpoint control method is proposed for optical manipulation of biological cells. The proposed method is able to manipulate the trapped cell to a desired position without using the high-order derivatives of the state variables such as acceleration and jerk. The stability of closed-loop system is analyzed by using LaSalle's invariance principle, with the consideration of the dynamics of both the cell and the robotic manipulator. The proposed control method is simple and easy to implement. Both simulation and experimental results are presented to illustrate the performance of the proposed control method.
{"title":"A simple setpoint controller for dynamic manipulation of biological cells using optical tweezers","authors":"C. Cheah, Xiang Li, Xiao Yan, Dong Sun, H. C. Liaw","doi":"10.1109/CDC.2013.6760400","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760400","url":null,"abstract":"Optical tweezers are one of the most common and useful tools in non-contact cell manipulation. While several control methods have been developed for cell manipulation using optical tweezers, the control input is commonly treated as the position of the laser beam and open-loop controllers are designed to move the laser source. Investigating the interaction between the robotic manipulator of laser source and biological cells can help us gain understanding into the dynamic manipulation problem using optical tweezers. However, the interaction between the cell dynamics and the manipulator dynamics leads to a fourth-order overall dynamics, and the use of high-order derivatives of the state variables is usually required in the overall control input. In this paper, a simple setpoint control method is proposed for optical manipulation of biological cells. The proposed method is able to manipulate the trapped cell to a desired position without using the high-order derivatives of the state variables such as acceleration and jerk. The stability of closed-loop system is analyzed by using LaSalle's invariance principle, with the consideration of the dynamics of both the cell and the robotic manipulator. The proposed control method is simple and easy to implement. Both simulation and experimental results are presented to illustrate the performance of the proposed control method.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"80 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124608480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2013-12-10DOI: 10.1109/CDC.2013.6760545
Thang Nguyen-Tien, S. G. Khan, C. Edwards, G. Herrmann, R. Harniman, S. Burgess, M. Antognozzi, M. Miles
In this paper, a scheme employing a super-twisting sliding mode observer is proposed to reconstruct the unknown shear forces at the tip of the vertically oriented cantilever (VOC) which is the key component of Bristol University's Vertically Oriented Probe Microscope (VOPM). The VOPM is unique in the sense that the distance of the VOC tip from the specimen can be measured using the known amplitude of the horizontal excitation of the cantilever. This amplitude is directly influenced by the interaction of the VOC with the fluid layer above the biological specimen. The shear forces caused by the fluid interaction increase with the proximity of the tip to the specimen. Thus, knowledge of the shear forces is another important measure of the closeness of the VOC tip to the specimen. Simulation examples show the feasibility of its eventual real-time application.
{"title":"Shear force reconstruction in a vertically oriented probe microscope using a super-twisting observer","authors":"Thang Nguyen-Tien, S. G. Khan, C. Edwards, G. Herrmann, R. Harniman, S. Burgess, M. Antognozzi, M. Miles","doi":"10.1109/CDC.2013.6760545","DOIUrl":"https://doi.org/10.1109/CDC.2013.6760545","url":null,"abstract":"In this paper, a scheme employing a super-twisting sliding mode observer is proposed to reconstruct the unknown shear forces at the tip of the vertically oriented cantilever (VOC) which is the key component of Bristol University's Vertically Oriented Probe Microscope (VOPM). The VOPM is unique in the sense that the distance of the VOC tip from the specimen can be measured using the known amplitude of the horizontal excitation of the cantilever. This amplitude is directly influenced by the interaction of the VOC with the fluid layer above the biological specimen. The shear forces caused by the fluid interaction increase with the proximity of the tip to the specimen. Thus, knowledge of the shear forces is another important measure of the closeness of the VOC tip to the specimen. Simulation examples show the feasibility of its eventual real-time application.","PeriodicalId":415568,"journal":{"name":"52nd IEEE Conference on Decision and Control","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128201203","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：