Proceedings of the ... American Control Conference. American Control Conference最新文献

In environments with uncertain dynamics, synthesis of optimal control policies mandates exploration. The applicability of classical learning algorithms to real-world problems is often limited by the number of time steps required for learning the environment model. Given some local side information about the differences in transition probabilities of the states, potentially obtained from the agent's onboard sensors, we generalize the idea of indirect sampling for accelerated learning to propose an algorithm that balances between exploration and exploitation. We formalize this idea by introducing the notion of the value of information in the context of a Markov decision process with unknown transition probabilities, as a measure of the expected improvement in the agent's current estimate of transition probabilities by taking a particular action. By exploiting available local side information and maximizing the estimated value of learned information at each time step, we accelerate the learning process and subsequent synthesis of the optimal control policy. Further, we define the notion of agent safety, a vital consideration for physical systems, in the context of our problem. Under certain assumptions, we provide guarantees on the safety of an agent exploring with our algorithm that exploits local side information. We illustrate agent safety and the improvement in learning speed using numerical experiments in the setting of a Mars rover, with data from onboard sensors acting as the local side information.

Single Particle Tracking (SPT) is a powerful class of tools for analyzing the dynamics of individual biological macromolecules moving inside living cells. The acquired data is typically in the form of a sequence of camera images that are then post-processed to reveal details about the motion. In this work, we develop a local time-varying estimation algorithm for estimating motion model parameters from the data considering nonlinear observations. Our approach uses several well-known existing tools, namely the Expectation Maximization (EM) algorithm combined with an Unscented Kalman filter (UKF) and an Unscented Rauch-Tung-Striebel smoother (URTSS), and applies them to the time-varying case through a sliding window methodology. Due to the shot noise characteristics of the photon generation process, this model uses a Poisson distribution to capture the measurement noise inherent in imaging. In order to apply our time-varying approach to the UKF, we first need to transform the measurements into a model with additive Gaussian noise. This is carried out using a variance stabilizing transform. Results from simulations show that our approach is successful in tracing time-varying diffusion constants at a range of physically relevant signal levels. We also discuss the initialization for the EM algorithm based on the available data.

Single particle tracking is a powerful tool for studying and understanding the motions of biological macromolecules integral to cellular processes. In the past three decades there has been continuous and rapid development of these techniques in both optical microscope design and in algorithms to estimate the statistics and positions of the molecule's trajectory. Although there has been great progress, comparison between different microscope configurations and estimation algorithms has been difficult beyond simulated data. In this paper we explore using a piezo actuated microscope stage to reproduce Brownian motion. Our goal is to use this as a tool to test performance of single particle tracking optical microscopes and estimation algorithms. In this study, Monte Carlo simulations were used to assess the ability of piezo actuated microscope stages for reproducing Brownian motion. Surprisingly, the dynamics of the stage together with configuration of the system allow for preservation of the Brownian motion statistics. Further, feed forward model inverse control allows for low error tracking of Brownian motion trajectories over a wide range of diffusion constants, varying stage response times, and trajectory discrete time steps. These results show great promise in using a piezo actuated microscope stage for testing single particle tracking experimental setups.

Cellular reprogramming is traditionally accomplished through an open loop (OL) control approach, wherein key transcription factors (TFs) are injected in cells to steer the state of the pluripotency (PL) gene regulatory network (GRN), as encoded by TFs concentrations, to the pluripotent state. Due to the OL nature of this approach, the concentration of TFs cannot be accurately controlled. Recently, a closed loop (CL) feedback control strategy was proposed to overcome this problem with promising theoretical results. However, previous analyses of the controller were based on deterministic models. It is well known that cellular systems are characterized by substantial stochasticity, especially when molecules are in low copy number as it is the case in reprogramming problems wherein the gene copy number is usually one or two. Hence, in this paper, we analyze the Chemical Master Equation (CME) for the reaction model of the PL GRN with and without the feedback controller. We computationally and analytically investigate the performance of the controller in biologically relevant parameter regimes where stochastic effects dictate system dynamics. Our results indicate that the feedback control approach still ensures reprogramming even when both the PL GRN and the controller are stochastic.

Combination Antiretroviral Therapy (cART) consists of a cocktail of drugs administered to HIV-infected patients that can suppress the amount of HIV in the patient's blood plasma to an undetectable level. Our previous work has suggested that some HIV-infected patients, despite being placed on cART, can still have ongoing viral replication occurring in self-sustaining inflamed lymph node follicle sanctuary sites. Spatial models of the putative sites show that inflammation is a necessary condition for ongoing HIV replication. In this study, we model the hypothesis that ongoing HIV replication may provide a sufficiently strong pro-inflammatory signal to maintain inflammation levels consistent with continued HIV replication. A system of ordinary differential equations integrated with a reactive-diffusion system is used to model the HIV dynamics and the diameter of a lymph node follicle as a function of time and external influence. The estimates of the parameters in our model come from prior data when available. The results of our study show that these dynamics have two stable steady-state solutions, one with low inflammation and no ongoing HIV replication in the site, and one with high inflammation and high levels of ongoing HIV replication in the site. We furthermore show that the system can transition between the two outcomes in response to a transient exogenous addition of pro-inflammatory signaling, consistent with the antigenic stimulus of a secondary infection. The spatial isolation of the sites results in a low viral load in the blood plasma for both conditions.

In this paper, we propose a general approach to derive runtime enforcement implementations for multi-agent systems, called shields, from temporal logical specifications. Each agent of the multi-agent system is monitored, and if needed corrected, by the shield, such that a global specification is always satisfied. The different ways of how a shield can interfere with each agent in the system in case of an error introduces the need for quantitative objectives. This work is the first to discuss the shield synthesis problem with quantitative objectives. We provide several cost functions that are utilized in the multi-agent setting and provide methods for the synthesis of cost-optimal shields and fair shields, under the given assumptions on the multi-agent system. We demonstrate the applicability of our approach via a detailed case study on UAV mission planning for warehouse logistics and simulating the shielded multi-agent system on ROS/Gazebo.