Applied Mathematics and Optimization最新文献

We study the (weak) equilibrium problem arising from the problem of optimally stopping a one-dimensional diffusion subject to an expectation constraint on the time until stopping. The weak equilibrium problem is realized with a set of randomized but purely state dependent stopping times as admissible strategies. We derive a verification theorem and necessary conditions for equilibria, which together basically characterize all equilibria. Furthermore, additional structural properties of equilibria are obtained to feed a possible guess-and-verify approach, which is then illustrated by an example.

This paper studies the q-learning, recently coined as the continuous time counterpart of Q-learning by Jia and Zhou (J Mach Learn Res 24:1–61, 2023), for continuous time mean-field control problems in the setting of entropy-regularized reinforcement learning. In contrast to the single agent’s control problem in Jia and Zhou (J Mach Learn Res 24:1–61, 2023), we reveal that two different q-functions naturally arise in mean-field control problems: (i) the integrated q-function (denoted by q) as the first-order approximation of the integrated Q-function introduced in Gu et al. (Oper Res 71(4):1040–1054, 2023), which can be learnt by a weak martingale condition using all test policies; and (ii) the essential q-function (denoted by (q_e)) that is employed in the policy improvement iterations. We show that two q-functions are related via an integral representation. Based on the weak martingale condition and our proposed searching method of test policies, some model-free learning algorithms are devised. In two examples, one in LQ control framework and one beyond LQ control framework, we can obtain the exact parameterization of the optimal value function and q-functions and illustrate our algorithms with simulation experiments.

We propose a comprehensive framework for policy gradient methods tailored to continuous time reinforcement learning. This is based on the connection between stochastic control problems and randomised problems, enabling applications across various classes of Markovian continuous time control problems, beyond diffusion models, including e.g. regular, impulse and optimal stopping/switching problems. By utilizing change of measure in the control randomisation technique, we derive a new policy gradient representation for these randomised problems, featuring parametrised intensity policies. We further develop actor-critic algorithms specifically designed to address general Markovian stochastic control issues. Our framework is demonstrated through its application to optimal switching problems, with two numerical case studies in the energy sector focusing on real options.

Coarse correlated equilibria (CCE) are a good alternative to Nash equilibria (NE), as they arise more naturally as outcomes of learning algorithms and as they may exhibit higher payoffs than NE. CCEs include a device which allows players’ strategies to be correlated without any cooperation, only through information sent by a mediator. We develop a methodology to concretely compute mean field CCEs in a linear-quadratic mean field game (MFG) framework. We compare their performance to mean field control solutions and mean field NE (usually named MFG solutions). Our approach is implemented in the mean field version of an emission abatement game between greenhouse gas emitters. In particular, we exhibit a simple and tractable class of mean field CCEs which allows to outperform very significantly the mean field NE payoff and abatement levels, bridging the gap between the mean field NE and the social optimum obtained by mean field control.

In this paper, we first give the definition of uniformly differentiable set and give the definitions of sets (P(A,eta , r)) and (P_{A,delta }(f)). Secondly, we prove that if the set A is bounded closed convex set, then A is uniformly differentiable if and only if for any (varepsilon , eta , r>0), there exists (delta =delta (varepsilon ,eta ,r )>0) such that (Vert x-yVert <varepsilon ) whenever (fin P(A,eta , r)), (yin P_{A,delta }(f)) and (xin P_{A}(f)). Moreover, we also prove that if A is a bounded closed convex set in a finite-dimensional space X, then A is differentiable if and only if A is uniformly differentiable. Finally, we give some examples of uniformly differentiable set. Therefore, we extend some conclusions (SIAM J. Optim. Vol. 30, No. 1, pp. 490–512).

In this paper, we investigate the initial boundary value problem of a three-species spatial food chain model with nonlinear taxis sensitivity in a bounded domain (Omega subset {mathbb {R}}^2) with a smooth boundary and homogeneous Neumann boundary conditions. By establishing a new energy functional, we demonstrate the global boundedness of classical solutions under appropriate initial data.

This paper studies finite-time optimal consumption–investment problems with power, logarithmic and exponential utilities, in a regime switching market with random coefficients and possibly subject to non-convex constraints. Compared to the existing models, one distinguish feature of our model is that the trading constraints put on the consumption and investment strategies are coupled together in the cases of power and logarithmic utilities, leading to new technical challenges. We provide explicit optimal consumption–investment strategies and optimal values for these consumption–investment problems, which are expressed in terms of the solutions to some multi-dimensional diagonally quadratic backward stochastic differential equation (BSDE) and linear BSDE with unbound coefficients. These BSDEs are new in the literature and solving them is one of the main theoretical contributions of this paper. We accomplish the latter by applying the truncation, approximation technique to get some a priori uniformly lower and upper bounds for their solutions.

In this paper, we study optimality conditions for a class of control problems driven by a cylindrical Wiener process, resulting in a stochastic maximum principle in differential form. The control acts on both the drift and volatility, potentially as unbounded operators, allowing for SPDEs with boundary control and/or noise. Through the factorization method, we establish a regularity property for the state equation, which, by duality, extends to the backward costate equation, understood in the transposition sense. Finally, we show that the cost functional is Gâteaux differentiable, with its derivative represented by the costate. The optimality condition is derived using results from set-valued analysis.

We study zero-sum stochastic games between a singular controller and a stopper when the (state-dependent) diffusion matrix of the underlying controlled diffusion process is degenerate. In particular, we show the existence of a value for the game and determine an optimal strategy for the stopper. The degeneracy of the dynamics prevents the use of analytical methods based on solution in Sobolev spaces of suitable variational problems. Therefore we adopt a probabilistic approach based on a perturbation of the underlying diffusion modulated by a parameter (gamma >0). For each (gamma >0) the approximating game is non-degenerate and admits a value (u^gamma ) and an optimal strategy (tau ^gamma _*) for the stopper. Letting (gamma rightarrow 0) we prove convergence of (u^gamma ) to a function v, which identifies the value of the original game. We also construct explicitly optimal stopping times (theta ^gamma _*) for (u^gamma ), related but not equal to (tau ^gamma _*), which converge almost surely to an optimal stopping time (theta _*) for the game with degenerate dynamics.

In this paper the resolvent family ({S_{alpha ,beta }(t)}_{tge 0}subset mathcal {L}(X,Y)) generated by an almost sectorial operator A,  where (alpha ,beta >0,) X, Y are complex Banach spaces and its Laplace transform satisfies (hat{S}_{alpha ,beta }(z)=z^{alpha -beta }(z^alpha -A)^{-1}) is studied. This family of operators allows to write the solution to an abstract initial value problem of time fractional type of order (1<alpha <2) as a variation of constants formula. Estimates of the norm (Vert S_{alpha ,beta }(t)Vert ,) as well as the continuity and compactness of (S_{alpha ,beta }(t)), for (t>0), are shown. Moreover, the Hölder regularity of its solutions is also studied.