Journal of Optimization Theory and Applications最新文献

We analyze the fulfillment of the simultaneous diagonalization (SD via congruence) property for any two real matrices, and develop sufficient conditions expressed in different way to those appeared in the last few years. These conditions are established under a different perspective, and in any case, they supplement and clarify other similar results published elsewhere. Following our point of view reflected in a previous work, we offer some necessary and sufficient conditions, different in nature to those in Jiang and Li (SIAM J Optim 26:1649–1668, 2016), for SD: roughly speaking our approach is more geometric and needs to compute images and kernels of matrices; whereas that in Jiang and Li (SIAM J Optim 26:1649–1668, 2016) requires to compute determinant and canonical forms. The bidimensional situation is particularly analyzed, providing new more precise characterizations than those in higher dimension and joint those given earlier by the authors. In addition, we also establish the connection of our characterization of SD with that provided in Jiang and Li (SIAM J Optim 26:1649–1668, 2016).

In this paper, we develop an algorithm for federated principal component analysis (PCA) with emphases on both communication efficiency and data privacy. Generally speaking, federated PCA algorithms based on direct adaptations of classic iterative methods, such as simultaneous subspace iterations, are unable to preserve data privacy, while algorithms based on variable-splitting and consensus-seeking, such as alternating direction methods of multipliers (ADMM), lack in communication-efficiency. In this work, we propose a novel consensus-seeking formulation by equalizing subspaces spanned by splitting variables instead of equalizing variables themselves, thus greatly relaxing feasibility restrictions and allowing much faster convergence. Then we develop an ADMM-like algorithm with several special features to make it practically efficient, including a low-rank multiplier formula and techniques for treating subproblems. We establish that the proposed algorithm can better protect data privacy than classic methods adapted to the federated PCA setting. We derive convergence results, including a worst-case complexity estimate, for the proposed ADMM-like algorithm in the presence of the nonlinear equality constraints. Extensive empirical results are presented to show that the new algorithm, while enhancing data privacy, requires far fewer rounds of communication than existing peer algorithms for federated PCA.

The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates has only been shown to be scale-invariant only under specific assumptions on the problem structure. The second shortfall is that the fast convergence rates of second-order methods have only been established by making assumptions regarding the input data. In this paper, we propose a randomized Newton method for self-concordant functions to address both shortfalls. We propose a Self-concordant Iterative-minimization-Galerkin-based Multilevel Algorithm (SIGMA) and establish its super-linear convergence rate using the theory of self-concordant functions. Our analysis is based on the connections between multigrid optimization methods, and the role of coarse-grained or reduced-order models in the computation of search directions. We take advantage of the insights from the analysis to significantly improve the performance of second-order methods in machine learning applications. We report encouraging initial experiments that suggest SIGMA outperforms other state-of-the-art sub-sampled/sketched Newton methods for both medium and large-scale problems.

The efficient optimization method for locally Lipschitz continuous multiobjective optimization problems from Gebken and Peitz (J Optim Theory Appl 188:696–723, 2021) is extended from finite-dimensional problems to general Hilbert spaces. The method iteratively computes Pareto critical points, where in each iteration, an approximation of the Clarke subdifferential is computed in an efficient manner and then used to compute a common descent direction for all objective functions. To prove convergence, we present some new optimality results for nonsmooth multiobjective optimization problems in Hilbert spaces. Using these, we can show that every accumulation point of the sequence generated by our algorithm is Pareto critical under common assumptions. Computational efficiency for finding Pareto critical points is numerically demonstrated for multiobjective optimal control of an obstacle problem.

In this paper we investigate a class of stochastic absolute value equations (SAVE). After establishing the relationship between the stochastic linear complementarity problem and SAVE, we study the expected residual minimization (ERM) formulation for SAVE and its Monte Carlo sample average approximation. In particular, we show that the ERM problem and its sample average approximation have optimal solutions under the condition of (R_0) pair, and the optimal value of the sample average approximation has uniform exponential convergence. Furthermore, we prove that the solutions to the ERM problem are robust for SAVE. For a class of SAVE problems, we use its special structure to construct a smooth residual and further study the convergence of the stationary points. Finally, a smoothing gradient method is proposed by simultaneously considering sample sampling and smooth techniques for solving SAVE. Numerical experiments exhibit the effectiveness of the method.

In this study, first-order necessary optimality conditions, in the form of a weak maximum principle, are derived for discrete optimal control problems with mixed equality and inequality constraints. Such conditions are achieved by using the Dubovitskii–Milyutin formalism approach. Nondegenerate conditions are obtained under the constant rank of the subspace component (CRSC) constraint qualification, which is an important generalization of both the Mangasarian–Fromovitz and constant rank constraint qualifications. Beyond its theoretical significance, CRSC has practical importance because it is closely related to the formulation of optimization algorithms. In addition, an instance of a discrete optimal control problem is presented in which CRSC holds while other stronger regularity conditions do not.

Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms bypass so-called strict saddle points of the loss function. However, in many modern machine learning applications, the required regularity conditions are not satisfied. In this paper, we prove a variant of the relevant dynamical systems result, a center-stable manifold theorem, in which we relax some of the regularity requirements. We explore its relevance for various machine learning tasks, with a particular focus on shallow rectified linear unit (ReLU) and leaky ReLU networks with scalar input. Building on a detailed examination of critical points of the square integral loss function for shallow ReLU and leaky ReLU networks relative to an affine target function, we show that gradient descent circumvents most saddle points. Furthermore, we prove convergence to global minima under favourable initialization conditions, quantified by an explicit threshold on the limiting loss.

This paper addresses the design of attitude controller for a fixed-wing unmanned aerial vehicle. To address the complexity of the coupled nonlinear model of a fixed-wing aircraft, this paper introduces a Fractional-Order Type-2 Fuzzy PID (FOTFPID) controller. The adoption of interval valued type-2 fuzzy sets, as an extension of conventional fuzzy sets, has endowed decision makers with the ability to assign membership and non-membership values as intervals. This enhanced capability facilitates more resilient decision-making processes. The Bat optimization algorithm is also employed to fine-tune the membership functions, scaling factors, and primary controller parameters, aiming to minimize the integrated absolute error index. Numerical simulations are conducted to demonstrate effectiveness of the proposed controllers in comparison to classical PID controllers, while subjecting the aircraft system to various disturbance conditions.

Two new accelerated fixed-time stable dynamic systems are proposed for solving absolute value equations (AVEs): (Ax-|x|-b=0). Under some mild conditions, the equilibrium point of the proposed dynamic systems is completely equivalent to the solution of the AVEs under consideration. Meanwhile, we have introduced a new relatively tighter global error bound for the AVEs. Leveraging this finding, we have separately established the globally fixed-time stability of the proposed methods, along with providing the conservative settling-time for each method. Compared with some existing state-of-the-art dynamical methods, preliminary numerical experiments show the effectiveness of our methods in solving the AVEs.

In this paper, we establish through an openness condition the metric subregularity of a multimapping with normal (omega (cdot ))-regularity of either the graph or values. Various preservation results for prox-regular and subsmooth sets are also provided.