Foundations of Computational Mathematics最新文献

Gradient-based learning in multi-layer neural networks displays a number of striking features. In particular, the decrease rate of empirical risk is non-monotone even after averaging over large batches. Long plateaus in which one observes barely any progress alternate with intervals of rapid decrease. These successive phases of learning often take place on very different time scales. Finally, models learnt in an early phase are typically ‘simpler’ or ‘easier to learn’ although in a way that is difficult to formalize. Although theoretical explanations of these phenomena have been put forward, each of them captures at best certain specific regimes. In this paper, we study the gradient flow dynamics of a wide two-layer neural network in high-dimension, when data are distributed according to a single-index model (i.e., the target function depends on a one-dimensional projection of the covariates). Based on a mixture of new rigorous results, non-rigorous mathematical derivations, and numerical simulations, we propose a scenario for the learning dynamics in this setting. In particular, the proposed evolution exhibits separation of timescales and intermittency. These behaviors arise naturally because the population gradient flow can be recast as a singularly perturbed dynamical system.

The exotic aromatic Butcher series were originally introduced for the calculation of order conditions for the high order numerical integration of ergodic stochastic differential equations in (mathbb {R} ^d) and on manifolds. We prove in this paper that exotic aromatic B-series satisfy a universal geometric property, namely that they are characterised by locality and equivariance with respect to orthogonal changes of coordinates. This characterisation confirms that exotic aromatic B-series are a fundamental geometric object that naturally generalises aromatic B-series and B-series, as they share similar equivariance properties. In addition, we provide a classification of the main subsets of the exotic aromatic B-series, in particular the exotic B-series, using different equivariance properties. Along the analysis, we present a generalised definition of exotic aromatic trees, dual vector fields, and we explore the impact of degeneracies on the classification.

We introduce a new class of numerical schemes which allow for low-regularity approximations to the expectation ( mathbb {E}(|u_{k}(t, v^{eta })|^2)), where (u_k) denotes the k-th Fourier coefficient of the solution u of the dispersive equation and ( v^{eta }(x) ) the associated random initial data. This quantity plays an important role in physics, in particular in the study of wave turbulence where one needs to adopt a statistical approach in order to obtain deep insight into the generic long-time behaviour of solutions to dispersive equations. Our new class of schemes is based on Wick’s theorem and Feynman diagrams together with a resonance-based discretisation (Bruned and Schratz in Forum Math Pi 10:E2, 2022) set in a more general context: we introduce a novel combinatorial structure called paired decorated forests which are two decorated trees whose decorations on the leaves come in pair. The character of the scheme draws its inspiration from the treatment of singular stochastic partial differential equations via regularity structures. In contrast to classical approaches, we do not discretise the PDE itself, but rather its expectation. This allows us to heavily exploit the optimal resonance structure and underlying gain in regularity on the finite dimensional (discrete) level.

Rapid convergence of the shifted QR algorithm on symmetric matrices was shown more than 50 years ago. Since then, despite significant interest and its practical relevance, an understanding of the dynamics and convergence properties of the shifted QR algorithm on nonsymmetric matrices has remained elusive. We introduce a new family of shifting strategies for the Hessenberg shifted QR algorithm. We prove that when the input is a diagonalizable Hessenberg matrix H of bounded eigenvector condition number (kappa _V(H))—defined as the minimum condition number of V over all diagonalizations (VDV^{-1}) of H—then the shifted QR algorithm with a certain strategy from our family is guaranteed to converge rapidly to a Hessenberg matrix with a zero subdiagonal entry, in exact arithmetic. Our convergence result is nonasymptotic, showing that the geometric mean of certain subdiagonal entries of H decays by a fixed constant in every QR iteration. The arithmetic cost of implementing each iteration of our strategy scales roughly logarithmically in the eigenvector condition number (kappa _V(H)), which is a measure of the nonnormality of H. The key ideas in the design and analysis of our strategy are: (1) we are able to precisely characterize when a certain shifting strategy based on Ritz values stagnates. We use this information to design certain “exceptional shifts” which are guaranteed to escape stagnation whenever it occurs. (2) We use higher degree shifts (of degree roughly (log kappa _V(H))) to dampen transient effects due to nonnormality, allowing us to treat nonnormal matrices in a manner similar to normal matrices.

We consider the Navier–Stokes–Fourier system governing the motion of a general compressible, heat conducting, Newtonian fluid driven by random initial/boundary data. Convergence of the stochastic collocation and Monte Carlo numerical methods is shown under the hypothesis that approximate solutions are bounded in probability. Abstract results are illustrated by numerical experiments for the Rayleigh–Bénard convection problem.

We describe fast algorithms for approximating the connection coefficients between a family of orthogonal polynomials and another family with a polynomially or rationally modified measure. The connection coefficients are computed via infinite-dimensional banded matrix factorizations and may be used to compute the modified Jacobi matrices all in linear complexity with respect to the truncation degree. A family of orthogonal polynomials with modified classical weights is constructed that support banded differentiation matrices, enabling sparse spectral methods with modified classical orthogonal polynomials. We present several applications and numerical experiments using an open source implementation which make direct use of these results.

On the reference tetrahedron (K), we construct, for each (k in {mathbb {N}}_0), a right inverse for the trace operator (u mapsto (u, partial _{textbf{n}} u, ldots , partial _{textbf{n}}^k u)|_{partial K}). The operator is stable as a mapping from the trace space of (W^{s, p}(K)) to (W^{s, p}(K)) for all (p in (1, infty )) and (s in (k+1/p, infty )). Moreover, if the data is the trace of a polynomial of degree (N in {mathbb {N}}_0), then the resulting lifting is a polynomial of degree N. One consequence of the analysis is a novel characterization for the range of the trace operator.

Classically, the continuous-time Langevin diffusion converges exponentially fast to its stationary distribution (pi ) under the sole assumption that (pi ) satisfies a Poincaré inequality. Using this fact to provide guarantees for the discrete-time Langevin Monte Carlo (LMC) algorithm, however, is considerably more challenging due to the need for working with chi-squared or Rényi divergences, and prior works have largely focused on strongly log-concave targets. In this work, we provide the first convergence guarantees for LMC assuming that (pi ) satisfies either a Latała–Oleszkiewicz or modified log-Sobolev inequality, which interpolates between the Poincaré and log-Sobolev settings. Unlike prior works, our results allow for weak smoothness and do not require convexity or dissipativity conditions.

A large toolbox of numerical schemes for dispersive equations has been established, based on different discretization techniques such as discretizing the variation-of-constants formula (e.g., exponential integrators) or splitting the full equation into a series of simpler subproblems (e.g., splitting methods). In many situations these classical schemes allow a precise and efficient approximation. This, however, drastically changes whenever non-smooth phenomena enter the scene such as for problems at low regularity and high oscillations. Classical schemes fail to capture the oscillatory nature of the solution, and this may lead to severe instabilities and loss of convergence. In this article we review a new class of resonance-based schemes. The key idea in the construction of the new schemes is to tackle and deeply embed the underlying nonlinear structure of resonances into the numerical discretization. As in the continuous case, these terms are central to structure preservation and offer the new schemes strong properties at low regularity.

Motivated by the problem of matching vertices in two correlated Erdős-Rényi graphs, we study the problem of matching two correlated Gaussian Wigner matrices. We propose an iterative matching algorithm, which succeeds in polynomial time as long as the correlation between the two Gaussian matrices does not vanish. Our result is the first polynomial time algorithm that solves a graph matching type of problem when the correlation is an arbitrarily small constant.