{"title":"非对称高阶荷尔德平滑性和均匀凸性下的严格下界","authors":"Site Bai, Brian Bullins","doi":"arxiv-2409.10773","DOIUrl":null,"url":null,"abstract":"In this paper, we provide tight lower bounds for the oracle complexity of\nminimizing high-order H\\\"older smooth and uniformly convex functions.\nSpecifically, for a function whose $p^{th}$-order derivatives are H\\\"older\ncontinuous with degree $\\nu$ and parameter $H$, and that is uniformly convex\nwith degree $q$ and parameter $\\sigma$, we focus on two asymmetric cases: (1)\n$q > p + \\nu$, and (2) $q < p+\\nu$. Given up to $p^{th}$-order oracle access,\nwe establish worst-case oracle complexities of $\\Omega\\left( \\left(\n\\frac{H}{\\sigma}\\right)^\\frac{2}{3(p+\\nu)-2}\\left(\n\\frac{\\sigma}{\\epsilon}\\right)^\\frac{2(q-p-\\nu)}{q(3(p+\\nu)-2)}\\right)$ with a\ntruncated-Gaussian smoothed hard function in the first case and\n$\\Omega\\left(\\left(\\frac{H}{\\sigma}\\right)^\\frac{2}{3(p+\\nu)-2}+\n\\log^2\\left(\\frac{\\sigma^{p+\\nu}}{H^q}\\right)^\\frac{1}{p+\\nu-q}\\right)$ in the\nsecond case, for reaching an $\\epsilon$-approximate solution in terms of the\noptimality gap. Our analysis generalizes previous lower bounds for functions\nunder first- and second-order smoothness as well as those for uniformly convex\nfunctions, and furthermore our results match the corresponding upper bounds in\nthe general setting.","PeriodicalId":501340,"journal":{"name":"arXiv - STAT - Machine Learning","volume":"89 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Tight Lower Bounds under Asymmetric High-Order Hölder Smoothness and Uniform Convexity\",\"authors\":\"Site Bai, Brian Bullins\",\"doi\":\"arxiv-2409.10773\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we provide tight lower bounds for the oracle complexity of\\nminimizing high-order H\\\\\\\"older smooth and uniformly convex functions.\\nSpecifically, for a function whose $p^{th}$-order derivatives are H\\\\\\\"older\\ncontinuous with degree $\\\\nu$ and parameter $H$, and that is uniformly convex\\nwith degree $q$ and parameter $\\\\sigma$, we focus on two asymmetric cases: (1)\\n$q > p + \\\\nu$, and (2) $q < p+\\\\nu$. Given up to $p^{th}$-order oracle access,\\nwe establish worst-case oracle complexities of $\\\\Omega\\\\left( \\\\left(\\n\\\\frac{H}{\\\\sigma}\\\\right)^\\\\frac{2}{3(p+\\\\nu)-2}\\\\left(\\n\\\\frac{\\\\sigma}{\\\\epsilon}\\\\right)^\\\\frac{2(q-p-\\\\nu)}{q(3(p+\\\\nu)-2)}\\\\right)$ with a\\ntruncated-Gaussian smoothed hard function in the first case and\\n$\\\\Omega\\\\left(\\\\left(\\\\frac{H}{\\\\sigma}\\\\right)^\\\\frac{2}{3(p+\\\\nu)-2}+\\n\\\\log^2\\\\left(\\\\frac{\\\\sigma^{p+\\\\nu}}{H^q}\\\\right)^\\\\frac{1}{p+\\\\nu-q}\\\\right)$ in the\\nsecond case, for reaching an $\\\\epsilon$-approximate solution in terms of the\\noptimality gap. Our analysis generalizes previous lower bounds for functions\\nunder first- and second-order smoothness as well as those for uniformly convex\\nfunctions, and furthermore our results match the corresponding upper bounds in\\nthe general setting.\",\"PeriodicalId\":501340,\"journal\":{\"name\":\"arXiv - STAT - Machine Learning\",\"volume\":\"89 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-16\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - STAT - Machine Learning\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.10773\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - STAT - Machine Learning","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.10773","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Tight Lower Bounds under Asymmetric High-Order Hölder Smoothness and Uniform Convexity
In this paper, we provide tight lower bounds for the oracle complexity of
minimizing high-order H\"older smooth and uniformly convex functions.
Specifically, for a function whose $p^{th}$-order derivatives are H\"older
continuous with degree $\nu$ and parameter $H$, and that is uniformly convex
with degree $q$ and parameter $\sigma$, we focus on two asymmetric cases: (1)
$q > p + \nu$, and (2) $q < p+\nu$. Given up to $p^{th}$-order oracle access,
we establish worst-case oracle complexities of $\Omega\left( \left(
\frac{H}{\sigma}\right)^\frac{2}{3(p+\nu)-2}\left(
\frac{\sigma}{\epsilon}\right)^\frac{2(q-p-\nu)}{q(3(p+\nu)-2)}\right)$ with a
truncated-Gaussian smoothed hard function in the first case and
$\Omega\left(\left(\frac{H}{\sigma}\right)^\frac{2}{3(p+\nu)-2}+
\log^2\left(\frac{\sigma^{p+\nu}}{H^q}\right)^\frac{1}{p+\nu-q}\right)$ in the
second case, for reaching an $\epsilon$-approximate solution in terms of the
optimality gap. Our analysis generalizes previous lower bounds for functions
under first- and second-order smoothness as well as those for uniformly convex
functions, and furthermore our results match the corresponding upper bounds in
the general setting.