D. Durfee, John Peebles, Richard Peng, Anup B. Rao
We show variants of spectral sparsification routines can preserve the totalspanning tree counts of graphs, which by Kirchhoffs matrix-tree theorem, isequivalent to determinant of a graph Laplacian minor, or equivalently, of any SDDM matrix. Our analyses utilizes this combinatorial connection to bridge between statisticalleverage scores / effective resistances and the analysis of random graphsby [Janson, Combinatorics, Probability and Computing 94]. This leads to a routine that in quadratic time, sparsifies a graph down to aboutn^(1.5) edges in ways that preserve both the determinant and the distributionof spanning trees (provided the sparsified graph is viewed as a random object). Extending this algorithm to work with Schur complements and approximateCholesky factorizations leads to algorithms for counting andsampling spanning trees which are nearly optimal for dense graphs.We give an algorithm that computes a (1 +/- δ) approximation to the determinantof any SDDM matrix with constant probability in about n^2 / δ^2 time. This is the first routine for graphs that outperforms general-purpose routines for computingdeterminants of arbitrary matrices. We also give an algorithm that generates in about n^2 / δ^2 time a spanning tree ofa weighted undirected graph from a distribution with total variationdistance of δ from the w-uniform distribution.
我们展示了谱稀疏化例程的变体可以保留图的总生成树计数，根据Kirchhoffs矩阵树定理，它等价于图拉普拉斯次矩阵的行列式，或等价于任何SDDM矩阵的行列式。我们的分析利用这种组合连接在统计平均分数/有效阻力和随机图分析之间架起一座桥梁[Janson, Combinatorics, Probability and Computing 94]。这导致了一个例程，在二次时间内，以保留生成树的行列式和分布的方式将图稀疏化到大约n^(1.5)条边(假设稀疏化的图被视为随机对象)。将该算法扩展到使用Schur补和近似echolesky分解导致生成树的计数和采样算法，这对于密集图来说几乎是最优的。我们给出了一种算法，可以在大约n^2 / δ^2的时间内计算任意SDDM矩阵的行列式的(1 +/- δ)近似。这是图的第一个例程，它优于计算任意矩阵的行列式的通用例程。我们还给出了一种算法，该算法在n^2 / δ^2时间内从总变异距离为δ的分布生成一棵加权无向图的生成树;从w均匀分布。
{"title":"Determinant-Preserving Sparsification of SDDM Matrices with Applications to Counting and Sampling Spanning Trees","authors":"D. Durfee, John Peebles, Richard Peng, Anup B. Rao","doi":"10.1109/FOCS.2017.90","DOIUrl":"https://doi.org/10.1109/FOCS.2017.90","url":null,"abstract":"We show variants of spectral sparsification routines can preserve the totalspanning tree counts of graphs, which by Kirchhoffs matrix-tree theorem, isequivalent to determinant of a graph Laplacian minor, or equivalently, of any SDDM matrix. Our analyses utilizes this combinatorial connection to bridge between statisticalleverage scores / effective resistances and the analysis of random graphsby [Janson, Combinatorics, Probability and Computing 94]. This leads to a routine that in quadratic time, sparsifies a graph down to aboutn^(1.5) edges in ways that preserve both the determinant and the distributionof spanning trees (provided the sparsified graph is viewed as a random object). Extending this algorithm to work with Schur complements and approximateCholesky factorizations leads to algorithms for counting andsampling spanning trees which are nearly optimal for dense graphs.We give an algorithm that computes a (1 +/- δ) approximation to the determinantof any SDDM matrix with constant probability in about n^2 / δ^2 time. This is the first routine for graphs that outperforms general-purpose routines for computingdeterminants of arbitrary matrices. We also give an algorithm that generates in about n^2 / δ^2 time a spanning tree ofa weighted undirected graph from a distribution with total variationdistance of δ from the w-uniform distribution.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116982030","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The celebrated Time Hierarchy Theorem for Turing machines states, informally, that more problems can be solved given more time. The extent to which a time hierarchy-type theorem holds in the classic distributed LOCAL model has been open for many years. In particular, it is consistent with previous results that all natural problems in the LOCAL model can be classified according to a small constant number of complexities, such as O(1), O(log* n), O(log n), 2^{O(sqrt{log n}), etc.In this paper we establish the first time hierarchy theorem for the LOCAL model and prove that several gaps exist in the LOCAL time hierarchy. Our main results are as follows:• We define an infinite set of simple coloring problems called Hierarchical 2½-Coloring. A correctly colored graph can be confirmed by simply checking the neighborhood of each vertex, so this problem fits into the class of locally checkable labeling (LCL) problems. However, the complexity of the k-level Hierarchical 2½-Coloring problem is Θ(n^{1/k}), for positive integer k. The upper and lower bounds hold for both general graphs and trees, and for both randomized and deterministic algorithms.• Consider any LCL problem on bounded degree trees. We prove an automatic-speedup theorem that states that any randomized n^{o(1)}-time algorithm solving the LCL can be transformed into a deterministic O(log n)-time algorithm. Together with a previous result, this establishes that on trees, there are no natural deterministic complexities in the ranges ω(log* n)—o(log n) or ω(log n)—n^{o(1)}.• We expose a gap in the randomized time hierarchy on general graphs. Roughly speaking, any randomized algorithm that solves an LCL problem in sublogarithmic time can be sped up to run in O(T_{LLL}) time, which is the complexity of the distributed Lovasz local lemma problem, currently known to be Ω(log log n) and 2^{O(sqrt{log log n})} on bounded degree graphs.Finally, we revisit Naor and Stockmeyers characterization of O(1)-time LOCAL algorithms for LCL problems (as order-invariant w.r.t. vertex IDs) and calculate the complexity gaps that are directly implied by their proof. For n-rings we see a ω(1)—o(log* n) complexity gap, for (sqrt{n} × √{n})-tori an ω(1)—o(sqrt{log* n}) gap, and for bounded degree trees and general graphs, an ω(1)—o(log(log* n)) complexity gap.
{"title":"A Time Hierarchy Theorem for the LOCAL Model","authors":"Yi-Jun Chang, S. Pettie","doi":"10.1109/FOCS.2017.23","DOIUrl":"https://doi.org/10.1109/FOCS.2017.23","url":null,"abstract":"The celebrated Time Hierarchy Theorem for Turing machines states, informally, that more problems can be solved given more time. The extent to which a time hierarchy-type theorem holds in the classic distributed LOCAL model has been open for many years. In particular, it is consistent with previous results that all natural problems in the LOCAL model can be classified according to a small constant number of complexities, such as O(1), O(log* n), O(log n), 2^{O(sqrt{log n}), etc.In this paper we establish the first time hierarchy theorem for the LOCAL model and prove that several gaps exist in the LOCAL time hierarchy. Our main results are as follows:• We define an infinite set of simple coloring problems called Hierarchical 2½-Coloring. A correctly colored graph can be confirmed by simply checking the neighborhood of each vertex, so this problem fits into the class of locally checkable labeling (LCL) problems. However, the complexity of the k-level Hierarchical 2½-Coloring problem is Θ(n^{1/k}), for positive integer k. The upper and lower bounds hold for both general graphs and trees, and for both randomized and deterministic algorithms.• Consider any LCL problem on bounded degree trees. We prove an automatic-speedup theorem that states that any randomized n^{o(1)}-time algorithm solving the LCL can be transformed into a deterministic O(log n)-time algorithm. Together with a previous result, this establishes that on trees, there are no natural deterministic complexities in the ranges ω(log* n)—o(log n) or ω(log n)—n^{o(1)}.• We expose a gap in the randomized time hierarchy on general graphs. Roughly speaking, any randomized algorithm that solves an LCL problem in sublogarithmic time can be sped up to run in O(T_{LLL}) time, which is the complexity of the distributed Lovasz local lemma problem, currently known to be Ω(log log n) and 2^{O(sqrt{log log n})} on bounded degree graphs.Finally, we revisit Naor and Stockmeyers characterization of O(1)-time LOCAL algorithms for LCL problems (as order-invariant w.r.t. vertex IDs) and calculate the complexity gaps that are directly implied by their proof. For n-rings we see a ω(1)—o(log* n) complexity gap, for (sqrt{n} × √{n})-tori an ω(1)—o(sqrt{log* n}) gap, and for bounded degree trees and general graphs, an ω(1)—o(log(log* n)) complexity gap.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130549219","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A weight-t halfspace} is a Boolean function f(x)=sign(w_1 x_1 + … + w_n x_n - θ) where each w_i is an integer in {-t,dots,t}. We give an explicit pseudorandom generator that δ-fools any intersection of k weight-t halfspaces with seed length poly(log n, log k,t,1/δ). In particular, our result gives an explicit PRG that fools any intersection of any quasipoly(n) number of halfspaces of any polylog(n) weight to any 1/polylog(n) accuracy using seed length polylog(n). Prior to this work no explicit PRG with non-trivial seed length was known even for fooling intersections of n weight-1 halfspaces to constant accuracy.The analysis of our PRG fuses techniques from two different lines of work on unconditional pseudorandomness for different kinds of Boolean functions. We extend the approach of Harsha, Klivans and Meka cite{HKM12} for fooling intersections of regular halfspaces, and combine this approach with results of Bazzi cite{Bazzi:07} and Razborov cite{Razborov:09} on bounded independence fooling CNF formulas. Our analysis introduces new coupling-based ingredients into the standard Lindeberg method for establishing quantitative central limit theorems and associated pseudorandomness results.
{"title":"Fooling Intersections of Low-Weight Halfspaces","authors":"R. Servedio, Li-Yang Tan","doi":"10.1109/FOCS.2017.81","DOIUrl":"https://doi.org/10.1109/FOCS.2017.81","url":null,"abstract":"A weight-t halfspace} is a Boolean function f(x)=sign(w_1 x_1 + … + w_n x_n - θ) where each w_i is an integer in {-t,dots,t}. We give an explicit pseudorandom generator that δ-fools any intersection of k weight-t halfspaces with seed length poly(log n, log k,t,1/δ). In particular, our result gives an explicit PRG that fools any intersection of any quasipoly(n) number of halfspaces of any polylog(n) weight to any 1/polylog(n) accuracy using seed length polylog(n). Prior to this work no explicit PRG with non-trivial seed length was known even for fooling intersections of n weight-1 halfspaces to constant accuracy.The analysis of our PRG fuses techniques from two different lines of work on unconditional pseudorandomness for different kinds of Boolean functions. We extend the approach of Harsha, Klivans and Meka cite{HKM12} for fooling intersections of regular halfspaces, and combine this approach with results of Bazzi cite{Bazzi:07} and Razborov cite{Razborov:09} on bounded independence fooling CNF formulas. Our analysis introduces new coupling-based ingredients into the standard Lindeberg method for establishing quantitative central limit theorems and associated pseudorandomness results.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121398252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We consider the Similarity Sketching problem: Given a universe [u] = {0,..., u-1} we want a random function S mapping subsets A of [u] into vectors S(A) of size t, such that similarity is preserved. More precisely: Given subsets A,B of [u], define X_i = [S(A)[i] = S(B)[i]] and X = sum_{i in [t]} X_i. We want to have E[X] = t*J(A,B), where J(A,B) = |A intersect B|/|A union B| and furthermore to have strong concentration guarantees (i.e. Chernoff-style bounds) for X. This is a fundamental problem which has found numerous applications in data mining, large-scale classification, computer vision, similarity search, etc. via the classic MinHash algorithm. The vectors S(A) are also called sketches. The seminal t x MinHash algorithm uses t random hash functions h_1,..., h_t, and stores (min_{a in A} h_1(A),..., min_{a in A} h_t(A)) as the sketch of A. The main drawback of MinHash is, however, its O(t*|A|) running time, and finding a sketch with similar properties and faster running time has been the subject of several papers. Addressing this, Li et al. [NIPS12] introduced one permutation hashing (OPH), which creates a sketch of size t in O(t + |A|) time, but with the drawback that possibly some of the t entries are empty when |A| = O(t). One could argue that sketching is not necessary in this case, however the desire in most applications is to have one sketching procedure that works for sets of all sizes. Therefore, filling out these empty entries is the subject of several follow-up papers initiated by Shrivastava and Li [ICML14]. However, these densification schemes fail to provide good concentration bounds exactly in the case |A| = O(t), where they are needed. In this paper we present a new sketch which obtains essentially the best of both worlds. That is, a fast O(t log t + |A|) expected running time while getting the same strong concentration bounds as MinHash. Our new sketch can be seen as a mix between sampling with replacement and sampling without replacement. We demonstrate the power of our new sketch by considering popular applications in large-scale classification with linear SVM as introduced by Li et al. [NIPS11] as well as approximate similarity search using the LSH framework of Indyk and Motwani [STOC98]. In particular, for the j_1, j_2-approximate similarity search problem on a collection of n sets we obtain a data-structure with space usage O(n^{1+rho} + sum_{A in C} |A|) and O(n^rho * log n + |Q|) expected time for querying a set Q compared to a O(n^rho * log n * |Q|) expected query time of the classic result of Indyk and Motwani.
我们考虑相似草图问题:给定一个宇宙[u] ={0，…， u-1}我们需要一个随机函数S将[u]的子集a映射到大小为t的向量S(a)，使得相似性保持不变。更精确地说:给定[u]的子集A,B，定义X_i = [S(A)[i] = S(B)[i]]和X = sum_{i in [t]} X_i。我们希望E[X] = t*J(A,B)，其中J(A,B) = |A相交B|/|A并集B|，并且对X具有强集中保证(即chernoff式边界)。这是一个基本问题，通过经典的MinHash算法在数据挖掘，大规模分类，计算机视觉，相似性搜索等方面找到了许多应用。向量S(A)也称为草图。开创性的t x MinHash算法使用t个随机哈希函数h_1，…， h_t, and stores (min_{a in a} h_1(a)，…)然而，MinHash的主要缺点是它的运行时间为O(t*| a |)，并且寻找具有类似属性和更快运行时间的草图已经成为几篇论文的主题。为了解决这个问题，Li等人[NIPS12]引入了一种排列哈希(OPH)，它在O(t + | a |)时间内创建了一个大小为t的草图，但缺点是当| a | = O(t)时，t项中可能有一些是空的。有人可能会说，在这种情况下没有必要绘制草图，然而，大多数应用程序都希望有一个适用于所有大小集合的草图过程。因此，填写这些空白条目是Shrivastava和Li [ICML14]等后续几篇论文的主题。然而，这些致密化方案在需要的情况下，却不能提供良好的浓度界限。在本文中，我们提出了一个新的草图，它基本上获得了两个世界的优点。也就是说，快速的O(t log t + | a |)预期运行时间，同时获得与MinHash相同的强集中界限。我们的新草图可以看作是抽样替换和抽样不替换的混合。通过考虑Li等人[NIPS11]引入的线性支持向量机在大规模分类中的流行应用，以及使用Indyk和Motwani [STOC98]的LSH框架的近似相似性搜索，我们展示了新草图的强大功能。特别地，对于n个集合集合上的j_1, j_2-近似相似搜索问题，我们得到了一个空间利用率为O(n^{1+rho} + sum_{a In C} | a |)的数据结构，与Indyk和Motwani经典结果的O(n^rho * log n * |Q|)的期望查询时间相比，查询集合Q的期望查询时间为O(n^rho * log n * |Q|)。
{"title":"Fast Similarity Sketching","authors":"Søren Dahlgaard, M. B. T. Knudsen, M. Thorup","doi":"10.1109/FOCS.2017.67","DOIUrl":"https://doi.org/10.1109/FOCS.2017.67","url":null,"abstract":"We consider the Similarity Sketching problem: Given a universe [u] = {0,..., u-1} we want a random function S mapping subsets A of [u] into vectors S(A) of size t, such that similarity is preserved. More precisely: Given subsets A,B of [u], define X_i = [S(A)[i] = S(B)[i]] and X = sum_{i in [t]} X_i. We want to have E[X] = t*J(A,B), where J(A,B) = |A intersect B|/|A union B| and furthermore to have strong concentration guarantees (i.e. Chernoff-style bounds) for X. This is a fundamental problem which has found numerous applications in data mining, large-scale classification, computer vision, similarity search, etc. via the classic MinHash algorithm. The vectors S(A) are also called sketches. The seminal t x MinHash algorithm uses t random hash functions h_1,..., h_t, and stores (min_{a in A} h_1(A),..., min_{a in A} h_t(A)) as the sketch of A. The main drawback of MinHash is, however, its O(t*|A|) running time, and finding a sketch with similar properties and faster running time has been the subject of several papers. Addressing this, Li et al. [NIPS12] introduced one permutation hashing (OPH), which creates a sketch of size t in O(t + |A|) time, but with the drawback that possibly some of the t entries are empty when |A| = O(t). One could argue that sketching is not necessary in this case, however the desire in most applications is to have one sketching procedure that works for sets of all sizes. Therefore, filling out these empty entries is the subject of several follow-up papers initiated by Shrivastava and Li [ICML14]. However, these densification schemes fail to provide good concentration bounds exactly in the case |A| = O(t), where they are needed. In this paper we present a new sketch which obtains essentially the best of both worlds. That is, a fast O(t log t + |A|) expected running time while getting the same strong concentration bounds as MinHash. Our new sketch can be seen as a mix between sampling with replacement and sampling without replacement. We demonstrate the power of our new sketch by considering popular applications in large-scale classification with linear SVM as introduced by Li et al. [NIPS11] as well as approximate similarity search using the LSH framework of Indyk and Motwani [STOC98]. In particular, for the j_1, j_2-approximate similarity search problem on a collection of n sets we obtain a data-structure with space usage O(n^{1+rho} + sum_{A in C} |A|) and O(n^rho * log n + |Q|) expected time for querying a set Q compared to a O(n^rho * log n * |Q|) expected query time of the classic result of Indyk and Motwani.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122944174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Huck Bennett, Alexander Golovnev, Noah Stephens-Davidowitz
For odd integers p ≥ 1 (and p = ∞), we show that the Closest Vector Problem in the ℓp norm (CVP_p) over rank n lattices cannot be solved in 2^(1-≥) n time for any constant ≥ 0 unless the Strong Exponential Time Hypothesis (SETH) fails. We then extend this result to almost all values of p ≥ 1, not including the even integers. This comes tantalizingly close to settling the quantitative time complexity of the important special case of CVP_2 (i.e., CVP in the Euclidean norm), for which a 2^{n +o(n)}-time algorithm is known. In particular, our result applies for any p = p(n) ≠ 2 that approaches 2 as n ↦ ∞.We also show a similar SETH-hardness result for SVP_∞; hardness of approximating CVP_p to within some constant factor under the so-called Gap-ETH assumption; and other hardness results for CVP_p and CVPP_p for any 1 ≤ p
{"title":"On the Quantitative Hardness of CVP","authors":"Huck Bennett, Alexander Golovnev, Noah Stephens-Davidowitz","doi":"10.1109/FOCS.2017.11","DOIUrl":"https://doi.org/10.1109/FOCS.2017.11","url":null,"abstract":"For odd integers p ≥ 1 (and p = ∞), we show that the Closest Vector Problem in the ℓp norm (CVP_p) over rank n lattices cannot be solved in 2^(1-≥) n time for any constant ≥ 0 unless the Strong Exponential Time Hypothesis (SETH) fails. We then extend this result to almost all values of p ≥ 1, not including the even integers. This comes tantalizingly close to settling the quantitative time complexity of the important special case of CVP_2 (i.e., CVP in the Euclidean norm), for which a 2^{n +o(n)}-time algorithm is known. In particular, our result applies for any p = p(n) ≠ 2 that approaches 2 as n ↦ ∞.We also show a similar SETH-hardness result for SVP_∞; hardness of approximating CVP_p to within some constant factor under the so-called Gap-ETH assumption; and other hardness results for CVP_p and CVPP_p for any 1 ≤ p","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121092766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We design a deterministic polynomial time cn approximation algorithm for the permanent of positive semidefinite matrices where c = e+1 ⋍ 4:84. We write a natural convex relaxation and show that its optimum solution gives a cn approximation of the permanent. We further show that this factor is asymptotically tight by constructing a family of positive semidefinite matrices. We also show that our result implies an approximate version of the permanent-ontop conjecture, which was recently refuted in its original form; we show that the permanent is within a cn factor of the top eigenvalue of the Schur power matrix.
{"title":"Simply Exponential Approximation of the Permanent of Positive Semidefinite Matrices","authors":"Nima Anari, L. Gurvits, S. Gharan, A. Saberi","doi":"10.1109/FOCS.2017.89","DOIUrl":"https://doi.org/10.1109/FOCS.2017.89","url":null,"abstract":"We design a deterministic polynomial time cn approximation algorithm for the permanent of positive semidefinite matrices where c = e+1 ⋍ 4:84. We write a natural convex relaxation and show that its optimum solution gives a cn approximation of the permanent. We further show that this factor is asymptotically tight by constructing a family of positive semidefinite matrices. We also show that our result implies an approximate version of the permanent-ontop conjecture, which was recently refuted in its original form; we show that the permanent is within a cn factor of the top eigenvalue of the Schur power matrix.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"60 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130578324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We show how to compute a relative-error low-rank approximation to any positive semidefinite (PSD) matrix in sublinear time, i.e., for any n x n PSD matrix A, in Õ(n ⋅ poly(k/ε)) time we output a rank-k matrix B, in factored form, for which kA – B║ 2 F ≤ (1 + ε)║A – Ak║2 F , where Ak is the best rank-k approximation to A. When k and 1/ε are not too large compared to the sparsity of A, our algorithm does not need to read all entries of the matrix. Hence, we significantly improve upon previous nnz(A) time algorithms based on oblivious subspace embeddings, and bypass an nnz(A) time lower bound for general matrices (where nnz(A) denotes the number of non-zero entries in the matrix). We prove time lower bounds for low-rank approximation of PSD matrices, showing that our algorithm is close to optimal. Finally, we extend our techniques to give sublinear time algorithms for lowrank approximation of A in the (often stronger) spectral norm metric ║A – B║2 2 and for ridge regression on PSD matrices.
我们展示了如何在亚线性时间内计算任意正半定(PSD)矩阵的相对误差低秩逼近，即对于任意n x n PSD矩阵a，在Õ(n ⋅poly(k/ε))时，我们以因子形式输出一个秩-k矩阵B，其中kA –b # x2551;2 F ≤(1 + ε)║A –k║2 F，其中Ak是a的最佳秩-秩近似。与A的稀疏度相比不是太大，我们的算法不需要读取矩阵的所有条目。因此，我们显著改进了先前基于遗忘子空间嵌入的nnz(A)时间算法，并绕过了一般矩阵的nnz(A)时间下界(其中nnz(A)表示矩阵中非零条目的数量)。我们证明了PSD矩阵的低秩逼近的时间下界，表明我们的算法是接近最优的。最后，我们扩展了我们的技术，给出了在(通常更强的)谱范数度量║ –中A的低秩近似的亚线性时间算法。B║2 2和用于PSD矩阵的脊回归。
{"title":"Sublinear Time Low-Rank Approximation of Positive Semidefinite Matrices","authors":"Cameron Musco, David P. Woodruff","doi":"10.1109/FOCS.2017.68","DOIUrl":"https://doi.org/10.1109/FOCS.2017.68","url":null,"abstract":"We show how to compute a relative-error low-rank approximation to any positive semidefinite (PSD) matrix in sublinear time, i.e., for any n x n PSD matrix A, in Õ(n ⋅ poly(k/ε)) time we output a rank-k matrix B, in factored form, for which kA – B║ 2 F ≤ (1 + ε)║A – Ak║2 F , where Ak is the best rank-k approximation to A. When k and 1/ε are not too large compared to the sparsity of A, our algorithm does not need to read all entries of the matrix. Hence, we significantly improve upon previous nnz(A) time algorithms based on oblivious subspace embeddings, and bypass an nnz(A) time lower bound for general matrices (where nnz(A) denotes the number of non-zero entries in the matrix). We prove time lower bounds for low-rank approximation of PSD matrices, showing that our algorithm is close to optimal. Finally, we extend our techniques to give sublinear time algorithms for lowrank approximation of A in the (often stronger) spectral norm metric ║A – B║2 2 and for ridge regression on PSD matrices.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"143 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116727654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We study an extension of active learning in which the learning algorithm may ask the annotator to compare the distances of two examples from the boundary of their label-class. For example, in a recommendation system application (say for restaurants), the annotator may be asked whether she liked or disliked a specific restaurant (a label query); or which one of two restaurants did she like more (a comparison query).We focus on the class of half spaces, and show that under natural assumptions, such as large margin or bounded bit-description of the input examples, it is possible to reveal all the labels of a sample of size n using approximately O(log n) queries. This implies an exponential improvement over classical active learning, where only label queries are allowed. We complement these results by showing that if any of these assumptions is removed then, in the worst case, Ω(n) queries are required.Our results follow from a new general framework of active learning with additional queries. We identify a combinatorial dimension, called the inference dimension, that captures the query complexity when each additional query is determined by O(1) examples (such as comparison queries, each of which is determined by the two compared examples). Our results for half spaces follow by bounding the inference dimension in the cases discussed above.
{"title":"Active Classification with Comparison Queries","authors":"D. Kane, Shachar Lovett, S. Moran, Jiapeng Zhang","doi":"10.1109/FOCS.2017.40","DOIUrl":"https://doi.org/10.1109/FOCS.2017.40","url":null,"abstract":"We study an extension of active learning in which the learning algorithm may ask the annotator to compare the distances of two examples from the boundary of their label-class. For example, in a recommendation system application (say for restaurants), the annotator may be asked whether she liked or disliked a specific restaurant (a label query); or which one of two restaurants did she like more (a comparison query).We focus on the class of half spaces, and show that under natural assumptions, such as large margin or bounded bit-description of the input examples, it is possible to reveal all the labels of a sample of size n using approximately O(log n) queries. This implies an exponential improvement over classical active learning, where only label queries are allowed. We complement these results by showing that if any of these assumptions is removed then, in the worst case, Ω(n) queries are required.Our results follow from a new general framework of active learning with additional queries. We identify a combinatorial dimension, called the inference dimension, that captures the query complexity when each additional query is determined by O(1) examples (such as comparison queries, each of which is determined by the two compared examples). Our results for half spaces follow by bounding the inference dimension in the cases discussed above.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"286 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133433452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The weighted k-server problem is a natural generalization of the k-server problem where each server has a different weight. We consider the problem on uniform metrics, which corresponds to a natural generalization of paging. Our main result is a doubly exponential lower bound on the competitive ratio of any deterministic online algorithm, that essentially matches the known upper bounds for the problem and closes a large and long-standing gap.The lower bound is based on relating the weighted k-server problem to a certain combinatorial problem and proving a Ramsey-theoretic lower bound for it. This combinatorial connection also reveals several structural properties of low cost feasible solutions to serve a sequence of requests. We use this to show that the generalized Work Function Algorithm achieves an almost optimum competitive ratio, and to obtain new refined upper bounds on the competitive ratio for the case of d different weight classes.
{"title":"Weighted k-Server Bounds via Combinatorial Dichotomies","authors":"N. Bansal, Marek Eliáš, G. Koumoutsos","doi":"10.1109/FOCS.2017.52","DOIUrl":"https://doi.org/10.1109/FOCS.2017.52","url":null,"abstract":"The weighted k-server problem is a natural generalization of the k-server problem where each server has a different weight. We consider the problem on uniform metrics, which corresponds to a natural generalization of paging. Our main result is a doubly exponential lower bound on the competitive ratio of any deterministic online algorithm, that essentially matches the known upper bounds for the problem and closes a large and long-standing gap.The lower bound is based on relating the weighted k-server problem to a certain combinatorial problem and proving a Ramsey-theoretic lower bound for it. This combinatorial connection also reveals several structural properties of low cost feasible solutions to serve a sequence of requests. We use this to show that the generalized Work Function Algorithm achieves an almost optimum competitive ratio, and to obtain new refined upper bounds on the competitive ratio for the case of d different weight classes.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125201061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A pervasive task in the differential privacy literature is to select the k items of highest quality out of a set of d items, where the quality of each item depends on a sensitive dataset that must be protected. Variants of this task arise naturally in fundamental problems like feature selection and hypothesis testing, and also as subroutines for many sophisticated differentially private algorithms.The standard approaches to these tasks—repeated use of the exponential mechanism or the sparse vector technique—approximately solve this problem given a dataset of n = O(√{k}log d) samples. We provide a tight lower bound for some very simple variants of the private selection problem. Our lower bound shows that a sample of size n = Ω(√{k} log d) is required even to achieve a very minimal accuracy guarantee.Our results are based on an extension of the fingerprinting method to sparse selection problems. Previously, the fingerprinting method has been used to provide tight lower bounds for answering an entire set of d queries, but often only some much smaller set of k queries are relevant. Our extension allows us to prove lower bounds that depend on both the number of relevant queries and the total number of queries.
{"title":"Tight Lower Bounds for Differentially Private Selection","authors":"T. Steinke, Jonathan Ullman","doi":"10.1109/FOCS.2017.57","DOIUrl":"https://doi.org/10.1109/FOCS.2017.57","url":null,"abstract":"A pervasive task in the differential privacy literature is to select the k items of highest quality out of a set of d items, where the quality of each item depends on a sensitive dataset that must be protected. Variants of this task arise naturally in fundamental problems like feature selection and hypothesis testing, and also as subroutines for many sophisticated differentially private algorithms.The standard approaches to these tasks—repeated use of the exponential mechanism or the sparse vector technique—approximately solve this problem given a dataset of n = O(√{k}log d) samples. We provide a tight lower bound for some very simple variants of the private selection problem. Our lower bound shows that a sample of size n = Ω(√{k} log d) is required even to achieve a very minimal accuracy guarantee.Our results are based on an extension of the fingerprinting method to sparse selection problems. Previously, the fingerprinting method has been used to provide tight lower bounds for answering an entire set of d queries, but often only some much smaller set of k queries are relevant. Our extension allows us to prove lower bounds that depend on both the number of relevant queries and the total number of queries.","PeriodicalId":311592,"journal":{"name":"2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122234364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：