首页 > 最新文献

Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation最新文献

英文 中文
Templates and recurrences: better together 模板和递归:更好地结合在一起
J. Breck, John Cyphert, Zachary Kincaid, T. Reps
This paper is the confluence of two streams of ideas in the literature on generating numerical invariants, namely: (1) template-based methods, and (2) recurrence-based methods. A template-based method begins with a template that contains unknown quantities, and finds invariants that match the template by extracting and solving constraints on the unknowns. A disadvantage of template-based methods is that they require fixing the set of terms that may appear in an invariant in advance. This disadvantage is particularly prominent for non-linear invariant generation, because the user must supply maximum degrees on polynomials, bases for exponents, etc. On the other hand, recurrence-based methods are able to find sophisticated non-linear mathematical relations, including polynomials, exponentials, and logarithms, because such relations arise as the solutions to recurrences. However, a disadvantage of past recurrence-based invariant-generation methods is that they are primarily loop-based analyses: they use recurrences to relate the pre-state and post-state of a loop, so it is not obvious how to apply them to a recursive procedure, especially if the procedure is non-linearly recursive (e.g., a tree-traversal algorithm). In this paper, we combine these two approaches and obtain a technique that uses templates in which the unknowns are functions rather than numbers, and the constraints on the unknowns are recurrences. The technique synthesizes invariants involving polynomials, exponentials, and logarithms, even in the presence of arbitrary control-flow, including any combination of loops, branches, and (possibly non-linear) recursion. For instance, it is able to show that (i) the time taken by merge-sort is O(n log(n)), and (ii) the time taken by Strassen’s algorithm is O(nlog2(7)).
本文是文献中关于生成数值不变量的两种思想的融合,即:(1)基于模板的方法,(2)基于递归的方法。基于模板的方法从包含未知量的模板开始,通过提取和求解未知量上的约束来找到与模板匹配的不变量。基于模板的方法的一个缺点是,它们需要预先固定可能出现在不变量中的一组术语。这个缺点对于非线性不变量生成尤其突出,因为用户必须提供多项式的最大度,指数的基数等。另一方面,基于递归的方法能够找到复杂的非线性数学关系,包括多项式、指数和对数,因为这些关系是递归的解。然而,过去基于递归的不变量生成方法的一个缺点是它们主要是基于循环的分析:它们使用递归来关联循环的前状态和后状态,因此如何将它们应用于递归过程并不明显,特别是如果该过程是非线性递归的(例如,树遍历算法)。在本文中,我们将这两种方法结合起来,获得了一种使用模板的技术,其中未知数是函数而不是数字,并且未知数的约束是递归的。该技术综合了涉及多项式、指数和对数的不变量,甚至在存在任意控制流的情况下,包括循环、分支和(可能是非线性的)递归的任何组合。例如,它能够证明(i)归并排序耗时为O(nlog (n)), (ii) Strassen算法耗时为O(nlog2(7))。
{"title":"Templates and recurrences: better together","authors":"J. Breck, John Cyphert, Zachary Kincaid, T. Reps","doi":"10.1145/3385412.3386035","DOIUrl":"https://doi.org/10.1145/3385412.3386035","url":null,"abstract":"This paper is the confluence of two streams of ideas in the literature on generating numerical invariants, namely: (1) template-based methods, and (2) recurrence-based methods. A template-based method begins with a template that contains unknown quantities, and finds invariants that match the template by extracting and solving constraints on the unknowns. A disadvantage of template-based methods is that they require fixing the set of terms that may appear in an invariant in advance. This disadvantage is particularly prominent for non-linear invariant generation, because the user must supply maximum degrees on polynomials, bases for exponents, etc. On the other hand, recurrence-based methods are able to find sophisticated non-linear mathematical relations, including polynomials, exponentials, and logarithms, because such relations arise as the solutions to recurrences. However, a disadvantage of past recurrence-based invariant-generation methods is that they are primarily loop-based analyses: they use recurrences to relate the pre-state and post-state of a loop, so it is not obvious how to apply them to a recursive procedure, especially if the procedure is non-linearly recursive (e.g., a tree-traversal algorithm). In this paper, we combine these two approaches and obtain a technique that uses templates in which the unknowns are functions rather than numbers, and the constraints on the unknowns are recurrences. The technique synthesizes invariants involving polynomials, exponentials, and logarithms, even in the presence of arbitrary control-flow, including any combination of loops, branches, and (possibly non-linear) recursion. For instance, it is able to show that (i) the time taken by merge-sort is O(n log(n)), and (ii) the time taken by Strassen’s algorithm is O(nlog2(7)).","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"66 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80364653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 15
Data-driven inference of representation invariants 数据驱动的表示不变量推理
Anders Miltner, Saswat Padhi, T. Millstein, D. Walker
A representation invariant is a property that holds of all values of abstract type produced by a module. Representation invariants play important roles in software engineering and program verification. In this paper, we develop a counterexample-driven algorithm for inferring a representation invariant that is sufficient to imply a desired specification for a module. The key novelty is a type-directed notion of visible inductiveness, which ensures that the algorithm makes progress toward its goal as it alternates between weakening and strengthening candidate invariants. The algorithm is parameterized by an example-based synthesis engine and a verifier, and we prove that it is sound and complete for first-order modules over finite types, assuming that the synthesizer and verifier are as well. We implement these ideas in a tool called Hanoi, which synthesizes representation invariants for recursive data types. Hanoi not only handles invariants for first-order code, but higher-order code as well. In its back end, Hanoi uses an enumerative synthesizer called Myth and an enumerative testing tool as a verifier. Because Hanoi uses testing for verification, it is not sound, though our empirical evaluation shows that it is successful on the benchmarks we investigated.
表示不变量是一种属性,它保存由模块生成的抽象类型的所有值。表示不变量在软件工程和程序验证中起着重要的作用。在本文中,我们开发了一种反例驱动算法,用于推断足以暗示模块所需规范的表示不变量。关键的新颖之处是一种类型导向的可见归纳性概念,它确保算法在削弱和增强候选不变量之间交替时朝着目标前进。该算法由一个基于实例的合成引擎和一个验证器参数化,并在假设合成引擎和验证器相同的情况下,证明了该算法对于有限类型上的一阶模块是健全完备的。我们在一个名为Hanoi的工具中实现了这些思想,该工具综合了递归数据类型的表示不变量。Hanoi不仅处理一阶代码的不变量,还处理高阶代码的不变量。在后端,Hanoi使用了一个名为Myth的枚举综合器和一个枚举测试工具作为验证器。因为河内使用测试进行验证,所以它是不健全的,尽管我们的经验评估表明它在我们调查的基准上是成功的。
{"title":"Data-driven inference of representation invariants","authors":"Anders Miltner, Saswat Padhi, T. Millstein, D. Walker","doi":"10.1145/3385412.3385967","DOIUrl":"https://doi.org/10.1145/3385412.3385967","url":null,"abstract":"A representation invariant is a property that holds of all values of abstract type produced by a module. Representation invariants play important roles in software engineering and program verification. In this paper, we develop a counterexample-driven algorithm for inferring a representation invariant that is sufficient to imply a desired specification for a module. The key novelty is a type-directed notion of visible inductiveness, which ensures that the algorithm makes progress toward its goal as it alternates between weakening and strengthening candidate invariants. The algorithm is parameterized by an example-based synthesis engine and a verifier, and we prove that it is sound and complete for first-order modules over finite types, assuming that the synthesizer and verifier are as well. We implement these ideas in a tool called Hanoi, which synthesizes representation invariants for recursive data types. Hanoi not only handles invariants for first-order code, but higher-order code as well. In its back end, Hanoi uses an enumerative synthesizer called Myth and an enumerative testing tool as a verifier. Because Hanoi uses testing for verification, it is not sound, though our empirical evaluation shows that it is successful on the benchmarks we investigated.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"27 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73766406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 19
Learning nonlinear loop invariants with gated continuous logic networks 用门控连续逻辑网络学习非线性环不变量
Jianan Yao, Gabriel Ryan, Justin Wong, S. Jana, Ronghui Gu
Verifying real-world programs often requires inferring loop invariants with nonlinear constraints. This is especially true in programs that perform many numerical operations, such as control systems for avionics or industrial plants. Recently, data-driven methods for loop invariant inference have shown promise, especially on linear loop invariants. However, applying data-driven inference to nonlinear loop invariants is challenging due to the large numbers of and large magnitudes of high-order terms, the potential for overfitting on a small number of samples, and the large space of possible nonlinear inequality bounds. In this paper, we introduce a new neural architecture for general SMT learning, the Gated Continuous Logic Network (G-CLN), and apply it to nonlinear loop invariant learning. G-CLNs extend the Continuous Logic Network (CLN) architecture with gating units and dropout, which allow the model to robustly learn general invariants over large numbers of terms. To address overfitting that arises from finite program sampling, we introduce fractional sampling—a sound relaxation of loop semantics to continuous functions that facilitates unbounded sampling on the real domain. We additionally design a new CLN activation function, the Piecewise Biased Quadratic Unit (PBQU), for naturally learning tight inequality bounds. We incorporate these methods into a nonlinear loop invariant inference system that can learn general nonlinear loop invariants. We evaluate our system on a benchmark of nonlinear loop invariants and show it solves 26 out of 27 problems, 3 more than prior work, with an average runtime of 53.3 seconds. We further demonstrate the generic learning ability of G-CLNs by solving all 124 problems in the linear Code2Inv benchmark. We also perform a quantitative stability evaluation and show G-CLNs have a convergence rate of 97.5% on quadratic problems, a 39.2% improvement over CLN models.
验证真实世界的程序通常需要推断具有非线性约束的循环不变量。在执行许多数值操作的程序中尤其如此,例如航空电子设备或工业工厂的控制系统。近年来,数据驱动的循环不变量推理方法,特别是线性循环不变量推理方法,显示出了良好的应用前景。然而,将数据驱动推理应用于非线性循环不变量是具有挑战性的,因为高阶项的数量和幅度都很大,在少数样本上存在过拟合的可能性,以及可能的非线性不等式边界的大空间。本文介绍了一种新的用于通用SMT学习的神经网络结构——门控连续逻辑网络(G-CLN),并将其应用于非线性环不变学习。g -CLN扩展了连续逻辑网络(CLN)体系结构,使用门控单元和dropout,使模型能够在大量项上鲁棒地学习一般不变量。为了解决由有限程序采样引起的过拟合问题,我们引入了分数采样-连续函数的循环语义的良好松弛,有助于在实域上进行无界采样。我们还设计了一个新的CLN激活函数,分段偏置二次单元(PBQU),用于自然学习紧不等式界。我们将这些方法整合到一个非线性环不变量推理系统中,该系统可以学习一般的非线性环不变量。我们在非线性循环不变量的基准上评估了我们的系统,并表明它解决了27个问题中的26个,比以前的工作多3个,平均运行时间为53.3秒。通过解决线性Code2Inv基准测试中的所有124个问题,我们进一步证明了g - cln的通用学习能力。我们还进行了定量稳定性评估,结果表明g -CLN模型在二次问题上的收敛率为97.5%,比CLN模型提高了39.2%。
{"title":"Learning nonlinear loop invariants with gated continuous logic networks","authors":"Jianan Yao, Gabriel Ryan, Justin Wong, S. Jana, Ronghui Gu","doi":"10.1145/3385412.3385986","DOIUrl":"https://doi.org/10.1145/3385412.3385986","url":null,"abstract":"Verifying real-world programs often requires inferring loop invariants with nonlinear constraints. This is especially true in programs that perform many numerical operations, such as control systems for avionics or industrial plants. Recently, data-driven methods for loop invariant inference have shown promise, especially on linear loop invariants. However, applying data-driven inference to nonlinear loop invariants is challenging due to the large numbers of and large magnitudes of high-order terms, the potential for overfitting on a small number of samples, and the large space of possible nonlinear inequality bounds. In this paper, we introduce a new neural architecture for general SMT learning, the Gated Continuous Logic Network (G-CLN), and apply it to nonlinear loop invariant learning. G-CLNs extend the Continuous Logic Network (CLN) architecture with gating units and dropout, which allow the model to robustly learn general invariants over large numbers of terms. To address overfitting that arises from finite program sampling, we introduce fractional sampling—a sound relaxation of loop semantics to continuous functions that facilitates unbounded sampling on the real domain. We additionally design a new CLN activation function, the Piecewise Biased Quadratic Unit (PBQU), for naturally learning tight inequality bounds. We incorporate these methods into a nonlinear loop invariant inference system that can learn general nonlinear loop invariants. We evaluate our system on a benchmark of nonlinear loop invariants and show it solves 26 out of 27 problems, 3 more than prior work, with an average runtime of 53.3 seconds. We further demonstrate the generic learning ability of G-CLNs by solving all 124 problems in the linear Code2Inv benchmark. We also perform a quantitative stability evaluation and show G-CLNs have a convergence rate of 97.5% on quadratic problems, a 39.2% improvement over CLN models.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"PP 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84308740","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 38
Automatic generation of efficient sparse tensor format conversion routines 自动生成高效的稀疏张量格式转换例程
Stephen Chou, Fredrik Kjolstad, Saman P. Amarasinghe
This paper shows how to generate code that efficiently converts sparse tensors between disparate storage formats (data layouts) such as CSR, DIA, ELL, and many others. We decompose sparse tensor conversion into three logical phases: coordinate remapping, analysis, and assembly. We then develop a language that precisely describes how different formats group together and order a tensor’s nonzeros in memory. This lets a compiler emit code that performs complex remappings of nonzeros when converting between formats. We also develop a query language that can extract statistics about sparse tensors, and we show how to emit efficient analysis code that computes such queries. Finally, we define an abstract interface that captures how data structures for storing a tensor can be efficiently assembled given specific statistics about the tensor. Disparate formats can implement this common interface, thus letting a compiler emit optimized sparse tensor conversion code for arbitrary combinations of many formats without hard-coding for any specific combination. Our evaluation shows that the technique generates sparse tensor conversion routines with performance between 1.00 and 2.01× that of hand-optimized versions in SPARSKIT and Intel MKL, two popular sparse linear algebra libraries. And by emitting code that avoids materializing temporaries, which both libraries need for many combinations of source and target formats, our technique outperforms those libraries by 1.78 to 4.01× for CSC/COO to DIA/ELL conversion.
本文展示了如何生成在不同存储格式(数据布局)之间有效转换稀疏张量的代码,例如CSR、DIA、ELL等。我们将稀疏张量转换分解为三个逻辑阶段:坐标重新映射、分析和组装。然后,我们开发了一种语言,精确地描述不同的格式如何组合在一起,并在内存中对张量的非零进行排序。这允许编译器发出代码,在格式之间转换时执行复杂的非零重新映射。我们还开发了一种可以提取关于稀疏张量的统计信息的查询语言,并展示了如何生成计算此类查询的高效分析代码。最后,我们定义了一个抽象接口,该接口捕获了在给定张量的特定统计信息的情况下,如何有效地组装用于存储张量的数据结构。不同的格式可以实现这个公共接口,从而让编译器为许多格式的任意组合发出优化的稀疏张量转换代码,而无需为任何特定的组合进行硬编码。我们的评估表明,该技术生成的稀疏张量转换例程的性能在SPARSKIT和Intel MKL这两个流行的稀疏线性代数库中手工优化版本的1.00到2.01倍之间。并且,通过发布代码避免实现临时代码(这两个库都需要许多源格式和目标格式的组合),我们的技术在CSC/COO到DIA/ELL的转换方面比那些库高出1.78到4.01倍。
{"title":"Automatic generation of efficient sparse tensor format conversion routines","authors":"Stephen Chou, Fredrik Kjolstad, Saman P. Amarasinghe","doi":"10.1145/3385412.3385963","DOIUrl":"https://doi.org/10.1145/3385412.3385963","url":null,"abstract":"This paper shows how to generate code that efficiently converts sparse tensors between disparate storage formats (data layouts) such as CSR, DIA, ELL, and many others. We decompose sparse tensor conversion into three logical phases: coordinate remapping, analysis, and assembly. We then develop a language that precisely describes how different formats group together and order a tensor’s nonzeros in memory. This lets a compiler emit code that performs complex remappings of nonzeros when converting between formats. We also develop a query language that can extract statistics about sparse tensors, and we show how to emit efficient analysis code that computes such queries. Finally, we define an abstract interface that captures how data structures for storing a tensor can be efficiently assembled given specific statistics about the tensor. Disparate formats can implement this common interface, thus letting a compiler emit optimized sparse tensor conversion code for arbitrary combinations of many formats without hard-coding for any specific combination. Our evaluation shows that the technique generates sparse tensor conversion routines with performance between 1.00 and 2.01× that of hand-optimized versions in SPARSKIT and Intel MKL, two popular sparse linear algebra libraries. And by emitting code that avoids materializing temporaries, which both libraries need for many combinations of source and target formats, our technique outperforms those libraries by 1.78 to 4.01× for CSC/COO to DIA/ELL conversion.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84216839","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 25
EVA: an encrypted vector arithmetic language and compiler for efficient homomorphic computation EVA:一个加密的矢量算术语言和编译器,用于高效的同态计算
Roshan Dathathri, Blagovesta Kostova, Olli Saarikivi, Wei Dai, Kim Laine, Madan Musuvathi
Fully-Homomorphic Encryption (FHE) offers powerful capabilities by enabling secure offloading of both storage and computation, and recent innovations in schemes and implementations have made it all the more attractive. At the same time, FHE is notoriously hard to use with a very constrained programming model, a very unusual performance profile, and many cryptographic constraints. Existing compilers for FHE either target simpler but less efficient FHE schemes or only support specific domains where they can rely on expert-provided high-level runtimes to hide complications. This paper presents a new FHE language called Encrypted Vector Arithmetic (EVA), which includes an optimizing compiler that generates correct and secure FHE programs, while hiding all the complexities of the target FHE scheme. Bolstered by our optimizing compiler, programmers can develop efficient general-purpose FHE applications directly in EVA. For example, we have developed image processing applications using EVA, with a very few lines of code. EVA is designed to also work as an intermediate representation that can be a target for compiling higher-level domain-specific languages. To demonstrate this, we have re-targeted CHET, an existing domain-specific compiler for neural network inference, onto EVA. Due to the novel optimizations in EVA, its programs are on average 5.3× faster than those generated by CHET. We believe that EVA would enable a wider adoption of FHE by making it easier to develop FHE applications and domain-specific FHE compilers.
完全同态加密(FHE)通过支持存储和计算的安全卸载提供了强大的功能,并且最近在方案和实现方面的创新使其更具吸引力。同时,众所周知,FHE很难用于非常受限的编程模型、非常不寻常的性能配置文件和许多加密约束。现有的FHE编译器要么针对更简单但效率较低的FHE方案,要么只支持特定领域,在这些领域中,它们可以依赖专家提供的高级运行时来隐藏复杂性。本文提出了一种新的FHE语言,称为加密矢量算法(EVA),它包含一个优化编译器,可以生成正确和安全的FHE程序,同时隐藏了目标FHE方案的所有复杂性。在优化编译器的支持下,程序员可以直接在EVA中开发高效的通用FHE应用程序。例如,我们使用EVA开发了图像处理应用程序,只需要很少的几行代码。EVA还被设计成一种中间表示,可以作为编译高级领域特定语言的目标。为了证明这一点,我们将现有的用于神经网络推理的特定领域编译器CHET重新定位到EVA上。由于EVA中新颖的优化,其程序比CHET生成的程序平均快5.3倍。我们相信,EVA将使开发FHE应用程序和特定领域的FHE编译器变得更容易,从而使FHE得到更广泛的采用。
{"title":"EVA: an encrypted vector arithmetic language and compiler for efficient homomorphic computation","authors":"Roshan Dathathri, Blagovesta Kostova, Olli Saarikivi, Wei Dai, Kim Laine, Madan Musuvathi","doi":"10.1145/3385412.3386023","DOIUrl":"https://doi.org/10.1145/3385412.3386023","url":null,"abstract":"Fully-Homomorphic Encryption (FHE) offers powerful capabilities by enabling secure offloading of both storage and computation, and recent innovations in schemes and implementations have made it all the more attractive. At the same time, FHE is notoriously hard to use with a very constrained programming model, a very unusual performance profile, and many cryptographic constraints. Existing compilers for FHE either target simpler but less efficient FHE schemes or only support specific domains where they can rely on expert-provided high-level runtimes to hide complications. This paper presents a new FHE language called Encrypted Vector Arithmetic (EVA), which includes an optimizing compiler that generates correct and secure FHE programs, while hiding all the complexities of the target FHE scheme. Bolstered by our optimizing compiler, programmers can develop efficient general-purpose FHE applications directly in EVA. For example, we have developed image processing applications using EVA, with a very few lines of code. EVA is designed to also work as an intermediate representation that can be a target for compiling higher-level domain-specific languages. To demonstrate this, we have re-targeted CHET, an existing domain-specific compiler for neural network inference, onto EVA. Due to the novel optimizations in EVA, its programs are on average 5.3× faster than those generated by CHET. We believe that EVA would enable a wider adoption of FHE by making it easier to develop FHE applications and domain-specific FHE compilers.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84195217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 79
A study of the learnability of relational properties: model counting meets machine learning (MCML) 关系属性的可学习性研究:模型计数与机器学习(MCML)
Muhammad Usman, Wenxi Wang, Kaiyuan Wang, Marko Vasic, H. Vikalo, S. Khurshid
This paper introduces the MCML approach for empirically studying the learnability of relational properties that can be expressed in the well-known software design language Alloy. A key novelty of MCML is quantification of the performance of and semantic differences among trained machine learning (ML) models, specifically decision trees, with respect to entire (bounded) input spaces, and not just for given training and test datasets (as is the common practice). MCML reduces the quantification problems to the classic complexity theory problem of model counting, and employs state-of-the-art model counters. The results show that relatively simple ML models can achieve surprisingly high performance (accuracy and F1-score) when evaluated in the common setting of using training and test datasets -- even when the training dataset is much smaller than the test dataset -- indicating the seeming simplicity of learning relational properties. However, MCML metrics based on model counting show that the performance can degrade substantially when tested against the entire (bounded) input space, indicating the high complexity of precisely learning these properties, and the usefulness of model counting in quantifying the true performance.
本文介绍了用MCML方法对关系属性的可学习性进行实证研究,这些关系属性可以用著名的软件设计语言Alloy来表示。MCML的一个关键新颖之处在于对经过训练的机器学习(ML)模型(特别是决策树)的性能和语义差异进行量化,这些模型相对于整个(有界的)输入空间,而不仅仅是针对给定的训练和测试数据集(这是常见的做法)。MCML将量化问题简化为模型计数的经典复杂性理论问题,并采用了最先进的模型计数器。结果表明,在使用训练和测试数据集的常见设置中进行评估时,相对简单的ML模型可以获得惊人的高性能(准确性和f1分数)——即使训练数据集比测试数据集小得多——这表明学习关系属性似乎很简单。然而,基于模型计数的MCML指标表明,当针对整个(有界)输入空间进行测试时,性能会大幅下降,这表明精确学习这些属性的复杂性很高,以及模型计数在量化真实性能方面的有用性。
{"title":"A study of the learnability of relational properties: model counting meets machine learning (MCML)","authors":"Muhammad Usman, Wenxi Wang, Kaiyuan Wang, Marko Vasic, H. Vikalo, S. Khurshid","doi":"10.1145/3385412.3386015","DOIUrl":"https://doi.org/10.1145/3385412.3386015","url":null,"abstract":"This paper introduces the MCML approach for empirically studying the learnability of relational properties that can be expressed in the well-known software design language Alloy. A key novelty of MCML is quantification of the performance of and semantic differences among trained machine learning (ML) models, specifically decision trees, with respect to entire (bounded) input spaces, and not just for given training and test datasets (as is the common practice). MCML reduces the quantification problems to the classic complexity theory problem of model counting, and employs state-of-the-art model counters. The results show that relatively simple ML models can achieve surprisingly high performance (accuracy and F1-score) when evaluated in the common setting of using training and test datasets -- even when the training dataset is much smaller than the test dataset -- indicating the seeming simplicity of learning relational properties. However, MCML metrics based on model counting show that the performance can degrade substantially when tested against the entire (bounded) input space, indicating the high complexity of precisely learning these properties, and the usefulness of model counting in quantifying the true performance.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"573 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77772013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
CacheQuery: learning replacement policies from hardware caches CacheQuery:从硬件缓存中学习替换策略
Pepe Vila, P. Ganty, M. Guarnieri, Boris Köpf
We show how to infer deterministic cache replacement policies using off-the-shelf automata learning and program synthesis techniques. For this, we construct and chain two abstractions that expose the cache replacement policy of any set in the cache hierarchy as a membership oracle to the learning algorithm, based on timing measurements on a silicon CPU. Our experiments demonstrate an advantage in scope and scalability over prior art and uncover two previously undocumented cache replacement policies.
我们展示了如何使用现成的自动学习和程序合成技术推断确定性缓存替换策略。为此,我们构建并链接了两个抽象,它们将缓存层次结构中任何集合的缓存替换策略作为成员oracle公开给学习算法,基于硅CPU上的定时测量。我们的实验证明了在范围和可伸缩性方面优于现有技术,并揭示了两个以前未记录的缓存替换策略。
{"title":"CacheQuery: learning replacement policies from hardware caches","authors":"Pepe Vila, P. Ganty, M. Guarnieri, Boris Köpf","doi":"10.1145/3385412.3386008","DOIUrl":"https://doi.org/10.1145/3385412.3386008","url":null,"abstract":"We show how to infer deterministic cache replacement policies using off-the-shelf automata learning and program synthesis techniques. For this, we construct and chain two abstractions that expose the cache replacement policy of any set in the cache hierarchy as a membership oracle to the learning algorithm, based on timing measurements on a silicon CPU. Our experiments demonstrate an advantage in scope and scalability over prior art and uncover two previously undocumented cache replacement policies.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88932242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 34
Proving data-poisoning robustness in decision trees 证明决策树的数据中毒鲁棒性
Samuel Drews, Aws Albarghouthi, Loris D'antoni
Machine learning models are brittle, and small changes in the training data can result in different predictions. We study the problem of proving that a prediction is robust to data poisoning, where an attacker can inject a number of malicious elements into the training set to influence the learned model. We target decision-tree models, a popular and simple class of machine learning models that underlies many complex learning techniques. We present a sound verification technique based on abstract interpretation and implement it in a tool called Antidote. Antidote abstractly trains decision trees for an intractably large space of possible poisoned datasets. Due to the soundness of our abstraction, Antidote can produce proofs that, for a given input, the corresponding prediction would not have changed had the training set been tampered with or not. We demonstrate the effectiveness of Antidote on a number of popular datasets.
机器学习模型是脆弱的,训练数据的微小变化可能导致不同的预测。我们研究了证明预测对数据中毒的鲁棒性问题,其中攻击者可以向训练集中注入许多恶意元素来影响学习模型。我们的目标是决策树模型,这是一种流行而简单的机器学习模型,是许多复杂学习技术的基础。我们提出了一种基于抽象解释的声音验证技术,并在一个名为Antidote的工具中实现。解毒剂抽象地训练决策树为一个棘手的大空间的可能有毒的数据集。由于我们抽象的合理性,Antidote可以证明,对于给定的输入,无论训练集是否被篡改,相应的预测都不会改变。我们在一些流行的数据集上展示了解毒剂的有效性。
{"title":"Proving data-poisoning robustness in decision trees","authors":"Samuel Drews, Aws Albarghouthi, Loris D'antoni","doi":"10.1145/3385412.3385975","DOIUrl":"https://doi.org/10.1145/3385412.3385975","url":null,"abstract":"Machine learning models are brittle, and small changes in the training data can result in different predictions. We study the problem of proving that a prediction is robust to data poisoning, where an attacker can inject a number of malicious elements into the training set to influence the learned model. We target decision-tree models, a popular and simple class of machine learning models that underlies many complex learning techniques. We present a sound verification technique based on abstract interpretation and implement it in a tool called Antidote. Antidote abstractly trains decision trees for an intractably large space of possible poisoned datasets. Due to the soundness of our abstraction, Antidote can produce proofs that, for a given input, the corresponding prediction would not have changed had the training set been tampered with or not. We demonstrate the effectiveness of Antidote on a number of popular datasets.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"39 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82220274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Automated derivation of parametric data movement lower bounds for affine programs 仿射程序参数数据移动下界的自动推导
Auguste Olivry, J. Langou, L. Pouchet, P. Sadayappan, F. Rastello
Researchers and practitioners have for long worked on improving the computational complexity of algorithms, focusing on reducing the number of operations needed to perform a computation. However the hardware trend nowadays clearly shows a higher performance and energy cost for data movements than computations: quality algorithms have to minimize data movements as much as possible. The theoretical operational complexity of an algorithm is a function of the total number of operations that must be executed, regardless of the order in which they will actually be executed. But theoretical data movement (or, I/O) complexity is fundamentally different: one must consider all possible legal schedules of the operations to determine the minimal number of data movements achievable, a major theoretical challenge. I/O complexity has been studied via complex manual proofs, e.g., refined from Ω(n3/√S) for matrix-multiply on a cache size S by Hong & Kung to 2n3/√S by Smith et al. While asymptotic complexity may be sufficient to compare I/O potential between broadly different algorithms, the accuracy of the reasoning depends on the tightness of these I/O lower bounds. Precisely, exposing constants is essential to enable precise comparison between different algorithms: for example the 2n3/√S lower bound allows to demonstrate the optimality of panel-panel tiling for matrix-multiplication. We present the first static analysis to automatically derive non-asymptotic parametric expressions of data movement lower bounds with scaling constants, for arbitrary affine computations. Our approach is fully automatic, assisting algorithm designers to reason about I/O complexity and make educated decisions about algorithmic alternatives.
长期以来,研究人员和实践者一直致力于提高算法的计算复杂性,重点是减少执行计算所需的操作数量。然而,如今的硬件趋势清楚地表明,与计算相比,数据移动的性能和能源成本更高:高质量的算法必须尽可能地减少数据移动。算法的理论操作复杂性是必须执行的操作总数的函数,而不管它们实际执行的顺序如何。但理论上的数据移动(或I/O)复杂性是根本不同的:必须考虑所有可能的合法操作时间表,以确定可实现的最小数据移动数量,这是一个主要的理论挑战。I/O复杂度已经通过复杂的手工证明进行了研究,例如,Hong & Kung从Ω(n3/√S)对缓存大小为S的矩阵乘法进行了改进,到Smith等人的2n3/√S。虽然渐近复杂性可能足以比较广泛不同算法之间的I/O潜力,但推理的准确性取决于这些I/O下界的紧密性。准确地说,暴露常数对于实现不同算法之间的精确比较至关重要:例如,2n3/√S下界允许演示面板-面板平铺矩阵乘法的最优性。我们提出了第一个静态分析,以自动导出具有缩放常数的数据移动下界的非渐近参数表达式,用于任意仿射计算。我们的方法是全自动的,帮助算法设计者推断I/O复杂性,并对算法替代方案做出明智的决策。
{"title":"Automated derivation of parametric data movement lower bounds for affine programs","authors":"Auguste Olivry, J. Langou, L. Pouchet, P. Sadayappan, F. Rastello","doi":"10.1145/3385412.3385989","DOIUrl":"https://doi.org/10.1145/3385412.3385989","url":null,"abstract":"Researchers and practitioners have for long worked on improving the computational complexity of algorithms, focusing on reducing the number of operations needed to perform a computation. However the hardware trend nowadays clearly shows a higher performance and energy cost for data movements than computations: quality algorithms have to minimize data movements as much as possible. The theoretical operational complexity of an algorithm is a function of the total number of operations that must be executed, regardless of the order in which they will actually be executed. But theoretical data movement (or, I/O) complexity is fundamentally different: one must consider all possible legal schedules of the operations to determine the minimal number of data movements achievable, a major theoretical challenge. I/O complexity has been studied via complex manual proofs, e.g., refined from Ω(n3/√S) for matrix-multiply on a cache size S by Hong & Kung to 2n3/√S by Smith et al. While asymptotic complexity may be sufficient to compare I/O potential between broadly different algorithms, the accuracy of the reasoning depends on the tightness of these I/O lower bounds. Precisely, exposing constants is essential to enable precise comparison between different algorithms: for example the 2n3/√S lower bound allows to demonstrate the optimality of panel-panel tiling for matrix-multiplication. We present the first static analysis to automatically derive non-asymptotic parametric expressions of data movement lower bounds with scaling constants, for arbitrary affine computations. Our approach is fully automatic, assisting algorithm designers to reason about I/O complexity and make educated decisions about algorithmic alternatives.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"35 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84627310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 19
Constant-time foundations for the new spectre era 新幽灵时代的恒定时间基础
S. Cauligi, Craig Disselkoen, K. V. Gleissenthall, D. Stefan, Tamara Rezk, G. Barthe
The constant-time discipline is a software-based countermeasure used for protecting high assurance cryptographic implementations against timing side-channel attacks. Constant-time is effective (it protects against many known attacks), rigorous (it can be formalized using program semantics), and amenable to automated verification. Yet, the advent of micro-architectural attacks makes constant-time as it exists today far less useful. This paper lays foundations for constant-time programming in the presence of speculative and out-of-order execution. We present an operational semantics and a formal definition of constant-time programs in this extended setting. Our semantics eschews formalization of microarchitectural features (that are instead assumed under adversary control), and yields a notion of constant-time that retains the elegance and tractability of the usual notion. We demonstrate the relevance of our semantics in two ways: First, by contrasting existing Spectre-like attacks with our definition of constant-time. Second, by implementing a static analysis tool, Pitchfork, which detects violations of our extended constant-time property in real world cryptographic libraries.
恒定时间原则是一种基于软件的对策,用于保护高保证加密实现免受定时旁信道攻击。恒定时间是有效的(它可以防止许多已知的攻击),严格的(它可以使用程序语义形式化),并且易于自动验证。然而,微体系结构攻击的出现使得现在存在的恒定时间变得不那么有用了。本文为存在推测性和乱序执行的恒时规划奠定了基础。在这个扩展的环境中,我们给出了恒定时间程序的操作语义和形式化定义。我们的语义避免了微架构特性的形式化(相反,这些特性是在对手的控制下假设的),并产生了一个恒定时间的概念,它保留了通常概念的优雅和可追溯性。我们通过两种方式展示了语义的相关性:首先,将现有的类幽灵攻击与我们的常量时间定义进行对比。其次,通过实现一个静态分析工具,Pitchfork,它检测违反我们的扩展常数时间属性在现实世界的加密库。
{"title":"Constant-time foundations for the new spectre era","authors":"S. Cauligi, Craig Disselkoen, K. V. Gleissenthall, D. Stefan, Tamara Rezk, G. Barthe","doi":"10.1145/3385412.3385970","DOIUrl":"https://doi.org/10.1145/3385412.3385970","url":null,"abstract":"The constant-time discipline is a software-based countermeasure used for protecting high assurance cryptographic implementations against timing side-channel attacks. Constant-time is effective (it protects against many known attacks), rigorous (it can be formalized using program semantics), and amenable to automated verification. Yet, the advent of micro-architectural attacks makes constant-time as it exists today far less useful. This paper lays foundations for constant-time programming in the presence of speculative and out-of-order execution. We present an operational semantics and a formal definition of constant-time programs in this extended setting. Our semantics eschews formalization of microarchitectural features (that are instead assumed under adversary control), and yields a notion of constant-time that retains the elegance and tractability of the usual notion. We demonstrate the relevance of our semantics in two ways: First, by contrasting existing Spectre-like attacks with our definition of constant-time. Second, by implementing a static analysis tool, Pitchfork, which detects violations of our extended constant-time property in real world cryptographic libraries.","PeriodicalId":20580,"journal":{"name":"Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation","volume":"51 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2019-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80061689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 68
期刊
Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1