首页 > 最新文献

arXiv - CS - Symbolic Computation最新文献

英文 中文
Towards Learning Abductive Reasoning using VSA Distributed Representations 利用 VSA 分布式表征学习归纳推理
Pub Date : 2024-06-27 DOI: arxiv-2406.19121
Giacomo Camposampiero, Michael Hersche, Aleksandar Terzić, Roger Wattenhofer, Abu Sebastian, Abbas Rahimi
We introduce the Abductive Rule Learner with Context-awareness (ARLC), amodel that solves abstract reasoning tasks based on Learn-VRF. ARLC features anovel and more broadly applicable training objective for abductive reasoning,resulting in better interpretability and higher accuracy when solving Raven'sprogressive matrices (RPM). ARLC allows both programming domain knowledge andlearning the rules underlying a data distribution. We evaluate ARLC on theI-RAVEN dataset, showcasing state-of-the-art accuracy across bothin-distribution and out-of-distribution (unseen attribute-rule pairs) tests.ARLC surpasses neuro-symbolic and connectionist baselines, including largelanguage models, despite having orders of magnitude fewer parameters. We showARLC's robustness to post-programming training by incrementally learning fromexamples on top of programmed knowledge, which only improves its performanceand does not result in catastrophic forgetting of the programmed solution. Wevalidate ARLC's seamless transfer learning from a 2x2 RPM constellation tounseen constellations. Our code is available athttps://github.com/IBM/abductive-rule-learner-with-context-awareness.
我们介绍了具有情境感知能力的归纳式规则学习器(ARLC),这是一种基于 Learn-VRF 解决抽象推理任务的模型。ARLC 的特点是为归纳推理提供了一个更高级、更广泛适用的训练目标,从而在求解瑞文渐进矩阵(RPM)时具有更好的可解释性和更高的准确性。ARLC 既可以编程领域知识,也可以学习数据分布的基本规则。我们在I-RAVEN数据集上对ARLC进行了评估,在分布内和分布外(未见属性-规则对)测试中展示了最先进的准确性。ARLC超越了神经符号和连接主义基线,包括大型语言模型,尽管其参数数量级要少得多。我们通过在编程知识的基础上增量学习示例,展示了 ARLC 对编程后训练的鲁棒性,这只会提高其性能,而不会导致编程解决方案的灾难性遗忘。我们验证了 ARLC 从 2x2 RPM 星座到所见星座的无缝迁移学习。我们的代码可在https://github.com/IBM/abductive-rule-learner-with-context-awareness。
{"title":"Towards Learning Abductive Reasoning using VSA Distributed Representations","authors":"Giacomo Camposampiero, Michael Hersche, Aleksandar Terzić, Roger Wattenhofer, Abu Sebastian, Abbas Rahimi","doi":"arxiv-2406.19121","DOIUrl":"https://doi.org/arxiv-2406.19121","url":null,"abstract":"We introduce the Abductive Rule Learner with Context-awareness (ARLC), a\u0000model that solves abstract reasoning tasks based on Learn-VRF. ARLC features a\u0000novel and more broadly applicable training objective for abductive reasoning,\u0000resulting in better interpretability and higher accuracy when solving Raven's\u0000progressive matrices (RPM). ARLC allows both programming domain knowledge and\u0000learning the rules underlying a data distribution. We evaluate ARLC on the\u0000I-RAVEN dataset, showcasing state-of-the-art accuracy across both\u0000in-distribution and out-of-distribution (unseen attribute-rule pairs) tests.\u0000ARLC surpasses neuro-symbolic and connectionist baselines, including large\u0000language models, despite having orders of magnitude fewer parameters. We show\u0000ARLC's robustness to post-programming training by incrementally learning from\u0000examples on top of programmed knowledge, which only improves its performance\u0000and does not result in catastrophic forgetting of the programmed solution. We\u0000validate ARLC's seamless transfer learning from a 2x2 RPM constellation to\u0000unseen constellations. Our code is available at\u0000https://github.com/IBM/abductive-rule-learner-with-context-awareness.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"144 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Large Language Models are Interpretable Learners 大型语言模型是可解释的学习者
Pub Date : 2024-06-25 DOI: arxiv-2406.17224
Ruochen Wang, Si Si, Felix Yu, Dorothea Wiesmann, Cho-Jui Hsieh, Inderjit Dhillon
The trade-off between expressiveness and interpretability remains a corechallenge when building human-centric predictive models for classification anddecision-making. While symbolic rules offer interpretability, they often lackexpressiveness, whereas neural networks excel in performance but are known forbeing black boxes. In this paper, we show a combination of Large LanguageModels (LLMs) and symbolic programs can bridge this gap. In the proposedLLM-based Symbolic Programs (LSPs), the pretrained LLM with natural languageprompts provides a massive set of interpretable modules that can transform rawinput into natural language concepts. Symbolic programs then integrate thesemodules into an interpretable decision rule. To train LSPs, we develop adivide-and-conquer approach to incrementally build the program from scratch,where the learning process of each step is guided by LLMs. To evaluate theeffectiveness of LSPs in extracting interpretable and accurate knowledge fromdata, we introduce IL-Bench, a collection of diverse tasks, including bothsynthetic and real-world scenarios across different modalities. Empiricalresults demonstrate LSP's superior performance compared to traditionalneurosymbolic programs and vanilla automatic prompt tuning methods. Moreover,as the knowledge learned by LSP is a combination of natural languagedescriptions and symbolic rules, it is easily transferable to humans(interpretable), and other LLMs, and generalizes well to out-of-distributionsamples.
在建立以人为中心的分类和决策预测模型时,表达能力和可解释性之间的权衡仍然是一个核心挑战。虽然符号规则提供了可解释性,但它们往往缺乏表现力,而神经网络虽然性能卓越,却以黑箱著称。在本文中,我们展示了大型语言模型(LLM)与符号程序的结合可以弥合这一差距。在我们提出的基于大型语言模型的符号程序(LSP)中,预训练的大型语言模型带有自然语言提示,提供了大量可解释的模块集,可以将原始输入转化为自然语言概念。然后,符号程序将这些模块整合到可解释的决策规则中。为了训练 LSP,我们开发了一种 "分而治之"(adivide-and-conquer)的方法,从零开始逐步构建程序,其中每一步的学习过程都由 LLMs 指导。为了评估 LSP 从数据中提取可解释的准确知识的效果,我们引入了 IL-Bench,这是一个多样化任务的集合,包括不同模式的合成任务和真实世界场景。实证结果表明,与传统的神经符号程序和虚构的自动提示调整方法相比,LSP 的性能更胜一筹。此外,由于 LSP 学习到的知识是自然语言描述和符号规则的结合,因此很容易将其移植到人类(可解释)和其他 LLM 中,并能很好地泛化到分布外样本中。
{"title":"Large Language Models are Interpretable Learners","authors":"Ruochen Wang, Si Si, Felix Yu, Dorothea Wiesmann, Cho-Jui Hsieh, Inderjit Dhillon","doi":"arxiv-2406.17224","DOIUrl":"https://doi.org/arxiv-2406.17224","url":null,"abstract":"The trade-off between expressiveness and interpretability remains a core\u0000challenge when building human-centric predictive models for classification and\u0000decision-making. While symbolic rules offer interpretability, they often lack\u0000expressiveness, whereas neural networks excel in performance but are known for\u0000being black boxes. In this paper, we show a combination of Large Language\u0000Models (LLMs) and symbolic programs can bridge this gap. In the proposed\u0000LLM-based Symbolic Programs (LSPs), the pretrained LLM with natural language\u0000prompts provides a massive set of interpretable modules that can transform raw\u0000input into natural language concepts. Symbolic programs then integrate these\u0000modules into an interpretable decision rule. To train LSPs, we develop a\u0000divide-and-conquer approach to incrementally build the program from scratch,\u0000where the learning process of each step is guided by LLMs. To evaluate the\u0000effectiveness of LSPs in extracting interpretable and accurate knowledge from\u0000data, we introduce IL-Bench, a collection of diverse tasks, including both\u0000synthetic and real-world scenarios across different modalities. Empirical\u0000results demonstrate LSP's superior performance compared to traditional\u0000neurosymbolic programs and vanilla automatic prompt tuning methods. Moreover,\u0000as the knowledge learned by LSP is a combination of natural language\u0000descriptions and symbolic rules, it is easily transferable to humans\u0000(interpretable), and other LLMs, and generalizes well to out-of-distribution\u0000samples.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"63 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Solving Hard Mizar Problems with Instantiation and Strategy Invention 用实例化和策略发明解决米扎难题
Pub Date : 2024-06-25 DOI: arxiv-2406.17762
Jan Jakubův, Mikoláš Janota, Josef Urban
In this work, we prove over 3000 previously ATP-unproved Mizar/MPTP problemsby using several ATP and AI methods, raising the number of ATP-solved Mizarproblems from 75% to above 80%. First, we start to experiment with the cvc5SMT solver which uses several instantiation-based heuristics that differ fromthe superposition-based systems, that were previously applied to Mizar,and addmany new solutions. Then we use automated strategy invention to develop cvc5strategies that largely improve cvc5's performance on the hard problems. Inparticular, the best invented strategy solves over 14% more problems than thebest previously available cvc5 strategy. We also show that differentclausification methods have a high impact on such instantiation-based methods,again producing many new solutions. In total, the methods solve 3021 (21.3%)of the 14163 previously unsolved hard Mizar problems. This is a new milestoneover the Mizar large-theory benchmark and a large strengthening of the hammermethods for Mizar.
在这项工作中,我们使用几种ATP和人工智能方法证明了3000多个以前ATP未证明的水蟾/MPTP问题,将ATP解决的水蟾问题的数量从75%提高到80%以上。首先,我们开始尝试使用 cvc5SMT 求解器,它使用了几种基于实例化的启发式方法,不同于之前应用于水泽的基于叠加的系统,并增加了许多新的解决方案。然后,我们使用自动策略发明来开发 cvc5 策略,这些策略在很大程度上提高了 cvc5 在难题上的性能。特别是,发明的最佳策略比以前可用的最佳 cvc5 策略多解决了超过 14% 的问题。我们还表明,不同的因果化方法对这种基于实例化的方法影响很大,同样产生了许多新的解决方案。这些方法总共解决了 14163 个以前未解决的米扎难题中的 3021 个(21.3%)。这是超越水蟾蜍大理论基准的一个新里程碑,也是对水蟾蜍锤击方法的极大加强。
{"title":"Solving Hard Mizar Problems with Instantiation and Strategy Invention","authors":"Jan Jakubův, Mikoláš Janota, Josef Urban","doi":"arxiv-2406.17762","DOIUrl":"https://doi.org/arxiv-2406.17762","url":null,"abstract":"In this work, we prove over 3000 previously ATP-unproved Mizar/MPTP problems\u0000by using several ATP and AI methods, raising the number of ATP-solved Mizar\u0000problems from 75% to above 80%. First, we start to experiment with the cvc5\u0000SMT solver which uses several instantiation-based heuristics that differ from\u0000the superposition-based systems, that were previously applied to Mizar,and add\u0000many new solutions. Then we use automated strategy invention to develop cvc5\u0000strategies that largely improve cvc5's performance on the hard problems. In\u0000particular, the best invented strategy solves over 14% more problems than the\u0000best previously available cvc5 strategy. We also show that different\u0000clausification methods have a high impact on such instantiation-based methods,\u0000again producing many new solutions. In total, the methods solve 3021 (21.3%)\u0000of the 14163 previously unsolved hard Mizar problems. This is a new milestone\u0000over the Mizar large-theory benchmark and a large strengthening of the hammer\u0000methods for Mizar.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"22 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Local Search Algorithm for MaxSMT(LIA) MaxSMT 的局部搜索算法(LIA)
Pub Date : 2024-06-22 DOI: arxiv-2406.15782
Xiang He, Bohan Li, Mengyu Zhao, Shaowei Cai
MaxSAT modulo theories (MaxSMT) is an important generalization ofSatisfiability modulo theories (SMT) with various applications. In this paper,we focus on MaxSMT with the background theory of Linear Integer Arithmetic,denoted as MaxSMT(LIA). We design the first local search algorithm forMaxSMT(LIA) called PairLS, based on the following novel ideas. A novel operatorcalled pairwise operator is proposed for integer variables. It extends theoriginal local search operator by simultaneously operating on two variables,enriching the search space. Moreover, a compensation-based picking heuristic isproposed to determine and distinguish the pairwise operations. Experiments areconducted to evaluate our algorithm on massive benchmarks. The results showthat our solver is competitive with state-of-the-art MaxSMT solvers.Furthermore, we also apply the pairwise operation to enhance the local searchalgorithm of SMT, which shows its extensibility.
MaxSAT 模态理论(MaxSMT)是可满足性模态理论(SMT)的一个重要泛化,有着广泛的应用。本文以线性整数算术为背景理论,重点研究 MaxSMT,简称 MaxSMT(LIA)。我们基于以下新思想,设计了第一个用于 MaxSMT(LIA)的局部搜索算法 PairLS。针对整数变量,我们提出了一种新颖的算子,称为成对算子。它扩展了最初的局部搜索算法,同时对两个变量进行操作,丰富了搜索空间。此外,还提出了一种基于补偿的选取启发式来确定和区分成对操作。我们在大量基准上进行了实验,以评估我们的算法。结果表明,我们的求解器与最先进的 MaxSMT 求解器相比具有很强的竞争力。此外,我们还将成对操作用于增强 SMT 的局部搜索算法,这显示了它的可扩展性。
{"title":"A Local Search Algorithm for MaxSMT(LIA)","authors":"Xiang He, Bohan Li, Mengyu Zhao, Shaowei Cai","doi":"arxiv-2406.15782","DOIUrl":"https://doi.org/arxiv-2406.15782","url":null,"abstract":"MaxSAT modulo theories (MaxSMT) is an important generalization of\u0000Satisfiability modulo theories (SMT) with various applications. In this paper,\u0000we focus on MaxSMT with the background theory of Linear Integer Arithmetic,\u0000denoted as MaxSMT(LIA). We design the first local search algorithm for\u0000MaxSMT(LIA) called PairLS, based on the following novel ideas. A novel operator\u0000called pairwise operator is proposed for integer variables. It extends the\u0000original local search operator by simultaneously operating on two variables,\u0000enriching the search space. Moreover, a compensation-based picking heuristic is\u0000proposed to determine and distinguish the pairwise operations. Experiments are\u0000conducted to evaluate our algorithm on massive benchmarks. The results show\u0000that our solver is competitive with state-of-the-art MaxSMT solvers.\u0000Furthermore, we also apply the pairwise operation to enhance the local search\u0000algorithm of SMT, which shows its extensibility.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
VeriFlow: Modeling Distributions for Neural Network Verification VeriFlow:为神经网络验证建立分布模型
Pub Date : 2024-06-20 DOI: arxiv-2406.14265
Faried Abu Zaid, Daniel Neider, Mustafa Yalçıner
Formal verification has emerged as a promising method to ensure the safetyand reliability of neural networks. Naively verifying a safety property amountsto ensuring the safety of a neural network for the whole input spaceirrespective of any training or test set. However, this also implies that thesafety of the neural network is checked even for inputs that do not occur inthe real-world and have no meaning at all, often resulting in spurious errors.To tackle this shortcoming, we propose the VeriFlow architecture as a flowbased density model tailored to allow any verification approach to restrict itssearch to the some data distribution of interest. We argue that ourarchitecture is particularly well suited for this purpose because of two majorproperties. First, we show that the transformation and log-density functionthat are defined by our model are piece-wise affine. Therefore, the modelallows the usage of verifiers based on SMT with linear arithmetic. Second,upper density level sets (UDL) of the data distribution take the shape of an$L^p$-ball in the latent space. As a consequence, representations of UDLsspecified by a given probability are effectively computable in latent space.This allows for SMT and abstract interpretation approaches with fine-grained,probabilistically interpretable, control regarding on how (a)typical the inputssubject to verification are.
形式验证已成为确保神经网络安全性和可靠性的一种有前途的方法。天真地验证安全属性相当于确保神经网络在整个输入空间的安全性,而不考虑任何训练集或测试集。为了解决这一缺陷,我们提出了 VeriFlow 架构,它是一种基于流的密度模型,专为允许任何验证方法将其搜索限制在某些感兴趣的数据分布上而定制。我们认为,我们的架构特别适合这一目的,因为它具有两大特性。首先,我们证明了我们的模型所定义的变换和对数密度函数是片断仿射的。因此,该模型允许使用基于线性运算的 SMT 校验器。其次,数据分布的上密度水平集(UDL)在潜空间中呈$L^p$球的形状。因此,由给定概率指定的 UDL 表示在潜在空间中是可以有效计算的。这使得 SMT 和抽象解释方法可以对需要验证的输入的典型程度进行细粒度的概率解释控制。
{"title":"VeriFlow: Modeling Distributions for Neural Network Verification","authors":"Faried Abu Zaid, Daniel Neider, Mustafa Yalçıner","doi":"arxiv-2406.14265","DOIUrl":"https://doi.org/arxiv-2406.14265","url":null,"abstract":"Formal verification has emerged as a promising method to ensure the safety\u0000and reliability of neural networks. Naively verifying a safety property amounts\u0000to ensuring the safety of a neural network for the whole input space\u0000irrespective of any training or test set. However, this also implies that the\u0000safety of the neural network is checked even for inputs that do not occur in\u0000the real-world and have no meaning at all, often resulting in spurious errors.\u0000To tackle this shortcoming, we propose the VeriFlow architecture as a flow\u0000based density model tailored to allow any verification approach to restrict its\u0000search to the some data distribution of interest. We argue that our\u0000architecture is particularly well suited for this purpose because of two major\u0000properties. First, we show that the transformation and log-density function\u0000that are defined by our model are piece-wise affine. Therefore, the model\u0000allows the usage of verifiers based on SMT with linear arithmetic. Second,\u0000upper density level sets (UDL) of the data distribution take the shape of an\u0000$L^p$-ball in the latent space. As a consequence, representations of UDLs\u0000specified by a given probability are effectively computable in latent space.\u0000This allows for SMT and abstract interpretation approaches with fine-grained,\u0000probabilistically interpretable, control regarding on how (a)typical the inputs\u0000subject to verification are.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"23 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The Liouville Generator for Producing Integrable Expressions 生成可积分表达式的柳维尔生成器
Pub Date : 2024-06-17 DOI: arxiv-2406.11631
Rashid Barket, Matthew England, Jürgen Gerhard
There has been a growing need to devise processes that can createcomprehensive datasets in the world of Computer Algebra, both for accuratebenchmarking and for new intersections with machine learning technology. Wepresent here a method to generate integrands that are guaranteed to beintegrable, dubbed the LIOUVILLE method. It is based on Liouville's theorem andthe Parallel Risch Algorithm for symbolic integration. We show that this data generation method retains the best qualities ofprevious data generation methods, while overcoming some of the issues builtinto that prior work. The LIOUVILLE generator is able to generate sufficientlycomplex and realistic integrands, and could be used for benchmarking or machinelearning training tasks related to symbolic integration.
在计算机代数领域,人们越来越需要设计出能够创建综合数据集的过程,以用于精确基准测试以及与机器学习技术的新交叉。我们在此介绍一种生成保证可积分的积分的方法,称为 "LIOUVILLE 方法"。该方法基于Liouville定理和符号积分并行Risch算法。我们证明,这种数据生成方法保留了以前数据生成方法的优点,同时克服了以前工作中存在的一些问题。LIOUVILLE生成器能够生成足够复杂和现实的积分,可用于与符号积分相关的基准测试或机器学习训练任务。
{"title":"The Liouville Generator for Producing Integrable Expressions","authors":"Rashid Barket, Matthew England, Jürgen Gerhard","doi":"arxiv-2406.11631","DOIUrl":"https://doi.org/arxiv-2406.11631","url":null,"abstract":"There has been a growing need to devise processes that can create\u0000comprehensive datasets in the world of Computer Algebra, both for accurate\u0000benchmarking and for new intersections with machine learning technology. We\u0000present here a method to generate integrands that are guaranteed to be\u0000integrable, dubbed the LIOUVILLE method. It is based on Liouville's theorem and\u0000the Parallel Risch Algorithm for symbolic integration. We show that this data generation method retains the best qualities of\u0000previous data generation methods, while overcoming some of the issues built\u0000into that prior work. The LIOUVILLE generator is able to generate sufficiently\u0000complex and realistic integrands, and could be used for benchmarking or machine\u0000learning training tasks related to symbolic integration.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"9 17 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546353","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Neural Concept Binder 神经概念活页夹
Pub Date : 2024-06-14 DOI: arxiv-2406.09949
Wolfgang Stammer, Antonia Wüst, David Steinmann, Kristian Kersting
The challenge in object-based visual reasoning lies in generating descriptiveyet distinct concept representations. Moreover, doing this in an unsupervisedfashion requires human users to understand a model's learned concepts andpotentially revise false concepts. In addressing this challenge, we introducethe Neural Concept Binder, a new framework for deriving discrete conceptrepresentations resulting in what we term "concept-slot encodings". Theseencodings leverage both "soft binding" via object-centric block-slot encodingsand "hard binding" via retrieval-based inference. The Neural Concept Binderfacilitates straightforward concept inspection and direct integration ofexternal knowledge, such as human input or insights from other AI models likeGPT-4. Additionally, we demonstrate that incorporating the hard bindingmechanism does not compromise performance; instead, it enables seamlessintegration into both neural and symbolic modules for intricate reasoningtasks, as evidenced by evaluations on our newly introduced CLEVR-Sudokudataset.
基于对象的视觉推理面临的挑战在于如何生成描述性的、独特的概念表征。此外,要在无监督的情况下做到这一点,需要人类用户理解模型学习到的概念,并有可能修改错误的概念。为了应对这一挑战,我们引入了神经概念绑定器(Neural Concept Binder),这是一种用于生成离散概念表征的新框架,我们称之为 "概念槽编码"(concept-slot encodings)。这些编码既可以通过以对象为中心的块槽编码实现 "软绑定",也可以通过基于检索的推理实现 "硬绑定"。神经概念绑定器有助于直接进行概念检查和直接整合外部知识,如人类输入或来自其他人工智能模型(如 GPT-4)的见解。此外,我们在新推出的 CLEVR-Sudokudataset 上进行的评估证明,采用硬绑定机制并不会影响性能;相反,它还能将复杂的推理任务无缝集成到神经和符号模块中。
{"title":"Neural Concept Binder","authors":"Wolfgang Stammer, Antonia Wüst, David Steinmann, Kristian Kersting","doi":"arxiv-2406.09949","DOIUrl":"https://doi.org/arxiv-2406.09949","url":null,"abstract":"The challenge in object-based visual reasoning lies in generating descriptive\u0000yet distinct concept representations. Moreover, doing this in an unsupervised\u0000fashion requires human users to understand a model's learned concepts and\u0000potentially revise false concepts. In addressing this challenge, we introduce\u0000the Neural Concept Binder, a new framework for deriving discrete concept\u0000representations resulting in what we term \"concept-slot encodings\". These\u0000encodings leverage both \"soft binding\" via object-centric block-slot encodings\u0000and \"hard binding\" via retrieval-based inference. The Neural Concept Binder\u0000facilitates straightforward concept inspection and direct integration of\u0000external knowledge, such as human input or insights from other AI models like\u0000GPT-4. Additionally, we demonstrate that incorporating the hard binding\u0000mechanism does not compromise performance; instead, it enables seamless\u0000integration into both neural and symbolic modules for intricate reasoning\u0000tasks, as evidenced by evaluations on our newly introduced CLEVR-Sudoku\u0000dataset.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"45 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Symbolic Computing Perspective on Software Systems 软件系统的符号计算视角
Pub Date : 2024-06-13 DOI: arxiv-2406.09085
Arthur C. Norman, Stephen M. Watt
Symbolic mathematical computing systems have served as a canary in the coalmine of software systems for more than sixty years. They have introduced orhave been early adopters of programming language ideas such ideas as dynamicmemory management, arbitrary precision arithmetic and dependent types. Thesesystems have the feature of being highly complex while at the same timeoperating in a domain where results are well-defined and clearly verifiable.These software systems span multiple layers of abstraction with concernsranging from instruction scheduling and cache pressure up to algorithmiccomplexity of constructions in algebraic geometry. All of the major symbolicmathematical computing systems include low-level code for arithmetic, memorymanagement and other primitives, a compiler or interpreter for a bespokeprogramming language, a library of high level mathematical algorithms, and someform of user interface. Each of these parts invokes multiple deep issues. We present some lessons learned from this environment and free flowingopinions on topics including: * Portability of software across architectures and decades; * Infrastructure to embrace and infrastructure to avoid; * Choosing base abstractions upon which to build; * How to get the most out of a small code base; * How developments in compilers both to optimise and to validate code havealways been and remain of critical importance, with plenty of remainingchallenges; * The way in which individuals including in particular Alan Mycroft who hasbeen able to span from hand-crafting Z80 machine code up to the most abstrusehigh level code analysis techniques are needed, and * Why it is important to teach full-stack thinking to the next generation.
六十多年来,符号数学计算系统一直是软件系统煤矿中的金丝雀。它们引入或较早采用了编程语言的思想,如动态内存管理、任意精度运算和依赖类型。这些软件系统具有高度复杂的特点,同时又在一个结果定义明确、可清晰验证的领域中运行。这些软件系统跨越多个抽象层,关注的问题从指令调度和高速缓存压力到代数几何构造的算法复杂性。所有主要的符号数学计算系统都包括运算、内存管理和其他基元的底层代码、定制编程语言的编译器或解释器、高层数学算法库以及某种形式的用户界面。其中每一部分都涉及多个深层次问题。我们将介绍从这一环境中汲取的一些经验教训,并就以下主题发表自由观点:* 软件在不同架构和不同年代的可移植性; * 需要采用的基础架构和需要避免的基础架构; * 选择构建基础抽象的方法; * 如何从小型代码库中获得最大收益; * 编译器在优化和验证代码方面的发展一直以来和现在都具有至关重要的意义,同时还面临着许多挑战;* 为什么向下一代传授全栈思维非常重要?
{"title":"A Symbolic Computing Perspective on Software Systems","authors":"Arthur C. Norman, Stephen M. Watt","doi":"arxiv-2406.09085","DOIUrl":"https://doi.org/arxiv-2406.09085","url":null,"abstract":"Symbolic mathematical computing systems have served as a canary in the coal\u0000mine of software systems for more than sixty years. They have introduced or\u0000have been early adopters of programming language ideas such ideas as dynamic\u0000memory management, arbitrary precision arithmetic and dependent types. These\u0000systems have the feature of being highly complex while at the same time\u0000operating in a domain where results are well-defined and clearly verifiable.\u0000These software systems span multiple layers of abstraction with concerns\u0000ranging from instruction scheduling and cache pressure up to algorithmic\u0000complexity of constructions in algebraic geometry. All of the major symbolic\u0000mathematical computing systems include low-level code for arithmetic, memory\u0000management and other primitives, a compiler or interpreter for a bespoke\u0000programming language, a library of high level mathematical algorithms, and some\u0000form of user interface. Each of these parts invokes multiple deep issues. We present some lessons learned from this environment and free flowing\u0000opinions on topics including: * Portability of software across architectures and decades; * Infrastructure to embrace and infrastructure to avoid; * Choosing base abstractions upon which to build; * How to get the most out of a small code base; * How developments in compilers both to optimise and to validate code have\u0000always been and remain of critical importance, with plenty of remaining\u0000challenges; * The way in which individuals including in particular Alan Mycroft who has\u0000been able to span from hand-crafting Z80 machine code up to the most abstruse\u0000high level code analysis techniques are needed, and * Why it is important to teach full-stack thinking to the next generation.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"37 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141528607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Amortized Equation Discovery in Hybrid Dynamical Systems 混合动力系统中的摊销方程发现
Pub Date : 2024-06-06 DOI: arxiv-2406.03818
Yongtuo Liu, Sara Magliacane, Miltiadis Kofinas, Efstratios Gavves
Hybrid dynamical systems are prevalent in science and engineering to expresscomplex systems with continuous and discrete states. To learn the laws ofsystems, all previous methods for equation discovery in hybrid systems follow atwo-stage paradigm, i.e. they first group time series into small clusterfragments and then discover equations in each fragment separately throughmethods in non-hybrid systems. Although effective, these methods do not fullytake advantage of the commonalities in the shared dynamics of multiplefragments that are driven by the same equations. Besides, the two-stageparadigm breaks the interdependence between categorizing and representingdynamics that jointly form hybrid systems. In this paper, we reformulate theproblem and propose an end-to-end learning framework, i.e. Amortized EquationDiscovery (AMORE), to jointly categorize modes and discover equationscharacterizing the dynamics of each mode by all segments of the mode.Experiments on four hybrid and six non-hybrid systems show that our methodoutperforms previous methods on equation discovery, segmentation, andforecasting.
混合动力系统在科学和工程领域非常普遍,用于表达具有连续和离散状态的复杂系统。为了学习系统的规律,以往在混合系统中发现方程的所有方法都遵循两阶段范式,即首先将时间序列分组为小的簇片段,然后通过非混合系统的方法分别发现每个片段中的方程。这些方法虽然有效,但不能充分利用由相同方程驱动的多个片段的共同动态的共性。此外,两阶段范式打破了共同构成混合系统的动力学分类和表示之间的相互依存关系。在本文中,我们重新阐述了这个问题,并提出了一个端到端的学习框架,即摊销方程发现(AMORE),以联合分类模式,并通过模式的所有分段发现描述每个模式动态的方程。在四个混合系统和六个非混合系统上的实验表明,我们的方法在方程发现、分段和预测方面优于之前的方法。
{"title":"Amortized Equation Discovery in Hybrid Dynamical Systems","authors":"Yongtuo Liu, Sara Magliacane, Miltiadis Kofinas, Efstratios Gavves","doi":"arxiv-2406.03818","DOIUrl":"https://doi.org/arxiv-2406.03818","url":null,"abstract":"Hybrid dynamical systems are prevalent in science and engineering to express\u0000complex systems with continuous and discrete states. To learn the laws of\u0000systems, all previous methods for equation discovery in hybrid systems follow a\u0000two-stage paradigm, i.e. they first group time series into small cluster\u0000fragments and then discover equations in each fragment separately through\u0000methods in non-hybrid systems. Although effective, these methods do not fully\u0000take advantage of the commonalities in the shared dynamics of multiple\u0000fragments that are driven by the same equations. Besides, the two-stage\u0000paradigm breaks the interdependence between categorizing and representing\u0000dynamics that jointly form hybrid systems. In this paper, we reformulate the\u0000problem and propose an end-to-end learning framework, i.e. Amortized Equation\u0000Discovery (AMORE), to jointly categorize modes and discover equations\u0000characterizing the dynamics of each mode by all segments of the mode.\u0000Experiments on four hybrid and six non-hybrid systems show that our method\u0000outperforms previous methods on equation discovery, segmentation, and\u0000forecasting.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"24 6 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141552446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Expressive Symbolic Regression for Interpretable Models of Discrete-Time Dynamical Systems 离散时间动态系统可解释模型的表达式符号回归
Pub Date : 2024-06-05 DOI: arxiv-2406.06585
Adarsh Iyer, Nibodh Boddupalli, Jeff Moehlis
Interpretable mathematical expressions defining discrete-time dynamicalsystems (iterated maps) can model many phenomena of scientific interest,enabling a deeper understanding of system behaviors. Since formulatinggoverning expressions from first principles can be difficult, it is ofparticular interest to identify expressions for iterated maps given only theirdata streams. In this work, we consider a modified Symbolic Artificial NeuralNetwork-Trained Expressions (SymANNTEx) architecture for this task, anarchitecture more expressive than others in the literature. We make amodification to the model pipeline to optimize the regression, thencharacterize the behavior of the adjusted model in identifying severalclassical chaotic maps. With the goal of parsimony, sparsity-inducing weightregularization and information theory-informed simplification are implemented.We show that our modified SymANNTEx model properly identifies single-state mapsand achieves moderate success in approximating a dual-state attractor. Theseperformances offer significant promise for data-driven scientific discovery andinterpretation.
定义离散时间动态系统(迭代图)的可解释数学表达式可以模拟许多科学现象,从而加深对系统行为的理解。由于从第一性原理出发制定管理表达式可能很困难,因此在仅给出迭代映射数据流的情况下识别其表达式就显得尤为重要。在这项工作中,我们考虑采用经过改进的符号人工神经网络训练表达式(SymANNTEx)架构来完成这项任务,这种架构比文献中的其他架构更具表现力。我们对模型管道进行了修改,以优化回归,然后描述了调整后的模型在识别几种经典混沌图时的行为。我们的研究表明,修改后的 SymANNTEx 模型可以正确识别单态图,并在近似双态吸引子方面取得了一定的成功。这些性能为数据驱动的科学发现和解释带来了重大希望。
{"title":"Expressive Symbolic Regression for Interpretable Models of Discrete-Time Dynamical Systems","authors":"Adarsh Iyer, Nibodh Boddupalli, Jeff Moehlis","doi":"arxiv-2406.06585","DOIUrl":"https://doi.org/arxiv-2406.06585","url":null,"abstract":"Interpretable mathematical expressions defining discrete-time dynamical\u0000systems (iterated maps) can model many phenomena of scientific interest,\u0000enabling a deeper understanding of system behaviors. Since formulating\u0000governing expressions from first principles can be difficult, it is of\u0000particular interest to identify expressions for iterated maps given only their\u0000data streams. In this work, we consider a modified Symbolic Artificial Neural\u0000Network-Trained Expressions (SymANNTEx) architecture for this task, an\u0000architecture more expressive than others in the literature. We make a\u0000modification to the model pipeline to optimize the regression, then\u0000characterize the behavior of the adjusted model in identifying several\u0000classical chaotic maps. With the goal of parsimony, sparsity-inducing weight\u0000regularization and information theory-informed simplification are implemented.\u0000We show that our modified SymANNTEx model properly identifies single-state maps\u0000and achieves moderate success in approximating a dual-state attractor. These\u0000performances offer significant promise for data-driven scientific discovery and\u0000interpretation.","PeriodicalId":501033,"journal":{"name":"arXiv - CS - Symbolic Computation","volume":"123 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546354","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
arXiv - CS - Symbolic Computation
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1