Pub Date : 2025-06-13DOI: 10.1016/j.artint.2025.104384
Ali Zakeri, Zhuowen Zou, Hanning Chen, Mohsen Imani
Graph analysis has emerged as a crucial field, offering versatile solutions for real-world data representation, from social networks to biological systems. However, the intricate nature of graphs often necessitates a degree of processing, such as learning mappings to a vector space, to perform analysis tasks like node classification and link prediction. A promising approach to this is Hyperdimensional Computing (HDC), inspired by neuroscience and mathematics. HDC utilizes high-dimensional vectors to efficiently manipulate complex data structures and perform operations like superposition and association, enhancing knowledge graph representations with contextual and semantic information. Nevertheless, addressing limitations in existing HDC-based approaches to graph representation is essential. This paper thoroughly explores these methods and presents ConfiGR: Configurable Graph Representation, a novel framework that introduces an adjustable design, enhancing its versatility across various graph types and tasks, ultimately boosting performance in multiple graph-related tasks.
{"title":"Configurable hyperdimensional graph representation","authors":"Ali Zakeri, Zhuowen Zou, Hanning Chen, Mohsen Imani","doi":"10.1016/j.artint.2025.104384","DOIUrl":"10.1016/j.artint.2025.104384","url":null,"abstract":"<div><div>Graph analysis has emerged as a crucial field, offering versatile solutions for real-world data representation, from social networks to biological systems. However, the intricate nature of graphs often necessitates a degree of processing, such as learning mappings to a vector space, to perform analysis tasks like node classification and link prediction. A promising approach to this is Hyperdimensional Computing (HDC), inspired by neuroscience and mathematics. HDC utilizes high-dimensional vectors to efficiently manipulate complex data structures and perform operations like superposition and association, enhancing knowledge graph representations with contextual and semantic information. Nevertheless, addressing limitations in existing HDC-based approaches to graph representation is essential. This paper thoroughly explores these methods and presents ConfiGR: Configurable Graph Representation, a novel framework that introduces an adjustable design, enhancing its versatility across various graph types and tasks, ultimately boosting performance in multiple graph-related tasks.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104384"},"PeriodicalIF":5.1,"publicationDate":"2025-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144305109","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-13DOI: 10.1016/j.artint.2025.104387
Tian-Zuo Wang , Lue Tao , Tian Qin , Zhi-Hua Zhou
Causal effect estimation from observational data is a fundamental task in artificial intelligence and has been widely studied given known causal relations. However, in the presence of latent confounders, only a part of causal relations can be identified from observational data, characterized by a partial ancestral graph (PAG), where some causal relations are indeterminate. In such cases, the causal effect is often unidentifiable, as there could be super-exponential number of potential causal graphs consistent with the identified PAG but associated with different causal effects. In this paper, we target on set determination within a PAG, i.e., determining the set of possible causal effects of a specified variable X on another variable Y via covariate adjustment. We develop the first set determination method that does not require enumerating any causal graphs. Furthermore, we present two novel orientation rules for incorporating structural background knowledge (BK) into a PAG, which facilitate the identification of additional causal relations given BK. Notably, we show that these rules can further enhance the efficiency of our set determination method, as certain transformed edges during the procedure can be interpreted as BK and enable the rules to reveal further causal information. Theoretically and empirically, we demonstrate that our set determination methods can yield the same results as the enumeration-based method with super-exponentially less computational complexity.
{"title":"Estimating possible causal effects with latent variables via adjustment and novel rule orientation","authors":"Tian-Zuo Wang , Lue Tao , Tian Qin , Zhi-Hua Zhou","doi":"10.1016/j.artint.2025.104387","DOIUrl":"10.1016/j.artint.2025.104387","url":null,"abstract":"<div><div>Causal effect estimation from observational data is a fundamental task in artificial intelligence and has been widely studied given known causal relations. However, in the presence of latent confounders, only a part of causal relations can be identified from observational data, characterized by a partial ancestral graph (PAG), where some causal relations are indeterminate. In such cases, the causal effect is often unidentifiable, as there could be super-exponential number of potential causal graphs consistent with the identified PAG but associated with different causal effects. In this paper, we target on <em>set determination</em> within a PAG, <em>i.e.</em>, determining the set of possible causal effects of a specified variable <em>X</em> on another variable <em>Y</em> via covariate adjustment. We develop the first set determination method that does not require enumerating any causal graphs. Furthermore, we present two novel orientation rules for incorporating structural background knowledge (BK) into a PAG, which facilitate the identification of additional causal relations given BK. Notably, we show that these rules can further enhance the efficiency of our set determination method, as certain transformed edges during the procedure can be interpreted as BK and enable the rules to reveal further causal information. Theoretically and empirically, we demonstrate that our set determination methods can yield the same results as the enumeration-based method with <em>super-exponentially less</em> computational complexity.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104387"},"PeriodicalIF":5.1,"publicationDate":"2025-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144298811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-13DOI: 10.1016/j.artint.2025.104383
Lianghe Shi, Weiwei Liu
Unsupervised domain adaptation (UDA) has been successfully applied in many contexts with domain shifts. However, we find that existing UDA methods are vulnerable to adversarial attacks. A direct modification of the existing UDA methods to improve adversarial robustness is to feed the algorithms with adversarial source examples. However, empirical results show that traditional discrepancy fails to measure the distance between adversarial examples, leading to poor alignment between adversarial examples of source and target domains and inefficient transfer of the robustness from source domain to target domain. And the traditional theoretical bounds do not always hold in adversarial scenarios. Accordingly, we first propose a novel adversarial discrepancy (AD) to narrow the gap between adversarial robustness and UDA. Based on AD, this paper provides a generalization error bound for adversarially robust unsupervised domain adaptation through the lens of Rademacher complexity, theoretically demonstrating that the expected adversarial target error can be bounded by empirical adversarial source error and AD. We also present the upper bounds of Rademacher complexity, with a particular focus on linear models and multi-layer neural networks under attack (). Inspired by this theory, we go on to develop an adversarially robust algorithm for UDA. We further conduct comprehensive experiments to support our theory and validate the robustness improvement of our proposed method on challenging domain adaptation tasks.
{"title":"Adversarially robust unsupervised domain adaptation","authors":"Lianghe Shi, Weiwei Liu","doi":"10.1016/j.artint.2025.104383","DOIUrl":"10.1016/j.artint.2025.104383","url":null,"abstract":"<div><div>Unsupervised domain adaptation (UDA) has been successfully applied in many contexts with domain shifts. However, we find that existing UDA methods are vulnerable to adversarial attacks. A direct modification of the existing UDA methods to improve adversarial robustness is to feed the algorithms with adversarial source examples. However, empirical results show that traditional discrepancy fails to measure the distance between adversarial examples, leading to poor alignment between adversarial examples of source and target domains and inefficient transfer of the robustness from source domain to target domain. And the traditional theoretical bounds do not always hold in adversarial scenarios. Accordingly, we first propose a novel adversarial discrepancy (AD) to narrow the gap between adversarial robustness and UDA. Based on AD, this paper provides a generalization error bound for adversarially robust unsupervised domain adaptation through the lens of Rademacher complexity, theoretically demonstrating that the expected adversarial target error can be bounded by empirical adversarial source error and AD. We also present the upper bounds of Rademacher complexity, with a particular focus on linear models and multi-layer neural networks under <span><math><msub><mrow><mi>ℓ</mi></mrow><mrow><mi>r</mi></mrow></msub></math></span> attack (<span><math><mi>r</mi><mo>≥</mo><mn>1</mn></math></span>). Inspired by this theory, we go on to develop an adversarially robust algorithm for UDA. We further conduct comprehensive experiments to support our theory and validate the robustness improvement of our proposed method on challenging domain adaptation tasks.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104383"},"PeriodicalIF":5.1,"publicationDate":"2025-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144305110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-06-04DOI: 10.1016/j.artint.2025.104372
S. Ardizzoni , L. Consolini , M. Locatelli , B. Nebel , I. Saccani
On an assigned graph, the problem of Multi-Agent Pathfinding (MAPF) consists in finding paths for multiple agents, avoiding collisions. Finding the minimum-length solution is known to be NP-hard, and computation times grows exponentially with the number of agents. However, in industrial applications, it is important to find feasible, suboptimal solutions, in a time that grows polynomially with the number of agents. Such algorithms exist for undirected and biconnected directed graphs. Our main contribution is to generalize these algorithms to the more general case of strongly connected directed graphs. In particular, we describe a procedure that checks the problem feasibility in linear time with respect to the number of vertices n, and we find a necessary and sufficient condition for feasibility of any MAPF instance. Moreover, we present an algorithm (diSC) that provides a feasible solution of length , where k is the number of agents and c the maximum length of the corridors of the graph.
{"title":"Multi-agent pathfinding on strongly connected digraphs: Feasibility and solution algorithms","authors":"S. Ardizzoni , L. Consolini , M. Locatelli , B. Nebel , I. Saccani","doi":"10.1016/j.artint.2025.104372","DOIUrl":"10.1016/j.artint.2025.104372","url":null,"abstract":"<div><div>On an assigned graph, the problem of Multi-Agent Pathfinding (MAPF) consists in finding paths for multiple agents, avoiding collisions. Finding the minimum-length solution is known to be NP-hard, and computation times grows exponentially with the number of agents. However, in industrial applications, it is important to find feasible, suboptimal solutions, in a time that grows polynomially with the number of agents. Such algorithms exist for undirected and biconnected directed graphs. Our main contribution is to generalize these algorithms to the more general case of strongly connected directed graphs. In particular, we describe a procedure that checks the problem feasibility in linear time with respect to the number of vertices <em>n</em>, and we find a necessary and sufficient condition for feasibility of any MAPF instance. Moreover, we present an algorithm (diSC) that provides a feasible solution of length <span><math><mi>O</mi><mo>(</mo><mi>k</mi><msup><mrow><mi>n</mi></mrow><mrow><mn>2</mn></mrow></msup><mi>c</mi><mo>)</mo></math></span>, where <em>k</em> is the number of agents and <em>c</em> the maximum length of the corridors of the graph.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104372"},"PeriodicalIF":5.1,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144241420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-23DOI: 10.1016/j.artint.2025.104362
Marco Mussi , Simone Drago , Marcello Restelli, Alberto Maria Metelli
In several real-world sequential decision problems, at every step, the learner is required to select different actions. Every action affects a specific part of the system and generates an observable intermediate effect. In this paper, we introduce the Factored-Reward Bandits (FRBs), a novel setting able to effectively capture and exploit the structure of this class of scenarios, where the reward is computed as the product of the action intermediate observations. We characterize the statistical complexity of the learning problem in the FRBs, by deriving worst-case and asymptotic instance-dependent regret lower bounds. Then, we devise and analyze two regret minimization algorithms. The former, F-UCB, is an anytime optimistic approach matching the worst-case lower bound (up to logarithmic factors) but fails to perform optimally from the instance-dependent perspective. The latter, F-Track, is a bound-tracking approach, that enjoys optimal asymptotic instance-dependent regret guarantees. Finally, we study the problem of performing best arm identification in this setting. We derive an error probability lower bound, and we develop F-SR, a nearly optimal rejection-based algorithm for identifying the best action vector, given a time budget.2
{"title":"Factored-reward bandits with intermediate observations: Regret minimization and best arm identification","authors":"Marco Mussi , Simone Drago , Marcello Restelli, Alberto Maria Metelli","doi":"10.1016/j.artint.2025.104362","DOIUrl":"10.1016/j.artint.2025.104362","url":null,"abstract":"<div><div>In several real-world sequential decision problems, at every step, the learner is required to select different actions. Every action affects a specific part of the system and generates an observable intermediate effect. In this paper, we introduce the Factored-Reward Bandits (FRBs), a novel setting able to effectively capture and exploit the structure of this class of scenarios, where the reward is computed as the product of the action intermediate observations. We characterize the statistical complexity of the learning problem in the FRBs, by deriving worst-case and asymptotic instance-dependent regret lower bounds. Then, we devise and analyze two regret minimization algorithms. The former, <span>F-UCB</span>, is an anytime optimistic approach matching the worst-case lower bound (up to logarithmic factors) but fails to perform optimally from the instance-dependent perspective. The latter, <span>F-Track</span>, is a bound-tracking approach, that enjoys optimal asymptotic instance-dependent regret guarantees. Finally, we study the problem of performing best arm identification in this setting. We derive an error probability lower bound, and we develop <span>F-SR</span>, a nearly optimal rejection-based algorithm for identifying the best action vector, given a time budget.<span><span><sup>2</sup></span></span></div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104362"},"PeriodicalIF":5.1,"publicationDate":"2025-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144263063","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-22DOI: 10.1016/j.artint.2025.104363
Wenjing Yang , Haoang Chi , Yibing Zhan , Bowen Hu , Xiaoguang Ren , Dapeng Tao , Long Lan
Few-shot domain adaptation (FDA) aims to train a target model with clean labeled data from the source domain and few labeled data from the target domain. Given a limited annotation budget, source data may contain many noisy labels, which can detrimentally impact the performance of models in real-world applications. This problem setting is denoted as wildly few-shot domain adaptation (WFDA), simultaneously taking care of label noise and data shortage. While previous studies have achieved some success, they typically rely on multiple adaptation models to collaboratively filter noisy labels, resulting in substantial computational overhead. To address WFDA more simply and elegantly, we offer a theoretical analysis of this problem and propose a comprehensive upper bound for the excess risk on the target domain. Our theoretical result reveals that correct domain-invariant representations can be obtained even in the presence of source noise and limited target data without incurring additional costs. In response, we propose a simple yet effective WFDA method, referred to as noise-tolerant few-shot adaptation network (NT-FAN). Experiments demonstrate that our method significantly outperforms all the state-of-the-art competitors while maintaining a more lightweight architecture. Notably, NT-FAN consistently exhibits robust performance when dealing with more realistic and intractable source noise (e.g., instance-dependent label noise) and severe source noise (e.g., a 40% noise rate) in the source domain.
{"title":"NT-FAN: A simple yet effective noise-tolerant few-shot adaptation network","authors":"Wenjing Yang , Haoang Chi , Yibing Zhan , Bowen Hu , Xiaoguang Ren , Dapeng Tao , Long Lan","doi":"10.1016/j.artint.2025.104363","DOIUrl":"10.1016/j.artint.2025.104363","url":null,"abstract":"<div><div><em>Few-shot domain adaptation</em> (FDA) aims to train a target model with <em>clean</em> labeled data from the source domain and <em>few</em> labeled data from the target domain. Given a limited annotation budget, source data may contain many noisy labels, which can detrimentally impact the performance of models in real-world applications. This problem setting is denoted as <em>wildly few-shot domain adaptation</em> (WFDA), simultaneously taking care of label noise and data shortage. While previous studies have achieved some success, they typically rely on multiple adaptation models to collaboratively filter noisy labels, resulting in substantial computational overhead. To address WFDA more simply and elegantly, we offer a theoretical analysis of this problem and propose a comprehensive upper bound for the excess risk on the target domain. Our theoretical result reveals that correct domain-invariant representations can be obtained even in the presence of source noise and limited target data without incurring additional costs. In response, we propose a simple yet effective WFDA method, referred to as <em>noise-tolerant few-shot adaptation network</em> (NT-FAN). Experiments demonstrate that our method significantly outperforms all the state-of-the-art competitors while maintaining a more <em>lightweight</em> architecture. Notably, NT-FAN consistently exhibits robust performance when dealing with more realistic and intractable source noise (e.g., instance-dependent label noise) and severe source noise (e.g., a 40% noise rate) in the source domain.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"346 ","pages":"Article 104363"},"PeriodicalIF":5.1,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144139326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hybrid Knowledge Bases (HKBs) successfully integrate Logic Programming (LP) and Description Logics (DL) under the Minimal Knowledge with Negation as Failure semantics. Both world closure assumptions (open and closed) can be used in the same HKB, a feature required in many domains, such as the legal and health-care ones. In previous work, we proposed (function-free) Probabilistic HKBs, whose semantics applied Sato's distribution semantics approach to the well-founded HKB semantics proposed by Knorr et al. and Lyu and You. This semantics relied on the fact that the grounding of a function-free Probabilistic HKB (PHKB) is finite. In this article, we extend the PHKB language to allow function symbols, obtaining PHKBFS. Because the grounding of a PHKBFS can be infinite, we propose a novel semantics which does not require the PHKBFS's grounding to be finite. We show that the proposed semantics extends the previously proposed semantics and that, for a large class of PHKBFS, every query can be assigned a probability.
{"title":"A semantics for probabilistic hybrid knowledge bases with function symbols","authors":"Marco Alberti , Evelina Lamma , Fabrizio Riguzzi , Riccardo Zese","doi":"10.1016/j.artint.2025.104361","DOIUrl":"10.1016/j.artint.2025.104361","url":null,"abstract":"<div><div>Hybrid Knowledge Bases (HKBs) successfully integrate Logic Programming (LP) and Description Logics (DL) under the Minimal Knowledge with Negation as Failure semantics. Both world closure assumptions (open and closed) can be used in the same HKB, a feature required in many domains, such as the legal and health-care ones. In previous work, we proposed (function-free) Probabilistic HKBs, whose semantics applied Sato's distribution semantics approach to the well-founded HKB semantics proposed by Knorr et al. and Lyu and You. This semantics relied on the fact that the grounding of a function-free Probabilistic HKB (PHKB) is finite. In this article, we extend the PHKB language to allow function symbols, obtaining PHKB<sup><em>FS</em></sup>. Because the grounding of a PHKB<sup><em>FS</em></sup> can be infinite, we propose a novel semantics which does not require the PHKB<sup><em>FS</em></sup>'s grounding to be finite. We show that the proposed semantics extends the previously proposed semantics and that, for a large class of PHKB<sup><em>FS</em></sup>, every query can be assigned a probability.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"346 ","pages":"Article 104361"},"PeriodicalIF":5.1,"publicationDate":"2025-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144098911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-19DOI: 10.1016/j.artint.2025.104357
Yanyu Liu , Yinghui Pan , Yifeng Zeng , Biyang Ma , Prashant Doshi
A multiagent sequential decision problem has been seen in many critical applications including urban transportation, autonomous driving cars, military operations, etc. Its widely known solution, namely multiagent reinforcement learning, has evolved tremendously in recent years. Among them, the solution paradigm of modeling other agents attracts our interest, which is different from traditional value decomposition or communication mechanisms. It enables agents to understand and anticipate others' behaviors and facilitates their collaboration. Inspired by recent research on the legibility that allows agents to reveal their intentions through their behavior, we propose a multiagent active legibility framework to improve their performance. The legibility-oriented framework drives agents to conduct legible actions so as to help others optimize their behaviors. In addition, we design a series of problem domains that emulate a common legibility-needed scenario and effectively characterize the legibility in multiagent reinforcement learning. The experimental results demonstrate that the new framework is more efficient and requires less training time compared to several multiagent reinforcement learning algorithms.
{"title":"Active legibility in multiagent reinforcement learning","authors":"Yanyu Liu , Yinghui Pan , Yifeng Zeng , Biyang Ma , Prashant Doshi","doi":"10.1016/j.artint.2025.104357","DOIUrl":"10.1016/j.artint.2025.104357","url":null,"abstract":"<div><div>A multiagent sequential decision problem has been seen in many critical applications including urban transportation, autonomous driving cars, military operations, etc. Its widely known solution, namely multiagent reinforcement learning, has evolved tremendously in recent years. Among them, the solution paradigm of modeling other agents attracts our interest, which is different from traditional value decomposition or communication mechanisms. It enables agents to understand and anticipate others' behaviors and facilitates their collaboration. Inspired by recent research on the legibility that allows agents to reveal their intentions through their behavior, we propose a <em>multiagent active legibility framework</em> to improve their performance. The legibility-oriented framework drives agents to conduct legible actions so as to help others optimize their behaviors. In addition, we design a series of problem domains that emulate a common legibility-needed scenario and effectively characterize the legibility in multiagent reinforcement learning. The experimental results demonstrate that the new framework is more efficient and requires less training time compared to several multiagent reinforcement learning algorithms.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"346 ","pages":"Article 104357"},"PeriodicalIF":5.1,"publicationDate":"2025-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144098910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-16DOI: 10.1016/j.artint.2025.104360
Pierre Baldi, Antonios Alexos, Ian Domingo, Alireza Rahmansetayesh
We develop a general theory of synaptic neural balance and how it can emerge or be enforced in neural networks. For a given additive cost function R (regularizer), a neuron is said to be in balance if the total cost of its input weights is equal to the total cost of its output weights. The basic example is provided by feedforward networks of ReLU units trained with regularizers, which exhibit balance after proper training. The theory explains this phenomenon and extends it in several directions. The first direction is the extension to bilinear and other activation functions. The second direction is the extension to more general regularizers, including all () regularizers. The third direction is the extension to non-layered architectures, recurrent architectures, convolutional architectures, as well as architectures with mixed activation functions and to different balancing algorithms. Gradient descent on the error function alone does not converge in general to a balanced state, where every neuron is in balance, even when starting from a balanced state. However, gradient descent on the regularized error function ought to converge to a balanced state, and thus network balance can be used to assess learning progress. The theory is based on two local neuronal operations: scaling which is commutative, and balancing which is not commutative. Finally, and most importantly, given any set of weights, when local balancing operations are applied to each neuron in a stochastic manner, global order always emerges through the convergence of the stochastic balancing algorithm to the same unique set of balanced weights. The reason for this convergence is the existence of an underlying strictly convex optimization problem where the relevant variables are constrained to a linear, only architecture-dependent, manifold. Simulations show that balancing neurons prior to learning, or during learning in alternation with gradient descent steps, can improve learning speed and performance thereby expanding the arsenal of available training tools. Scaling and balancing operations are entirely local and thus physically plausible in biological and neuromorphic neural networks.
{"title":"A theory of synaptic neural balance: From local to global order","authors":"Pierre Baldi, Antonios Alexos, Ian Domingo, Alireza Rahmansetayesh","doi":"10.1016/j.artint.2025.104360","DOIUrl":"10.1016/j.artint.2025.104360","url":null,"abstract":"<div><div>We develop a general theory of synaptic neural balance and how it can emerge or be enforced in neural networks. For a given additive cost function <em>R</em> (regularizer), a neuron is said to be in balance if the total cost of its input weights is equal to the total cost of its output weights. The basic example is provided by feedforward networks of ReLU units trained with <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub></math></span> regularizers, which exhibit balance after proper training. The theory explains this phenomenon and extends it in several directions. The first direction is the extension to bilinear and other activation functions. The second direction is the extension to more general regularizers, including all <span><math><msub><mrow><mi>L</mi></mrow><mrow><mi>p</mi></mrow></msub></math></span> (<span><math><mi>p</mi><mo>></mo><mn>0</mn></math></span>) regularizers. The third direction is the extension to non-layered architectures, recurrent architectures, convolutional architectures, as well as architectures with mixed activation functions and to different balancing algorithms. Gradient descent on the error function alone does not converge in general to a balanced state, where every neuron is in balance, even when starting from a balanced state. However, gradient descent on the regularized error function ought to converge to a balanced state, and thus network balance can be used to assess learning progress. The theory is based on two local neuronal operations: scaling which is commutative, and balancing which is not commutative. Finally, and most importantly, given any set of weights, when local balancing operations are applied to each neuron in a stochastic manner, global order always emerges through the convergence of the stochastic balancing algorithm to the same unique set of balanced weights. The reason for this convergence is the existence of an underlying strictly convex optimization problem where the relevant variables are constrained to a linear, only architecture-dependent, manifold. Simulations show that balancing neurons prior to learning, or during learning in alternation with gradient descent steps, can improve learning speed and performance thereby expanding the arsenal of available training tools. Scaling and balancing operations are entirely local and thus physically plausible in biological and neuromorphic neural networks.</div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"346 ","pages":"Article 104360"},"PeriodicalIF":5.1,"publicationDate":"2025-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144106474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-05-15DOI: 10.1016/j.artint.2025.104359
Asahi Ushio, Jose Camacho-Collados, Steven Schockaert
Many applications need access to background knowledge about how different concepts and entities are related. Although Large Language Models (LLM) can address this need to some extent, LLMs are inefficient and difficult to control. As an alternative, we propose to extract relation embeddings from relatively small language models. In particular, we show that masked language models such as RoBERTa can be straightforwardly fine-tuned for this purpose, using only a small amount of training data. The resulting model, which we call RelBERT, captures relational similarity in a surprisingly fine-grained way, allowing us to set a new state-of-the-art in analogy benchmarks. Crucially, RelBERT is capable of modelling relations that go well beyond what the model has seen during training. For instance, we obtained strong results on relations between named entities with a model that was only trained on lexical relations between concepts, and we observed that RelBERT can recognise morphological analogies despite not being trained on such examples. Overall, we find that RelBERT significantly outperforms strategies based on prompting language models that are several orders of magnitude larger, including recent GPT-based models and open source models.1
{"title":"RelBERT: Embedding relations with language models","authors":"Asahi Ushio, Jose Camacho-Collados, Steven Schockaert","doi":"10.1016/j.artint.2025.104359","DOIUrl":"10.1016/j.artint.2025.104359","url":null,"abstract":"<div><div>Many applications need access to background knowledge about how different concepts and entities are related. Although Large Language Models (LLM) can address this need to some extent, LLMs are inefficient and difficult to control. As an alternative, we propose to extract relation embeddings from relatively small language models. In particular, we show that masked language models such as RoBERTa can be straightforwardly fine-tuned for this purpose, using only a small amount of training data. The resulting model, which we call RelBERT, captures relational similarity in a surprisingly fine-grained way, allowing us to set a new state-of-the-art in analogy benchmarks. Crucially, RelBERT is capable of modelling relations that go well beyond what the model has seen during training. For instance, we obtained strong results on relations between named entities with a model that was only trained on lexical relations between concepts, and we observed that RelBERT can recognise morphological analogies despite not being trained on such examples. Overall, we find that RelBERT significantly outperforms strategies based on prompting language models that are several orders of magnitude larger, including recent GPT-based models and open source models.<span><span><sup>1</sup></span></span></div></div>","PeriodicalId":8434,"journal":{"name":"Artificial Intelligence","volume":"347 ","pages":"Article 104359"},"PeriodicalIF":5.1,"publicationDate":"2025-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144190068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号