Neural Computing & Applications最新文献

Policy gradient methods have become one of the most popular classes of algorithms for multi-agent reinforcement learning. A key challenge, however, that is not addressed by many of these methods is multi-agent credit assignment: assessing an agent's contribution to the overall performance, which is crucial for learning good policies. We propose a novel algorithm called Dr.Reinforce that explicitly tackles this by combining difference rewards with policy gradients to allow for learning decentralized policies when the reward function is known. By differencing the reward function directly, Dr.Reinforce avoids difficulties associated with learning the Q-function as done by counterfactual multi-agent policy gradients (COMA), a state-of-the-art difference rewards method. For applications where the reward function is unknown, we show the effectiveness of a version of Dr.Reinforce that learns an additional reward network that is used to estimate the difference rewards.

While deep reinforcement learning (RL) agents outperform humans on an increasing number of tasks, training them requires data equivalent to decades of human gameplay. Recent hierarchical RL methods have increased sample efficiency by incorporating information inherent to the structure of the decision problem but at the cost of having to discover or use human-annotated sub-goals that guide the learning process. We show that intentions of human players, i.e. the precursor of goal-oriented decisions, can be robustly predicted from eye gaze even for the long-horizon sparse rewards task of Montezuma's Revenge-one of the most challenging RL tasks in the Atari2600 game suite. We propose Int-HRL: Hierarchical RL with intention-based sub-goals that are inferred from human eye gaze. Our novel sub-goal extraction pipeline is fully automatic and replaces the need for manual sub-goal annotation by human experts. Our evaluations show that replacing hand-crafted sub-goals with automatically extracted intentions leads to an HRL agent that is significantly more sample efficient than previous methods.

In this paper, we introduce multi-objective deep centralized multi-agent actor-critic (MO-DCMAC), a multi-objective reinforcement learning method for infrastructural maintenance optimization, an area traditionally dominated by single-objective reinforcement learning (RL) approaches. Previous single-objective RL methods combine multiple objectives, such as probability of collapse and cost, into a singular reward signal through reward-shaping. In contrast, MO-DCMAC can optimize a policy for multiple objectives directly, even when the utility function is nonlinear. We evaluated MO-DCMAC using two utility functions, which use probability of collapse and cost as input. The first utility function is the threshold utility, in which MO-DCMAC should minimize cost so that the probability of collapse is never above the threshold. The second is based on the failure mode, effects, and criticality analysis methodology used by asset managers to assess maintenance plans. We evaluated MO-DCMAC, with both utility functions, in multiple maintenance environments, including ones based on a case study of the historical quay walls of Amsterdam. The performance of MO-DCMAC was compared against multiple rule-based policies based on heuristics currently used for constructing maintenance plans. Our results demonstrate that MO-DCMAC outperforms traditional rule-based policies across various environments and utility functions.

This investigation explores whether machine learning can predict recidivism while addressing societal biases. To investigate this, we obtained conviction data from the UK's Police National Computer (PNC) on 346,685 records between January 1, 2000, and February 3, 2006 (His Majesty's Inspectorate of Constabulary in Use of the Police National Computer: An inspection of the ACRO Criminal Records Office. His Majesty's Inspectorate of Constabulary, Birmingham, https://assets-hmicfrs.justiceinspectorates.gov.uk/uploads/police-national-computer-use-acro-criminal-records-office.pdf, 2017). We generate twelve machine learning models-six to forecast general recidivism, and six to forecast violent recidivism-over a 3-year period, evaluated via fivefold cross-validation. Our best-performing models outperform the existing state-of-the-arts, receiving an area under curve (AUC) score of 0.8660 and 0.8375 for general and violent recidivism, respectively. Next, we construct a fairness scale that communicates the semantic and technical trade-offs associated with debiasing a criminal justice forecasting model. We use this scale to debias our best-performing models. Results indicate both models can achieve all five fairness definitions because the metrics measuring these definitions-the statistical range of recall, precision, positive rate, and error balance between demographics-indicate that these scores are within a one percentage point difference of each other. Deployment recommendations and implications are discussed. These include recommended safeguards against false positives, an explication of how these models addressed societal biases, and a case study illustrating how these models can improve existing criminal justice practices. That is, these models may help police identify fewer people in a way less impacted by structural bias while still reducing crime. A randomized control trial is proposed to test this illustrated case study, and further directions explored.

Supplementary information: The online version contains supplementary material available at 10.1007/s00521-025-11478-x.

The research in Materials Science and Engineering focuses on the design, synthesis, properties, and performance of materials. An important class of materials that is widely investigated are crystalline materials, including metals and semiconductors. Crystalline material typically contains a specific type of defect called "dislocation". This defect significantly affects various material properties, including bending strength, fracture toughness, and ductility. Researchers have devoted a significant effort in recent years to understanding dislocation behaviour through experimental characterization techniques and simulations, e.g., dislocation dynamics simulations. This paper presents how data from dislocation dynamics simulations can be modelled using semantic web technologies through annotating data with ontologies. We extend the dislocation ontology by adding missing concepts and aligning it with two other domain-related ontologies (i.e., the Elementary Multi-perspective Material Ontology and the Materials Design Ontology), allowing for efficiently representing the dislocation simulation data. Moreover, we present a real-world use case for representing the discrete dislocation dynamics data as a knowledge graph (DisLocKG) which can depict the relationship between them. We also developed a SPARQL endpoint that brings extensive flexibility for querying DisLocKG.

Pattern recognition models, particularly neural networks, often focus on maximising classification accuracy. However, in practice, the types of errors made (misclassification between different classes) can have varying associated costs. Current methods overlook varying misclassification error types. Misclassification labels can either be available from expert knowledge or derived from semantics of textual descriptions of class labels. Exploiting such misclassification costs can have significant implications when deploying machine learning systems. Here, using five examples from image and tabular domains, we show how a deep neural architecture trained in a nested layer-wise fashion (cascade learning) in which early layers solve easier problems than later ones could exploit such hierarchical aspects of class labels. We employ a measure of performance called "severity" of errors and show how emphasis could be placed on classes that are deeper in the hierarchy, ignoring errors that arise between semantic neighbours.

Supplementary information: The online version contains supplementary material available at 10.1007/s00521-025-11613-8.

Reconstructing images from observational data is a complex and time-consuming process, particularly in astronomy, where traditional algorithms like CLEAN require extensive computational resources and expert interpretation to distinguish genuine features from artifacts, especially without ground truth data. To address these challenges, we developed the Fourier convolutional decoder (FCD), a custom-made overcomplete autoencoder trained on simulated data with available ground truth. This enables the network to generate outputs that closely approximate expected ground truth. The model's versatility was demonstrated on both simulated and observational datasets, with a specific application to data from the spectrometer/telescope for imaging X-rays (STIX) on the solar orbiter. In the simulated environment, FCD's performance was evaluated using multiple-image reconstruction metrics, demonstrating its ability to produce accurate images with minimal artifacts. For observational data, FCD was compared with benchmark algorithms, focusing on reconstruction metrics related to Fourier components. Our evaluation found that FCD is the fastest imaging method, with runtimes on the order of milliseconds. Its computational cost is comparable to the most efficient reconstruction algorithm and 280 $\times$ faster than the slowest imaging method for single-image reconstruction on a CPU. Additionally, its runtime can be reduced by an order of magnitude for multiple-image reconstruction on a GPU. FCD outperforms or matches state-of-the-art methods on simulated data, achieving a mean MS-SSIM of 0.97, LPIPS of 0.04, PSNR of 35.70 dB, a Dice coefficient of 0.83, and a Hausdorff distance of 5.08. Finally, on experimental STIX observations, FCD remains competitive with top methods despite reduced performance compared to simulated data.

This paper proposes a general method of creating an accurate neural network-based surrogate model for postprocessing a topologically optimized structure. When topology optimization results are converted into computer-aided design (CAD) files with smooth boundaries for manufacturability, finite element method (FEM) based stresses often do not agree with the topology optimized results due to changes of surface and mesh density. The conversion between topology optimization derived results and CAD files often requires postprocessing, an additional fine tuning of the geometry parameters to reconcile the change of the stress values. In this work, a feedforward, deep artificial neural network (DANN) is presented with varying architecture parameters that are found for each stress output of interest. This network is trained with the data based on a combination of Design of Experiments (DoE) models that have the geometry dimensions as inputs and stress readings under various loads as the outputs. A DANN-based surrogate model is constructed to enable fine tuning of all relevant stress performance metrics. This method of constructing an artificial network-based surrogate model minimizes the number of FEM computations required to generate an optimized, post-processed design. We present a case study of postprocessing a wind tunnel balance, a measurement device that yields the six force and moment components of a test aircraft. It needs to be designed considering multiple stress measures under combinations of the six loading conditions. Excellent performance of a neural network is presented in this paper in terms of accurate prediction of the highly nonlinear stresses under combinations of the six loads. Von Mises stress predictions are within 10% and axial force sensor stress predictions are within 2% for the final post-processed topology. The results support its usefulness for postprocessing of topology optimized structures.

Communication is a widely used mechanism to promote cooperation in multi-agent systems. In the field of emergent communication, agents are typically trained in specific environments: cooperative, competitive or mixed-motive. Motivated by the idea that real-world settings are characterized by incomplete information and that humans face daily interactions under a wide spectrum of incentives, we aim to explore the role of emergent communication when simultaneously exploited across all these contexts. In this work, we pursue this line of research by focusing on social dilemmas. To do this, we developed an extended version of the Public Goods Game, which allows us to train independent reinforcement learning agents simultaneously in different scenarios where incentives are (mis)aligned to various extents. Additionally, agents experience uncertainty in terms of the alignment of their incentives with those of others. We equip agents with the ability to learn a communication policy and study the impact of emergent communication in the face of uncertainty among agents. Our findings show that in settings where all agents have the same level of uncertainty, communication can enhance the cooperation of the whole group. However, in cases of asymmetric uncertainty, the agents that do not face uncertainty learn to use communication to deceive and exploit their uncertain peers.

The concept "Internet of Things" (IoT), which facilitates communication between linked devices, is relatively new. It refers to the next generation of the Internet. IoT supports healthcare and is essential to numerous applications for tracking medical services. By examining the pattern of observed parameters, the type of the disease can be anticipated. For people with a range of diseases, health professionals and technicians have developed an excellent system that employs commonly utilized techniques like wearable technology, wireless channels, and other remote equipment to give low-cost healthcare monitoring. Whether put in living areas or worn on the body, network-related sensors gather detailed data to evaluate the patient's physical and mental health. The main objective of this study is to examine the current e-health monitoring system using integrated systems. Automatically providing patients with a prescription based on their status is the main goal of the e-health monitoring system. The doctor can keep an eye on the patient's health without having to communicate with them. The purpose of the study is to examine how IoT technologies are applied in the medical industry and how they help to raise the bar of healthcare delivered by healthcare institutions. The study will also include the uses of IoT in the medical area, the degree to which it is used to enhance conventional practices in various health fields, and the degree to which IoT may raise the standard of healthcare services. The main contributions in this paper are as follows: (1) importing signals from wearable devices, extracting signals from non-signals, performing peak enhancement; (2) processing and analyzing the incoming signals; (3) proposing a new stress monitoring algorithm (SMA) using wearable sensors; (4) comparing between various ML algorithms; (5) the proposed stress monitoring algorithm (SMA) is composed of four main phases: (a) data acquisition phase, (b) data and signal processing phase, (c) prediction phase, and (d) model performance evaluation phase; and (6) grid search is used to find the optimal values for hyperparameters of SVM (C and gamma). From the findings, it is shown that random forest is best suited for this classification, with decision tree and XGBoost following closely behind.