Journal of Systems and Software最新文献

Microservice architectures and design patterns enhance the development of large-scale applications by promoting flexibility. Industrial practitioners perceive the importance of applying architectural patterns but they struggle to quantify their impact on system quality requirements. Our research aims to quantify the effect of design patterns on system performance metrics, e.g., service latency and resource utilization, even more so when the patterns operate in real-world environments subject to heterogeneous workloads. We built a cloud infrastructure to host a well-established benchmark system that represents our test bed, complemented by the implementation of three design patterns: Gateway Aggregation, Gateway Offloading, Pipe and Filters. Real performance measurements are collected and compared with model-based predictions that we derived as part of our previous research, thus further consolidating the actual impact of these patterns. Our results demonstrate that, despite the difficulty to parameterize our benchmark system, model-based predictions are in line with real experimentation, since the performance behaviors of patterns, e.g., bottleneck switches, are mostly preserved. In summary, this is the first work that experimentally demonstrates the performance behavior of microservices-based architectural patterns. Results highlight the complexity of evaluating the performance of design patterns and emphasize the need for complementing theoretical models with empirical data.

Architecture smells are a widely established concept to describe symptoms of software degradation by measuring perceived violations of software-design principles. As such, architecture smells can help developers assess and understand the architectural quality of their software system. However, research has rarely been concerned with how architecture smells evolve and whether they actually foster software degradation during a system’s evolution. Building on our previous work in this direction, we present extended techniques for measuring architecture smells, novel visualizations, as well as an empirical study of how architecture smells evolve and what typical patterns they exhibit in 485 releases of 14 open-source systems. Among others, the results of our study indicate that especially cyclic dependencies on the class-level are prone to becoming highly complex over time, with one of the reasons being the continued merging of smells, most often resulting in tangled multi-hubs. Moreover, we found unstable dependencies to mostly grow slowly over time, whereas hub-like dependencies remain rather stable during a system’s evolution. These findings are valuable for practitioners to identify and tackle system degeneration, as well as for researchers to scope new research on managing architecture smells and technical debt.

In the past couple of decades, significant research efforts have been devoted to the prediction of software bugs (i.e., defects). In general, these works leverage a diverse set of metrics, tools, and techniques to predict which classes, methods, lines, or commits are buggy. However, most existing work in this domain treats all bugs the same, which is not the case in practice. The more severe the bugs the higher their consequences. Therefore, it is important for a defect prediction method to estimate the severity of the identified bugs, so that the higher severity ones get immediate attention. In this paper, we provide a quantitative and qualitative study on two popular datasets (Defects4J and Bugs.jar), using 10 common source code metrics, and two popular static analysis tools (SpotBugs and Infer) for analyzing their capability to predict defects and their severity. We studied 3,358 buggy methods with different severity labels from 19 Java open-source projects. Results show that although code metrics are useful in predicting buggy code (Lines of the Code, Maintainable Index, FanOut, and Effort metrics are the best), they cannot estimate the severity level of the bugs. In addition, we observed that static analysis tools have weak performance in both predicting bugs (F1 score range of 3.1%–7.1%) and their severity label (F1 score under 2%). We also manually studied the characteristics of the severe bugs to identify possible reasons behind the weak performance of code metrics and static analysis tools in estimating their severity. Also, our categorization shows that Security bugs have high severity in most cases while Edge/Boundary faults have low severity. Finally, we discuss the practical implications of the results and propose new directions for future research.

Just-in-time (JIT) defect prediction helps rationally allocate testing resources and reduce testing costs. However, most JIT defect prediction models lack explainability, which significantly affects their credibility. Recently, the local interpretable model-agnostic explanations (LIME) method has been used widely in model-explainable research, and many improved LIME-based methods have been proposed. However, problems with respect to explanation effectiveness and reliability remain, which seriously affects the practical use of LIME. To address this problem, CfExplainer, a local rule-based model-agnostic approach, is proposed. The approach first applies counterfactuals to generate synthetic instances. It then mines weighted class association rules based on synthetic instances, and it optimises the process of generating, ranking, pruning, and predicting the class association rules. Next, it employs the rules with the highest priority to explain the prediction results of the model. Experiments were conducted using the public datasets employed in related studies. Compared to other state-of-the-art methods, in terms of explanation effectiveness, CfExplainer's instance similarity improves by 26.5 %-31.2 %, and local model fittness improves by 2.0 %-3.5 %, 2.3 %-3 %, and 0.7 %-7.5 % on the AUC, F1-score, and Popt metrics, respectively. In terms of the reliability of the explanation, explanations that are 2.6 %-4.7 % more unique and 2.5 %-5.9 % more consistent with the actual characteristics of defect-introducing commits than other state-of-the-art methods. Thus, the explanations of the proposed approach can enhance the model credibility and help guide developers in fixing defects and reducing the risk of introducing them.

The increasing complexity of modern systems poses numerous challenges at all stages of system development and operation. Continuous software and system engineering processes, e.g., DevOps, are increasingly adopted and spread across organizations. In parallel, many leading companies have begun to apply artificial intelligence (AI) principles and techniques, including Machine Learning (ML), to improve their products. However, there is no holistic approach that can support and enhance the growing challenges of DevOps. In this paper, we propose a software architecture that provides the foundations of a model-based framework for the development of AI-augmented solutions incorporating methods and tools for continuous software and system engineering and validation. The key characteristic of the proposed architecture is that it allows leveraging the advantages of both AI/ML and Model Driven Engineering (MDE) approaches and techniques in a DevOps context. This architecture has been designed, developed and applied in the context of the European large collaborative project named AIDOaRt. In this paper, we also report on the practical evaluation of this architecture. This evaluation is based on a significant set of technical solutions implemented and applied in the context of different real industrial case studies coming from the AIDOaRt project. Moreover, we analyze the collected results and discuss them according to both architectural and technical challenges we intend to tackle with the proposed architecture.

Digital technologies are an essential resource for maximizing education opportunities, yet the COVID-19 pandemic has exposed learning inequities, particularly among underrepresented groups such as children with autism. In this study, we have evaluated the quality of a distinctive dataset of multiplatform software applications, encompassing both assistive and mainstream software, first-hand acquired from special education professionals and families of children with autism. Through a heuristic evaluation based on a system indicators, and an in-depth analysis, we aimed to (1) assess the quality and effectiveness of assistive technologies in supporting the education of children with autism; (2) determine the adaptability of mainstream applications to the unique educational needs of children with autism; and (3) explore the features and constraints of applications targeting children with ASD, categorized according to the needs they cover. The resulting quality ranking, organized by cognitive domains, provides insights into the engagement and effectiveness of applications supporting the learning of children with autism. Furthermore, the findings delineating the functionalities and limitations of these applications contribute to the identification of necessary software engineering best practices. These practices align with user-centered design principles and drive the development of accessible software, thereby fostering high-quality inclusive education for children with autism.

Software aging is one of the significant factors affecting the reliability and availability of long-running software systems, such as Android, Cloud systems, etc. The time to aging failure (TTAF) prediction for software systems plays a crucial role in proactive rejuvenation scheduling through machine learning or statistical analysis techniques, due to its ability to determine when to perform rejuvenation to mitigate the aging effects. However, software aging characterization is relatively complicated, and only fitting the variations for a single aging indicator cannot grasp the comprehensive degradation process across different case systems; moreover, since software systems often exhibit long and short-term inherent degradation characteristics, existing prediction models possess a poor ability for modeling both global and local information simultaneously. To tackle the above problems, a novel TTAF prediction framework based on the parallel multi-scale temporal convolution network and transformer (named PMTT) is proposed, by mapping various system running indicators reflecting the software aging to TTAF. PMTT possesses the following distinctive characteristics. First, a local feature extraction module that contains multiple channel TCNs with different scales is developed to extract inherent local information from the raw input. Second, in a parallel manner, a global feature extraction module integrating transformer blocks is built to extract global information representation synchronously using the self-attention mechanism. Afterward, high-level global–local features extracted from different channels are fused, and TTAF is estimated through two fully connected regression layers using the fused features. The proposed PMTT has been compared to seven competitors using run-to-failure data collected from Android and OpenStack systems. The experiments have demonstrated the superiority of PMTT, showing an average improvement of 11.2%, 9.0%, and 9.3% in performance across three evaluation metrics compared with the optimal baseline model.

The majority of Information Technology (IT)/Software Engineering (SE) professionals are male. A potential reason for the low number of female IT/SE professionals might be that the roles and the way they are advertised are biased towards male candidates. The aim of this research is to collect information about the present state of practice of gender inclusiveness within IT/SE job advertisements and how, if needed, we might improve this. We conducted a survey of hiring managers and IT/SE professionals (who are employed in IT/SE roles). The survey collected their general views on gender bias within job advertisements. According to their opinions, job advertisements are often biased towards male candidates. Based on the review and suggestions from our participants we developed a set of recommendations to help hiring managers design more gender inclusive SE job advertisements. This will be a first step toward developing a gender balanced SE workforce.

Quantum computing is considered a revolutionary technology due to its ability to solve computational problems that are beyond the capabilities of classical computers. However, quantum computing requires great amounts of energy to run. Therefore, a factor in deciding whether to use quantum computing should be not only the complexity of the problem to be solved, but also the energy required to solve it. This paper presents an empirical study developed with the aim of comparing classical and quantum computing in terms of energy efficiency to determine whether the increased power of quantum computers is offset by their higher energy consumption. To achieve this, a variety of problems with different levels of complexity were tested on both types of computers. Specifically, we used the IBM Quantum computers with a maximum of 5 qubits and an Intel i7, as a classical computer. In addition to this we have also analysed the evolution of the quantum computers, performing measurements on three time periods. Our empirical study showed that there is a variability of results obtained in the three time periods and that quantum computing is not recommended for low-complexity problems, given its high energy consumption, particularly when compared to traditional computing.

The advancement of machine learning (ML) models has led to the development of ML-based approaches to improve numerous software engineering tasks in software maintenance and evolution. Nevertheless, research indicates that despite their potential successes, ML models may not be employed in real-world scenarios because they often remain a black box to practitioners, lacking explainability in their reasoning. Recently, various rule-based model-agnostic Explainable AI (XAI) techniques, such as PyExplainer and LIME, have been employed to explain the predictions of ML models in software analytics tasks. In this paper, we assess the ability of these techniques, particularly the state-of-the-art PyExplainer and LIME, to generate reliable and consistent explanations for ML models across various software analytics tasks, including Just-in-Time (JIT) defect prediction, clone detection, and the classification of useful code review comments. Our manual investigations find inconsistencies and anomalies in the explanations generated by these techniques. Therefore, we design a novel framework: Evaluation of Explainable AI (EvaluateXAI), along with granular-level evaluation metrics, to automatically assess the effectiveness of rule-based XAI techniques in generating reliable and consistent explanations for ML models in software analytics tasks. After conducting in-depth experiments involving seven state-of-the-art ML models trained on five datasets and six evaluation metrics, we find that none of the evaluation metrics reached 100%, indicating the unreliability of the explanations generated by XAI techniques. Additionally, PyExplainer and LIME failed to provide consistent explanations for 86.11% and 77.78% of the experimental combinations, respectively. Therefore, our experimental findings emphasize the necessity for further research in XAI to produce reliable and consistent explanations for ML models in software analytics tasks.