Journal of Systems and Software最新文献

Cryptography is known as a challenging topic for developers. We studied StackOverflow posts to identify the problems that developers encounter when using Java Cryptography Architecture (JCA) for symmetric encryption. We investigated security risks that are disseminated in these posts, and we examined whether ChatGPT helps avoid cryptography issues. We found that developers frequently struggle with key and IV generations, as well as padding. Security is a top concern among developers, but security issues are pervasive in code snippets. ChatGPT can effectively aid developers when they engage with it properly. Nevertheless, it does not substitute human expertise, and developers should remain alert.

Wavelet shrinkage estimation received considerable attentions to estimate stochastic processes such as a non-homogeneous Poisson process in a non-parametric way, and was applied to software reliability estimation/prediction. However, it lacks the prediction ability for unknown future patterns in long term and penalizes assessing the software reliability in practice. In this paper, we focus on the long-term prediction of the number of software faults detected in the testing phase and propose many novel long-term prediction methods based on the wavelet shrinkage estimation. The fundamental idea is to adopt both the denoised fault-count data and prediction values, and to minimize several kinds of loss functions to make effective predictions. We also develop an automated wavelet-based software reliability assessment tool, W-SRAT2, which is a drastic extension of the existing tool, W-SRAT, by adding those prediction algorithms. In numerical experiments with 6 actual software development project data, we investigate the predictive performance of our long-term prediction approaches, which consist of 2,640 combinations, and compare them with the common software reliability growth models with the maximum likelihood estimation. It is shown that our wavelet shrinkage estimation/prediction methods outperform the existing software reliability growth models.

Mutation testing is a fault-based and powerful software testing technique, but the large number of mutations can result in extremely high costs. To reduce the cost of mutation testing, researchers attempt to identify stubborn mutants and generate test data to kill them, in order to achieve the same testing effect. However, existing methods suffer from inaccurate identification of stubborn mutants and low productiveness in generating test data, which will seriously affect the effectiveness and efficiency of mutation testing. Therefore, we propose a new method of generating test data for killing stubborn mutants based on set evolution, namely TDGMSE. We first propose an integrated indicator to identify stubborn mutants. Then, we establish a constrained multi-objective model for generating test data of killing stubborn mutants. Finally, we develop a new genetic algorithm based on set evolution to solve the mathematical model. The results on 14 programs depict that the average false positive (or negative) rate of TDGMSE is decreased about 81.87% (or 32.34%); the success rate of TDGMSE is 99.22%; and the average number of iterations of TDGMSE is 16132.23, which is lowest of all methods. The research highlights several potential research directions for mutation testing.

This paper discusses one of the most challenging practical engineering problems in speaker recognition systems — the version control of models and user profiles. A typical speaker recognition system consists of two stages: the enrollment stage, where a profile is generated from user-provided enrollment audio; and the runtime stage, where the voice identity of the runtime audio is compared against the stored profiles. As technology advances, the speaker recognition system needs to be updated for better performance. However, if the stored user profiles are not updated accordingly, version mismatch will result in meaningless recognition results. In this paper, we describe different version control strategies for speaker recognition systems that had been carefully studied at Google from years of engineering practice. These strategies are categorized into three groups according to how they are deployed in the production environment: device-side deployment, server-side deployment, and hybrid deployment. To compare different strategies with quantitative metrics under various network configurations, we present SpeakerVerSim, an easily-extensible Python-based simulation framework for different server-side deployment strategies of speaker recognition systems.

Requirements quality research, which is dedicated to assessing and improving the quality of requirements specifications, is dependent on research artifacts like data sets (containing information about quality defects) and implementations (automatically detecting and removing these defects). However, recent research exposed that the majority of these research artifacts have become unavailable or have never been disclosed, which inhibits progress in the research domain. In this work, we aim to improve the availability of research artifacts in requirements quality research. To this end, we (1) extend an artifact recovery initiative, (2) empirically evaluate the reasons for artifact unavailability using Bayesian data analysis, and (3) compile a concise guideline for open science artifact disclosure. Our results include 10 recovered data sets and 7 recovered implementations, empirical support for artifact availability improving over time and the positive effect of public hosting services, and a pragmatic artifact management guideline open for community comments. With this work, we hope to encourage and support adherence to open science principles and improve the availability of research artifacts for the requirements research quality community.

Performance engineering is a proactive and systematic approach aimed at designing, building, and enhancing software systems to ensure their efficient and reliable operation. It involves observing and measuring the operational behavior of a software system without interference, assessing performance metrics like response times, throughput, and resource utilization. This entails delving into kernel-level events related to performance monitoring, which play a significant role in understanding system behavior and diagnosing performance-related issues. Kernel-level events offer insights into how both the operating system and hardware resources are utilized. This information empowers system administrators, developers, and performance analysts to optimize and troubleshoot the system effectively.

A critical aspect of performance analysis is root cause analysis, which involves delving deep into kernel-level events connected to performance monitoring. These events provide valuable insights into the utilization of operating system and hardware resources, equipping system administrators, developers, and performance analysts with tools to effectively troubleshoot and optimize the system. Our study introduces an innovative artifact that captures kernel-level events using Elasticsearch and Kibana, facilitating comprehensive performance analysis under diverse scenarios. By defining both Light-load and Heavy-load scenarios and simulating CPU, I/O, Network, and Memory noise, we offer researchers a realistic environment to explore innovative approaches to system performance enhancement.

The artifact comprises both kernel events and system calls, resulting in a cumulative count of 24,263,691 events. The proposed artifact can serve three distinct applications. The first application emphasizes performance analysis by utilizing kernel events for monitoring. The second application targets noise detection and root cause analysis, again using kernel events. Finally, the third application investigates software phase detection through monitoring at the kernel level. These applications demonstrate that through our artifact, researchers can effectively analyze performance, detect and address performance noise, and identify software phases, contributing to the advancement of performance engineering methodologies.

All the system configurations, scripts, and traces can be found in the artifact GitHub repository.¹

Automatic program repair (APR) is a promising technique to fix program defects by generating patches. In the current APR techniques, template-based and learning-based techniques have demonstrated different advantages. Template-based APR techniques rely on pre-defined fix templates, providing higher controllability but limited by the variety of templates and edit expressiveness. In contrast, learning-based APR techniques treat repair as a neural machine translation task, improving the edit expressiveness through training neural networks. However, this technique also faces the influence of quality and variety of training data, leading to numerous errors and redundant code generation. To overcome their limitations, this paper proposes an innovative APR technique called Confix. Confix first constructs a code information tree to assist in mining edit changes during historical repair. It then further enriches the types of fix templates using node information in the tree. Afterward, Confix defines masked lines based on node-level fix templates to control the scope of patch generation, avoiding redundant semantic code generation. Finally, Confix leverages the powerful edit expressiveness of the masked language model and combines it with fix strategies to generate correct patches more efficiently and accurately. Experimental results show that Confix exhibits state-of-the-art performance on the Defects4J 1.2 and QuixBugs benchmarks.

Vulnerability detection in smart contracts is critical to secure blockchain systems. Existing methods represent the bytecode as a graph structure and leverage graph neural networks to learn graph features for vulnerability detection. However, these methods are limited to handling the long-range dependencies between nodes. This means that they might focus on learning local node feature while ignoring global node information. In this paper, we propose a novel vulnerability detection framework with Enhanced Graph Feature Learning (EGFL), which aims to extract the global node information and utilize it to improve vulnerability detection in smart contracts. Specifically, we first represent the bytecode as a Control Flow Graph (CFG). To extract global node information, EGFL constructs a linear node feature matrix from CFG, and uses the feature-aware and relationship-aware modules to handle long-range dependencies between nodes. Meanwhile, a graph neural network is adopted to extract the local node feature from CFG. Subsequently, we fuse the global node information and local node feature to generate an enhanced graph feature for capturing more vulnerability features. We evaluate EGFL on the benchmark dataset with six types of smart contract vulnerabilities. Results show that EGFL outperforms fourteen state-of-the-art vulnerability detection methods by 10.83%–60.28% in F1 score.

Software product line engineering aims to systematically generate similar products or services within a given domain to reduce cost and time to market while increasing reuse. Various studies recognize the success of product line engineering in different domains. Software variability have increased over the years in many different domains such as mobile applications, cyber–physical systems or car control systems to just mention a few. However, software product line engineering is not as widely adopted as other software development technologies. In this paper, we present an empirical study conducted through a survey distributed to many software development companies. Our goal is to understand their need of software variability management and the level of knowledge the companies have regarding software product line engineering. The survey was answered by 127 participants from more than a hundred of different software development companies. Our study reveals that most of companies manage a catalog of similar products in a way or another (e.g. clone-and-own, common modules that are statically imported,etc.), they mostly document the features of products using text or spreed sheet based documents and more than 66% of companies identify a base product from which they derive other similar products. We also found a correlation between the lack of Software Product Line (SPL) knowledge and the absence of reuse practices. Notably, this is the first study that explore software variability needs regardless of a company’s prior knowledge of SPL. The results encourages further research to understand the reason for the limited knowledge and application of software product line engineering practices, despite the growing demand of variability management.