Automated Software Engineering最新文献

Search-Based Software Testing (SBST) has seen several success stories in academia and industry. The effectiveness of a search algorithm at solving a software engineering problem strongly depends on how such algorithm can navigate the fitness landscape of the addressed problem. The fitness landscape depends on the used fitness function. Understanding the properties of a fitness landscape can help to provide insight on how a search algorithm behaves on it. Such insight can provide valuable information to researchers to being able to design novel, more effective search algorithms and fitness functions tailored for a specific problem. Due to its importance, few fitness landscape analyses have been carried out in the scientific literature of SBST. However, those have been focusing on the problem of unit test generation, e.g., with state-of-the-art tools such as EvoSuite. In this paper, we replicate one such existing study. However, in our work we focus on system test generation, with the state-of-the-art tool EvoMaster. Based on an empirical study involving the testing of 23 web services, this enables us to provide valuable insight into this important testing domain of practical industrial relevance. Our results indicate that fitness landscapes are largely dominated by neutral regions (e.g., plateaus), which make the search process challenging. We observe that the presence of information content in the landscape can improve search guidance, while boolean flags are a primary contributor to neutrality. These findings confirm prior results in unit testing but also reveal system-level differences, particularly in how branch types impact search effectiveness. These insights suggest the need for improved fitness functions, testability transformations, and search operators tailored to system-level testing.

As the Virtual Reality (VR) industry expands, the need for automated GUI testing is growing rapidly. Large Language Models (LLMs), capable of retaining information long-term and analyzing both visual and textual data, are emerging as a potential key to deciphering the complexities of VR’s evolving user interfaces. In this paper, we conduct a case study to investigate the capability of using LLMs, particularly GPT-4o, for field of view (FOV) analysis in VR exploration testing. Specifically, we validate that LLMs can identify test entities in FOVs and that prompt engineering can effectively enhance the accuracy of test entity identification from (varvec{41.67%}) to (varvec{71.30%}). Our study also shows that LLMs can accurately describe identified entities’ features with at least a (varvec{90%}) accuracy rate. We further find out that the core features that effectively represent an entity are color, placement, and shape. Furthermore, the combination of the three features can especially be used to improve the accuracy of determining identical entities in multiple FOVs with the highest F1-score of (varvec{0.70}). Additionally, our study demonstrates that LLMs are capable of scene recognition and spatial understanding in VR with precisely designed structured prompts. Finally, we find that LLMs fail to label the identified test entities, and we discuss potential solutions as future research directions.

The Internet of Things (IoT) protocols are a core element of IoT systems, providing the fundamental support for communication and data exchange between devices. These protocols enable various devices to connect and work together. However, potential errors and vulnerabilities in IoT protocol implementations can make devices easily attacked. Therefore, ensuring the security of IoT protocols is of utmost importance. Common vulnerability detection methods, such as fuzzing, encounter significant challenges in evaluating these implementations, mainly due to the need for extensive protocol knowledge, high time and resource consumption, as well as the difficulty of generating high-quality and targeted test cases. In order to solve the above issues, this paper presents an intelligent fuzzer, LIPFuzzer, for testing IoT protocols. Unlike common methods that heavily rely on the user’s understanding of the protocol to generate test cases, LIPFuzzer, with the assistance of Large Language Models (LLMs), mutates real IoT protocol communication messages to automatically generate more targeted test cases. Specifically, it utilizes LLMs to understand the relative knowledge of protocols, analyze different categories of protocol messages, and identify recommended mutation fields in combination with the characteristics of IoT protocols, providing targeted mutation strategies for each category. In addition, we evaluate LIPFuzzer on several widely-used implementations of well-known IoT protocols (e.g., Modbus-TCP, MQTT, and CoAP). Experimental results indicate that, compared to widely-used protocol fuzzers such as Peach, LIPFuzzer generates test cases more conveniently and efficiently, while also discovering vulnerabilities more effectively.

Smart contracts have revolutionized the credit landscape. However, their security remains intensely scrutinized due to numerous hacking incidents and inherent logical challenges. One well-known issue is reentrancy vulnerability, exemplified by DAO attacks that lead to substantial economic losses. Previous approaches have employed rule-based and deep learning-based (DL) algorithms to detect and repair reentrancy vulnerability. Large language models (LLM) have been distinguished in recent years for their excellent understanding of text and code. However, less attention has been paid to LLM-based reentrancy vulnerability detection and repair, and direct prompt-based approaches often suffer from inefficiencies and high false positives. To overcome the above shortcomings, this paper proposes a hybrid model framework combining LLM with DL to enhance the detection and repair of reentrancy vulnerabilities. This unified framework comprises three crucial phases: the data processing phase, the vulnerability detection phase, and the vulnerability repair phase. Extensive experimental results validate the superiority of our approach over state-of-the-art baselines, and ablation studies demonstrate the effectiveness of each component. Our approach demonstrates significant improvements in vulnerability detection, with increases of 3.51% in accuracy, 2.31% in recall, 0.42% in precision, and 0.85% in F1-score. Furthermore, our approach can achieve a notable 9.62% enhancement in the repair rate. Finally, we also conducted a user study to emphasize its potential to fortify the security of smart contracts.

As modern software systems become complex and the demand for rapid development cycles increases, automatic code generation techniques have attained a prominent focus in academic research and industrial practice. These techniques can significantly reduce human error, increase productivity, and ensure consistency across large codebases. However, the task of generating code automatically presents significant challenges. In this study, we investigate, identify, and analyze the existing automatic techniques for generating code from various input formats, highlighting their efficiencies and areas for potential improvement. A Systematic Literature Review (SLR) is conducted to systematically summarize and review 76 primary studies related to automatic code generation in the software engineering domain. The selected studies are investigated from several dimensions: paradigms, techniques, input types, intermediate representations, tool support, targeted programming languages, and validation methods, including performance metrics, datasets, and benchmarking status. Our investigation identified 12 main techniques, categorized into five paradigms, where the Model-to-Code paradigm and model-driven techniques are the most prevalent. Notably, 57% of the studies utilized Java, and a limited number of studies showed multilingual support. Furthermore, 72% of the selected studies did not compare their results with existing techniques, and 17% lacked validation of the proposed techniques. We also noticed a lack of detailed information about the datasets used in the validation process, where 52% of the studies omitted these details. This SLR provides several recommendations to enhance methodological rigor in future research, and it highlights opportunities for leveraging emerging technologies to improve the efficiency of the identified automatic code generation techniques.

Traditional vulnerability detection methods based on rules or learning primarily focus on coarse-grained predictions, often lacking precise localization and interpretability regarding the root causes of vulnerabilities. The growing availability of open-source vulnerability databases calls for advanced methods that can reason about vulnerabilities at a finer slice-level granularity. GPTVD, which leverages large language models’ (LLMs) in-context learning (ICL) and chain-of-thought (COT) reasoning capabilities. The goal is to enhance both detection performance and explainability. GPTVD extracts threat code slices through static code analysis, focusing on data and control dependencies. Positive and negative samples are clustered based on heuristic features and semantic feature vectors, and representative samples are manually annotated with reasoning processes to build COT prompts. These prompts are combined with target samples to form LLM input queries, enabling slice-level vulnerability inference and explanation using LLM. The method was evaluated on 18,062 programs from a public dataset. GPTVD achieved superior performance compared to existing methods, with 92.21% accuracy, 93.20% precision, and 92.28% recall. Ablation studies confirm that clustering-based prompt selection, explicit threat code slices, and human expert reasoning significantly improve detection effectiveness and interpretability. GPTVD demonstrates that combining static code analysis with LLM-based COT reasoning can effectively detect vulnerabilities at the slice level with high accuracy and interpretability.

Spreadsheets are widely used in business and scientific domains, yet they are prone to input errors that can lead to significant risks. Faults often occur due to the use of formulas that are syntactically correct but semantically incorrect. This issue is particularly challenging for formula cells that are physically close and exhibit minor logical differences, which traditional fault prediction methods struggle to detect. To address these challenges, this paper introduces an enhanced neighborhood metric approach, which extends traditional formula-based metrics by incorporating neighborhood-based metrics. This approach analyzes the dependencies between adjacent formula cells, considering factors such as formula diversity, content dissimilarity, and structural consistency. This study introduces eight new neighborhood-based spreadsheet indicators to improve fault prediction, building on previous metric-based methods. Extensive experiments conducted on three widely used datasets–Enron, INFO1, and EUSES–demonstrated that integrating the enhanced neighborhood metrics with traditional ones significantly improves fault prediction performance. The approach shows notable improvements in precision, recall, and F1-scores, particularly for medium and large datasets. This study highlights the importance of incorporating neighborhood metrics for spreadsheet fault detection. The enhanced neighborhood metric approach improves fault detection accuracy by capturing subtle logical variations between formula cells that are physically close. This method offers a robust and effective approach for improving the reliability of spreadsheets and can be applied in various real-world data analysis tasks.

In the code generation field, Large Language Models (LLMs) pre-trained on numerous open-source code fragments show powerful reasoning abilities and remarkable downstream performance. They assist code generation by combining retrieval techniques like retrieving relevant code fragments as templates or using retrieval results to supplement natural language descriptions and get code examples. However, in domains like aerospace equipment, existing code generation technologies perform suboptimally. Different aerospace equipment has different functions and significant data processing and loading differences. There is a lack of effective retrieval methods to provide semantically similar code contexts for LLMs, hindering code generation from meeting complex task requirements. To address this, we propose CodeCLARE, a retrieval-augmented code generation framework. It first fine-tunes UniXcoder via contrastive learning and uses it as a semantic encoder for code fragment retrieval. Then, the NL2Code search strategy is adopted with program requirements as queries. In the final stage of the code generation process, through a “Few-Shots Selection” mechanism, the prompt templates effectively integrate both the retrieved code examples and the specific requirement information, enabling the successful generation of highly accurate C++ code through the advanced capabilities of LLMs. Experimental results show that our approach significantly improves code quality compared to traditional ones and provides an effective solution for spacecraft control code generation.

The use of diverse mobile applications among senior users is becoming increasingly widespread. However, many of these apps contain accessibility problems that result in negative user experiences for seniors. A key reason is that software practitioners often lack the time or resources to address the broad spectrum of age-related accessibility and personalisation needs. As current developer tools and practices encourage one-size-fits-all interfaces with limited potential to address the diversity of senior needs, there is a growing demand for approaches that support the systematic creation of adaptive, accessible app experiences. To this end, we present AdaptForge, a novel model-driven engineering (MDE) approach that enables advanced design-time adaptations of mobile application interfaces and behaviours tailored to the accessibility needs of senior users. AdaptForge uses two domain-specific languages (DSLs) to address age-related accessibility needs. The first model defines users’ context-of-use parameters, while the second defines conditional accessibility scenarios and corresponding UI adaptation rules. These rules are interpreted by an MDE workflow to transform an app’s original source code into personalised instances. We also report evaluations with professional software developers and senior end-users, demonstrating the feasibility and practical utility of AdaptForge.

The growing reuse of third-party libraries in software supply chains increases the risk of being affected by the involved vulnerabilities. To strengthen software security, security vendors such as Snyk manage up-to-date vulnerability databases by associating reported vulnerabilities with their affected libraries, and contemporary digital organizations such as banking and software enterprises detect the third-party libraries they use if affected by these reported vulnerabilities. Existing studies focus on automating the detection process but make few efforts on detecting newly affected libraries, although new libraries (previously healthy) are constantly disclosed to be affected by new vulnerabilities. Moreover, existing studies do not seriously consider digital organizations’ concerns only about the libraries they use. In this paper, we propose an approach LibAlarm to address these challenges. We implement LibAlarm as a large language model-powered approach and compare it with the baseline approaches from multiple perspectives. Our experimental evaluation using 16,238 NVD reports indicates that LibAlarm improves the F1 by over 14% compared with baselines and detects over 40% newly affected libraries. For contemporary digital organizations, LibAlarm performs better than the baseline approaches with the F1 above 70% and the reduced false alarm ratio to 20%. Our case analysis using 540 NVD reports and 20 projects from Microsoft and Google demonstrates the effectiveness of LibAlarm. These results indicate that LibAlarm can help security vendors and digital organizations detect affected libraries from vulnerability reports.