Automated Software Engineering最新文献

Mobile malware has become one of the most critical security threats in the era of ubiquitous mobile computing. Despite the intensive efforts from security experts to counteract it, recent years have still witnessed a rapid growth of identified malware samples. This could be partly attributed to the newly-emerged technologies that may constantly open up under-studied attack surfaces for adversaries. One typical example is the recently-developed mobile machine learning (ML) framework that enables storing and running deep learning (DL) models on mobile devices. Despite obvious advantages, this new feature also inadvertently introduces potential vulnerabilities (e.g., on-device models may be modified for malicious purposes). In this work, we propose a method to generate or transform mobile malware by hiding malicious payloads inside DL models’ parameters based on a strategy considering four factors (layer type, layer number, layer coverage, and the number of bytes to replace). Utilizing the proposed method, we can run malware in DL mobile applications covertly with little impact on the model performance (i.e., as little as 0.35% drop in accuracy and at most 39ms latency overhead). We can successfully trigger malicious functions, such as getting SMS records and screenshots in a real-world application. The generated malware can evade state-of-the-art detection techniques (i.e., none detected by VirusTotal), and the malware-based attack exhibits high practical feasibility (i.e., successfully attack 41% of the apps with on-device DL models). Our work should alert security experts on malware injection attacks on mobile devices, and further raise more awareness towards the deep-learning-assisted attacks in the mobile ecosystem.

Large Language Models (LLMs) have made significant advancements in code-related tasks. In the field of automated debugging, fault localization (FL) and automated program repair (APR) are two prevalent topics attracting significant research effort. Recently, in the field of FL and APR, many novel LLM-based approaches have emerged. However, most existing LLM-based studies primarily focus on the GPT models from OpenAI or open-source LLMs. With the rapid development of LLMs, various internet giants have introduced new closed-source models. In addition, due to policy restrictions, some regions can only access the commercial LLMs provided by specified companies. Despite the LLMs of OpenAI, the effectiveness of the other closed-source LLMs in FL and APR remains unknown. To better understand the effectiveness of contemporary closed-source models, we conduct a large-scale empirical study on their performance with respect to FL and APR. Specifically, our study involves 4 recent commercial closed-source LLMs (i.e., GPT-4o-Mini, Ernie-3.5, Qwen-turbo, and Doubao-pro) and 1 open-source LLM (i.e., DeepSeek-V3-chat). Note that only the GPT models have region restrictions among all LLMs we studied. We designed a total of 12 distinct prompt templates, 6 each for FL and APR, incorporating various formats and information sources. We conducted experiments to evaluate the effectiveness of FL and APR on 1036 real Java bugs from two datasets, Defects4J 2.0 and ConDefects. The key findings of the experiments indicate that (1) different LLMs tend to succeed on different sets of bugs in both FL and APR, with relatively little overlap among successful cases, implying the models possess distinct strengths in handling specific kinds of bugs, (2) the effectiveness of prompt templates varies across different models, and (3) the effectiveness of FL and APR capabilities of the studied models is significantly correlated with the bug type. We summarized all 14 findings obtained into 3 implications, which could help researchers further improve the performance of LLMs on FL and APR.

Generating programs using large language models (LLMs) for fuzz testing has emerged as a significant testing methodology. While traditional fuzzers can produce correct programs, their effectiveness is limited by excessive constraints and restricted API combinations, resulting in insufficient coverage of the target system’s code and impacting testing efficiency. Unlike traditional methods, large language model based fuzzers can generate more diverse code, effectively addressing key issues of conventional fuzzers. However, the lack of constraints on API combinations during the generation process often leads to reduced program validity. Therefore, a crucial challenge is to enhance the validity of generated code while maintaining its diversity. To address this issue, we propose a novel and universal fuzzer, LMFuzz. To ensure the fuzzer’s generation capability, we utilize a large language model as the primary generator and model the operator selection problem within the fuzzing loop as a multi-armed bandit problem. We introduce the Thompson Sampling algorithm to enhance both the diversity and validity of program generation. To improve the validity of the generated code, we incorporate a program repair loop that iteratively corrects the generated programs, thereby reducing errors caused by the lack of API combination constraints. Experimental results demonstrate that LMFuzz significantly surpasses existing state-of-the-art large language model based fuzzers in terms of coverage and validity, and also exhibits notable advantages in generating diverse programs. Furthermore, LMFuzz has identified 24 bugs across five popular programming languages and their corresponding systems.

Software defect prediction (SDP) is a critical task for improving software quality and reducing development costs by identifying faults early. While machine learning models, particularly XGBoost, have been widely adopted for SDP, their performance is highly dependent on optimal hyperparameter tuning. Furthermore, existing state-of-the-art methods—including deep learning-based approaches leveraging semantic code features—often suffer from high computational complexity, and extensive training requirements. To address these challenges, this paper proposes a hybrid optimization approach, RL-SWO, which integrates the Spider Wasp Optimizer (SWO) with reinforcement learning (RL) to refine XGBoost’s parameters. RL-SWO was first validated on CEC’22 benchmark functions, where it outperformed several state-of-the-art metaheuristics. It was then applied to five defect prediction datasets from the AEEEM repository, demonstrating superior performance in detecting defective and non-defective instances, particularly in imbalanced data scenarios. Compared to traditional optimization methods, RL-SWO significantly improved XGBoost’s classification accuracy and robustness. Experimental results highlight RL-SWO’s potential in enhancing SDP models by balancing exploration and exploitation during optimization. This study advances automated defect prediction by leveraging metaheuristics and reinforcement learning, offering a promising approach for improving software reliability.

Smart contract vulnerability detection has attracted increasing attention due to billions of economic losses caused by vulnerabilities. Existing smart contract vulnerability detection methods have high false negative and high false positive rates. To address these issues, we present ByteEye, a bytecode level smart contract vulnerability detection framework with Graph Neural Networks (GNNs). ByteEye first constructs an edge-enhanced Control Flow Graph (CFG) to maintain rich information from the low-level bytecode with low latency. ByteEye also designs and incorporates both general information and vulnerability-specific information into its detection method as bytecode level features. Furthermore, ByteEye flexibly supports machine/deep learning models, especially with graph neural networks, which can facilitate vulnerability detection precisely. The extensive experimental results highlight that ByteEye outperforms the state-of-the-art approaches on all three types of vulnerability detection. ByteEye can achieve an average of 35.29%, 43.95%, and 6.38% higher on F1 than the bytecode level best-performed baseline on reentrancy vulnerability, timestamp dependency vulnerability, and integer overflow/underflow vulnerability, respectively. Moreover, ByteEye can detect 361 new vulnerabilities in real-world smart contracts, which are reported for the first time. ByteEye enhances control flow information, designs general bytecode-level features with expert knowledge, and flexibly supports deep learning models, particularly GNNs, thus achieving high detection effectiveness.

This paper explores the integration of Artificial Intelligence (AI) and automation within the Behavior-Driven Development (BDD) paradigm, using the PyTest-BDD framework, to enhance Software Quality Assurance (SQA) processes. Traditional SQA methods struggle with the increasing complexity and rapid release cycles of modern software development. This research demonstrates how AI can address these challenges through intelligent test generation, prioritization, and anomaly detection. The proposed framework utilizes Natural Language Processing (NLP) to analyze requirements, machine learning (ML) to generate and prioritize test scenarios, and deep learning (DL) for anomaly detection, all within the PyTest-BDD ecosystem. This approach fosters a collaborative environment between human testers and AI agents, leading to more robust testing with reduced human overhead.The framework offers reduced human error, faster feedback loops, and increased team collaboration, thereby reducing development time and improving software reliability. AI-powered test prioritization and anomaly detection are shown to be effective in identifying subtle defects. The modular and extensible nature of the framework allows for a flexible and scalable testing system.

Software size is a key input for project planning, effort estimation, and productivity analysis. While pre-trained language models have shown promise in deriving functional size from natural-language requirements, measuring size directly from source code remains under-explored. Yet, code-based size measurement is critical in modern workflows where requirement documents are often incomplete or unavailable, especially in Agile development environments. This exploratory study investigates the use of CodeBERT, a pre-trained bimodal transformer model, for measuring software size directly from Python source code according to two measurement methods: COSMIC Function Points and MicroM. We construct two curated datasets from the Python subset of the CodeSearchNet corpus, and manually annotate each function with its corresponding size. Our experimental results show that CodeBERT can successfully measure COSMIC data movements with up to 91.4% accuracy and generalize to the functional, architectural, and algorithmic event types defined in MicroM, reaching up to 81.5% accuracy. These findings highlight the potential of code-based language models for automated functional size measurement when requirement artifacts are absent or unreliable.

C is widely used in system programming due to its low-level flexibility. However, as demands for memory safety and code reliability grow, Rust has become a more favorable alternative owing to its modern design principles. Migrating existing C code to Rust has therefore emerged as a key approach for enhancing the security and maintainability of software systems. Nevertheless, automating such migrations remains challenging due to fundamental differences between the two languages in terms of language design philosophy, type systems, and levels of abstraction. Most current code transformation tools focus on mappings of basic data types and syntactic replacements, such as handling pointers or conversion of lock mechanisms. These approaches often fail to deeply model the semantic features and programming paradigms of the target language. To address this limitation, this paper proposes RustFlow, a C-to-Rust code translation framework based on large language models (LLMs), designed to generate idiomatic and semantically accurate Rust code. This framework employs a multi-stage progressive architecture, which decomposes the overall translation task into several sequential stages, namely translation, validation, and repair. During the translation phase, a collaborative prompting strategy is employed to guide the LLM in achieving cross-language semantic alignment, thereby improving the accuracy of the generated code. Subsequently, a validation mechanism is introduced to perform syntactic and semantic checks on the generated output, and a conversational iterative repair strategy is employed to further enhance the quality of the final result. Experimental results show that RustFlow outperforms most of the latest baseline approaches, achieving an average improvement of 50.67% in translation performance compared to the base LLM. This work offers a novel technical approach and practical support for efficient and reliable cross-language code migration.

GitHub and Jira projects typically contain many issues and issue comments used to track project tasks and defects. An important class of issues that needs appropriate consideration is called “human-centric issues”. These issues relate to different human characteristics of end users that need to be identified, tracked and managed differently from traditional technical-related issues. Current management of these human-centric issues during defect management is limited. We introduce a novel dashboard – the (Human-centric Issue Visualiser – HCIV) that categorises and tags these HCIss. We built HCIV prototypes for the two platforms, GitHub and Jira. These tag issues and present them in various visual forms to software practitioners. Using the dashboard, human-centric issues can be prioritised and tracked, and machine learning-generated classifications can be overridden. To reflect these interactions, associated GitHub and Jira issue tags are updated while the user interacts with our dashboard. The user evaluations of our dashboard prototypes show their potential for human-centric issue management. A demo of the GitHub version of the tool being used can be viewed at https://youtu.be/v49aiRiDIPs, and the Jira version can be viewed at https://youtu.be/qQM72SErmqs.

High-quality repositories containing real-world defects are essential for developing defect-related algorithms. Although plenty of defect repositories exist, they often fail to capture the context of inter-procedural defects, which include all methods in the propagation path from the defect-source method to the defect-triggering method. This limitation is particularly critical for the Null Pointer Exception (NPE), a common defect that often propagates across multiple methods in Java systems. To address this problem, we propose a novel and automatic approach, called JDExtractor, to extract defect-related methods from real applications. The main challenge is how to identify all defect-related methods efficiently and accurately. JDExtractor tackles this challenge by constructing a method-level data graph using the principle of Java type compatibility and simplifying the data graph using filtering criteria. Data flow analysis helps construct a coarse-grained method-level data graph, which reflects the potential patterns of inter-procedural data interaction, thereby ensuring analysis efficiency. Afterward, filtering analysis simplifies the data graph based on the propagation properties of inter-procedural defects, thus ensuring analysis accuracy. Evaluation results suggest that both the static slicing tool WALA and the dynamic slicing tool Slicer4J yield several false positives, whereas JDExtractor successfully extracts defect-related methods and defect propagation paths with fewer false positives in a short time. Moreover, JDExtractor has been applied to open source projects on GitHub, ultimately extracting defect-related methods for 67 defects from 319 compiled open source applications.