Journal of Systems and Software最新文献

When modeling topics from chat messages of developer instant messaging communication, individual chat messages are short text documents. Our study aims at understanding how short text topic models perform with conversations from developer instant messaging. We applied four models to nine Gitter chat rooms (with sizes ranging from $\approx$100 to $\approx$160,000 messages). To assess the quality of topics and identify the best performing models, we compared topics based on four metrics for topic coherence. Furthermore, for a subset of Gitter chat rooms we used two human-based assessments: intrusion tasks with 18 experts analyzing 40 topics each, and topic naming (assigning a name to a topic that summarizes its main concept) with eight additional experts naming 60 topics each. Models performed differently in terms of coherence metrics and human assessment depending on the corpus (small, medium or large chat room). Our findings offer recommendations for the selection and use of short text topic models with developer chat messages based on characteristics of models and their performance with different sizes of corpora, and based on different strategies to assess topic quality.

Traceability links between issues and commits record valuable information about the evolutionary history of software projects. Unfortunately, these links are often missing. While deep learning stands as the current state-of-the-art (SOTA) in automated traceability links recovery (TLR), its effectiveness is faced with the practical problem of limited labeled data during training. To overcome this challenge, in this paper, we propose DSSLink, a novel method based on deep semi-supervised learning, enhancing deep-learning-based link recovery tasks. DSSLink first learns knowledge from labeled data through pre-trained model and then leverages deep semi-supervised learning to infer pseudo-labels on unlabeled data. The extended dataset of pseudo-labeled and labeled data re-trains the deep learning model in an iterative process. Our extensive evaluations are conducted on two SOTA traceability methods (T-BERT and BTLink) across four GitHub projects and 11 Apache projects. Specifically, the maximum F1-score improvements for GitHub and Apache projects reached 22.9% and 43.5%, respectively. Evaluation results show that DSSLink is effective in enhancing TLR performance and outperforms TraceFUN, a recent approach that utilizes unlabeled data for TLR. The source code of DSSLink is available at https://github.com/DSSLink.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Evaluating the correctness of code generated by AI is a challenging open problem. In this paper, we propose a fully automated method, named ACCA, to evaluate the correctness of AI-generated code for security purposes. The method uses symbolic execution to assess whether the AI-generated code behaves as a reference implementation. We use ACCA to assess four state-of-the-art models trained to generate security-oriented assembly code and compare the results of the evaluation with different baseline solutions, including output similarity metrics, widely used in the field, and the well-known ChatGPT, the AI-powered language model developed by OpenAI.

Our experiments show that our method outperforms the baseline solutions and assesses the correctness of the AI-generated code similar to the human-based evaluation, which is considered the ground truth for the assessment in the field. Moreover, ACCA has a very strong correlation with the human evaluation (Pearson’s correlation coefficient $r=0.84$ on average). Finally, since it is a full y automated solution that does not require any human intervention, the proposed method performs the assessment of every code snippet in $\sim 0.17$ s on average, which is definitely lower than the average time required by human analysts to manually inspect the code, based on our experience.

Context:

Cloud computing’s rise as the primary platform for software development and delivery is largely driven by the potential cost savings. However, it is surprising that no empirical evidence has been collected to determine whether cost awareness permeates the development process and how it manifests in practice.

Objective:

This study aims to provide empirical evidence of cost awareness by mining open source repositories of cloud-based applications. The focus is on Infrastructure-as-Code artifacts that automate software (re)deployment on the cloud.

Methods:

A systematic examination of $152735$ repositories yielded $2010$ relevant hits. We then analyzed 538 relevant commits and 208 relevant issues using inductive and deductive coding and corroborated findings with discussions from Stack Overflow.

Results:

The findings indicate that developers are not only concerned with the cost of their application deployments but also take actions to reduce these costs beyond selecting cheaper cloud services. We also identify research areas for future consideration.

Conclusion:

Although we focus on a particular Infrastructure-as-Code technology (Terraform), the findings can be applicable to cloud-based application development in general. The provided empirical grounding can serve developers seeking to reduce costs through service selection, resource allocation, deployment optimization, and other techniques.

User feedback on software usage is utilised by developers to improve their software. Software product forums are platforms rich in software-related user feedback, such as forum threads containing bug reports or requests for new features. However, previous studies have mainly focused on analysing user feedback from software product forums as individual sentences, which can lead to missing insights and a lack of understanding of the overall context of forum posts. To fill this gap in research, this work examines user feedback found in software product forum posts to investigate the differences between content classifications found in forum sentences and posts. We manually evaluated software product forum posts collected from two open-sourced software product forums and discovered five new types of user feedback that can only be identified when examining user feedback in the form of forum posts. Additionally, we examined the association between sentence classifications found within software product forums. Our results indicate that contextual information complimenting product improvement insights can be found in software product forums, with a confidence of 0.75 and 0.69 for the association between apparent bug and application usage sentences. This information can be used to reduce manual efforts required to chase up missing contextual information when attempting to understand or fix software issues. We also provide insights into the progression of posts in software product forums at the thread-level, and our progression flowchart can be used to summarise the sequence of events in software product forum threads. Our findings reveal the importance of looking at user feedback within software product forums in the format of forum posts to identify new insights on user feedback for software improvements.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

The COVID-19 virus has caused a global pandemic that has heavily impacted daily life. Rapid advances in testing and vaccinating led to an additional use case besides the well-known contact-tracing apps: certificate-verification systems. Verification systems are often commissioned by local authorities to enable more public life, and are often developed by smaller organizations or startups. So, the development of verification systems differs from other software projects, featuring interesting and unique properties. In this article, we present an experience report on the development of one verification system by a German startup, focusing on three properties: working in a pandemic, developing a product for handling a pandemic, and the startup context. To this end, we surveyed nine startup developers and analyzed the results with two experts from the startup. We found that the developers focused on fast delivery to cope with the time pressure of releasing the verification system, which is why some phases of typical development processes were hardly carried out. As a result, while the verification system is successful, we also identified negative effects of the properties (e.g., programming mistakes, well-being). We discuss our findings to guide researchers and practitioners in preparing for software engineering in future emergencies.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Reentrancy, the most notorious vulnerability in smart contracts, has attracted extensive attention. To eliminate reentrancy before deploying contracts, there is a need to locate and repair the contracts. However, existing tools suffer from false positive localization, original semantics destruction, and high gas overhead. In this work, we propose a template-based gas-optimized reentrancy repair method with semantic maintenance. We avoid false positive locations from verifying the attack’s effectiveness, using connectivity and read–write dependencies. We design the semantic equivalence check algorithm based on the def-use chain. We optimize the lock and reordering templates for reentrancy repair and add semantic maintenance operations. We implement our tool, ReenRepair, and compare it with two state-of-the-art detection tools and two repair tools. The results show that ReenRepair yields good location precision, the highest repair rate, and the lowest gas overhead. All semantic changes caused by lock and 89.66% of semantic changes caused by reordering are successfully maintained.

Nowadays, many companies design and develop their software systems as a set of loosely coupled microservices that communicate via their Application Programming Interfaces (APIs). While the loose coupling improves maintainability, scalability, and fault tolerance, it poses new challenges to the API evolution process. Related works identified communication and integration as major API evolution challenges but did not provide the underlying reasons and research directions to mitigate them. In this paper, we aim to identify microservice API evolution strategies and challenges in practice and gain a broader perspective of their relationships. We conducted 17 semi-structured interviews with developers, architects, and managers in 11 companies and analyzed the interviews with open coding used in grounded theory. In total, we identified six strategies and six challenges for REpresentational State Transfer (REST) and event-driven communication via message brokers. The strategies mainly focus on API backward compatibility, versioning, and close collaboration between teams. The challenges include change impact analysis efforts, ineffective communication of changes, and consumer reliance on outdated versions, leading to API design degradation. We defined two important problems in microservice API evolution resulting from the challenges and their coping strategies: tight organizational coupling and consumer lock-in. To mitigate these two problems, we propose automating the change impact analysis and investigating effective communication of changes as open research directions.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Refactoring is a well-known technique to improve software quality. However, there are relevant domains where refactoring has not been studied in-depth before, such as JavaScript front-end frameworks. To fill this gap, we empirically study refactorings that developers perform when maintaining and evolving React-based Web applications. By manually inspecting 320 refactoring commits performed in open source projects, we catalog 69 distinct refactoring operations of which 25 are specific to React code, 17 are adaptations of traditional refactorings for the React context, 22 are traditional refactorings, and six are specific to JavaScript and CSS code. The catalog of refactorings proposed in this article might support practitioners when improving the maintainability of React applications.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

In this work, we introduce Basilisk, a high-level architectural pattern designed to facilitate interoperability among various languages, platforms, and ecosystems. The pursuit of language-independent software development is highly desirable, enabling developers to utilize existing software products with most programming languages. Achieving platform independence is equally advantageous, allowing code deployment on different platforms effortlessly. While the development community has often aimed for either language or platform independence, Basilisk aims to combine both into a single product. To realize this dual objective, Basilisk employs two fundamental components. The first is a transpilation infrastructure used to render software products language-independent. The second is an abstraction layer over platforms, enabling the creation of platform-independent software products. To illustrate Basilisk’s potential, we introduce Hydra, a one-to-many, declarative transpilation infrastructure. Hydra has been utilized to develop transpilers from HydraKernel (source language) to various target languages, including D, C++, C#, Scala, Ruby, Hy, and Python. Additionally, we instantiate the abstraction layer in Wyvern, a low-level embedded domain-specific language for GPU programming, supporting any Vulkan-compatible GPU. With the Hydra transpilation infrastructure, Wyvern becomes available for D, C++, C#, Scala, Ruby, Hy, and Python. We evaluate Basilisk through the instantiation of Hydra and Wyvern, writing five algorithms from the Rodinia suite for the seven available languages, totaling 35 benchmarks. These benchmarks are executed on four different hardware platforms.