There are a number of empirical studies that assess the benefit deriving from the use of documentation and models in the execution of maintenance tasks. The greater part of these studies are quantitative and fail to analyze the values, beliefs, and assumptions that inform and shape source code comprehensibility and maintainability. We designed and conducted a qualitative study to understand the role of source code comments and identifiers in source code comprehensibility and maintainability. In particular, we sought to understand how novice and young professional developers perceive comments and identifier names after they have inspected the system behavior visible in its user interfaces. Novice developers were 18 third-year Bachelor students in Computer Science. The young professional developers were 12 and had work experience in between 3 months and 2 and half years. The used qualitative methodological approach is ethnographic. We asked the participants to inspect the behavior of a Java application visible in its user interfaces and then to comprehend and modify the source code of that application. We immersed ourselves and participated to the study, while collecting data by means of contemporaneous field notes, audio recordings, and copies of various artifacts. From the collected data, we have identified insights into comprehension and maintenance practices. The main insights can be summarized as follows: (i) with respect to novice developers, professional developers prefer to deal with identifier names rather than comments, (ii) all the participants indicate as essential the use of naming convention techniques for identifiers, and (iii) for all the participants the names of identifiers are important and should be properly chosen. Summarizing, independently from the kind of developer, it is advisable to use naming convention techniques and to properly choose identifiers.
{"title":"Dealing with identifiers and comments in source code comprehension and maintenance: results from an ethnographically-informed study with students and professionals","authors":"Felice Salviulo, G. Scanniello","doi":"10.1145/2601248.2601251","DOIUrl":"https://doi.org/10.1145/2601248.2601251","url":null,"abstract":"There are a number of empirical studies that assess the benefit deriving from the use of documentation and models in the execution of maintenance tasks. The greater part of these studies are quantitative and fail to analyze the values, beliefs, and assumptions that inform and shape source code comprehensibility and maintainability. We designed and conducted a qualitative study to understand the role of source code comments and identifiers in source code comprehensibility and maintainability. In particular, we sought to understand how novice and young professional developers perceive comments and identifier names after they have inspected the system behavior visible in its user interfaces. Novice developers were 18 third-year Bachelor students in Computer Science. The young professional developers were 12 and had work experience in between 3 months and 2 and half years. The used qualitative methodological approach is ethnographic. We asked the participants to inspect the behavior of a Java application visible in its user interfaces and then to comprehend and modify the source code of that application. We immersed ourselves and participated to the study, while collecting data by means of contemporaneous field notes, audio recordings, and copies of various artifacts. From the collected data, we have identified insights into comprehension and maintenance practices. The main insights can be summarized as follows: (i) with respect to novice developers, professional developers prefer to deal with identifier names rather than comments, (ii) all the participants indicate as essential the use of naming convention techniques for identifiers, and (iii) for all the participants the names of identifiers are important and should be properly chosen. Summarizing, independently from the kind of developer, it is advisable to use naming convention techniques and to properly choose identifiers.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125833613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We propose, in this paper, a new holonic agent testing technique. The technique is based on models and uses genetic algorithms. It considers the successive versions of an agent. The approach is organized in two main phases that are conducted iteratively. The first phase is concerned with detecting a new version of an agent under test. The second phase focuses on testing each new detected version. The new version is analyzed in order to generate a behavioral model on which is based the generation of test cases. The test case generation process focuses on the new parts of the agent behavior. In this way, the technique supports an incremental update of test cases.
{"title":"Towards a novel testing approach for holonic agents","authors":"N. H. Dehimi","doi":"10.1145/2601248.2613079","DOIUrl":"https://doi.org/10.1145/2601248.2613079","url":null,"abstract":"We propose, in this paper, a new holonic agent testing technique. The technique is based on models and uses genetic algorithms. It considers the successive versions of an agent. The approach is organized in two main phases that are conducted iteratively. The first phase is concerned with detecting a new version of an agent under test. The second phase focuses on testing each new detected version. The new version is analyzed in order to generate a behavioral model on which is based the generation of test cases. The test case generation process focuses on the new parts of the agent behavior. In this way, the technique supports an incremental update of test cases.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125407115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Genero, J. A. Vargas, Lilia García-Mundo, M. Piattini
Context: A Serious Game (SG) is a game for purposes other than entertainment [12]. SGs are currently in widespread use and their popularity has begun to steadily increase; their application areas now extend not only to education, but also to military, health and corporate [9] [12] sectors. SGs are of vital importance at present, as they can be a means to achieve relevant goals from both a personal and an institutional point of view. This may take place in fields as diverse as defense, education, scientific exploration, health care, emergency management, city planning, engineering, religion, and politics. The number of users of these systems grows each day, signifying that their impact is very high, and it is precisely for this reason that more extensive research on SG quality is needed. Objective: The aim of this study is to discover the current state of SG quality initiatives, identifying gaps that merit future investigation. Method: We conducted a systematic mapping study (SMS) on SG quality, following the guidelines proposed by Kitchenham and Charters [7]. We selected 112 papers found in six digital libraries until April of 2013. Results: Since 2007, research on SG quality proves to have grown very significantly. Research has focused mostly on addressing the effectiveness of SGs (78.57%), in addition to several entertainment characteristics that are principally related to pleasure (62.50%). The most widely-researched software artifact was the final product (97.32%), with design coming very far behind (7.14%). Less than half of all the research reviewed had been validated by means of experiments, and in most of these cases, experiments were conducted by the same researchers who had proposed the SG. The majority of experiments have not been replicated. The most common research outcome was questionnaires, closely followed by the confirmation of knowledge. Most of these outcomes evaluated the quality of a particular SG. Conclusion: Results show that SG quality has undergone a very important growth, thus making SG quality an area of opportunity for future research. Researchers are mainly concerned with demonstrating or confirming the effectiveness of SGs, but very little research has been conducted as regards the characteristics of playability that make SGs more effective. Since effectiveness and playability are evaluated in the final product there is a need to provide quality assurance methods that incorporate quality issues from the early stages of SG development. Further empirical validation is also needed, and in particular, external replications must be performed in order to corroborate and generalize the findings obtained.
{"title":"A systematic mapping study on serious game quality","authors":"M. Genero, J. A. Vargas, Lilia García-Mundo, M. Piattini","doi":"10.1145/2601248.2601261","DOIUrl":"https://doi.org/10.1145/2601248.2601261","url":null,"abstract":"Context: A Serious Game (SG) is a game for purposes other than entertainment [12]. SGs are currently in widespread use and their popularity has begun to steadily increase; their application areas now extend not only to education, but also to military, health and corporate [9] [12] sectors. SGs are of vital importance at present, as they can be a means to achieve relevant goals from both a personal and an institutional point of view. This may take place in fields as diverse as defense, education, scientific exploration, health care, emergency management, city planning, engineering, religion, and politics. The number of users of these systems grows each day, signifying that their impact is very high, and it is precisely for this reason that more extensive research on SG quality is needed. Objective: The aim of this study is to discover the current state of SG quality initiatives, identifying gaps that merit future investigation. Method: We conducted a systematic mapping study (SMS) on SG quality, following the guidelines proposed by Kitchenham and Charters [7]. We selected 112 papers found in six digital libraries until April of 2013. Results: Since 2007, research on SG quality proves to have grown very significantly. Research has focused mostly on addressing the effectiveness of SGs (78.57%), in addition to several entertainment characteristics that are principally related to pleasure (62.50%). The most widely-researched software artifact was the final product (97.32%), with design coming very far behind (7.14%). Less than half of all the research reviewed had been validated by means of experiments, and in most of these cases, experiments were conducted by the same researchers who had proposed the SG. The majority of experiments have not been replicated. The most common research outcome was questionnaires, closely followed by the confirmation of knowledge. Most of these outcomes evaluated the quality of a particular SG. Conclusion: Results show that SG quality has undergone a very important growth, thus making SG quality an area of opportunity for future research. Researchers are mainly concerned with demonstrating or confirming the effectiveness of SGs, but very little research has been conducted as regards the characteristics of playability that make SGs more effective. Since effectiveness and playability are evaluated in the final product there is a need to provide quality assurance methods that incorporate quality issues from the early stages of SG development. Further empirical validation is also needed, and in particular, external replications must be performed in order to corroborate and generalize the findings obtained.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122131834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: Ownership metrics measure how the workload of software modules is shared among their developers. They have been shown to be accurate indicators of software quality. Objective: Since ownership metrics studies were done only on industrial software projects, we replicated such a study on Java free/libre and open source software (FLOSS) projects. Our goal was to generalize an "ownership law" that stated that minor developers should be avoided. Method: We explored the relationship between ownership metrics and fault-proneness on seven FLOSS projects, using publicly available corpora to retrieve the fault-related information. Results: In our corpus, the relationship between ownership metrics and module faults is weak. At best, less than half of projects exhibit a significant correlation, and at worst, no projects at all. Moreover, fault-proneness seems to be much more influenced by module size than by ownership. Conclusion: The results of ownership studies done on closed-source projects do not generalize to FLOSS projects. To understand the reasons for that, we performed an in-depth analysis and found that the lack of correlation between ownership metrics and module faults is due to the distributions of contributions among developers and the presence of "heroes" in FLOSS projects.
{"title":"Code ownership in open-source software","authors":"Matthieu Foucault, Jean-Rémy Falleri, Xavier Blanc","doi":"10.1145/2601248.2601283","DOIUrl":"https://doi.org/10.1145/2601248.2601283","url":null,"abstract":"Context: Ownership metrics measure how the workload of software modules is shared among their developers. They have been shown to be accurate indicators of software quality. Objective: Since ownership metrics studies were done only on industrial software projects, we replicated such a study on Java free/libre and open source software (FLOSS) projects. Our goal was to generalize an \"ownership law\" that stated that minor developers should be avoided. Method: We explored the relationship between ownership metrics and fault-proneness on seven FLOSS projects, using publicly available corpora to retrieve the fault-related information. Results: In our corpus, the relationship between ownership metrics and module faults is weak. At best, less than half of projects exhibit a significant correlation, and at worst, no projects at all. Moreover, fault-proneness seems to be much more influenced by module size than by ownership. Conclusion: The results of ownership studies done on closed-source projects do not generalize to FLOSS projects. To understand the reasons for that, we performed an in-depth analysis and found that the lack of correlation between ownership metrics and module faults is due to the distributions of contributions among developers and the presence of \"heroes\" in FLOSS projects.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123855801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents the main goals, questions, and methods of my PhD research. This research focuses on facilitating the conduction of distributed human-oriented experiments in software engineering (SE). First, I am studying how the experiments are being conducted in SE and what are the challenges in conducting distributed experiments. Second, I plan to propose a platform able to mitigate the challenges encountered. Finally, I am going to evaluate the proposed solution using both academic-industrial experiments. The research was divided in phases and each phase is based on a research question. Moreover, I propose a set of empirical studies that intends to provide answer to each research question.
{"title":"Together we are stronger: facilitating the conduction of distributed human-oriented experiments","authors":"Waldemar Ferreira","doi":"10.1145/2601248.2613083","DOIUrl":"https://doi.org/10.1145/2601248.2613083","url":null,"abstract":"This paper presents the main goals, questions, and methods of my PhD research. This research focuses on facilitating the conduction of distributed human-oriented experiments in software engineering (SE). First, I am studying how the experiments are being conducted in SE and what are the challenges in conducting distributed experiments. Second, I plan to propose a platform able to mitigate the challenges encountered. Finally, I am going to evaluate the proposed solution using both academic-industrial experiments. The research was divided in phases and each phase is based on a research question. Moreover, I propose a set of empirical studies that intends to provide answer to each research question.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"323 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"113989647","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Test-driven development (TDD) is an iterative software development technique where unit-tests are defined before production code. The proponents of TDD claim that it improves both external quality and developers' productivity. In particular, Erdogmus et al. (i.e., original study) proposed a two-stage model to investigate these claims regarding TDD's effects. Our aim is to enhance the model proposed in the original study by investigating an additional factor: TDD process conformance. We conducted a close, external replication of the original study accompanied by a correlation analysis to check whether process conformance is related to improvements for the subjects using TDD. We partially confirmed the results of the original study. Moreover, we observed a correlation between process conformance and quality, but not productivity. We found no evidence to support the claim that external quality and productivity are improved by the adoption of TDD compared to test-last development. Finally, conformance to TDD process improves the quality and does not affect productivity. We conclude that the role of process conformance is relevant in studying the quality and productivity-related effects of TDD.
{"title":"Conformance factor in test-driven development: initial results from an enhanced replication","authors":"D. Fucci, Burak Turhan, M. Oivo","doi":"10.1145/2601248.2601272","DOIUrl":"https://doi.org/10.1145/2601248.2601272","url":null,"abstract":"Test-driven development (TDD) is an iterative software development technique where unit-tests are defined before production code. The proponents of TDD claim that it improves both external quality and developers' productivity. In particular, Erdogmus et al. (i.e., original study) proposed a two-stage model to investigate these claims regarding TDD's effects. Our aim is to enhance the model proposed in the original study by investigating an additional factor: TDD process conformance. We conducted a close, external replication of the original study accompanied by a correlation analysis to check whether process conformance is related to improvements for the subjects using TDD. We partially confirmed the results of the original study. Moreover, we observed a correlation between process conformance and quality, but not productivity. We found no evidence to support the claim that external quality and productivity are improved by the adoption of TDD compared to test-last development. Finally, conformance to TDD process improves the quality and does not affect productivity. We conclude that the role of process conformance is relevant in studying the quality and productivity-related effects of TDD.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125983313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: In modern systems, like cyber-physical systems, where software and physical services are interacting, safety, security or performance play an important role. In order to guarantee the correct interoperability of such systems, with respect to functional and non-functional requirements, integration testing is an effective measure to achieve this. Model-based testing moreover not only enables early definition and validation, but also test automation. This makes it a good choice to overcome urgent challenges of integration testing. Objective: Many publications on model-based integration testing (MBIT) approaches can be found. Nevertheless, a study giving a systematic overview on the underlying software paradigms, measures for guiding the integration testing process as well as non-functional requirements they are suitable for, is missing. The aim of this paper is to find and synthesize the relevant primary studies to gain a comprehensive understanding of the current state of model-based integration testing. Method: For synthesizing the relevant studies, we conducted a systematic literature review (SLR) according to the guidelines of Kitchenham. Results: The systematic search and selection retrieved 83 relevant studies from which data has been extracted. Our review identified three assessment criteria for guiding the testing process, namely static metrics, dynamic metrics and stochastic &random. In addition it shows that just a small fraction considers non-functional requirements. Most approaches are for component-oriented systems. Conclusion: Results from the SLR show that there are two major research gaps. First, there is an accumulated need for approaches in the MBIT field that support non-functional requirements, as they are gaining importance. Second, means for steering the integration testing process, especially together with automation, need to evolve.
{"title":"Software paradigms, assessment types and non-functional requirements in model-based integration testing: a systematic literature review","authors":"Florian Häser, M. Felderer, R. Breu","doi":"10.1145/2601248.2601257","DOIUrl":"https://doi.org/10.1145/2601248.2601257","url":null,"abstract":"Context: In modern systems, like cyber-physical systems, where software and physical services are interacting, safety, security or performance play an important role. In order to guarantee the correct interoperability of such systems, with respect to functional and non-functional requirements, integration testing is an effective measure to achieve this. Model-based testing moreover not only enables early definition and validation, but also test automation. This makes it a good choice to overcome urgent challenges of integration testing. Objective: Many publications on model-based integration testing (MBIT) approaches can be found. Nevertheless, a study giving a systematic overview on the underlying software paradigms, measures for guiding the integration testing process as well as non-functional requirements they are suitable for, is missing. The aim of this paper is to find and synthesize the relevant primary studies to gain a comprehensive understanding of the current state of model-based integration testing. Method: For synthesizing the relevant studies, we conducted a systematic literature review (SLR) according to the guidelines of Kitchenham. Results: The systematic search and selection retrieved 83 relevant studies from which data has been extracted. Our review identified three assessment criteria for guiding the testing process, namely static metrics, dynamic metrics and stochastic &random. In addition it shows that just a small fraction considers non-functional requirements. Most approaches are for component-oriented systems. Conclusion: Results from the SLR show that there are two major research gaps. First, there is an accumulated need for approaches in the MBIT field that support non-functional requirements, as they are gaining importance. Second, means for steering the integration testing process, especially together with automation, need to evolve.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132374364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Context: Although agile software development (ASD) method use has grown dramatically in recent years, there is lack of empirical evidence for their effective use and predicted improvements in software development. This suggests that there is an imprecise understanding of their use and practice in organisations beyond the adoption phase. There is a lack of clarity on what 'usage' or 'effectiveness' mean in the context of ASD. In particular, the relationship between agile usage and agile effectiveness has not been investigated. Objective: To study the factors that impact usage of ASD methods and analyse the relationship between post-adoption use of ASD methods, i.e. agile usage and the results effectively achieved in terms of improvements in development outcomes, i.e. agile effectiveness. Method: An agile usage model of post-adoption or continued agile usage is tested using a survey instrument. The study employs variance-based or partial least squares structural equation modelling (PLS-SEM) implemented in SmartPLS version 2.0. Special attention is given to the quality of the measures used by developing reliable and valid measures of study variables. Results: The results from a partial least squares (PLS) analysis shows that relative advantage and agile coach significantly influence the usage of agile methods. Results also support the thesis that agile usage as measured by intensity and extent of agile method use significantly impacts agile effectiveness. Conclusions: The overall contribution of this research is to improve the effectiveness of ASD methods by increasing the likelihood that ASD methods will be effectively used and sustained beyond the adoption phase.
{"title":"An empirical investigation of the factors affecting agile usage","authors":"Mali Senapathi, A. Srinivasan","doi":"10.1145/2601248.2601253","DOIUrl":"https://doi.org/10.1145/2601248.2601253","url":null,"abstract":"Context: Although agile software development (ASD) method use has grown dramatically in recent years, there is lack of empirical evidence for their effective use and predicted improvements in software development. This suggests that there is an imprecise understanding of their use and practice in organisations beyond the adoption phase. There is a lack of clarity on what 'usage' or 'effectiveness' mean in the context of ASD. In particular, the relationship between agile usage and agile effectiveness has not been investigated. Objective: To study the factors that impact usage of ASD methods and analyse the relationship between post-adoption use of ASD methods, i.e. agile usage and the results effectively achieved in terms of improvements in development outcomes, i.e. agile effectiveness. Method: An agile usage model of post-adoption or continued agile usage is tested using a survey instrument. The study employs variance-based or partial least squares structural equation modelling (PLS-SEM) implemented in SmartPLS version 2.0. Special attention is given to the quality of the measures used by developing reliable and valid measures of study variables. Results: The results from a partial least squares (PLS) analysis shows that relative advantage and agile coach significantly influence the usage of agile methods. Results also support the thesis that agile usage as measured by intensity and extent of agile method use significantly impacts agile effectiveness. Conclusions: The overall contribution of this research is to improve the effectiveness of ASD methods by increasing the likelihood that ASD methods will be effectively used and sustained beyond the adoption phase.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130686456","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E. Mendes, Marcos Kalinowski, Daves Martins, F. Ferrucci, Federica Sarro
[Objective] The objective of this paper is to extend a previously conducted systematic literature review (SLR) that investigated under what circumstances individual organizations would be able to rely on cross-company based estimation models. [Method] We applied the same methodology used in the SLR we are extending herein (covering the period 2006-2013) based on primary studies that compared predictions from cross-company models with predictions from within-company models constructed from analysis of project data. [Results] We identified 11 additional papers; however two of these did not present independent results and one had inconclusive findings. Two of the remaining eight papers presented both, trials where cross-company predictions were not significantly different from within-company predictions and others where they were significantly different. Four found that cross-company models gave prediction accuracy significantly different from within-company models (one of them in favor of cross-company models), while two found no significant difference. The main pattern when examining the study related factors was that studies where cross-company predictions were significantly different from within-company predictions employed larger within-company data sets. [Conclusions] Overall, half of the analyzed evidence indicated that cross-company estimation models are not significantly worse than within-company estimation models. Moreover, there is some evidence that sample size does not imply in higher estimation accuracy, and that samples for building estimation models should be carefully selected/filtered based on quality control and project similarity aspects. The results need to be combined with the findings from the SLR we are extending to allow further investigating this topic.
{"title":"Cross- vs. within-company cost estimation studies revisited: an extended systematic review","authors":"E. Mendes, Marcos Kalinowski, Daves Martins, F. Ferrucci, Federica Sarro","doi":"10.1145/2601248.2601284","DOIUrl":"https://doi.org/10.1145/2601248.2601284","url":null,"abstract":"[Objective] The objective of this paper is to extend a previously conducted systematic literature review (SLR) that investigated under what circumstances individual organizations would be able to rely on cross-company based estimation models. [Method] We applied the same methodology used in the SLR we are extending herein (covering the period 2006-2013) based on primary studies that compared predictions from cross-company models with predictions from within-company models constructed from analysis of project data. [Results] We identified 11 additional papers; however two of these did not present independent results and one had inconclusive findings. Two of the remaining eight papers presented both, trials where cross-company predictions were not significantly different from within-company predictions and others where they were significantly different. Four found that cross-company models gave prediction accuracy significantly different from within-company models (one of them in favor of cross-company models), while two found no significant difference. The main pattern when examining the study related factors was that studies where cross-company predictions were significantly different from within-company predictions employed larger within-company data sets. [Conclusions] Overall, half of the analyzed evidence indicated that cross-company estimation models are not significantly worse than within-company estimation models. Moreover, there is some evidence that sample size does not imply in higher estimation accuracy, and that samples for building estimation models should be carefully selected/filtered based on quality control and project similarity aspects. The results need to be combined with the findings from the SLR we are extending to allow further investigating this topic.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"48 16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130999683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents an empirical evaluation of the team reflection support provided by the Software Engineering Method and Theory (SEMAT) Essence framework, and compares Essence reflection meetings to other types of team reflection meetings. The researchers conducted a field study involving seven graduate master student teams running Essence reflection meetings throughout their practicum projects aiming at delivering a working product for an industry client. The main result validates that Essence meetings generate reflective team discussions through a thinking framework that is holistic, state-based, goal-driven, and method-agnostic. Student teams benefit from stepping back and assessing the project holistically throughout its lifecycle. The goals set by the framework's checklists lead the teams to address critical aspects of the project that have not been considered. All team members are encouraged to express their views and influence the various project dimensions. Essence reflection meetings are comparable and complementary to Agile retrospectives, and project teams might want to leverage both techniques. The value added by Essence reflections is to surface unknown issues, help monitor progress, steer the project to a higher state, and prevent retrospectives from being repetitive by varying styles.
{"title":"Essence reflection meetings: field study","authors":"Cécile Péraire, Todd Sedano","doi":"10.1145/2601248.2601296","DOIUrl":"https://doi.org/10.1145/2601248.2601296","url":null,"abstract":"This paper presents an empirical evaluation of the team reflection support provided by the Software Engineering Method and Theory (SEMAT) Essence framework, and compares Essence reflection meetings to other types of team reflection meetings. The researchers conducted a field study involving seven graduate master student teams running Essence reflection meetings throughout their practicum projects aiming at delivering a working product for an industry client. The main result validates that Essence meetings generate reflective team discussions through a thinking framework that is holistic, state-based, goal-driven, and method-agnostic. Student teams benefit from stepping back and assessing the project holistically throughout its lifecycle. The goals set by the framework's checklists lead the teams to address critical aspects of the project that have not been considered. All team members are encouraged to express their views and influence the various project dimensions. Essence reflection meetings are comparable and complementary to Agile retrospectives, and project teams might want to leverage both techniques. The value added by Essence reflections is to surface unknown issues, help monitor progress, steer the project to a higher state, and prevent retrospectives from being repetitive by varying styles.","PeriodicalId":115935,"journal":{"name":"International Conference on Evaluation & Assessment in Software Engineering","volume":"217 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115103458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：