Analogy-based cost estimation provides a useful and intuitive means to support decision making in software project management. It derives a cost estimate required for completing a project from information about similar past projects, namely the analogues. While on average this method provides a relatively accurate cost estimate there remains a possibility of large estimation errors. In this paper, we empirically tested the hypothesis that "using more homogeneous analogues produces a more reliable cost estimate" using a software engineering data repository established by the software engineering center (SEC), Information-technology Promotion Agency, Japan. This testing showed that low and high homogeneity projects had a large variation in estimation reliability. For instance, the difference was 22.9% (p = 0.021) in terms of percentage to get accurate estimates (better than Median of Magnitude of Relative Error).
{"title":"Is This Cost Estimate Reliable? -- The Relationship between Homogeneity of Analogues and Estimation Reliability","authors":"Naoki Ohsugi, Akito Monden, Nahomi Kikuchi, Mike Barker, Masateru Tsunoda, Takeshi Kakimoto, Ken-ichi Matsumoto","doi":"10.1109/ESEM.2007.31","DOIUrl":"https://doi.org/10.1109/ESEM.2007.31","url":null,"abstract":"Analogy-based cost estimation provides a useful and intuitive means to support decision making in software project management. It derives a cost estimate required for completing a project from information about similar past projects, namely the analogues. While on average this method provides a relatively accurate cost estimate there remains a possibility of large estimation errors. In this paper, we empirically tested the hypothesis that \"using more homogeneous analogues produces a more reliable cost estimate\" using a software engineering data repository established by the software engineering center (SEC), Information-technology Promotion Agency, Japan. This testing showed that low and high homogeneity projects had a large variation in estimation reliability. For instance, the difference was 22.9% (p = 0.021) in terms of percentage to get accurate estimates (better than Median of Magnitude of Relative Error).","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115702122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper explores the cognitive limits of estimation in the context of software cost estimation. Two heuristics, representativeness and anchoring, motivate two experiments involving psychology students, engineering students, and engineering practitioners. The first experiment, designed to determine if there is a difference in estimating ability in everyday quantities, demonstrates that the three populations estimate with relatively equal accuracy. The results shed light on the distribution of estimates and the process of subjective judgment. The second experiment, designed to explore abilities for estimating the cost of software-intensive systems given incomplete information, shows that predictions by engineering students and practitioners are within 3-12% of each other. The value of this work is in helping better understand how software engineers make decisions based on limited information. The manifestation of the two heuristics is discussed together with the implications for the development of software cost estimation models in light of the findings from the two experiments.
{"title":"Cognitive Limits of Software Cost Estimation","authors":"Ricardo Valerdi","doi":"10.1109/ESEM.2007.85","DOIUrl":"https://doi.org/10.1109/ESEM.2007.85","url":null,"abstract":"This paper explores the cognitive limits of estimation in the context of software cost estimation. Two heuristics, representativeness and anchoring, motivate two experiments involving psychology students, engineering students, and engineering practitioners. The first experiment, designed to determine if there is a difference in estimating ability in everyday quantities, demonstrates that the three populations estimate with relatively equal accuracy. The results shed light on the distribution of estimates and the process of subjective judgment. The second experiment, designed to explore abilities for estimating the cost of software-intensive systems given incomplete information, shows that predictions by engineering students and practitioners are within 3-12% of each other. The value of this work is in helping better understand how software engineers make decisions based on limited information. The manifestation of the two heuristics is discussed together with the implications for the development of software cost estimation models in light of the findings from the two experiments.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"80 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123411676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Information retrieval is an important problem in any evidence-based discipline. Although evidence- based software engineering (EBSE) is not immune to this fact, this question has not been examined at length. The goal of this paper is to analyse the optimality of search strategies for use in systematic reviews. We tried out 29 search strategies using different terms and combinations of terms. We evaluated their sensitivity and precision with a view to finding an optimum strategy. From this study of search strategies we were able to analyse trends and weaknesses in terminology use in articles reporting experiments.
{"title":"Developing Search Strategies for Detecting Relevant Experiments for Systematic Reviews","authors":"Ó. Dieste, O.A.G. Padua","doi":"10.1109/ESEM.2007.39","DOIUrl":"https://doi.org/10.1109/ESEM.2007.39","url":null,"abstract":"Information retrieval is an important problem in any evidence-based discipline. Although evidence- based software engineering (EBSE) is not immune to this fact, this question has not been examined at length. The goal of this paper is to analyse the optimality of search strategies for use in systematic reviews. We tried out 29 search strategies using different terms and combinations of terms. We evaluated their sensitivity and precision with a view to finding an optimum strategy. From this study of search strategies we were able to analyse trends and weaknesses in terminology use in articles reporting experiments.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130551156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This research investigates the effects of linear and non-linear feature extraction methods on the cost estimation performance. We use principal component analysis (PCA) and Isomap for extracting new features from observed ones and evaluate these methods with support vector regression (SVR) on publicly available datasets. Our results for these datasets indicate there is no significant difference between the performances of these linear and non-linear feature extraction methods.
{"title":"Evaluation of Feature Extraction Methods on Software Cost Estimation","authors":"Burak Turhan, Onur Kutlubay, A. Bener","doi":"10.1109/ESEM.2007.57","DOIUrl":"https://doi.org/10.1109/ESEM.2007.57","url":null,"abstract":"This research investigates the effects of linear and non-linear feature extraction methods on the cost estimation performance. We use principal component analysis (PCA) and Isomap for extracting new features from observed ones and evaluate these methods with support vector regression (SVR) on publicly available datasets. Our results for these datasets indicate there is no significant difference between the performances of these linear and non-linear feature extraction methods.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"31 5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129103774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a controlled experiment comparing the defect detection efficiency of exploratory testing (ET) and test case based testing (TCT). While traditional testing literature emphasizes test cases, ET stresses the individual tester's skills during test execution and does not rely upon predesigned test cases. In the experiment, 79 advanced software engineering students performed manual functional testing on an open-source application with actual and seeded defects. Each student participated in two 90-minute controlled sessions, using ET in one and TCT in the other. We found no significant differences in defect detection efficiency between TCT and ET. The distributions of detected defects did not differ significantly regarding technical type, detection difficulty, or severity. However, TCT produced significantly more false defect reports than ET. Surprisingly, our results show no benefit of using predesigned test cases in terms of defect detection efficiency, emphasizing the need for further studies of manual testing.
{"title":"Defect Detection Efficiency: Test Case Based vs. Exploratory Testing","authors":"Juha Itkonen, M. Mäntylä, C. Lassenius","doi":"10.1109/ESEM.2007.56","DOIUrl":"https://doi.org/10.1109/ESEM.2007.56","url":null,"abstract":"This paper presents a controlled experiment comparing the defect detection efficiency of exploratory testing (ET) and test case based testing (TCT). While traditional testing literature emphasizes test cases, ET stresses the individual tester's skills during test execution and does not rely upon predesigned test cases. In the experiment, 79 advanced software engineering students performed manual functional testing on an open-source application with actual and seeded defects. Each student participated in two 90-minute controlled sessions, using ET in one and TCT in the other. We found no significant differences in defect detection efficiency between TCT and ET. The distributions of detected defects did not differ significantly regarding technical type, detection difficulty, or severity. However, TCT produced significantly more false defect reports than ET. Surprisingly, our results show no benefit of using predesigned test cases in terms of defect detection efficiency, emphasizing the need for further studies of manual testing.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"49 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131916420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This poster reports on a solution to ERP project cost estimation and on results from its first experimental application.
这张海报报告了ERP项目成本估算的解决方案及其首次实验应用的结果。
{"title":"Approaching the ERP Project Cost Estimation Problem: an Experiment","authors":"M. Daneva","doi":"10.1109/ESEM.2007.72","DOIUrl":"https://doi.org/10.1109/ESEM.2007.72","url":null,"abstract":"This poster reports on a solution to ERP project cost estimation and on results from its first experimental application.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116985331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Preserving data privacy is posing new challenges to software engineering researchers. Current technologies can be too cumbersome, pervasive or costly to be successfully applied in dynamic and complex scenarios where data exchange occurs among a large number of applications. Anonymization techniques seem to be a promising candidate, even if preliminary investigations suggest that they could deteriorate the quality of data. An empirical study has been carried out in order to understand the relationship between the anonymization level and the degradation of data quality.
{"title":"Tuning anonymity level for assuring high data quality: an empirical study.","authors":"G. Canfora, C. A. Visaggio","doi":"10.1109/ESEM.2007.23","DOIUrl":"https://doi.org/10.1109/ESEM.2007.23","url":null,"abstract":"Preserving data privacy is posing new challenges to software engineering researchers. Current technologies can be too cumbersome, pervasive or costly to be successfully applied in dynamic and complex scenarios where data exchange occurs among a large number of applications. Anonymization techniques seem to be a promising candidate, even if preliminary investigations suggest that they could deteriorate the quality of data. An empirical study has been carried out in order to understand the relationship between the anonymization level and the degradation of data quality.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124080646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With today's ever increasing demands on software, developers must produce software that can be changed without the risk of degrading the software architecture. Degraded software architecture is problematic because it makes the system more prone to defects and increases the cost of making future changes. The effects of making changes to software can be difficult to measure. One way to address software changes is to characterize their causes and effects. This paper introduces an initial architecture change characterization scheme created to assist developers in measuring the impact of a change on the architecture of the system. It also presents an initial study conducted to gain insight into the validity of the scheme. The results of this study indicated a favorable view of the viability of the scheme by the subjects, and the scheme increased the ability of novice developers to assess and adequately estimate change effort.
{"title":"Characterizing Software Architecture Changes: An Initial Study","authors":"Byron J. Williams, Jeffrey C. Carver","doi":"10.1109/ESEM.2007.26","DOIUrl":"https://doi.org/10.1109/ESEM.2007.26","url":null,"abstract":"With today's ever increasing demands on software, developers must produce software that can be changed without the risk of degrading the software architecture. Degraded software architecture is problematic because it makes the system more prone to defects and increases the cost of making future changes. The effects of making changes to software can be difficult to measure. One way to address software changes is to characterize their causes and effects. This paper introduces an initial architecture change characterization scheme created to assist developers in measuring the impact of a change on the architecture of the system. It also presents an initial study conducted to gain insight into the validity of the scheme. The results of this study indicated a favorable view of the viability of the scheme by the subjects, and the scheme increased the ability of novice developers to assess and adequately estimate change effort.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129489407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Effort estimation by analogy (EBA) is often confronted with missing values. Our former analogy- based method AUQA is able to tolerate missing values in the data set, but it is unclear how the percentage of missing values impacts the prediction accuracy and if there is an upper bound for how big this percentage might become in order to guarantee the applicability of AQUA. This paper investigates these questions through an impact analysis. The impact analysis is conducted for seven data sets being of different size and having different initial percentages of missing values. The major results are that (i) we confirm the intuition that the more missing values, the poorer the prediction accuracy of AQUA; (ii) there is a quadratic dependency between the prediction accuracy and the percentage of missing values; and (Hi) the upper limit of missing values for the applicability of AQUA is determined as 40%. These results are obtained in the context of AQUA. Further analysis is necessary for other ways of applying EBA, such as using different similarity measures or analogy adaptation methods from those used in AQUA. For that purpose, the experimental design in this study can be adapted.
{"title":"Impact Analysis of Missing Values on the Prediction Accuracy of Analogy-based Software Effort Estimation Method AQUA","authors":"Jingzhou Li, Ahmed Al-Emran, G. Ruhe","doi":"10.1109/ESEM.2007.10","DOIUrl":"https://doi.org/10.1109/ESEM.2007.10","url":null,"abstract":"Effort estimation by analogy (EBA) is often confronted with missing values. Our former analogy- based method AUQA is able to tolerate missing values in the data set, but it is unclear how the percentage of missing values impacts the prediction accuracy and if there is an upper bound for how big this percentage might become in order to guarantee the applicability of AQUA. This paper investigates these questions through an impact analysis. The impact analysis is conducted for seven data sets being of different size and having different initial percentages of missing values. The major results are that (i) we confirm the intuition that the more missing values, the poorer the prediction accuracy of AQUA; (ii) there is a quadratic dependency between the prediction accuracy and the percentage of missing values; and (Hi) the upper limit of missing values for the applicability of AQUA is determined as 40%. These results are obtained in the context of AQUA. Further analysis is necessary for other ways of applying EBA, such as using different similarity measures or analogy adaptation methods from those used in AQUA. For that purpose, the experimental design in this study can be adapted.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"118 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134222396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The fault-prone module detection in source code is of importance for assurance of software quality. Most of previous conventional fault-prone detection approaches have been based on using software metrics. Such approaches, however, have difficulties in collecting the metrics and constructing mathematical models based on the metrics. In order to mitigate such difficulties, we propose a novel approach for detecting fault-prone modules using a spam filtering technique. Because of the increase of needs for spam e-mail detection, the spam filtering technique has been progressed as a convenient and effective technique for text mining. In our approach, fault-prone modules are detected in a way that the source code modules are considered as text files and are applied to the spam filter directly. In order to show the usefulness of our approach, we conducted an experiment using source code repository of a Java based open source development. The result of experiment shows that our approach can classify more than 70% of software modules correctly.
{"title":"Fault-Prone Filtering: Detection of Fault-Prone Modules Using Spam Filtering Technique","authors":"O. Mizuno, Shiro Ikami, Shuya Nakaichi, T. Kikuno","doi":"10.1109/ESEM.2007.29","DOIUrl":"https://doi.org/10.1109/ESEM.2007.29","url":null,"abstract":"The fault-prone module detection in source code is of importance for assurance of software quality. Most of previous conventional fault-prone detection approaches have been based on using software metrics. Such approaches, however, have difficulties in collecting the metrics and constructing mathematical models based on the metrics. In order to mitigate such difficulties, we propose a novel approach for detecting fault-prone modules using a spam filtering technique. Because of the increase of needs for spam e-mail detection, the spam filtering technique has been progressed as a convenient and effective technique for text mining. In our approach, fault-prone modules are detected in a way that the source code modules are considered as text files and are applied to the spam filter directly. In order to show the usefulness of our approach, we conducted an experiment using source code repository of a Java based open source development. The result of experiment shows that our approach can classify more than 70% of software modules correctly.","PeriodicalId":124420,"journal":{"name":"First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133142478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：