Jayshankar Sankarasetty, Kevin Mobley, Libby Foster, Tad Hammer, Terri Calderone
In the nine years that we have been involved in software performance engineering (SPE) and performance testing engagements we have learned several things. Across numerous eCommerce applications and an enterprise CRM product suite, our knowledge base about the field of Software Performance Engineering is constantly evolving. The focus of this paper is what we have learned in the areas of SPE project management, performance testing, defining the scope of SPE projects, ITIL, post production performance support, and exploration of the boundaries of applied SPE. Is it really just about performance.
{"title":"Software performance in the real world: personal lessons from the performance trauma team","authors":"Jayshankar Sankarasetty, Kevin Mobley, Libby Foster, Tad Hammer, Terri Calderone","doi":"10.1145/1216993.1217027","DOIUrl":"https://doi.org/10.1145/1216993.1217027","url":null,"abstract":"In the nine years that we have been involved in software performance engineering (SPE) and performance testing engagements we have learned several things. Across numerous eCommerce applications and an enterprise CRM product suite, our knowledge base about the field of Software Performance Engineering is constantly evolving. The focus of this paper is what we have learned in the areas of SPE project management, performance testing, defining the scope of SPE projects, ITIL, post production performance support, and exploration of the boundaries of applied SPE. Is it really just about performance.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126940524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents Puppet (Pick UP Performance Evaluation Test-bed), an approach for the automatic generation of test-beds to empirically evaluate different QoS features of a Web Service under development. Specifically, the generation exploits the information about the coordinating scenario, the service description and the specification of the agreements that the roles will abide. The approach is supported by a proof-of-concept tool to validate the feasibility of the idea.
本文提出了Puppet (Pick - UP Performance Evaluation Test-bed)，一种自动生成测试平台的方法，用于对正在开发的Web服务的不同QoS特性进行经验评估。具体地说，生成利用了关于协调场景、服务描述和角色将遵守的协议规范的信息。该方法由一个概念验证工具来支持，以验证该想法的可行性。
{"title":"Automatic generation of test-beds for pre-deployment QoS evaluation of web services","authors":"A. Bertolino, G. D. Angelis, A. Polini","doi":"10.1145/1216993.1217017","DOIUrl":"https://doi.org/10.1145/1216993.1217017","url":null,"abstract":"This paper presents Puppet (Pick UP Performance Evaluation Test-bed), an approach for the automatic generation of test-beds to empirically evaluate different QoS features of a Web Service under development. Specifically, the generation exploits the information about the coordinating scenario, the service description and the specification of the agreements that the roles will abide. The approach is supported by a proof-of-concept tool to validate the feasibility of the idea.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"506 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124461613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we present the mechanics of the approach that is at the core of our framework for developing and deploying adaptable applications. By leveraging this approach we are able to perform a quantitative resource-oriented analysis of Java applications. The analysis, considering the execution context of the application, allows the framework to decide what adaptation has to be chosen before the actual deployment and execution. The paper describes the resource model which is the foundation of the approach and the abstract resource analyzer. The latter performs, through an abstract operational semantics of the Java language, a static analysis of the adaptable application that enables the actual resource-oriented analysis.
{"title":"Quantitative resource-oriented analysis of Java (Adaptable) applications","authors":"F. Mancinelli, P. Inverardi","doi":"10.1145/1216993.1216998","DOIUrl":"https://doi.org/10.1145/1216993.1216998","url":null,"abstract":"In this paper we present the mechanics of the approach that is at the core of our framework for developing and deploying adaptable applications. By leveraging this approach we are able to perform a quantitative resource-oriented analysis of Java applications. The analysis, considering the execution context of the application, allows the framework to decide what adaptation has to be chosen before the actual deployment and execution. The paper describes the resource model which is the foundation of the approach and the abstract resource analyzer. The latter performs, through an abstract operational semantics of the Java language, a static analysis of the adaptable application that enables the actual resource-oriented analysis.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2007-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116392773","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper describes a method for evolutionary component-based development of families of parallel programs to attain performance goals on multiple execution environments for multiple family instances and an implementation of the method. It is based upon combining component-oriented development with integration of parallel/distributed execution and parallel/distributed simulation. Each component may have multiple representations at multiple levels of realization from analytical timing models to production code. Each component is encapsulated with an associative interface specifying its properties and behaviors which enables distinguishing among different implementations (or abstractions) of the same functional behavior which may have different performance behavior. Evolutionary development evolves a program from an abstract performance model to a complete program and may continue evolution during runtime. Performance can be estimated at any stage of realization. The implementation is a compiler which composes parallel/distributed programs from components encapsulated with associative interfaces and a runtime system which supports integrated execution/simulation of parallel programs composed from components at different levels of abstraction and program evolution at runtime by component replacement. Case studies in the application of the evolutionary development method including performance results are given.
{"title":"Evolutionary performance-oriented development of parallel programs by composition of components","authors":"N. Mahmood, Yusheng Feng, J. Browne","doi":"10.1145/1071021.1071048","DOIUrl":"https://doi.org/10.1145/1071021.1071048","url":null,"abstract":"This paper describes a method for evolutionary component-based development of families of parallel programs to attain performance goals on multiple execution environments for multiple family instances and an implementation of the method. It is based upon combining component-oriented development with integration of parallel/distributed execution and parallel/distributed simulation. Each component may have multiple representations at multiple levels of realization from analytical timing models to production code. Each component is encapsulated with an associative interface specifying its properties and behaviors which enables distinguishing among different implementations (or abstractions) of the same functional behavior which may have different performance behavior. Evolutionary development evolves a program from an abstract performance model to a complete program and may continue evolution during runtime. Performance can be estimated at any stage of realization. The implementation is a compiler which composes parallel/distributed programs from components encapsulated with associative interfaces and a runtime system which supports integrated execution/simulation of parallel programs composed from components at different levels of abstraction and program evolution at runtime by component replacement. Case studies in the application of the evolutionary development method including performance results are given.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116239968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Large distributed client-server systems often contain subsystems which are either identical to each other, or very nearly so, and this simplifies the system description for planning purposes. These replicated components and subsystems all have the same workload and performance parameters. It is known how to exploit this symmetry to simplify the solution of some kinds of performance models, using state aggregation in Markov Chains. This work considers the same problem for layered queueing models, using mean value analysis. The mean values are found for each group of replicas just once, and then are inserted appropriately into the solution of the system as a whole. An algorithm has been implemented in the Layered Queueing Network Solver (LQNS), including approximations to deal with interactions among the replicas, and is evaluated for accuracy and for efficiency. The resulting solver is insensitive (in time of solution) to the number of replicas in a group, and can efficiently calculate waiting times and throughputs for systems with tens of thousands of nodes and processes.
{"title":"Solving layered queueing networks of large client-server systems with symmetric replication","authors":"T. Omari, G. Franks, M. Woodside, Amy Pan","doi":"10.1145/1071021.1071038","DOIUrl":"https://doi.org/10.1145/1071021.1071038","url":null,"abstract":"Large distributed client-server systems often contain subsystems which are either identical to each other, or very nearly so, and this simplifies the system description for planning purposes. These replicated components and subsystems all have the same workload and performance parameters. It is known how to exploit this symmetry to simplify the solution of some kinds of performance models, using state aggregation in Markov Chains. This work considers the same problem for layered queueing models, using mean value analysis. The mean values are found for each group of replicas just once, and then are inserted appropriately into the solution of the system as a whole. An algorithm has been implemented in the Layered Queueing Network Solver (LQNS), including approximations to deal with interactions among the replicas, and is evaluated for accuracy and for efficiency. The resulting solver is insensitive (in time of solution) to the number of replicas in a group, and can efficiently calculate waiting times and throughputs for systems with tens of thousands of nodes and processes.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121490918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this talk, I discuss practical approaches to designing high performance data access systems and how design choices impact performance measurement methodology. The xrootd data access system is used as an example of how far the performance curve can be pushed, invariable tradeoffs that occur, and when one should stop pursuing the ultimate ideal.
{"title":"Designing high performance data access systems: invited talk abstract","authors":"A. Hanushevsky, Bill Weeks","doi":"10.1145/1071021.1071053","DOIUrl":"https://doi.org/10.1145/1071021.1071053","url":null,"abstract":"In this talk, I discuss practical approaches to designing high performance data access systems and how design choices impact performance measurement methodology. The xrootd data access system is used as an example of how far the performance curve can be pushed, invariable tradeoffs that occur, and when one should stop pursuing the ultimate ideal.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122271277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E-commerce systems are composed of many components with several configurable software parameters which, if properly configured, can optimize system performance. The cost and burden of implementing upgrades can be deferred by judicial tuning of these parameters. Tuning is a time consuming effort and energy has to be spent on the most relevant parameters. This paper provides a method to evaluate the statistical significance of configurable parameters in e-commerce systems, the interaction effects between each parameter and e-commerce function types, and a method to rank key configurable e-commerce system parameters that significantly impact overall system performance, and the performance of the most significant Web function types. Both on-line and off-line parameters at each of the e-commerce system layers---Web server, application server, and database server---are considered. The paper discusses the design of a practical and ad-hoc approach that involves conducting experiments on an e-commerce site compliant with the TPC-W benchmark. The performance metrics of interest include server's response time, system throughput, and probability of rejecting a customer's request. The paper also provides a set of useful guidelines for tuning the performance of e-commerce sites.
{"title":"A method for evaluating the impact of software configuration parameters on e-commerce sites","authors":"Monchai Sopitkamol, D. Menascé","doi":"10.1145/1071021.1071027","DOIUrl":"https://doi.org/10.1145/1071021.1071027","url":null,"abstract":"E-commerce systems are composed of many components with several configurable software parameters which, if properly configured, can optimize system performance. The cost and burden of implementing upgrades can be deferred by judicial tuning of these parameters. Tuning is a time consuming effort and energy has to be spent on the most relevant parameters. This paper provides a method to evaluate the statistical significance of configurable parameters in e-commerce systems, the interaction effects between each parameter and e-commerce function types, and a method to rank key configurable e-commerce system parameters that significantly impact overall system performance, and the performance of the most significant Web function types. Both on-line and off-line parameters at each of the e-commerce system layers---Web server, application server, and database server---are considered. The paper discusses the design of a practical and ad-hoc approach that involves conducting experiments on an e-commerce site compliant with the TPC-W benchmark. The performance metrics of interest include server's response time, system throughput, and probability of rejecting a customer's request. The paper also provides a set of useful guidelines for tuning the performance of e-commerce sites.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"404 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122943555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In order to effectively validate the performance of software systems throughout their development cycle it is necessary to continuously build performance models from software models and then use the obtained models to check whether the system is being developed according to its performance requirements. The model building activity is a critical and effort-consuming activity. Several approaches have been envisaged to endow software designers with tools that automatically build ready-to-evaluate performance models from software development models. One essential requirement of such tools, often disregarded by current approaches, is a high degree of interoperability with software development tools, which has the positive effect of reducing both the level of required expertise in performance theory and the burden of learning separate tools. This paper introduces a framework for transforming source software models into target performance models. The transformation requires a clear understanding of the abstract syntax and semantics of both the source and target models, which is obtained by use of metamodeling techniques for defining the abstract syntax of models, the interrelationships between model elements and the model transformation rules. In the paper case, the framework is applied to the transformation of source models of UML type into target models of LQN (layered queueing network) type. The proposed approach is founded on the precepts recently introduced by model-driven development (MDA) and makes use of the set of related standards (MOF, QVT, XMI). This allows to obtain a high degree of automation, so that interoperable model transformation tools can be implemented in a timely and efficient way, leading to improvements in terms of software designers' productivity and system quality.
{"title":"A model transformation framework for the automated building of performance models from UML models","authors":"A. D’Ambrogio","doi":"10.1145/1071021.1071029","DOIUrl":"https://doi.org/10.1145/1071021.1071029","url":null,"abstract":"In order to effectively validate the performance of software systems throughout their development cycle it is necessary to continuously build performance models from software models and then use the obtained models to check whether the system is being developed according to its performance requirements. The model building activity is a critical and effort-consuming activity. Several approaches have been envisaged to endow software designers with tools that automatically build ready-to-evaluate performance models from software development models. One essential requirement of such tools, often disregarded by current approaches, is a high degree of interoperability with software development tools, which has the positive effect of reducing both the level of required expertise in performance theory and the burden of learning separate tools. This paper introduces a framework for transforming source software models into target performance models. The transformation requires a clear understanding of the abstract syntax and semantics of both the source and target models, which is obtained by use of metamodeling techniques for defining the abstract syntax of models, the interrelationships between model elements and the model transformation rules. In the paper case, the framework is applied to the transformation of source models of UML type into target models of LQN (layered queueing network) type. The proposed approach is founded on the precepts recently introduced by model-driven development (MDA) and makes use of the set of related standards (MOF, QVT, XMI). This allows to obtain a high degree of automation, so that interoperable model transformation tools can be implemented in a timely and efficient way, leading to improvements in terms of software designers' productivity and system quality.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"476 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133470904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Designers of distributed systems often rely on replicas for increased robustness, scalability, and performance. Replicated server architectures require some technique to select a target replica for each client transaction. In this paper, we use simulation to characterize performance (response time, selection error, probability of server overload) for four common replica-selection algorithms (random, greedy, partitioned, weighted) when applied in a decentralized form to client queries in a distributed object system deployed on a local network. We introduce two new selection algorithms (balanced and balanced-partitioned) that give improved performance over the more common algorithms. We find the weighted algorithm performs best among the common algorithms and the balanced algorithm performs best among all those we considered. Our findings should help designers of distributed object systems to make informed decisions when choosing among available replica-selection algorithms.
{"title":"Performance characterization of decentralized algorithms for replica selection in distributed object systems","authors":"C. Tan, K. Mills","doi":"10.1145/1071021.1071050","DOIUrl":"https://doi.org/10.1145/1071021.1071050","url":null,"abstract":"Designers of distributed systems often rely on replicas for increased robustness, scalability, and performance. Replicated server architectures require some technique to select a target replica for each client transaction. In this paper, we use simulation to characterize performance (response time, selection error, probability of server overload) for four common replica-selection algorithms (random, greedy, partitioned, weighted) when applied in a decentralized form to client queries in a distributed object system deployed on a local network. We introduce two new selection algorithms (balanced and balanced-partitioned) that give improved performance over the more common algorithms. We find the weighted algorithm performs best among the common algorithms and the balanced algorithm performs best among all those we considered. Our findings should help designers of distributed object systems to make informed decisions when choosing among available replica-selection algorithms.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115568432","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Recently, growing attention is focused on run-time management of Quality of Service (QoS) of complex software systems. In this context, self-adaptation of applications, based on runtime monitoring and dynamic reconfiguration, is considered a useful technique to manage/negotiate QoS. Several defined approaches to dynamic performance management propose the use of on-line model-based evaluation of the managed application in order to choose the new application configuration for improving the QoS. In a previous work we defined a methodology to manage performance of soft-ware system at run-time based on model-based performance evaluation. To avoid unnecessary details the performance model represents the managed application at the Software Architecture level. In this work we apply and implement this approach to dynamic manage the performance of the SIENA publish/subscribe event notification service.
{"title":"Run-time performance management of the Siena publish/subscribe middleware","authors":"M. Caporuscio, A. Marco, P. Inverardi","doi":"10.1145/1071021.1071028","DOIUrl":"https://doi.org/10.1145/1071021.1071028","url":null,"abstract":"Recently, growing attention is focused on run-time management of Quality of Service (QoS) of complex software systems. In this context, self-adaptation of applications, based on runtime monitoring and dynamic reconfiguration, is considered a useful technique to manage/negotiate QoS. Several defined approaches to dynamic performance management propose the use of on-line model-based evaluation of the managed application in order to choose the new application configuration for improving the QoS. In a previous work we defined a methodology to manage performance of soft-ware system at run-time based on model-based performance evaluation. To avoid unnecessary details the performance model represents the managed application at the Software Architecture level. In this work we apply and implement this approach to dynamic manage the performance of the SIENA publish/subscribe event notification service.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2005-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125751237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：