While the use of network intrusion detection systems (nIDS) is becoming pervasive, evaluating nIDS performance has been found to be challenging. The goal of this study is to determine how to generate realistic workloads for nIDS performance evaluation. We develop a workload model that appears to provide reasonably accurate estimates compared to real workloads. The model attempts to emulate a traffic mix of different applications, reflecting characteristics of each application and the way these interact with the system. We have implemented this model as part of a traffic generator that can be extended and tuned to reflect the needs of different scenarios. We also present an approach to measuring the capacity of a nIDS that does not require the setup of a full network testbed.
{"title":"Generating realistic workloads for network intrusion detection systems","authors":"S. Antonatos, K. Anagnostakis, E. Markatos","doi":"10.1145/974044.974078","DOIUrl":"https://doi.org/10.1145/974044.974078","url":null,"abstract":"While the use of network intrusion detection systems (nIDS) is becoming pervasive, evaluating nIDS performance has been found to be challenging. The goal of this study is to determine how to generate realistic workloads for nIDS performance evaluation. We develop a workload model that appears to provide reasonably accurate estimates compared to real workloads. The model attempts to emulate a traffic mix of different applications, reflecting characteristics of each application and the way these interact with the system. We have implemented this model as part of a traffic generator that can be extended and tuned to reflect the needs of different scenarios. We also present an approach to measuring the capacity of a nIDS that does not require the setup of a full network testbed.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114153844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Performance characteristics, such as response time, through put and scalability, are key quality attributes of distributed applications. Current practice, however, rarely applies systematic techniques to evaluate performance characteristics. We argue that evaluation of performance is particularly crucial in early development stages, when important architectural choices are made. At first glance, this contradicts the use of testing techniques, which are usually applied towards the end of a project. In this paper, we assume that many distributed systems are built with middleware technologies, such as the Java 2 Enterprise Edition (J2EE) or the Common Object Request Broker Architecture (CORBA). These provide services and facilities whose implementations are available when architectures are defined. We also note that it is the middleware functionality, such as transaction and persistence services, remote communication primitives and threading policy primitives, that dominate distributed system performance Drawing on these observations, this paper presents a novel approach to performance testing of distributed applications. We propose to derive application-specific test cases from architecture designs so that performance of a distributed application can be tested using the middleware software at early stages of a development process. We report empirical results that support the viability of the approach.
{"title":"Early performance testing of distributed software applications","authors":"G. Denaro, A. Polini, W. Emmerich","doi":"10.1145/974044.974059","DOIUrl":"https://doi.org/10.1145/974044.974059","url":null,"abstract":"Performance characteristics, such as response time, through put and scalability, are key quality attributes of distributed applications. Current practice, however, rarely applies systematic techniques to evaluate performance characteristics. We argue that evaluation of performance is particularly crucial in early development stages, when important architectural choices are made. At first glance, this contradicts the use of testing techniques, which are usually applied towards the end of a project. In this paper, we assume that many distributed systems are built with middleware technologies, such as the Java 2 Enterprise Edition (J2EE) or the Common Object Request Broker Architecture (CORBA). These provide services and facilities whose implementations are available when architectures are defined. We also note that it is the middleware functionality, such as transaction and persistence services, remote communication primitives and threading policy primitives, that dominate distributed system performance Drawing on these observations, this paper presents a novel approach to performance testing of distributed applications. We propose to derive application-specific test cases from architecture designs so that performance of a distributed application can be tested using the middleware software at early stages of a development process. We report empirical results that support the viability of the approach.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115494327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Non-functional attributes of software/hardware systems are aspects of interest to embed in modeling notations. In the last few years, extensions of UML in this direction have been proposed, and some of them have been recently adopted as final specifications, such as the UML Profile for Schedulability, Performance, and Time. In this paper we intend to further contribute to the integration of UML with non-functional aspects, and we devise a lightweight extension of UML (i.e., stereotypes, tagged values and constraints) to represent issues related to the reliability modeling of component-based systems. To a certain extent we lay on other profiles to make easier the embedding of such issues and to work toward a unifying UML profile for Quality of Service and Fault Tolerance. Our extensions mostly concern the fault forecasting task, which enables the specification of models describing the failure rates of components and combines them to obtain a reliability factor for the whole system. Example diagrams from an elevator system help to show how the concepts introduced here can be used to design reliability-integrated UML models of component-based systems. In order to demonstrate the practical benefits of our approach we also show how to fit the proposed tagged values into a reliability model.
{"title":"Towards a UML profile for QoS: a contribution in the reliability domain","authors":"V. Cortellessa, A. Pompei","doi":"10.1145/974044.974076","DOIUrl":"https://doi.org/10.1145/974044.974076","url":null,"abstract":"Non-functional attributes of software/hardware systems are aspects of interest to embed in modeling notations. In the last few years, extensions of UML in this direction have been proposed, and some of them have been recently adopted as final specifications, such as the UML Profile for Schedulability, Performance, and Time. In this paper we intend to further contribute to the integration of UML with non-functional aspects, and we devise a lightweight extension of UML (i.e., stereotypes, tagged values and constraints) to represent issues related to the reliability modeling of component-based systems. To a certain extent we lay on other profiles to make easier the embedding of such issues and to work toward a unifying UML profile for Quality of Service and Fault Tolerance. Our extensions mostly concern the fault forecasting task, which enables the specification of models describing the failure rates of components and combines them to obtain a reliability factor for the whole system. Example diagrams from an elevator system help to show how the concepts introduced here can be used to design reliability-integrated UML models of component-based systems. In order to demonstrate the practical benefits of our approach we also show how to fit the proposed tagged values into a reliability model.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125965991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we report a survey examining the approach to performance and software engineering in courses at highly ranked computer science schools in the United States. An analysis of the survey shows serious shortcomings including inadequate or missing definitions of performance, reactive "fix it later" mentality, vague performance requirements, and a general lack of awareness of the practices developed by the Software Performance Engineering (SPE) community. The survey is followed by guidelines for teaching SPE to undergraduates based on a semester long course we have developed. It is our plan to incorporate these guidelines into the curriculum of our senior capstone software engineering course.
{"title":"Performance lies my professor told me: the case for teaching Software Performance Engineering to undergraduates","authors":"R. Dugan","doi":"10.1145/974044.974050","DOIUrl":"https://doi.org/10.1145/974044.974050","url":null,"abstract":"In this paper we report a survey examining the approach to performance and software engineering in courses at highly ranked computer science schools in the United States. An analysis of the survey shows serious shortcomings including inadequate or missing definitions of performance, reactive \"fix it later\" mentality, vague performance requirements, and a general lack of awareness of the practices developed by the Software Performance Engineering (SPE) community. The survey is followed by guidelines for teaching SPE to undergraduates based on a semester long course we have developed. It is our plan to incorporate these guidelines into the curriculum of our senior capstone software engineering course.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129842329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We describe a structured, hierarchic approach to exploring the scalability of IT systems architectures. An architecture is considered to be scalable over a particular set of requirements if the physical resource usage per unit of capacity remains roughly constant. For completeness, both requirements and capacity must be defined in the three dimensions of processing, storage and connectivity. Interactions between the three dimensions are considered, as are various forms of departure from non-uniform scaling. Scalability is explored via a combination of measurement and static and dynamic models. Appropriate scale-invariants are introduced to eliminate congestion effects and packaging issues from the analysis. This paper focuses on processing and to a lesser extent, on storage. The method is applied to a practical case study of Transigo, a J2EE-based software platform used in the Norwegian banking industry. We find that understanding the relationship between replication and upgrade for systems, subsystems and devices is key to guiding the exploration of scalability.
{"title":"Exploring architectural scalability","authors":"Gunnar Brataas, P. H. Hughes","doi":"10.1145/974044.974064","DOIUrl":"https://doi.org/10.1145/974044.974064","url":null,"abstract":"We describe a structured, hierarchic approach to exploring the scalability of IT systems architectures. An architecture is considered to be scalable over a particular set of requirements if the physical resource usage per unit of capacity remains roughly constant. For completeness, both requirements and capacity must be defined in the three dimensions of processing, storage and connectivity. Interactions between the three dimensions are considered, as are various forms of departure from non-uniform scaling. Scalability is explored via a combination of measurement and static and dynamic models. Appropriate scale-invariants are introduced to eliminate congestion effects and packaging issues from the analysis. This paper focuses on processing and to a lesser extent, on storage. The method is applied to a practical case study of Transigo, a J2EE-based software platform used in the Norwegian banking industry. We find that understanding the relationship between replication and upgrade for systems, subsystems and devices is key to guiding the exploration of scalability.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129590948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Publish/subscribe middlewares are becoming popular for distributed applications because of their flexible and scalable nature. Anonymous and loosely-coupled communication between publisher and subscriber, along with the inherently asynchronous nature of these systems, help them adapt quickly to changing environments, making them a good choice for mobile cellular networks. This paper studies publish/subscribe middleware performance in such networks in detail. As a first step, the paper characterizes a popular implementation of publish/subscribe system for the mobile domain, studying and analyzing the effect of different mobility parameters, which to the best of our knowledge is the first experimental research on the performance behavior of publish/subscribe systems in a mobile wireless domain. As a second step the paper presents the design, implementation and evaluation of middleware level handoffs, a well known solution to extend publish/subscribe systems to a mobile domain, and identifies the performance concerns of such extensions. The results show that such handoff protocols involving two brokers are impractical from a performance perspective under highly dynamic and unreliable mobile wireless settings. The paper identifies the basic reason for the limitations of middleware level handoffs.
{"title":"Performance of publish/subscribe middleware in mobile wireless networks","authors":"U. Farooq, E. Parsons, S. Majumdar","doi":"10.1145/974044.974088","DOIUrl":"https://doi.org/10.1145/974044.974088","url":null,"abstract":"Publish/subscribe middlewares are becoming popular for distributed applications because of their flexible and scalable nature. Anonymous and loosely-coupled communication between publisher and subscriber, along with the inherently asynchronous nature of these systems, help them adapt quickly to changing environments, making them a good choice for mobile cellular networks. This paper studies publish/subscribe middleware performance in such networks in detail. As a first step, the paper characterizes a popular implementation of publish/subscribe system for the mobile domain, studying and analyzing the effect of different mobility parameters, which to the best of our knowledge is the first experimental research on the performance behavior of publish/subscribe systems in a mobile wireless domain. As a second step the paper presents the design, implementation and evaluation of middleware level handoffs, a well known solution to extend publish/subscribe systems to a mobile domain, and identifies the performance concerns of such extensions. The results show that such handoff protocols involving two brokers are impractical from a performance perspective under highly dynamic and unreliable mobile wireless settings. The paper identifies the basic reason for the limitations of middleware level handoffs.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128227728","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A data model, or UML class diagram of a software system, is of central importance to the computing time and the disk usage a software system exhibits. It can be normalized or denormalized to better utilize the components of a computer system or to offer acceptable performance values.This paper describes an approach to software performance analysis that is based on the UML class diagram. It predicts the performance values for use cases operating on persistent data collections as well as whether and how much normalization or denormalization a UML class diagram may need, in order to ensure that these use cases meet their performance objectives.
{"title":"A performance analysis approach based on the UML class diagram","authors":"Ahmad Alsaadi","doi":"10.1145/974044.974084","DOIUrl":"https://doi.org/10.1145/974044.974084","url":null,"abstract":"A data model, or UML class diagram of a software system, is of central importance to the computing time and the disk usage a software system exhibits. It can be normalized or denormalized to better utilize the components of a computer system or to offer acceptable performance values.This paper describes an approach to software performance analysis that is based on the UML class diagram. It predicts the performance values for use cases operating on persistent data collections as well as whether and how much normalization or denormalization a UML class diagram may need, in order to ensure that these use cases meet their performance objectives.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128405419","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper we are primarily concerned with embedded real-time systems such as mobile phones or personal communication devices. Today personal communication devices are more than voice call terminals. Mobile phones serve as platforms for a variety of mobile applications including text and picture messaging as well as personal information management, including data synchronization with remote servers and desktop computers. Mobile phones host a range of communication-centered applications most of which have real-time constraints. During a voice call speech data must be processed in a timely fashion to avoid jitter. During a GPRS session lower layer packets arrive every 10 ms. These are just few examples of system requirements that lead to tight software performance constraints.
{"title":"Applying fixed priority scheduling in practice","authors":"Raimondas Lencevicius, A. Ran","doi":"10.1145/974044.974069","DOIUrl":"https://doi.org/10.1145/974044.974069","url":null,"abstract":"In this paper we are primarily concerned with embedded real-time systems such as mobile phones or personal communication devices. Today personal communication devices are more than voice call terminals. Mobile phones serve as platforms for a variety of mobile applications including text and picture messaging as well as personal information management, including data synchronization with remote servers and desktop computers. Mobile phones host a range of communication-centered applications most of which have real-time constraints. During a voice call speech data must be processed in a timely fashion to avoid jitter. During a GPRS session lower layer packets arrive every 10 ms. These are just few examples of system requirements that lead to tight software performance constraints.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131700448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Mills, S. Rose, S. Quirolgico, M. Britton, C. Tan
Designs for distributed systems must consider the possibility that failures will arise and must adopt specific failure detection strategies. We describe and analyze a self-regulating failure-detection algorithm that bounds resource usage and failure-detection latency, while automatically reassigning resources to improve failure-detection latency as system size decreases. We apply the algorithm to (1) Jini leasing, (2) service registration in the Service Location Protocol (SLP), and (3) SLP service polling
{"title":"An autonomic failure-detection algorithm","authors":"K. Mills, S. Rose, S. Quirolgico, M. Britton, C. Tan","doi":"10.1145/974044.974056","DOIUrl":"https://doi.org/10.1145/974044.974056","url":null,"abstract":"Designs for distributed systems must consider the possibility that failures will arise and must adopt specific failure detection strategies. We describe and analyze a self-regulating failure-detection algorithm that bounds resource usage and failure-detection latency, while automatically reassigning resources to improve failure-detection latency as system size decreases. We apply the algorithm to (1) Jini leasing, (2) service registration in the Service Location Protocol (SLP), and (3) SLP service polling","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"70 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126375564","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The next generation of software systems will be highly distributed, component-based, service-oriented, will need to operate in unattended mode and possibly in hostile environments, will be composed of a large number of "replaceable" components discoverable at run-time, and will have to run on a multitude of unknown and heterogeneous hardware and network platforms. This paper focuses on service oriented-architectures in which each component provides a set of interrelated services to other components. These components are QoS-aware (i.e., aware of Quality of Service requirements) and are capable of engaging in QoS negotiations with other components of a distributed application. The main contributions of this paper are: i) the description of an architecture for QoS-aware software components that are able to negotiate QoS requirements with other components, ii) the specification of the protocols used for QoS negotiation and admission control at the QoS-aware components, iii) a report on the implementation of a QoS-aware component, and iv) the experimental validation of the ideas presented in the paper.
{"title":"A framework for QoS-aware software components","authors":"D. Menascé, Honglei Ruan, H. Gomaa","doi":"10.1145/974044.974075","DOIUrl":"https://doi.org/10.1145/974044.974075","url":null,"abstract":"The next generation of software systems will be highly distributed, component-based, service-oriented, will need to operate in unattended mode and possibly in hostile environments, will be composed of a large number of \"replaceable\" components discoverable at run-time, and will have to run on a multitude of unknown and heterogeneous hardware and network platforms. This paper focuses on service oriented-architectures in which each component provides a set of interrelated services to other components. These components are QoS-aware (i.e., aware of Quality of Service requirements) and are capable of engaging in QoS negotiations with other components of a distributed application. The main contributions of this paper are: i) the description of an architecture for QoS-aware software components that are able to negotiate QoS requirements with other components, ii) the specification of the protocols used for QoS negotiation and admission control at the QoS-aware components, iii) a report on the implementation of a QoS-aware component, and iv) the experimental validation of the ideas presented in the paper.","PeriodicalId":235512,"journal":{"name":"Workshop on Software and Performance","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2004-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123598541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：