Performance Evaluation最新文献

Age-of-information is a metric that quantifies the freshness of information obtained by sampling a remote sensor. In signal-agnostic sampling, sensor updates are triggered at certain times without being conditioned on the actual sensor signal. Optimal update policies have been researched and it is accepted that periodic updates achieve smaller age-of-information than random updates. We contribute a study of a signal-aware policy, where updates are triggered randomly by a defined sensor event. By definition, this implies random updates and as a consequence inferior age-of-information. Considering a notion of deviation-of-information as a signal-aware metric, our results show, however, that event-triggered systems can perform equally well as time-triggered systems while causing smaller mean network utilization. We use the stochastic network calculus to derive bounds of age- and deviation-of-information that are exceeded at most with a small, defined probability. We include simulation results that confirm the tail decay of the bounds. We also evaluate a hybrid time- and event-triggered policy where the event-triggered system is complemented by a minimal and a maximal update interval.

Due to scalability requirements and the split of large software development projects into small agile teams, there is a current trend toward the migration of monolith systems to the microservice architecture. However, the split of the monolith into microservices, its encapsulation through well-defined interfaces, and the introduction of inter-microservice communication add a cost in terms of performance. In this paper, we describe a case study of the migration of a monolith to a microservice architecture, where a modular monolith architecture is used as an intermediate step. The impact on migration effort and performance is measured for both steps. Current state-of-the-art analyses the migration of monolith systems to a microservice architecture, but we observed that migration effort and performance issues are already significant in the migration to a modular monolith. Therefore, a clear distinction is established for each of the steps, which may inform software architects on the planning of the migration of monolith systems. In particular, we consider the trade-offs of doing all the migration process or just migrating to a modular monolith.

We consider a system with $n$ unit-rate servers where jobs arrive according a Poisson process with rate $n\lambda$ ($\lambda <1$). In the standard Power-of-$d$ or Pod scheme with $d\ge 2$, for each incoming job, a dispatcher samples $d$ servers uniformly at random and sends the incoming job to the least loaded of the $d$ sampled servers. However, in practice, load comparisons may not always be accurate. In this paper, we analyse the effects of noisy load comparisons on the performance of the Pod scheme. To test the robustness of the Pod scheme against load comparison errors, we assume an adversarial setting where, in the event of an error, the adversary assigns the incoming job to the worst possible server, i.e., the server with the maximum load among the $d$ sampled servers. We consider two error models: load-dependent and load-independent errors. In the load-dependent error model, the adversary has limited power in that it is able to cause an error with probability $ϵ\in [0,1]$ only when the difference in the minimum and the maximum queue lengths of the $d$ sampled servers is bounded by a constant threshold $g\ge 0$. For this type of errors, we show that, in the large system limit, the benefits of the Pod scheme are retained even if $g$ and $ϵ$ are arbitrarily large as long as the system is heavily loaded, i.e., $\lambda$ is close to 1. In the load-independent error model, the adversary is assumed to be more powerful in that it can cause an error with probability $ϵ$ independent of the loads of the sampled servers. For this model, we show that the performance benefits of the Pod scheme are retained only if $ϵ\le 1/d$; for $ϵ>1/d$ we show that the stability region of the system reduces and the system performs poorly in comparison to the random scheme. Our mean-field analysis uses a new approach to characterise fixed points which neither have closed form solutions nor admit any recursion. Furthermore, we develop a generic approach to prove tightness and stability for any state-dependent load balancing scheme.

As the complexity of processor microarchitecture and applications increases, obtaining performance optimization knowledge, such as critical dependent chains, becomes more challenging. To tackle this issue, this paper employs pattern mining methods to analyze the critical path of processor micro-execution dependence graphs. We propose a high average utility pattern mining algorithm called Dependence Graph Miner (DG-Miner) based on the characteristics of dependence graphs. DG-Miner overcomes the limitations of current pattern mining algorithms for dependence graph pattern mining by offering support for variable utility, candidate generation using endpoint matching, the adjustable upper bound, and the concise pattern judgment mechanism. Experiments reveal that, compared with existing upper bound candidate generation methods, the adjustable upper bound reduces the number of candidate patterns by 28.14% and the running time by 27% on average. The concise pattern judgment mechanism enhances the conciseness of mining results by 16.31% and reduces the running time by 39.82%. Furthermore, DG-Miner aids in identifying critical dependent chains, critical program regions, and performance exceptions.

Storage cache hierarchies include diverse topologies, assorted parameters and policies, and devices with varied performance characteristics. Simulation enables efficient exploration of their configuration space while avoiding expensive physical experiments. Miss Ratio Curves (MRCs) efficiently characterize the performance of a cache over a range of cache sizes, revealing “key points” for cache simulation, such as knees in the curve that immediately follow sharp cliffs. Unfortunately, there are no automated techniques for efficiently finding key points in MRCs, and the cross-application of existing knee-detection algorithms yields inaccurate results.

We present a multi-stage framework that identifies key points in any MRC, for both stack-based (e.g., LRU) and more sophisticated eviction algorithms (e.g., ARC). Our approach quickly locates candidates using efficient hash-based sampling, curve simplification, knee detection, and novel post-processing filters. We introduce Z-Method, a new multi-knee detection algorithm that employs statistical outlier detection to choose promising points robustly and efficiently.

We evaluated our framework against seven other knee-detection algorithms, identifying key points in multi-tier MRCs with both ARC and LRU policies for 106 diverse real-world workloads. Compared to naïve approaches, our framework reduced the total number of points needed to accurately identify the best two-tier cache hierarchies by an average factor of approximately $5.5\times$ for ARC and $7.7\times$ for LRU.

We also show how our framework can be used to seed the initial population for evolutionary algorithms. We ran 32,616 experiments requiring over three million cache simulations, on 151 samples, from three datasets, using a diverse set of population initialization techniques, evolutionary algorithms, knee-detection algorithms, cache replacement algorithms, and stopping criteria. Our results showed an overall acceleration rate of 34% across all configurations.

Exchanging time-critical information is prevalent in various industrial applications where low latency and timely delivery are paramount. Through this work, we consider a cognitive radio network comprised of multiple secondary users with time-sensitive traffic, and they can access the licensed channel under the hybrid interweave/underlay scheme to enhance spectrum utilization. Traffic in the secondary system is divided into two distinct categories: deadline-constrained data and status updates. Quality of service of data with expiration time, such as multimedia streams, is assessed through the timely throughput metric. However, the age of information metric is used to characterize the freshness of the status update packets, which is vital in several emerging applications. Within an interference constraint imposed by the primary user, a dynamic scheduling policy is proposed to optimize the weighted sum of the average age of information of the status update users under a strict timely throughput requirement for each user with deadline-constrained traffic. We formulate the optimization problem as a constrained Markov decision process. Then, through the drift-plus-penalty method, the problem is reduced into a series of unconstrained Markov decision problems. Finally, each subproblem is tackled using the backward dynamic programming technique. Simulation results illustrate the effect of the main system parameters, such as the PU transmitted power and transmission rate level, on the performance of the secondary system. Moreover, the model feasibility regarding the fulfillment of the constraints against PU activity is experimentally investigated under the proposed hybrid mode and classical interweave mode. The performance of the proposed policy is compared to two other low-complexity scheduling schemes, which ensure the satisfaction of the constraints; results show the performance superiority of our proposed policy.

The FlexRay communication protocol provides high bandwidth for supporting both hard deadline and soft deadline traffic in in-vehicle communication networks. In this paper, we carry out delay analysis of soft deadline traffic which is handled by the dynamic segment of FlexRay. We model the arrival of these messages as Poisson processes, and use queuing theory to evaluate the average delay that they experience. Initially, we consider three nodes competing for service, assuming that two out of three can transmit messages in any FlexRay cycle and obtain expressions for the evolution of the corresponding queues. We also determine the range of message arrival rates for which the queues are stable. These results are then extended to the general case of N queues. The analytical results are compared with those obtained by simulation for a typical system.

Markov regenerative process (MRGP) is favored for modeling and evaluating system dependability due to its high power and flexibility. However, its analysis presents challenges because of its inherent renewal nature. The embedded Markov chain (EMC) method offers a stationary solution to the MRGP, while the phase expansion approach delivers both stationary and transient solutions. From these solutions, one can derive performance or dependability measures as outputs from the MRGP model. It is crucial to conduct a sensitivity analysis on MRGP to understand the influence of input factor changes on model outputs, aiding efficient system optimization. Yet, a clear analytical method for sensitivity analysis of MRGP models is currently lacking. Filling this gap, this paper introduces an analytical approach to assess parametric sensitivity for steady-state MRGP, utilizing the EMC method for obtaining the stationary solution. Specifically, since system availability closely correlates with the average system available duration, this paper also shifts its focus from mere model parameters to representative values, like the average available time of a system.

In this paper, we study dispatching systems that appear in manufacturing, service, healthcare systems, as well as, in various information, communication and computer systems. Such systems comprise a dispatcher and a pool of parallel servers, to which jobs are assigned upon arrival. A common objective is to minimize the mean waiting or response time. In large systems, due to the state-space explosion and scalability reasons, it is impossible to utilize full state information of the system. We therefore consider systems with a small number of servers, and assume that the job sizes become known upon arrival. In such settings, it is plausible to carefully evaluate each server for every new job. First we study a system with a Poisson arrival process, and derive Bellman equations. Then we generalize to the case with general i.i.d. inter-arrival times. The Bellman equations are essentially functional equations that can be solved numerically via value iteration. From their solutions, the optimal dispatching policy and corresponding mean performance can be determined. Our solution framework is illustrated with examples, which show that significant performance gains compared to popular heuristic policies are available in our setting.

In this paper, we study the source (event) localization problem in decentralized wireless sensor networks (WSNs) under faulty sensor nodes without knowledge of the sensor parameters. Source localization has many applications, such as localizing WiFi hotspots and mobile users. Some works in the literature localize the source by utilizing the knowledge or estimates of the fault probability of each sensor node or the region of influence of the source. However, this paper proposes two approaches: the hitting set and feature selection for estimating the source location without any knowledge of the sensor parameters under faulty sensor nodes in WSN. The proposed approaches provide better or comparable source localization performances. For the hitting set approach, we also derive a lower bound on the required number of samples. In addition, we extend the proposed methods for localizing multiple sources. Finally, we provide extensive simulations to illustrate the performances of the proposed methods against the centroid, maximum likelihood (ML), fault-tolerant ML (FTML), and subtract on negative add on positive (SNAP) estimators. The proposed approaches significantly outperform the centroid and maximum likelihood estimators for faulty sensor nodes while providing comparable or better performance to FTML or SNAP algorithm. In addition, we use real-world WiFi data set to localize the source in comparison to the support vector machine based estimator in the literature, where the proposed methods outperformed the estimator.