Performance Evaluation最新文献

This paper introduces TorchAxf¹, a framework for fast simulation of diverse approximate deep neural network (DNN) models, including spiking neural networks (SNNs). The proposed framework utilizes various approximate adders and multipliers, supports industrial standard reduced precision floating-point formats, such as bfloat16, and accommodates user-customized precision representations. Leveraging GPU acceleration on the PyTorch framework, TorchAxf accelerates approximate DNN training and inference. In addition, it allows seamless integration of arbitrary approximate arithmetic algorithms with C/C++ behavioral models to emulate approximate DNN hardware accelerators.

We utilize the proposed TorchAxf framework to assess twelve popular DNN models under approximate multiply-and-accumulate (MAC) operations. Through comprehensive experiments, we determine the suitable degree of floating-point arithmetic approximation for these DNN models without significant accuracy loss and offer the optimal reduced precision formats for each DNN model. Additionally, we demonstrate that approximate-aware re-training can rectify errors and enhance pre-trained DNN models under reduced precision formats. Furthermore, TorchAxf, operating on GPU, remarkably reduces simulation time for complex DNN models using approximate arithmetic by up to 131.38$\times$ compared to the baseline optimized CPU implementation. Finally, we compare the proposed framework with state-of-the-art frameworks to highlight its superiority.

We consider a Markovian retrial queueing system with customer collisions and abandonment in the context of carrier-sense multiple access systems. Using $z$-transform techniques, we find a set of first-order differential equations for the probability generating functions of the orbit size when the server is empty, busy, or in the collision phase. We then rely on series expansion techniques to extract approximations for relevant performance measures from this set of differential equations. More precisely, we construct a numerical algorithm to calculate the terms in the series expansions of various factorial moments of the orbit size. To improve the accuracy of our series expansion approach, we apply Wynn’s epsilon algorithm which not only speeds up convergence, but also extends the region of convergence. We illustrate the accuracy of our approach by means of some numerical examples, and find that the method is both fast and accurate for a wide range of the parameter values.

Energy Packet Networks (EPNs) model the interaction between renewable sources generating energy following a random process and communication devices that consume energy. This network is formed by cells and, in each cell, there is a queue that handles energy packets and another queue that handles data packets. We assume Poisson arrivals of energy packets and of data packets to all the cells and exponential service times. We consider an EPN model with a dynamic load balancing where a cell without data packets can poll other cells to migrate jobs. This migration can only take place when there is enough energy in both interacting cells, in which case a batch of data packets is transferred and the required energy is consumed (i.e. it disappears). We consider that data packet also consume energy to be routed to the next station. Our main result shows that the steady-state distribution of jobs in the queues admits a product form solution provided that a stable solution of a fixed point equation exists. We prove sufficient conditions for irreducibility. Under these conditions and when the fixed point equation has a solution, the Markov chain is ergodic. We also provide sufficient conditions for the existence of a solution of the fixed point equation. We then focus on layered networks and we study the polling rates that must be set to achieve a fair load balancing, i.e., such that, in the same layer, the load of the queues handling data packets is the same. Our numerical experiments illustrate that dynamic load balancing satisfies several interesting properties such as performance improvement or fair load balancing.

Network slicing is a key component of 5G-and-beyond networks but induces many questions related to an associated business model and its need to be regulated due to its difficult co-existence with the network neutrality debate. We propose in this paper a slicing model in the case of heterogeneous users/applications where a service provider may purchase a slice in a wireless network and offer a “premium” service where the improved quality stems from higher prices leading to less demand and less congestion than the basic service offered by the network owner, a scheme known as Paris Metro Pricing. We obtain thanks to game theory the economically-optimal slice size and prices charged by all actors. We also compare with the case of a unique “pipe” (no premium service) corresponding to a fully-neutral scenario and with the case of vertical integration to evaluate the impact of slicing on all actors and identify the “best” economic scenario and the eventual need for regulation.

Motivated by real-life applications, a special research-work interest has been recently directed towards the prioritized status update systems, which prioritize the update streams according to their timeliness constraints. The preferential service treatment between priority classes is commonly based on classical disciplines, preemption and non-preemption. However, both disciplines fail to give an even satisfaction between all classes. In our work, an interruption-based hybrid preemptive/non-preemptive discipline is proposed under a single-buffer system modeled as an M/M/1/2 priority queueing system. Each class being served (resp. buffered) can be preempted unless its recorded number of service preemptions reaches the predetermined in-service (resp. in-waiting) threshold. All thresholds between classes are the controlling parameters of the whole system’s performance. Using the stochastic hybrid system approach, the age of information (AoI) performance metric is analyzed in terms of its statistical average along with the higher-order moments, considering a general number of priority classes. Closed-form results are also obtained for some special cases, giving analytical insights about the AoI stability in heavy loading conditions. The average AoI and its dispersion are numerically investigated for the case of a three-class network. The significance of the proposed model is manifested in achieving a compromise satisfaction between all priority classes by a thorough adjustment of its threshold parameters. Two approaches are proposed to clarify the adjustment of these parameters. It turned out that the proposed hybrid discipline compensates for the limited buffer resource, achieving more promising performance with low design complexity and low cost. Moreover, the proposed scheme can operate under a wider span of the total offered load, through which the whole network satisfaction can be optimized under some legitimate constraints on the age-sensitive classes.

Model transformation languages are domain-specific languages used to define transformations of models. These transformations consist of the translation from one modeling formalism into another or just the updating of a given model. Such transformations are often described declaratively and are often implemented based on very small models that cover the language of the input model. As a result, transformation developers are often unable to assess the time required to transform a larger model.

Hence, we propose a prediction approach based on machine learning which uses a set of model characteristics as input and provides a prediction of the execution time of a transformation defined in the Atlas Transformation Language (ATL). In our previous work (Groner et al., 2023), we already showed that support vector regression in combination with a model characterization based on the number of model elements, the number of references, and the number of attributes is the best choice in terms of usability and prediction accuracy for the transformations considered in our experiments.

A major weakness of our previous approach is that it fails to predict the performance of transformations that also transform attribute values of arbitrary length, such as string values. Therefore, we investigate in this work whether an extension of our feature sets that describes the average size of string attributes can help to overcome this weakness.

Our results show that the random forest approach in combination with model characterizations based on the number of model elements, the number of references, the number of attributes, and the average size of string attributes filtered by the 85th percentile of their variance is the best choice in terms of the simple way to describe a model and the quality of the obtained prediction. With this combination, we obtained a mean absolute percentage error (MAPE) of 5.07% over all modules and a MAPE of 4.82% over all modules excluding the transformation for which our previous approach failed. Whereas, we obtained previously a MAPE of 38.48% over all modules and a MAPE of 4.45% over all modules excluding the transformation for which our previous approach failed.

Age-of-information is a metric that quantifies the freshness of information obtained by sampling a remote sensor. In signal-agnostic sampling, sensor updates are triggered at certain times without being conditioned on the actual sensor signal. Optimal update policies have been researched and it is accepted that periodic updates achieve smaller age-of-information than random updates. We contribute a study of a signal-aware policy, where updates are triggered randomly by a defined sensor event. By definition, this implies random updates and as a consequence inferior age-of-information. Considering a notion of deviation-of-information as a signal-aware metric, our results show, however, that event-triggered systems can perform equally well as time-triggered systems while causing smaller mean network utilization. We use the stochastic network calculus to derive bounds of age- and deviation-of-information that are exceeded at most with a small, defined probability. We include simulation results that confirm the tail decay of the bounds. We also evaluate a hybrid time- and event-triggered policy where the event-triggered system is complemented by a minimal and a maximal update interval.

Due to scalability requirements and the split of large software development projects into small agile teams, there is a current trend toward the migration of monolith systems to the microservice architecture. However, the split of the monolith into microservices, its encapsulation through well-defined interfaces, and the introduction of inter-microservice communication add a cost in terms of performance. In this paper, we describe a case study of the migration of a monolith to a microservice architecture, where a modular monolith architecture is used as an intermediate step. The impact on migration effort and performance is measured for both steps. Current state-of-the-art analyses the migration of monolith systems to a microservice architecture, but we observed that migration effort and performance issues are already significant in the migration to a modular monolith. Therefore, a clear distinction is established for each of the steps, which may inform software architects on the planning of the migration of monolith systems. In particular, we consider the trade-offs of doing all the migration process or just migrating to a modular monolith.

We consider a system with $n$ unit-rate servers where jobs arrive according a Poisson process with rate $n\lambda$ ($\lambda <1$). In the standard Power-of-$d$ or Pod scheme with $d\ge 2$, for each incoming job, a dispatcher samples $d$ servers uniformly at random and sends the incoming job to the least loaded of the $d$ sampled servers. However, in practice, load comparisons may not always be accurate. In this paper, we analyse the effects of noisy load comparisons on the performance of the Pod scheme. To test the robustness of the Pod scheme against load comparison errors, we assume an adversarial setting where, in the event of an error, the adversary assigns the incoming job to the worst possible server, i.e., the server with the maximum load among the $d$ sampled servers. We consider two error models: load-dependent and load-independent errors. In the load-dependent error model, the adversary has limited power in that it is able to cause an error with probability $ϵ\in [0,1]$ only when the difference in the minimum and the maximum queue lengths of the $d$ sampled servers is bounded by a constant threshold $g\ge 0$. For this type of errors, we show that, in the large system limit, the benefits of the Pod scheme are retained even if $g$ and $ϵ$ are arbitrarily large as long as the system is heavily loaded, i.e., $\lambda$ is close to 1. In the load-independent error model, the adversary is assumed to be more powerful in that it can cause an error with probability $ϵ$ independent of the loads of the sampled servers. For this model, we show that the performance benefits of the Pod scheme are retained only if $ϵ\le 1/d$; for $ϵ>1/d$ we show that the stability region of the system reduces and the system performs poorly in comparison to the random scheme. Our mean-field analysis uses a new approach to characterise fixed points which neither have closed form solutions nor admit any recursion. Furthermore, we develop a generic approach to prove tightness and stability for any state-dependent load balancing scheme.

As the complexity of processor microarchitecture and applications increases, obtaining performance optimization knowledge, such as critical dependent chains, becomes more challenging. To tackle this issue, this paper employs pattern mining methods to analyze the critical path of processor micro-execution dependence graphs. We propose a high average utility pattern mining algorithm called Dependence Graph Miner (DG-Miner) based on the characteristics of dependence graphs. DG-Miner overcomes the limitations of current pattern mining algorithms for dependence graph pattern mining by offering support for variable utility, candidate generation using endpoint matching, the adjustable upper bound, and the concise pattern judgment mechanism. Experiments reveal that, compared with existing upper bound candidate generation methods, the adjustable upper bound reduces the number of candidate patterns by 28.14% and the running time by 27% on average. The concise pattern judgment mechanism enhances the conciseness of mining results by 16.31% and reduces the running time by 39.82%. Furthermore, DG-Miner aids in identifying critical dependent chains, critical program regions, and performance exceptions.