Performance Evaluation最新文献_第2页

Strategic pricing and ranking in recommendation systems with seller competition 考虑卖家竞争的推荐系统策略定价与排名

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-13 DOI: 10.1016/j.peva.2025.102518

Tushar Shankar Walunj , Veeraruna Kavitha , Jayakrishnan Nair , Priyank Agarwal

We study a recommendation system where sellers compete for visibility by strategically offering commissions to a platform that optimally curates a ranked menu of items and their respective prices for each customer. Customers interact sequentially with the menu following a cascade click model, and their purchase decisions are influenced by price sensitivity and positions of various items in the menu. We model the seller-platform interaction as a Stackelberg game with sellers as leaders and consider two different games depending on whether the prices are set by the platform or prefixed by the sellers.

It is complicated to find the optimal policy of the platform in complete generality; hence, we solve the problem in an important asymptotic regime. In fact, both the games coincide in this regime, obtained by decreasing the customer exploration rates

γ

to zero (in this regime, the customers explore fewer items). Through simulations, we illustrate that the limit game well approximates the original game(s) even for exploration probabilities as high as 0.4 (the differences are around 2.54%). Further, the second game (where the sellers prefix the prices) coincides with the approximate game for all values of

γ

.

The core contribution of this paper lies in characterizing the equilibrium structure of the limit game. We show that when sellers are of different strengths, the standard Nash equilibrium does not exist due to discontinuities in utilities. We instead establish the existence of a novel equilibrium solution, namely ‘

μ

-connected equilibrium cycle’ (

μ

-EC), which captures oscillatory strategic responses at the equilibrium. Unlike the (pure) Nash equilibrium, which defines a fixed point of mutual best responses, this is a set-valued solution concept of connected components. This novel equilibrium concept identifies a Cartesian product set of connected action profiles in the continuous action space that satisfies four important properties: stability against external deviations, no external chains, instability against internal deviations, and minimality. We extend a recently introduced solution concept equilibrium cycle to include stability against measure-zero violations and avoid some topological difficulties to propose

μ

-EC.

我们研究了一个推荐系统，在这个系统中，卖家通过有策略地向一个平台提供佣金来竞争知名度，该平台为每个客户最佳地策划了一个商品排名菜单及其各自的价格。客户按照级联点击模型顺序与菜单交互，他们的购买决策受到价格敏感性和菜单中各种项目位置的影响。我们将卖家与平台的互动建模为Stackelberg游戏，其中卖家是领导者，并根据价格是由平台设定还是由卖家设定来考虑两种不同的游戏。在完全一般情况下寻找平台的最优策略比较复杂；因此，我们在一个重要的渐近区域内解决了这个问题。事实上，这两款游戏都符合这一机制，即通过将用户探索率γ降低至零而获得（在此机制中，用户探索的道具更少）。通过模拟，我们发现即使勘探概率高达0.4（差异约为2.54%），极限博弈也很接近原始博弈。此外，第二个博弈（卖家在价格前加上前缀）与所有γ值的近似博弈一致。本文的核心贡献在于刻画了极限对策的均衡结构。结果表明，当卖者具有不同的优势时，由于效用的不连续，标准纳什均衡不存在。相反，我们建立了一个新的平衡解的存在性，即“μ-连接的平衡循环”（μ-EC），它捕获了平衡处的振荡策略响应。不像（纯粹的）纳什均衡，它定义了一个相互最佳响应的固定点，这是一个连接组件的集值解决概念。这种新的平衡概念确定了连续作用空间中相互连接的作用轮廓的笛卡尔积集，它满足四个重要性质：抗外部偏差的稳定性、无外部链、抗内部偏差的不稳定性和极小性。我们扩展了最近引入的解决方案概念平衡循环，以包括对测度零违反的稳定性，并避免了一些拓扑困难，提出了μ-EC。

{"title":"Strategic pricing and ranking in recommendation systems with seller competition","authors":"Tushar Shankar Walunj , Veeraruna Kavitha , Jayakrishnan Nair , Priyank Agarwal","doi":"10.1016/j.peva.2025.102518","DOIUrl":"10.1016/j.peva.2025.102518","url":null,"abstract":"<div><div>We study a recommendation system where sellers compete for visibility by strategically offering commissions to a platform that optimally curates a ranked menu of items and their respective prices for each customer. Customers interact sequentially with the menu following a cascade click model, and their purchase decisions are influenced by price sensitivity and positions of various items in the menu. We model the seller-platform interaction as a Stackelberg game with sellers as leaders and consider two different games depending on whether the prices are set by the platform or prefixed by the sellers.</div><div>It is complicated to find the optimal policy of the platform in complete generality; hence, we solve the problem in an important asymptotic regime. In fact, both the games coincide in this regime, obtained by decreasing the customer exploration rates <span><math><mi>γ</mi></math></span> to zero (in this regime, the customers explore fewer items). Through simulations, we illustrate that the limit game well approximates the original game(s) even for exploration probabilities as high as 0.4 (the differences are around 2.54%). Further, the second game (where the sellers prefix the prices) coincides with the approximate game for all values of <span><math><mi>γ</mi></math></span>.</div><div>The core contribution of this paper lies in characterizing the equilibrium structure of the limit game. We show that when sellers are of different strengths, the standard Nash equilibrium does not exist due to discontinuities in utilities. We instead establish the existence of a novel equilibrium solution, namely ‘<span><math><mi>μ</mi></math></span>-connected equilibrium cycle’ (<span><math><mi>μ</mi></math></span>-EC), which captures oscillatory strategic responses at the equilibrium. Unlike the (pure) Nash equilibrium, which defines a fixed point of mutual best responses, this is a set-valued solution concept of connected components. This novel equilibrium concept identifies a Cartesian product set of connected action profiles in the continuous action space that satisfies four important properties: stability against external deviations, no external chains, instability against internal deviations, and minimality. We extend a recently introduced solution concept <em>equilibrium cycle</em> to include stability against measure-zero violations and avoid some topological difficulties to propose <span><math><mi>μ</mi></math></span>-EC.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102518"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145516997","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Bayesian optimization for dynamic pricing and learning 动态定价与学习的贝叶斯优化

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-12 DOI: 10.1016/j.peva.2025.102519

Anush Anand, Pranav Agrawal, Tejas Bodas

Dynamic pricing is the practice of adjusting the selling price of a product to maximize a firm’s revenue by responding to market demand. The literature typically distinguishes between two settings: infinite inventory, where the firm has unlimited stock and time to sell, and finite inventory, where both inventory and selling horizon are limited. In both cases, the central challenge lies in the fact that the demand function — how sales respond to price — is unknown and must be learned from data. Traditional approaches often assume a specific parametric form for the demand function, enabling the use of reinforcement learning (RL) to identify near-optimal pricing strategies. However, such assumptions may not hold in real-world scenarios, limiting the applicability of these methods.

In this work, we propose a Gaussian Process (GP) based nonparametric approach to dynamic pricing that avoids restrictive modeling assumptions. We treat the demand function as a black-box function of the price and develop pricing algorithms based on Bayesian Optimization (BO)—a sample-efficient method for optimizing unknown functions. We present BO-based algorithms tailored for both infinite and finite inventory settings and provide regret guarantees for both regimes, thereby quantifying the learning efficiency of our methods. Through extensive experiments, we demonstrate that our BO-based methods outperform several state-of-the-art RL algorithms in terms of revenue, while requiring fewer assumptions and offering greater robustness. This highlights Bayesian Optimization as a powerful and practical tool for dynamic pricing in complex, uncertain environments.

动态定价是指根据市场需求调整产品销售价格，使企业收益最大化的做法。文献通常区分两种情况：无限库存，公司有无限的库存和时间来销售；有限库存，库存和销售范围都是有限的。在这两种情况下，核心挑战都在于这样一个事实：需求函数——销售对价格的反应——是未知的，必须从数据中学习。传统方法通常假设需求函数具有特定的参数形式，从而能够使用强化学习（RL）来识别近乎最优的定价策略。然而，这些假设在实际场景中可能不成立，从而限制了这些方法的适用性。在这项工作中，我们提出了一种基于高斯过程（GP）的非参数动态定价方法，避免了限制性建模假设。我们将需求函数视为价格的黑盒函数，并开发了基于贝叶斯优化（BO）的定价算法-一种优化未知函数的样本效率方法。我们提出了针对无限和有限库存设置的基于bo的算法，并为这两种制度提供了后悔保证，从而量化了我们方法的学习效率。通过广泛的实验，我们证明基于bo的方法在收益方面优于几种最先进的强化学习算法，同时需要更少的假设并提供更强的鲁棒性。这突出了贝叶斯优化作为一个强大而实用的工具，在复杂的，不确定的环境中动态定价。

{"title":"Bayesian optimization for dynamic pricing and learning","authors":"Anush Anand, Pranav Agrawal, Tejas Bodas","doi":"10.1016/j.peva.2025.102519","DOIUrl":"10.1016/j.peva.2025.102519","url":null,"abstract":"<div><div>Dynamic pricing is the practice of adjusting the selling price of a product to maximize a firm’s revenue by responding to market demand. The literature typically distinguishes between two settings: infinite inventory, where the firm has unlimited stock and time to sell, and finite inventory, where both inventory and selling horizon are limited. In both cases, the central challenge lies in the fact that the demand function — how sales respond to price — is unknown and must be learned from data. Traditional approaches often assume a specific parametric form for the demand function, enabling the use of reinforcement learning (RL) to identify near-optimal pricing strategies. However, such assumptions may not hold in real-world scenarios, limiting the applicability of these methods.</div><div>In this work, we propose a Gaussian Process (GP) based nonparametric approach to dynamic pricing that avoids restrictive modeling assumptions. We treat the demand function as a black-box function of the price and develop pricing algorithms based on Bayesian Optimization (BO)—a sample-efficient method for optimizing unknown functions. We present BO-based algorithms tailored for both infinite and finite inventory settings and provide regret guarantees for both regimes, thereby quantifying the learning efficiency of our methods. Through extensive experiments, we demonstrate that our BO-based methods outperform several state-of-the-art RL algorithms in terms of revenue, while requiring fewer assumptions and offering greater robustness. This highlights Bayesian Optimization as a powerful and practical tool for dynamic pricing in complex, uncertain environments.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102519"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145568452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Comparing approximations in the ASIP tandem queue 比较ASIP串联队列中的近似

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-04 DOI: 10.1016/j.peva.2025.102523

Wesley Geelen , Maria Vlasiou , Yaron Yeger

The Asymmetric Inclusion Process (ASIP) models unidirectional transport with particle clustering, yet remains analytically intractable for systems beyond small sizes. To address this, we develop two approximation methods: the replica mean-field (RMF) limit, providing a first-order approximation, and the power series algorithm (PSA), a numerical scheme based on traffic intensity expansions. We evaluate these approximations against Monte Carlo simulations for general systems and prior exact results for homogeneous ASIP systems. Both methods yield accurate estimates, with PSA closely matching simulations for both homogeneous and heterogeneous systems, while RMF performing well for early sites but being slightly impacted downstream or as load increases. These approximations offer practical and computationally efficient alternatives to simulation, enabling detailed performance analysis of ASIP tandem queues where exact solutions are unavailable.

不对称包合过程（ASIP）模拟颗粒聚类的单向输运，但对于小尺寸以上的系统仍然难以分析。为了解决这个问题，我们开发了两种近似方法：复制平均场（RMF）极限，提供一阶近似，以及幂级数算法（PSA），一种基于交通强度展开的数值方案。我们对一般系统的蒙特卡罗模拟和齐次ASIP系统的先前精确结果评估了这些近似。这两种方法都产生了准确的估计，PSA与均匀和非均匀系统的模拟密切匹配，而RMF在早期站点表现良好，但在下游或负载增加时受到轻微影响。这些近似提供了实用且计算效率高的模拟替代方案，可以在无法获得精确解决方案的情况下对ASIP串联队列进行详细的性能分析。

引用次数: 0

User equilibria in heterogeneous discriminatory processor sharing queues 异构歧视性处理器共享队列中的用户均衡

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-09-24 DOI: 10.1016/j.peva.2025.102510

Dieter Fiems , Balakrishna J. Prabhu

We consider a strategic routing game for a two-class discriminatory processor-sharing queue with an additional cost for joining the premium class. We show that, depending on the specific parameters of the system, various equilibria can coexist, including equilibria where the queueing system is not ergodic for the equilibrium traffic split. We also investigate how the server can select the priority of the classes and the fees charged to the customers to maximise its revenue. We then investigate learning strategies that converge to particular equilibria. Finally, we study how the elasticity of the traffic demand affects the equilibrium solutions.

我们考虑了一个两类歧视性处理器共享队列的策略路由博弈，该队列具有加入优质类的额外成本。我们证明，根据系统的特定参数，各种均衡可以共存，包括排队系统对于均衡流量分割不遍历的均衡。我们还研究了服务器如何选择课程的优先级和向客户收取的费用，以使其收入最大化。然后我们研究收敛于特定均衡的学习策略。最后，研究了交通需求弹性对均衡解的影响。

引用次数: 0

The last, the least, and the urgent: Fluid modeling and performance equivalence for scheduling policies in partial service queues with abandonment 最后，最少，也是最紧迫的：放弃部分服务队列中调度策略的流体建模和性能等效

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-08 DOI: 10.1016/j.peva.2025.102517

Andres Ferragut, Diego Goldsztajn, Fernando Paganini

In several queueing systems, arriving tasks or customers have both service and timing requirements, the latter expressed as a deadline for the task to be served. These systems with customer abandonment have a long and rich history in queueing theory, and have several applications in task scheduling in computer systems, operations research problems, etc. A common feature in all of these works is that they deal with customers reneging from the system only while in the queue, and not during service. However, in several applications, customers may also leave during service, and the partial work performed by the system during their stay is still useful.

In this paper we analyze these partial service queues with abandonment in a many-server setting, characterizing the equilibrium performance of several policies in terms of the amount of service attained by tasks. For this purpose, we develop fluid models with two-dimensional independent variables, corresponding to service and sojourn times, which take the form of partial differential equations expressed in weak form. These fluid models allow us to consider general and possibly correlated service and timing requirements, as well as a wide range of service disciplines. In particular, we focus on Earliest-Deadline-First, Least-Attained-Service and Last-Come-First-Served, and establish that all three policies have the same equilibrium performance, even though the latter two do not need any information about deadlines. This striking property means that designers may avoid the difficult job of estimating deadlines without incurring a performance penalty. The fluid model conclusions are validated by extensive numerical experiments.

在一些排队系统中，到达的任务或客户同时具有服务和时间要求，后者表示为要服务的任务的截止日期。这些客户放弃系统在排队理论中有着悠久而丰富的历史，在计算机系统的任务调度、运筹学问题等方面有着广泛的应用。所有这些工作的一个共同特点是，它们只在排队时处理违背系统的客户，而不是在服务期间。然而，在一些应用程序中，客户也可能在服务期间离开，系统在他们逗留期间执行的部分工作仍然是有用的。在本文中，我们在多服务器设置中分析了这些带有放弃的部分服务队列，并根据任务获得的服务量描述了几种策略的均衡性能。为此，我们开发了具有二维自变量的流体模型，对应于服务和逗留时间，采用弱形式表示的偏微分方程的形式。这些流体模型使我们能够考虑一般的和可能相关的服务和时间要求，以及广泛的服务学科。特别是，我们关注最早截止日期优先、最少获得服务和最后先到先得，并确定所有三种策略具有相同的均衡性能，即使后两种策略不需要任何关于截止日期的信息。这个惊人的特性意味着设计师可以避免估算截止日期的困难工作，而不会导致性能损失。大量的数值实验验证了流体模型的结论。

{"title":"The last, the least, and the urgent: Fluid modeling and performance equivalence for scheduling policies in partial service queues with abandonment","authors":"Andres Ferragut, Diego Goldsztajn, Fernando Paganini","doi":"10.1016/j.peva.2025.102517","DOIUrl":"10.1016/j.peva.2025.102517","url":null,"abstract":"<div><div>In several queueing systems, arriving tasks or customers have both service and timing requirements, the latter expressed as a deadline for the task to be served. These systems with customer abandonment have a long and rich history in queueing theory, and have several applications in task scheduling in computer systems, operations research problems, etc. A common feature in all of these works is that they deal with customers reneging from the system only while in the queue, and not during service. However, in several applications, customers may also leave during service, and the partial work performed by the system during their stay is still useful.</div><div>In this paper we analyze these partial service queues with abandonment in a many-server setting, characterizing the equilibrium performance of several policies in terms of the amount of service attained by tasks. For this purpose, we develop fluid models with two-dimensional independent variables, corresponding to service and sojourn times, which take the form of partial differential equations expressed in weak form. These fluid models allow us to consider general and possibly correlated service and timing requirements, as well as a wide range of service disciplines. In particular, we focus on Earliest-Deadline-First, Least-Attained-Service and Last-Come-First-Served, and establish that all three policies have the same equilibrium performance, even though the latter two do not need any information about deadlines. This striking property means that designers may avoid the difficult job of estimating deadlines without incurring a performance penalty. The fluid model conclusions are validated by extensive numerical experiments.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102517"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145568453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Optimizing resource allocation for geographically-distributed inference by large language models 基于大型语言模型的地理分布推理资源优化分配

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-08 DOI: 10.1016/j.peva.2025.102527

Tingyang Sun , Ting He , Bo Ji , Parimal Parag

Large language models (LLMs) have demonstrated extraordinary performance in many artificial intelligence (AI) tasks but are expensive to use, even after training, due to their requirement of high-end GPUs. Recently, a distributed system called PETALS was developed to lower the barrier for deploying LLMs by splitting the model blocks across multiple servers with low-end GPUs distributed over the Internet, which was much faster than swapping the model parameters between the GPU memory and other cheaper but slower local storage media. However, the performance of such a distributed system critically depends on the resource allocation, and how to do so optimally remains unknown. In this work, we present the first systematic study of the resource allocation problem in distributed LLM inference, with focus on two important decisions: block placement and request routing. Our main results include: (i) experimentally validated performance models that can predict the inference performance under given block placement and request routing decisions, (ii) a formulation of the offline optimization of block placement and request routing as a mixed integer linear programming (MILP) problem together with the NP-hardness proof and a polynomial-complexity algorithm with guaranteed performance, and (iii) an adaptation of the offline algorithm for the online setting with the same performance guarantee under bounded load. Through both experiments and experimentally-validated simulations, we have verified that the proposed solution can substantially reduce the inference time compared to the state-of-the-art solution in diverse settings with geographically-distributed servers. As a byproduct, we have also developed a light-weighted CPU-only simulator capable of predicting the performance of distributed LLM inference on GPU servers, which can evaluate large deployments and facilitate future research for researchers with limited GPU access.

大型语言模型（llm）在许多人工智能（AI）任务中表现出非凡的性能，但由于对高端gpu的要求，即使经过训练，使用起来也很昂贵。最近，一种名为PETALS的分布式系统被开发出来，通过在互联网上分布的低端GPU的多个服务器上分割模型块来降低部署llm的障碍，这比在GPU内存和其他更便宜但速度较慢的本地存储介质之间交换模型参数要快得多。然而，这种分布式系统的性能严重依赖于资源分配，如何实现最佳分配仍然是未知的。在这项工作中，我们首次系统地研究了分布式LLM推理中的资源分配问题，重点关注两个重要的决策：块放置和请求路由。我们的主要结果包括：(i)实验验证的性能模型，可以预测给定块放置和请求路由决策下的推理性能；（ii）将块放置和请求路由的离线优化表述为混合整数线性规划（MILP）问题，以及具有保证性能的np硬度证明和多项式复杂度算法；（iii）有界负载下具有相同性能保证的离线算法对在线设置的自适应。通过实验和实验验证的模拟，我们已经验证了与地理分布服务器的不同设置下的最先进解决方案相比，所提出的解决方案可以大大减少推理时间。作为副产品，我们还开发了一个轻量级的仅cpu模拟器，能够预测GPU服务器上分布式LLM推理的性能，它可以评估大型部署，并为GPU访问受限的研究人员提供便利。

{"title":"Optimizing resource allocation for geographically-distributed inference by large language models","authors":"Tingyang Sun , Ting He , Bo Ji , Parimal Parag","doi":"10.1016/j.peva.2025.102527","DOIUrl":"10.1016/j.peva.2025.102527","url":null,"abstract":"<div><div>Large language models (LLMs) have demonstrated extraordinary performance in many artificial intelligence (AI) tasks but are expensive to use, even after training, due to their requirement of high-end GPUs. Recently, a distributed system called PETALS was developed to lower the barrier for deploying LLMs by splitting the model blocks across multiple servers with low-end GPUs distributed over the Internet, which was much faster than swapping the model parameters between the GPU memory and other cheaper but slower local storage media. However, the performance of such a distributed system critically depends on the resource allocation, and how to do so optimally remains unknown. In this work, we present the first systematic study of the resource allocation problem in distributed LLM inference, with focus on two important decisions: block placement and request routing. Our main results include: (i) experimentally validated performance models that can predict the inference performance under given block placement and request routing decisions, (ii) a formulation of the offline optimization of block placement and request routing as a mixed integer linear programming (MILP) problem together with the NP-hardness proof and a polynomial-complexity algorithm with guaranteed performance, and (iii) an adaptation of the offline algorithm for the online setting with the same performance guarantee under bounded load. Through both experiments and experimentally-validated simulations, we have verified that the proposed solution can substantially reduce the inference time compared to the state-of-the-art solution in diverse settings with geographically-distributed servers. As a byproduct, we have also developed a light-weighted CPU-only simulator capable of predicting the performance of distributed LLM inference on GPU servers, which can evaluate large deployments and facilitate future research for researchers with limited GPU access.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102527"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145517000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Designing asymptotically optimal policies for continuous-time weakly coupled MDPs 连续时间弱耦合mdp的渐近最优策略设计

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-05 DOI: 10.1016/j.peva.2025.102528

Matthieu Perbal, Balakrishna Prabhu, Ina Maria Verloop

We study the continuous-time Weakly Coupled Markov Decision Process (WCMDP), a class of decision problems involving multiple interacting Markov processes (or “arms”) subject to shared resource constraints. We present a general framework for policy design using a combination of an underlying Markov process and a sequence of mappings. Our main theoretical result establishes sufficient conditions on the Markov process and mapping defining the policy, such that it is asymptotically optimal as the number of arms grows.

We construct both deterministic and randomized policies based on a solution to a linear program (LP). These policies initially assign actions to arms — either proportionally (deterministic) or randomly — based on conditional measures derived from the LP. As this initial allocation may violate feasibility constraints, we introduce a mapping to enforce the resource constraints are satisfied. Finally, we numerically evaluate and compare the performance of our proposed policies, both deterministic and randomized, under different choices of mappings.

我们研究了连续时间弱耦合马尔可夫决策过程（WCMDP），这是一类涉及多个受共享资源约束的相互作用的马尔可夫过程（或“手臂”）的决策问题。我们提出了一个使用底层马尔可夫过程和映射序列相结合的策略设计的一般框架。我们的主要理论结果建立了马尔可夫过程和映射定义策略的充分条件，使得该策略随着武器数量的增长是渐近最优的。我们基于线性规划（LP）的一个解构造了确定性和随机策略。这些政策最初根据LP衍生的条件措施，按比例（确定性）或随机地将行动分配给武器。由于这种初始分配可能违反可行性约束，我们引入映射来强制满足资源约束。最后，在不同的映射选择下，我们数值评估和比较了我们所提出的策略的性能，包括确定性策略和随机策略。

引用次数: 0

ASIP tandem queues with Lévy input and consumption 带有lims输入和消耗的ASIP串联队列

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-10-27 DOI: 10.1016/j.peva.2025.102513

Onno Boxma , Offer Kella , Jacques Resing

We consider an ASIP (asymmetric inclusion process) tandem queue, in which the first queue receives a fluid input according to a nondecreasing Lévy process. Each queue has a gate that opens after independent, exponentially distributed periods for an infinitesimal amount of time, allowing the queue content to move to the next queue. In addition, again at independent exponentially distributed instants, a fixed fraction of a queue content is removed from the system.

For this model, restricting ourselves to steady state, we obtain the following results. (i) We derive the buffer content distribution of the first queue. (ii) For the 2-queue model, we obtain relatively simple explicit expressions for the Laplace transform of the joint buffer content in several special cases. (iii) Asymptotic results are obtained for the 2-queue model when the above-mentioned buffer content removal process approaches a shot-noise process. (iv) For the general

n

-queue case, we show how all moments of the buffer contents at all queues can be obtained. (v) For the general

n

-queue case, we sketch an approximation method that allows one in principle to derive tractable expressions for the Laplace transform of the buffer content at each queue, with exact mean buffer contents at all queues.

我们考虑一个ASIP（非对称包含过程）串联队列，其中第一个队列根据非递减的lsamvy过程接收流体输入。每个队列都有一个门，在独立的、指数分布的时间段内打开一个无限小的时间，允许队列内容移动到下一个队列。此外，同样在独立的指数分布时刻，从系统中删除队列内容的固定部分。对于这个模型，我们将自己限制在稳态，得到如下结果。(i)导出了第一个队列的缓冲区内容分布。（ii）对于2队列模型，我们得到了几种特殊情况下联合缓冲区内容的拉普拉斯变换的比较简单的显式表达式。（iii）对于2队列模型，当上述缓冲区内容移除过程接近于一个短噪声过程时，得到了渐近结果。（iv）对于一般的n队列情况，我们展示了如何获得所有队列上缓冲区内容的所有矩。(v)对于一般的n队列情况，我们概述了一种近似方法，该方法原则上允许人们推导出每个队列上缓冲区内容的拉普拉斯变换的易于处理的表达式，并具有所有队列上的精确平均缓冲区内容。

{"title":"ASIP tandem queues with Lévy input and consumption","authors":"Onno Boxma , Offer Kella , Jacques Resing","doi":"10.1016/j.peva.2025.102513","DOIUrl":"10.1016/j.peva.2025.102513","url":null,"abstract":"<div><div>We consider an ASIP (asymmetric inclusion process) tandem queue, in which the first queue receives a fluid input according to a nondecreasing Lévy process. Each queue has a gate that opens after independent, exponentially distributed periods for an infinitesimal amount of time, allowing the queue content to move to the next queue. In addition, again at independent exponentially distributed instants, a fixed fraction of a queue content is removed from the system.</div><div>For this model, restricting ourselves to steady state, we obtain the following results. (i) We derive the buffer content distribution of the first queue. (ii) For the 2-queue model, we obtain relatively simple explicit expressions for the Laplace transform of the joint buffer content in several special cases. (iii) Asymptotic results are obtained for the 2-queue model when the above-mentioned buffer content removal process approaches a shot-noise process. (iv) For the general <span><math><mi>n</mi></math></span>-queue case, we show how all moments of the buffer contents at all queues can be obtained. (v) For the general <span><math><mi>n</mi></math></span>-queue case, we sketch an approximation method that allows one in principle to derive tractable expressions for the Laplace transform of the buffer content at each queue, with exact mean buffer contents at all queues.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102513"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145417138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Switching constrained OCO with predictions and feedback delays 具有预测和反馈延迟的切换约束OCO

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-05 DOI: 10.1016/j.peva.2025.102524

Weici Pan, Zhenhua Liu

We examine Online Convex Optimization (OCO) problems with feedback delay and a strict limit on decision switching, which exists in applications such as smart grid and learning. Existing algorithms developed for traditional OCO struggle in this setting, often violating switching constraints or incurring high regrets, as evidenced by simulations. In this paper, we establish a new algorithm, Follow-the-Maximally-Coupled-Latest-Leader (FMCLL), achieving a near-optimal regret of

O (T / S)

for such problems with delayed feedbacks and a bound of

O (T / S - τ)

for problems with predictions of

τ

rounds, even though the player is only allowed to move at most

S

times in expectation across

T

rounds. FMCLL meets performance bounds in scenarios with delays and predictions by using maximal coupling sampling to inform algorithm design for switching-constrained problems. To better apply our framework to practical applications, we also extend the algorithm and results to the bandit feedback setting. Simulations demonstrate FMCLL’s superiority over traditional Gradient Descent or Follow-the-Leader algorithms, excelling under adversarial or stochastic losses and reducing constraint violations.

我们研究了在智能电网和学习等应用中存在的带有反馈延迟和严格限制决策切换的在线凸优化（OCO）问题。仿真结果表明，针对传统OCO开发的现有算法往往会违反切换约束或导致高遗憾。在本文中，我们建立了一个新的算法，跟随最大耦合最新领导者（FMCLL），对于具有延迟反馈的问题实现了O（T/S）的近最优后悔，对于具有τ轮预测的问题实现了O（T/S−τ）的界，即使玩家在T轮中最多只允许在期望中移动S次。FMCLL通过使用最大耦合采样来为切换约束问题的算法设计提供信息，从而满足有延迟和预测场景下的性能界限。为了更好地将我们的框架应用于实际应用，我们还将算法和结果扩展到强盗反馈设置中。仿真结果表明，FMCLL优于传统的梯度下降算法或跟随者算法，在对抗或随机损失和减少约束违反方面表现出色。

{"title":"Switching constrained OCO with predictions and feedback delays","authors":"Weici Pan, Zhenhua Liu","doi":"10.1016/j.peva.2025.102524","DOIUrl":"10.1016/j.peva.2025.102524","url":null,"abstract":"<div><div>We examine Online Convex Optimization (OCO) problems with feedback delay and a strict limit on decision switching, which exists in applications such as smart grid and learning. Existing algorithms developed for traditional OCO struggle in this setting, often violating switching constraints or incurring high regrets, as evidenced by simulations. In this paper, we establish a new algorithm, Follow-the-Maximally-Coupled-Latest-Leader (FMCLL), achieving a near-optimal regret of <span><math><mrow><mi>O</mi><mrow><mo>(</mo><mi>T</mi><mo>/</mo><mi>S</mi><mo>)</mo></mrow></mrow></math></span> for such problems with delayed feedbacks and a bound of <span><math><mrow><mi>O</mi><mrow><mo>(</mo><mi>T</mi><mo>/</mo><mi>S</mi><mo>−</mo><mi>τ</mi><mo>)</mo></mrow></mrow></math></span> for problems with predictions of <span><math><mi>τ</mi></math></span> rounds, even though the player is only allowed to move at most <span><math><mi>S</mi></math></span> times in expectation across <span><math><mi>T</mi></math></span> rounds. FMCLL meets performance bounds in scenarios with delays and predictions by using maximal coupling sampling to inform algorithm design for switching-constrained problems. To better apply our framework to practical applications, we also extend the algorithm and results to the bandit feedback setting. Simulations demonstrate FMCLL’s superiority over traditional Gradient Descent or Follow-the-Leader algorithms, excelling under adversarial or stochastic losses and reducing constraint violations.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102524"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145517001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

LLMEmu: A lightweight performance emulator for high-fidelity distributed LLM training LLMEmu：用于高保真分布式LLM训练的轻量级性能模拟器

IF 0.8 4区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

Performance Evaluation

Pub Date : 2025-11-01 Epub Date: 2025-11-12 DOI: 10.1016/j.peva.2025.102526

Siyuan Yang , Enda Yu , Pingjing Lu , Dezun Dong

The prohibitive cost of training trillion-parameter large language models (LLMs) necessitates low-cost emulation tools for distributed system optimization. In modern large-scale clusters, communication often becomes the primary bottleneck to scalability. However, existing emulators, such as vTrain and ASTRA-Sim, overlook dynamic network factors that significantly impact performance at scale, resulting in limited emulation accuracy. This work offers an efficient and reliable tool for training system optimization and parallel strategy exploration, considerably lowering the barrier to large-scale AI research. We present LLMEmu, a distributed training emulator that combines real kernel profiling and actual communication execution. First, computation is profiled through real CUDA kernel traces on GPU nodes to construct an operator-level latency lookup table, enabling GPU-like execution on CPU clusters. Second, inter-node communication is executed using communication library primitives (e.g., AllReduce, Send/Recv), triggered by communication anchors embedded in the execution graph, and implemented using a pluggable communication backend. LLMEmu can seamlessly model hybrid parallelism strategies and supports multiple collective algorithms. Its lightweight design incorporates gradient bucketing with latency reuse to minimize overhead while maintaining extensibility to various network interconnects. The effectiveness of LLMEmu is validated through its performance results, demonstrating an average prediction error of only 2.17% on 24-GPU clusters, which outperforms vTrain by 21.09%, and confirming its scalability in modeling training cost distributions across 128-node CPU emulations under varying network conditions.

训练具有万亿参数的大型语言模型（llm）的高昂成本需要用于分布式系统优化的低成本仿真工具。在现代大规模集群中，通信往往成为可伸缩性的主要瓶颈。然而，现有的仿真器，如vTrain和ASTRA-Sim，忽略了在规模上显著影响性能的动态网络因素，导致仿真精度有限。这项工作为训练系统优化和并行策略探索提供了高效可靠的工具，大大降低了大规模人工智能研究的门槛。我们提出了LLMEmu，一个分布式训练仿真器，结合了真实的内核分析和实际的通信执行。首先，通过GPU节点上的真实CUDA内核跟踪对计算进行分析，以构建一个操作级延迟查找表，从而在CPU集群上实现类似GPU的执行。其次，节点间通信使用通信库原语（例如，AllReduce, Send/Recv）执行，由嵌入在执行图中的通信锚触发，并使用可插拔的通信后端实现。LLMEmu可以无缝地建模混合并行策略，并支持多种集体算法。它的轻量级设计结合了梯度桶和延迟重用，以尽量减少开销，同时保持对各种网络互连的可扩展性。通过性能结果验证了LLMEmu的有效性，在24个gpu集群上的平均预测误差仅为2.17%，比vTrain高出21.09%，并证实了其在不同网络条件下跨128节点CPU模拟的训练成本分布建模方面的可扩展性。

{"title":"LLMEmu: A lightweight performance emulator for high-fidelity distributed LLM training","authors":"Siyuan Yang , Enda Yu , Pingjing Lu , Dezun Dong","doi":"10.1016/j.peva.2025.102526","DOIUrl":"10.1016/j.peva.2025.102526","url":null,"abstract":"<div><div>The prohibitive cost of training trillion-parameter large language models (LLMs) necessitates low-cost emulation tools for distributed system optimization. In modern large-scale clusters, communication often becomes the primary bottleneck to scalability. However, existing emulators, such as vTrain and ASTRA-Sim, overlook dynamic network factors that significantly impact performance at scale, resulting in limited emulation accuracy. This work offers an efficient and reliable tool for training system optimization and parallel strategy exploration, considerably lowering the barrier to large-scale AI research. We present LLMEmu, a distributed training emulator that combines real kernel profiling and actual communication execution. First, computation is profiled through real CUDA kernel traces on GPU nodes to construct an operator-level latency lookup table, enabling GPU-like execution on CPU clusters. Second, inter-node communication is executed using communication library primitives (e.g., AllReduce, Send/Recv), triggered by communication anchors embedded in the execution graph, and implemented using a pluggable communication backend. LLMEmu can seamlessly model hybrid parallelism strategies and supports multiple collective algorithms. Its lightweight design incorporates gradient bucketing with latency reuse to minimize overhead while maintaining extensibility to various network interconnects. The effectiveness of LLMEmu is validated through its performance results, demonstrating an average prediction error of only 2.17% on 24-GPU clusters, which outperforms vTrain by 21.09%, and confirming its scalability in modeling training cost distributions across 128-node CPU emulations under varying network conditions.</div></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":"170 ","pages":"Article 102526"},"PeriodicalIF":0.8,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145516998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0