Performance Evaluation最新文献

英文中文

Can we use deliberation to change evaluation systems? How an advisory group contributed to policy change 我们可以通过审议来改变评估系统吗?咨询小组如何促进政策变化

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-03-06 DOI: 10.1177/13563890231156955

Peter Dahler-Larsen

One of the most dominant trends in the field of evaluation in recent years is the institutionalization of evaluation under headlines such as “evaluation culture,” “evaluation policy,” and “evaluation systems.” There has been less interest in how evaluation systems can be changed, modified, or improved, not to mention deinstitutionalized, if necessary. Can a variety of stakeholders deliberate about the consequences of an evaluation system, and can it lead to policy change? A case study of a ministerial advisory group on national tests in Denmark shows how specific challenges were dealt with, such as the design of the deliberative process, the potential dominance of experts, and the distinction between technical-evaluative and practical-political arguments, and how these maneuvers paved the way for policy change. Based on the case study, the article discusses the prospects for democratic deliberation about evaluation systems.

近年来评估领域最主要的趋势之一是在“评估文化”、“评估政策”和“评估体系”等标题下的评估制度化。人们对如何改变、修改或改进评估系统的兴趣越来越少，更不用说在必要时将其去机构化了。各种利益相关者是否能够考虑评估系统的后果，是否能够导致政策变化?对丹麦国家考试部长级咨询小组的案例研究显示了如何处理具体挑战，例如审议程序的设计、专家的潜在主导地位、技术评估和实际政治论点之间的区别，以及这些操作如何为政策变革铺平道路。本文以案例研究为基础，探讨了评价制度民主审议的前景。

引用次数: 0

The use of mediation analysis in evaluation of complex health interventions 在评价复杂卫生干预措施中使用中介分析

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-02-07 DOI: 10.1177/13563890221149452

Deborah D. DiLiberto, C. Opondo, S. Staedke, Clare I. R. Chandler, E. Allen

This article presents an application of the causal inference approach to mediation analysis using the example of a complex intervention that aimed to improve the quality of care at health centres in Uganda. Mediation analysis is a statistical method that aims to isolate the causal mechanisms that make an intervention work in a given context. We combined data from a cluster randomized control trial and a mixed-methods process evaluation. We developed two causal models following our hypotheses of how the intervention was intended to work through mechanisms at health centres to improve health outcomes in the community. In adjusted analyses, there was evidence of an effect of the intervention on some health centre mechanisms; however, these did not lead to improvements in community health outcomes. We discuss the practical and epistemological challenges encountered when using mediation analysis to evaluate a complex intervention. These findings will inform future evaluations. Trial registration: The trial reported in this article is registered at: clinicaltrials.gov, NCT01024426. Registered 2 December 2009, https://clinicaltrials.gov/ct2/show/record/NCT01024426?term=NCT01024426&draw=2&rank=1

本文介绍了因果推理方法在调解分析中的应用，使用旨在提高乌干达保健中心护理质量的复杂干预措施的例子。中介分析是一种统计方法，旨在隔离使干预在给定上下文中起作用的因果机制。我们结合了来自集群随机对照试验和混合方法过程评估的数据。根据我们的假设，我们开发了两个因果模型，假设干预是如何通过卫生中心的机制来改善社区的健康结果的。在调整后的分析中，有证据表明干预对某些保健中心机制产生了影响;然而，这些并没有导致社区健康结果的改善。我们讨论在使用中介分析评估复杂干预时遇到的实践和认识论挑战。这些发现将为今后的评价提供信息。试验注册:本文报道的试验注册在:clinicaltrials.gov, NCT01024426。2009年12月2日注册，https://clinicaltrials.gov/ct2/show/record/NCT01024426?term=NCT01024426&draw=2&rank=1

{"title":"The use of mediation analysis in evaluation of complex health interventions","authors":"Deborah D. DiLiberto, C. Opondo, S. Staedke, Clare I. R. Chandler, E. Allen","doi":"10.1177/13563890221149452","DOIUrl":"https://doi.org/10.1177/13563890221149452","url":null,"abstract":"This article presents an application of the causal inference approach to mediation analysis using the example of a complex intervention that aimed to improve the quality of care at health centres in Uganda. Mediation analysis is a statistical method that aims to isolate the causal mechanisms that make an intervention work in a given context. We combined data from a cluster randomized control trial and a mixed-methods process evaluation. We developed two causal models following our hypotheses of how the intervention was intended to work through mechanisms at health centres to improve health outcomes in the community. In adjusted analyses, there was evidence of an effect of the intervention on some health centre mechanisms; however, these did not lead to improvements in community health outcomes. We discuss the practical and epistemological challenges encountered when using mediation analysis to evaluate a complex intervention. These findings will inform future evaluations. Trial registration: The trial reported in this article is registered at: clinicaltrials.gov, NCT01024426. Registered 2 December 2009, https://clinicaltrials.gov/ct2/show/record/NCT01024426?term=NCT01024426&draw=2&rank=1","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75357772","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Editorial Message from the New Editor-in-Chief 新主编的社论

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-01-01 DOI: 10.1016/S0166-5316(23)00007-X

引用次数: 0

French language abstracts 法语摘要

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-01-01 DOI: 10.1177/13563890221142105

引用次数: 0

A learning-based scheme for channel allocation to vehicular users in wireless networks 一种基于学习的无线网络车载用户信道分配方案

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-01-01 DOI: 10.1016/j.peva.2023.102331

Thi Thuy Nga Nguyen , Olivier Brun , Balakrishna J. Prabhu

Resource allocation algorithms in wireless networks can require solving complex optimization problems at every decision epoch. For large scale networks, when decisions need to be taken on time scales of milliseconds, using standard convex optimization solvers for computing the optimum can be a time-consuming affair that may impair real-time decision making. In this paper, we propose to use Data-driven and Deep Feedforward Neural Networks (DFNN) for learning the relation between the inputs and the outputs of two such resource allocation algorithms that were proposed in Nguyen et al. (2019, 2020). On numerical examples with realistic mobility patterns, we show that the learning algorithm yields an approximate yet satisfactory solution with much less computation time.

无线网络中的资源分配算法在每个决策时刻都需要求解复杂的优化问题。对于大型网络，当需要在毫秒的时间尺度上做出决策时，使用标准的凸优化求解器来计算最优值可能是一件耗时的事情，可能会损害实时决策。在本文中，我们建议使用数据驱动和深度前馈神经网络(DFNN)来学习Nguyen等人(2019,2020)提出的两种资源分配算法的输入和输出之间的关系。在具有实际移动模式的数值示例中，我们证明了该学习算法以更少的计算时间产生了近似但令人满意的解。

引用次数: 0

Practical anonymization for data streams: z-anonymity and relation with k-anonymity 数据流的实用匿名化:z匿名及其与k匿名的关系

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-01-01 DOI: 10.1016/j.peva.2022.102329

Nikhil Jha , Luca Vassio , Martino Trevisan , Emilio Leonardi , Marco Mellia

With the advent of big data and the emergence of data markets, preserving individuals’ privacy has become of utmost importance. The classical response to this need is anonymization, i.e., sanitizing the information that, directly or indirectly, can allow users’ re-identification. Among the various approaches, $k$ -anonymity provides a simple and easy-to-understand protection. However, $k$ -anonymity is challenging to achieve in a continuous stream of data and scales poorly when the number of attributes becomes high.

In this paper, we study a novel anonymization property called $z$ -anonymity that we explicitly design to deal with data streams, i.e., where the decision to publish a given attribute (atomic information) is made in real time. The idea at the base of $z$ -anonymity is to release such attribute about a user only if at least $z - 1$ other users have exposed the same attribute in a past time window. Depending on the value of $z$ , the output stream results $k$ -anonymized with a certain probability. To this end, we present a probabilistic model to map the $z$ -anonymity into the $k$ -anonymity property. The model is not only helpful in studying the $z$ -anonymity property, but also general enough to evaluate the probability of achieving $k$ -anonymity in data streams, resulting in a generic contribution.

随着大数据的出现和数据市场的出现，保护个人隐私变得至关重要。对这一需求的典型回应是匿名化，即对可以直接或间接允许用户重新识别的信息进行消毒。在各种方法中，k-匿名提供了一种简单易懂的保护。然而，在连续的数据流中实现k-匿名是具有挑战性的，并且当属性数量变得很高时扩展性很差。在本文中，我们研究了一种新的匿名化属性，称为z-匿名，我们显式地设计它来处理数据流，即在实时做出发布给定属性(原子信息)的决定。z匿名的基本思想是，只有当至少z−1个其他用户在过去的时间窗口中暴露了相同的属性时，才释放有关用户的此类属性。根据z的值，输出流以一定的概率进行k匿名化。为此，我们提出了一个将z匿名性映射到k匿名性的概率模型。该模型不仅有助于研究z-匿名性，而且具有通用性，足以评估数据流中实现k-匿名的概率，从而产生一般性贡献。

{"title":"Practical anonymization for data streams: z-anonymity and relation with k-anonymity","authors":"Nikhil Jha , Luca Vassio , Martino Trevisan , Emilio Leonardi , Marco Mellia","doi":"10.1016/j.peva.2022.102329","DOIUrl":"https://doi.org/10.1016/j.peva.2022.102329","url":null,"abstract":"<div>With the advent of big data and the emergence of data markets, preserving individuals’ privacy has become of utmost importance. The classical response to this need is anonymization, i.e., sanitizing the information that, directly or indirectly, can allow users’ re-identification. Among the various approaches, <math><mi>k</mi></math>-anonymity provides a simple and easy-to-understand protection. However, <math><mi>k</mi></math>-anonymity is challenging to achieve in a continuous stream of data and scales poorly when the number of attributes becomes high.In this paper, we study a novel anonymization property called <math><mi>z</mi></math>-anonymity that we explicitly design to deal with data streams, i.e., where the decision to publish a given attribute (atomic information) is made in real time. The idea at the base of <math><mi>z</mi></math>-anonymity is to release such attribute about a user only if at least <math><mrow><mi>z</mi><mo>−</mo><mn>1</mn></mrow></math> other users have exposed the same attribute in a past time window. Depending on the value of <math><mi>z</mi></math>, the output stream results <math><mi>k</mi></math>-anonymized with a certain probability. To this end, we present a probabilistic model to map the <math><mi>z</mi></math>-anonymity into the <math><mi>k</mi></math>-anonymity property. The model is not only helpful in studying the <math><mi>z</mi></math>-anonymity property, but also general enough to evaluate the probability of achieving <math><mi>k</mi></math>-anonymity in data streams, resulting in a generic contribution.</div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49901370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Stability analysis of two-class retrial systems with constant retrial rates and general service times 具有恒定再审率和一般服役时间的两级再审系统稳定性分析

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2023-01-01 DOI: 10.1016/j.peva.2022.102330

Konstantin Avrachenkov , Evsey Morozov , Ruslana Nekrasova

We establish stability criterion for a two-class retrial system with Poisson inputs, general class-dependent service times and class-dependent constant retrial rates. We also characterize an interesting phenomenon of partial stability when one orbit is tight but the other orbit goes to infinity in probability. All theoretical results are illustrated by numerical experiments.

我们建立了一类具有泊松输入、一般依赖于类的服务时间和依赖于类的常数重审率的两类重审系统的稳定性判据。我们还描述了一个有趣的现象，部分稳定性，当一个轨道是紧的，而另一个轨道在概率上趋于无穷大。所有的理论结果都得到了数值实验的验证。

引用次数: 4

Combining process tracing and synthetic control method: Bridging two ways of making causal inference in evaluation research 过程追溯与综合控制相结合:在评价研究中架起两种进行因果推理的桥梁

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2022-12-05 DOI: 10.1177/13563890221139511

F. Podestà

This article discusses potential ways of combining two methods of evaluation in single-case studies: the synthetic control method and the process tracing method. Both are designed to examine certain events/programmes that take place in given cases but view these events/programmes from different causal perspectives. Seeing an event/programme as a cause, synthetic control estimates its impact on one or more outcomes. Conversely, starting from a certain outcome, process tracing uncovers the causes responsible. One can start from the causal explanation reached via one of the two methods and then proceed to examine that explanation through the other method. Once the causes of an outcome are traced via a process tracing analysis, that account can be validated by estimating the effects of those causes via synthetic control. Equally, once the impact of a certain event is estimated through synthetic control, causal mechanisms traceable via process tracing can be exploited to refine that impact evaluation.

本文讨论了在单案例研究中结合两种评估方法的潜在方法:综合控制方法和过程追踪方法。两者都旨在研究在特定情况下发生的某些事件/项目，但从不同的因果角度看待这些事件/项目。综合控制将事件/项目视为原因，评估其对一个或多个结果的影响。相反，从某个结果开始，过程跟踪揭示了负责任的原因。人们可以从通过两种方法中的一种得到的因果解释开始，然后通过另一种方法进一步检验这种解释。一旦通过过程跟踪分析追踪到结果的原因，就可以通过综合控制估计这些原因的影响来验证该说明。同样，一旦通过综合控制估计了某一事件的影响，就可以利用可通过过程跟踪跟踪的因果机制来改进影响评估。

引用次数: 0

Evaluating the impact of business coaching programmes by taking perceptions seriously 通过认真对待看法来评估商业培训计划的影响

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2022-11-27 DOI: 10.1177/13563890221137611

G. Ton, F. van Rijn, H. Pamuk

The paper addresses the challenges of evaluating the impact of business coaching programmes with a varied portfolio of firms working across sectors and countries. Observable indicators of changes in business management practices are rarely relevant across sectors. Therefore, evaluators need to rely on the perceptions of the managers who have received coaching. We designed an online survey to compare the effectiveness of business coaching within a portfolio and across programmes. The survey was applied to the portfolio of two private sector development programmes. We derived so-called ‘contribution scores’ from individuals’ perceptions of how business management practices had changed and their perceptions of the role of business coaching in bringing about these changes. The survey included some features to reflect on response reliability. We show that the tool seems fairly reliable for comparative analysis and helped to identify the types of firms and contexts where business coaching support appears more effective.

本文探讨了评估跨部门和国家的不同公司投资组合的商业培训计划的影响所面临的挑战。企业管理实践变化的可观察指标很少与各个部门相关。因此，评估者需要依赖接受过培训的管理者的看法。我们设计了一项在线调查，以比较一个投资组合内和跨项目的商业指导的有效性。这项调查适用于两个私营部门发展方案的组合。我们从个人对企业管理实践如何发生变化的看法以及他们对企业指导在带来这些变化中的作用的看法中得出了所谓的“贡献分数”。该调查包括一些反映反应可靠性的特征。我们表明，该工具对于比较分析似乎相当可靠，并有助于确定企业类型和商业指导支持更有效的背景。

引用次数: 0

Don’t you forget about me: Independence of evaluations from the perspective of US evaluation clients—An exploratory study 别忘了我:从美国评估客户的角度看评估的独立性——一项探索性研究

IF 2.2 4区计算机科学 Q2 Mathematics

Performance Evaluation

Pub Date : 2022-11-26 DOI: 10.1177/13563890221138876

F. Schmidli, Lyn E. Pleger, Susanne Hadorn

Policy evaluation literature has stressed the importance of independence of evaluations to guarantee objective evidence collection. The evaluator–client relationship is critical in this respect, since it contains inherent tensions due to the necessity for independent assessments alongside requirements for increased responsiveness to clients’ interests. Despite this distinct relationship, the client perspective has only recently received attention in research. This article presents findings from a survey among US evaluation clients and compares these to existing evidence from Switzerland. Unlike previous studies, we distinguish between constructive and destructive client influences. We show that professional experience and client familiarity with evaluation standards increase the likelihood of constructive influences aimed at improving evaluation results. Nevertheless, the findings indicate that dissatisfaction with an evaluation increases client’s attempts at influence that may be destructive. By discussing both motives behind influence and potential preventive measures, this article seeks to contribute to the increased social impact of policy evaluations.

政策评估文献强调了评估的独立性对保证客观证据收集的重要性。在这方面，评估人员与客户的关系是至关重要的，因为它包含固有的紧张关系，这是由于独立评估的必要性以及对客户利益增加响应的要求。尽管存在这种独特的关系，但客户视角直到最近才受到研究的关注。本文介绍了一项针对美国评估客户的调查结果，并将其与来自瑞士的现有证据进行了比较。与以往的研究不同，我们区分了建设性和破坏性的客户影响。我们表明，专业经验和客户对评估标准的熟悉程度增加了旨在改善评估结果的建设性影响的可能性。然而，研究结果表明，对评估的不满增加了客户施加影响的企图，这可能是破坏性的。通过讨论影响背后的动机和潜在的预防措施，本文试图为政策评估增加社会影响作出贡献。

{"title":"Don’t you forget about me: Independence of evaluations from the perspective of US evaluation clients—An exploratory study","authors":"F. Schmidli, Lyn E. Pleger, Susanne Hadorn","doi":"10.1177/13563890221138876","DOIUrl":"https://doi.org/10.1177/13563890221138876","url":null,"abstract":"Policy evaluation literature has stressed the importance of independence of evaluations to guarantee objective evidence collection. The evaluator–client relationship is critical in this respect, since it contains inherent tensions due to the necessity for independent assessments alongside requirements for increased responsiveness to clients’ interests. Despite this distinct relationship, the client perspective has only recently received attention in research. This article presents findings from a survey among US evaluation clients and compares these to existing evidence from Switzerland. Unlike previous studies, we distinguish between constructive and destructive client influences. We show that professional experience and client familiarity with evaluation standards increase the likelihood of constructive influences aimed at improving evaluation results. Nevertheless, the findings indicate that dissatisfaction with an evaluation increases client’s attempts at influence that may be destructive. By discussing both motives behind influence and potential preventive measures, this article seeks to contribute to the increased social impact of policy evaluations.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2022-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76584634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

首页上一页

下一页尾页

类型

全部化学•材料生命科学医学物理工程技术环境•农林材料科学地球科学法学管理学化学环境科学与生态学计算机科学教育学经济学农林科学人文科学生物学数学物理与天体物理心理学综合性期刊其他工业工程理学历史学农学文学信息工程

数据库

全部 ACS Publications Elsevier ieeexplore Springer The Royal Society of Chemistry Wiley

期刊

Performance Evaluation

全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.

﹀