首页 > 最新文献

2013 Proceedings IEEE INFOCOM最新文献

英文 中文
Pegasus: Precision hunting for icebergs and anomalies in network flows 飞马:精确搜寻网络流中的冰山和异常
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566936
S. Gangam, P. Sharma, S. Fahmy
Accurate online network monitoring is crucial for detecting attacks, faults, and anomalies, and determining traffic properties across the network. With high bandwidth links and consequently increasing traffic volumes, it is difficult to collect and analyze detailed flow records in an online manner. Traditional solutions that decouple data collection from analysis resort to sampling and sketching to handle large monitoring traffic volumes. We propose a new system, Pegasus, to leverage commercially available co-located compute and storage devices near routers and switches. Pegasus adaptively manages data transfers between monitors and aggregators based on traffic patterns and user queries. We use Pegasus to detect global icebergs or global heavy-hitters. Icebergs are flows with a common property that contribute a significant fraction of network traffic. For example, DDoS attack detection is an iceberg detection problem with a common destination IP. Other applications include identification of “top talkers,” top destinations, and detection of worms and port scans. Experiments with Abilene traces, sFlow traces from an enterprise network, and deployment of Pegasus as a live monitoring service on PlanetLab show that our system is accurate and scales well with increasing traffic and number of monitors.
准确的在线网络监控对于发现网络中的攻击、故障和异常,判断网络中的流量属性至关重要。由于链路带宽大,流量也随之增加,因此很难在线收集和分析详细的流量记录。传统的解决方案将数据收集与分析分离,采用采样和草图来处理大量的监控流量。我们提出了一个新的系统,Pegasus,利用商业上可用的位于路由器和交换机附近的计算和存储设备。Pegasus基于流量模式和用户查询自适应地管理监视器和聚合器之间的数据传输。我们用帕伽索斯探测全球冰山或全球重磅炸弹。冰山是具有共同属性的流,它贡献了网络流量的很大一部分。例如,DDoS攻击检测是具有共同目的IP的冰山检测问题。其他应用程序包括识别“顶级通话者”、顶级目的地、蠕虫检测和端口扫描。在PlanetLab上使用Abilene跟踪、sFlow跟踪和部署Pegasus作为实时监控服务的实验表明,我们的系统是准确的,并且随着流量和监视器数量的增加而扩展良好。
{"title":"Pegasus: Precision hunting for icebergs and anomalies in network flows","authors":"S. Gangam, P. Sharma, S. Fahmy","doi":"10.1109/INFCOM.2013.6566936","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566936","url":null,"abstract":"Accurate online network monitoring is crucial for detecting attacks, faults, and anomalies, and determining traffic properties across the network. With high bandwidth links and consequently increasing traffic volumes, it is difficult to collect and analyze detailed flow records in an online manner. Traditional solutions that decouple data collection from analysis resort to sampling and sketching to handle large monitoring traffic volumes. We propose a new system, Pegasus, to leverage commercially available co-located compute and storage devices near routers and switches. Pegasus adaptively manages data transfers between monitors and aggregators based on traffic patterns and user queries. We use Pegasus to detect global icebergs or global heavy-hitters. Icebergs are flows with a common property that contribute a significant fraction of network traffic. For example, DDoS attack detection is an iceberg detection problem with a common destination IP. Other applications include identification of “top talkers,” top destinations, and detection of worms and port scans. Experiments with Abilene traces, sFlow traces from an enterprise network, and deployment of Pegasus as a live monitoring service on PlanetLab show that our system is accurate and scales well with increasing traffic and number of monitors.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131222082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 19
Schedule first, manage later: Network-aware load balancing 先调度,后管理:网络感知负载均衡
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566825
Amir Nahir, A. Orda, D. Raz
Load balancing in large distributed server systems is a complex optimization problem of critical importance in cloud systems and data centers. Existing schedulers often incur a high overhead in communication when collecting the data required to make the scheduling decision, hence delaying the job request on its way to the executing server. We propose a novel scheme that incurs no communication overhead between the users and the servers upon job arrival, thus removing any scheduling overhead from the job's critical path. Our approach is based on creating several replicas of each job and sending each replica to a different server. Upon the arrival of a replica to the head of the queue at its server, the latter signals the servers holding replicas of that job, so as to remove them from their queues. We show, through analysis and simulations, that this scheme improves the expected queuing overhead over traditional schemes by a factor of 9 (or more) under various load conditions. In addition, we show that our scheme remains efficient even when the inter-server signal propagation delay is significant (relative to the job's execution time). We provide heuristic solutions to the performance degradation that occurs in such cases and show, by simulations, that they efficiently mitigate the detrimental effect of propagation delays. Finally, we demonstrate the efficiency of our proposed scheme in a real-world environment by implementing a load balancing system based on it, deploying the system on the Amazon Elastic Compute Cloud (EC2), and measuring its performance.
大型分布式服务器系统的负载平衡是云系统和数据中心中一个非常重要的复杂优化问题。在收集制定调度决策所需的数据时,现有的调度器通常会在通信方面产生很高的开销,因此会延迟发送到执行服务器的作业请求。我们提出了一种新的方案,该方案在作业到达时不会引起用户和服务器之间的通信开销,从而消除了作业关键路径上的任何调度开销。我们的方法基于为每个作业创建多个副本,并将每个副本发送到不同的服务器。当副本到达其服务器上的队列头部时,后者会向持有该作业副本的服务器发出信号,以便将其从队列中删除。我们通过分析和模拟表明,在各种负载条件下,该方案比传统方案将预期队列开销提高了9倍(或更多)。此外,我们还表明,即使服务器间信号传播延迟很大(相对于作业的执行时间),我们的方案仍然有效。我们提供了在这种情况下发生的性能下降的启发式解决方案,并通过模拟显示,它们有效地减轻了传播延迟的有害影响。最后,我们通过在实际环境中实现基于该方案的负载平衡系统,在Amazon Elastic Compute Cloud (EC2)上部署该系统,并测量其性能,来证明我们所提出方案的效率。
{"title":"Schedule first, manage later: Network-aware load balancing","authors":"Amir Nahir, A. Orda, D. Raz","doi":"10.1109/INFCOM.2013.6566825","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566825","url":null,"abstract":"Load balancing in large distributed server systems is a complex optimization problem of critical importance in cloud systems and data centers. Existing schedulers often incur a high overhead in communication when collecting the data required to make the scheduling decision, hence delaying the job request on its way to the executing server. We propose a novel scheme that incurs no communication overhead between the users and the servers upon job arrival, thus removing any scheduling overhead from the job's critical path. Our approach is based on creating several replicas of each job and sending each replica to a different server. Upon the arrival of a replica to the head of the queue at its server, the latter signals the servers holding replicas of that job, so as to remove them from their queues. We show, through analysis and simulations, that this scheme improves the expected queuing overhead over traditional schemes by a factor of 9 (or more) under various load conditions. In addition, we show that our scheme remains efficient even when the inter-server signal propagation delay is significant (relative to the job's execution time). We provide heuristic solutions to the performance degradation that occurs in such cases and show, by simulations, that they efficiently mitigate the detrimental effect of propagation delays. Finally, we demonstrate the efficiency of our proposed scheme in a real-world environment by implementing a load balancing system based on it, deploying the system on the Amazon Elastic Compute Cloud (EC2), and measuring its performance.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130985406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 9
Joint request mapping and response routing for geo-distributed cloud services 用于地理分布式云服务的联合请求映射和响应路由
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566873
Hong Xu, Baochun Li
Many cloud services are running on geographically distributed datacenters for better reliability and performance. We consider the emerging problem of joint request mapping and response routing with distributed datacenters in this paper. We formulate the problem as a general workload management optimization. A utility function is used to capture various performance goals, and the location diversity of electricity and bandwidth costs are realistically modeled. To solve the large-scale optimization, we develop a distributed algorithm based on the alternating direction method of multipliers (ADMM). Following a decomposition-coordination approach, our algorithm allows for a parallel implementation in a datacenter where each server solves a small sub-problem. The solutions are coordinated to find an optimal solution to the global problem. Our algorithm converges to near optimum within tens of iterations, and is insensitive to step sizes. We empirically evaluate our algorithm based on real-world workload traces and latency measurements, and demonstrate its effectiveness compared to conventional methods.
许多云服务运行在地理上分布的数据中心上,以获得更好的可靠性和性能。本文研究了分布式数据中心的联合请求映射和响应路由问题。我们将这个问题表述为一般的工作负载管理优化。使用效用函数来捕获各种性能目标,并对电力和带宽成本的位置多样性进行了实际建模。为了解决大规模优化问题,我们开发了一种基于乘法器交替方向法(ADMM)的分布式算法。遵循分解-协调方法,我们的算法允许在数据中心中并行实现,其中每个服务器解决一个小的子问题。这些解决方案相互协调,以找到全球问题的最优解决方案。该算法在几十次迭代中收敛到接近最优,并且对步长不敏感。我们根据实际工作负载跟踪和延迟测量经验评估了我们的算法,并与传统方法相比证明了它的有效性。
{"title":"Joint request mapping and response routing for geo-distributed cloud services","authors":"Hong Xu, Baochun Li","doi":"10.1109/INFCOM.2013.6566873","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566873","url":null,"abstract":"Many cloud services are running on geographically distributed datacenters for better reliability and performance. We consider the emerging problem of joint request mapping and response routing with distributed datacenters in this paper. We formulate the problem as a general workload management optimization. A utility function is used to capture various performance goals, and the location diversity of electricity and bandwidth costs are realistically modeled. To solve the large-scale optimization, we develop a distributed algorithm based on the alternating direction method of multipliers (ADMM). Following a decomposition-coordination approach, our algorithm allows for a parallel implementation in a datacenter where each server solves a small sub-problem. The solutions are coordinated to find an optimal solution to the global problem. Our algorithm converges to near optimum within tens of iterations, and is insensitive to step sizes. We empirically evaluate our algorithm based on real-world workload traces and latency measurements, and demonstrate its effectiveness compared to conventional methods.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132850981","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 163
HBaseMQ: A distributed message queuing system on clouds with HBase HBaseMQ:基于HBase的云上分布式消息队列系统
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566731
Chen Zhang, Xue Liu
Message queuing systems can be used to support a plethora of fundamental services in distributed systems. This paper presents HBaseMQ, the first distributed message queuing system based on bare-bones HBase. HBaseMQ directly inherits HBase's properties such as scalability and fault tolerance, enabling HBase users to rapidly instantiate a message queuing system with no extra program deployment or modification to HBase. As a result, HBaseMQ effectively enhances the data processing capability of an existing HBase installation. HBaseMQ supports reliable and total order message delivery with “at least once” or “at most once” message delivery guarantees with no limit on message size. Furthermore, HBaseMQ provides atomicity, consistency, isolation and durability on any operation over messages.
消息队列系统可用于支持分布式系统中的大量基础服务。本文提出了第一个基于HBase的分布式消息队列系统HBaseMQ。HBaseMQ直接继承了HBase的可扩展性和容错等特性,使得HBase用户无需额外的程序部署或修改HBase即可快速实例化一个消息队列系统。因此,HBaseMQ有效地增强了现有HBase安装的数据处理能力。HBaseMQ支持可靠的全订单消息传递,“至少一次”或“最多一次”消息传递保证,没有消息大小限制。此外,HBaseMQ为消息上的任何操作提供了原子性、一致性、隔离性和持久性。
{"title":"HBaseMQ: A distributed message queuing system on clouds with HBase","authors":"Chen Zhang, Xue Liu","doi":"10.1109/INFCOM.2013.6566731","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566731","url":null,"abstract":"Message queuing systems can be used to support a plethora of fundamental services in distributed systems. This paper presents HBaseMQ, the first distributed message queuing system based on bare-bones HBase. HBaseMQ directly inherits HBase's properties such as scalability and fault tolerance, enabling HBase users to rapidly instantiate a message queuing system with no extra program deployment or modification to HBase. As a result, HBaseMQ effectively enhances the data processing capability of an existing HBase installation. HBaseMQ supports reliable and total order message delivery with “at least once” or “at most once” message delivery guarantees with no limit on message size. Furthermore, HBaseMQ provides atomicity, consistency, isolation and durability on any operation over messages.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"225 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133301453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
Capacity provisioning for schedulers with tiny buffers 为具有微小缓冲区的调度器提供容量
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6567050
Y. Ghiassi-Farrokhfal, J. Liebeherr
Capacity and buffer sizes are critical design parameters in schedulers which multiplex many flows. Previous studies show that in an asymptotic regime, when the number of traffic flows N goes to infinity, the choice of scheduling algorithm does not have a big impact on performance. We raise the question whether or not the choice of scheduling algorithm impacts the capacity and buffer sizing for moderate values of N (e.g., few hundred). For Markov-modulated On-Off sources and for finite N, we show that the choice of scheduling is influential on (1) buffer overflow probability, (2) capacity provisioning, and (3) the viability of network decomposition in a non-asymptotic regime. This conclusion is drawn based on numerical examples and by a comparison of the scaling properties of different scheduling algorithms. In particular, we show that the per-flow capacity converges to the per-flow long-term average rate of the arrivals with convergence speeds ranging from O (√log N/N) to O(1/N) depending on the scheduling algorithm. This speed of convergences of the required capacities for different schedulers (to meet a target buffer overflow probability) is perceptible even for moderate values of N in our numerical examples.
在多流调度中,容量和缓冲区大小是关键的设计参数。以往的研究表明,在渐近区域,当交通流的数量N趋于无穷时,调度算法的选择对性能的影响不大。我们提出了调度算法的选择是否会影响中等N值(例如,几百)的容量和缓冲区大小的问题。对于马尔可夫调制的on- off源和有限N,我们证明了调度的选择对(1)缓冲区溢出概率,(2)容量供应,以及(3)非渐近状态下网络分解的可行性有影响。通过数值算例,比较了不同调度算法的尺度特性,得出了这一结论。特别是,我们证明了每流容量收敛到每流到达的长期平均速率,收敛速度从O(√log N/N)到O(1/N)不等,具体取决于调度算法。在我们的数值示例中,即使对于中等的N值,不同调度器所需容量的收敛速度(以满足目标缓冲区溢出概率)也是可以察觉的。
{"title":"Capacity provisioning for schedulers with tiny buffers","authors":"Y. Ghiassi-Farrokhfal, J. Liebeherr","doi":"10.1109/INFCOM.2013.6567050","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6567050","url":null,"abstract":"Capacity and buffer sizes are critical design parameters in schedulers which multiplex many flows. Previous studies show that in an asymptotic regime, when the number of traffic flows N goes to infinity, the choice of scheduling algorithm does not have a big impact on performance. We raise the question whether or not the choice of scheduling algorithm impacts the capacity and buffer sizing for moderate values of N (e.g., few hundred). For Markov-modulated On-Off sources and for finite N, we show that the choice of scheduling is influential on (1) buffer overflow probability, (2) capacity provisioning, and (3) the viability of network decomposition in a non-asymptotic regime. This conclusion is drawn based on numerical examples and by a comparison of the scaling properties of different scheduling algorithms. In particular, we show that the per-flow capacity converges to the per-flow long-term average rate of the arrivals with convergence speeds ranging from O (√log N/N) to O(1/N) depending on the scheduling algorithm. This speed of convergences of the required capacities for different schedulers (to meet a target buffer overflow probability) is perceptible even for moderate values of N in our numerical examples.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133312427","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A contract-based approach for ancillary services in V2G networks: Optimality and learning 基于契约的V2G网络辅助服务方法:最优性与学习
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566906
Yang Gao, Yan Chen, Chih-Yu Wang, K. Liu
With the foreseeable large scale deployment of electric vehicles (EVs) and the development of vehicle-to-grid (V2G) technologies, it is possible to provide ancillary services to the power grid in a cost efficient way, i.e., through the bidirectional power flow of EVs. A key issue in such kind of schemes is how to stimulate a large number of EVs to act coordinately to achieve the service request. This is challenging since EVs are self-interested and generally have different preferences toward charging and discharging based on their own constraints. In this paper, we propose a contract-based mechanism to tackle this challenge. Through the design of an optimal contract, the aggregator can provide incentives for EVs to participate in ancillary services to power grid, match the aggregated energy rate with the service request and maximize its own profits. We prove that under mild conditions, the optimal contract-based mechanism takes a very simple form, i.e., the aggregator only needs to publish an optimal unit price to EVs, which is determined based on the statistical distribution of EVs' preferences. We then consider a more practical scenario where the aggregator has no prior knowledge regarding the statistical distribution and study how should the aggregator learn the optimal unit price from its interactions with EVs. Simulation results are shown to verify the effectiveness of the proposed contract-based mechanism.
随着可预见的电动汽车的大规模部署和V2G技术的发展,通过电动汽车的双向电力流动,以经济有效的方式为电网提供辅助服务是可能的。这类方案的关键问题是如何激励大量电动汽车协同行动以实现服务请求。这是一个挑战,因为电动汽车是自私自利的,通常会根据自身的限制对充电和放电有不同的偏好。在本文中,我们提出了一种基于合约的机制来应对这一挑战。通过最优契约的设计,聚合器可以为电动汽车参与电网辅助服务提供激励,使聚合电价与服务请求相匹配,使自身利润最大化。我们证明了在温和条件下,基于契约的最优机制采用一种非常简单的形式,即聚合器只需向电动汽车发布一个最优单价,该单价是根据电动汽车偏好的统计分布确定的。然后,我们考虑一个更实际的场景,即聚合器对统计分布没有先验知识,并研究聚合器如何从与电动汽车的交互中学习最优单价。仿真结果验证了所提出的基于契约机制的有效性。
{"title":"A contract-based approach for ancillary services in V2G networks: Optimality and learning","authors":"Yang Gao, Yan Chen, Chih-Yu Wang, K. Liu","doi":"10.1109/INFCOM.2013.6566906","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566906","url":null,"abstract":"With the foreseeable large scale deployment of electric vehicles (EVs) and the development of vehicle-to-grid (V2G) technologies, it is possible to provide ancillary services to the power grid in a cost efficient way, i.e., through the bidirectional power flow of EVs. A key issue in such kind of schemes is how to stimulate a large number of EVs to act coordinately to achieve the service request. This is challenging since EVs are self-interested and generally have different preferences toward charging and discharging based on their own constraints. In this paper, we propose a contract-based mechanism to tackle this challenge. Through the design of an optimal contract, the aggregator can provide incentives for EVs to participate in ancillary services to power grid, match the aggregated energy rate with the service request and maximize its own profits. We prove that under mild conditions, the optimal contract-based mechanism takes a very simple form, i.e., the aggregator only needs to publish an optimal unit price to EVs, which is determined based on the statistical distribution of EVs' preferences. We then consider a more practical scenario where the aggregator has no prior knowledge regarding the statistical distribution and study how should the aggregator learn the optimal unit price from its interactions with EVs. Simulation results are shown to verify the effectiveness of the proposed contract-based mechanism.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"124 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133340855","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 38
Near-optimal approximate membership query over time-decaying windows 在时间衰减窗口上的近似最优成员查询
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566939
Yang Liu, Wenji Chen, Y. Guan
There has been a long history of finding a spaceefficient data structure to support approximate membership queries, started from Bloom's work in the 1970's. Given a set A of n items and an additional item x from the same universe U of a size m ≫ n, we want to distinguish whether x ∈ A or not, using small (limited) space. The solutions for the membership query are needed for many network applications, such as cache directory, load-balancing, security, etc. If A is static, there exist optimal algorithms to find a randomized data structure to represent A using only (1+ o(1))n log 1/δ bits, which only allows for a small false positive δ but no false negative. However, existing optimal algorithms are not practical for many Internet applications, e.g., social network services, peer-to-peer systems, network traffic monitoring, etc. They are too spaceand time-expensive due to the frequent changes in the set A, because all items are needed to recompute the optimal data structure for each change using a linear running time. In this paper, we propose a novel data structure to support the approximate membership query in the time-decaying window model. In this model, items are inserted one-by-one over a data stream, and we want to determine whether an item is among the most recent w items for any given window size w ≤ n. Our data structure only requires O(n(log 1/δ+logn)) bits and O(1) running time. We also prove a non-trivial space lower bound, i.e. (n - δm) log(n - δm) bits, which guarantees that our data structure is near-optimal. Our data structure has been evaluated using both synthetic and real data sets.
从20世纪70年代Bloom的工作开始,人们一直在寻找一种空间高效的数据结构来支持近似成员查询。给定一个有n个项目的集合a和一个附加的项目x,它们来自大小为m比n的同一个宇宙U,我们想用很小的(有限的)空间来区分x是否∈a。许多网络应用都需要成员查询的解决方案,如缓存目录、负载平衡、安全性等。如果A是静态的,存在最优算法来找到一个随机数据结构来表示A,只使用(1+ o(1))n log 1/δ位,这只允许一个小的假正δ,但没有假负。然而,现有的最优算法并不适用于许多互联网应用,例如社交网络服务、点对点系统、网络流量监控等。由于集合A中的频繁更改,它们的空间和时间开销太大,因为所有项都需要使用线性运行时间为每次更改重新计算最佳数据结构。本文提出了一种新的数据结构来支持时间衰减窗口模型中的近似隶属度查询。在这个模型中,条目在数据流上一个接一个地插入,我们想要确定一个条目是否在任何给定窗口大小w≤n的最近w项中。我们的数据结构只需要O(n(log 1/δ+logn))位和O(1)运行时间。我们还证明了一个非平凡空间下界,即(n - δm) log(n - δm)位,这保证了我们的数据结构是接近最优的。我们的数据结构已经使用合成数据集和真实数据集进行了评估。
{"title":"Near-optimal approximate membership query over time-decaying windows","authors":"Yang Liu, Wenji Chen, Y. Guan","doi":"10.1109/INFCOM.2013.6566939","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566939","url":null,"abstract":"There has been a long history of finding a spaceefficient data structure to support approximate membership queries, started from Bloom's work in the 1970's. Given a set A of n items and an additional item x from the same universe U of a size m ≫ n, we want to distinguish whether x ∈ A or not, using small (limited) space. The solutions for the membership query are needed for many network applications, such as cache directory, load-balancing, security, etc. If A is static, there exist optimal algorithms to find a randomized data structure to represent A using only (1+ o(1))n log 1/δ bits, which only allows for a small false positive δ but no false negative. However, existing optimal algorithms are not practical for many Internet applications, e.g., social network services, peer-to-peer systems, network traffic monitoring, etc. They are too spaceand time-expensive due to the frequent changes in the set A, because all items are needed to recompute the optimal data structure for each change using a linear running time. In this paper, we propose a novel data structure to support the approximate membership query in the time-decaying window model. In this model, items are inserted one-by-one over a data stream, and we want to determine whether an item is among the most recent w items for any given window size w ≤ n. Our data structure only requires O(n(log 1/δ+logn)) bits and O(1) running time. We also prove a non-trivial space lower bound, i.e. (n - δm) log(n - δm) bits, which guarantees that our data structure is near-optimal. Our data structure has been evaluated using both synthetic and real data sets.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"139 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133790250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 20
O(ε)-Approximation to physical world by sensor networks O(ε)-传感器网络对物理世界的逼近
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6567121
Siyao Cheng, Jianzhong Li, Zhipeng Cai
To observe the complicate physical world by a WSN, the sensors in the WSN senses and samples the data from the physical world. Currently, most of the existing work use equi-frequency sampling methods (EFS) or EFS based sampling methods for data acquisition in sensor networks. However, the accuracies of EFS and EFS based sampling methods cannot be guaranteed in practice since the physical world usually varies continuously, and these methods does not support reconstructing of the monitored physical world. To overcome the shortages of EFS and EFS based sampling methods, this paper focuses on designing physical-world-aware data acquisition algorithms to support O(ϵ)-approximation to the physical world for any ϵ ≥ 0. Two physical-world-aware data acquisition algorithms based on Hermit and Spline interpolation are proposed in the paper. Both algorithms can adjust the sensing frequency automatically based on the changing trend of the physical world and given c. The thorough analysis on the performance of the algorithms are also provided, including the accuracies, the smooth of the outputted curves, the error bounds for computing first and second derivatives, the number of the sampling times and complexities of the algorithms. It is proven that the error bounds of the algorithms are O(ϵ) and the complexities of the algorithms are O(1/ϵ1/4). Based on the new data acquisition algorithms, an algorithm for reconstructing physical world is also proposed and analyzed. The theoretical analysis and experimental results show that all the proposed algorithms have high performance in items of accuracy and energy consumption.
为了通过WSN观察复杂的物理世界,WSN中的传感器对物理世界中的数据进行感知和采样。目前,传感器网络中的数据采集工作大多采用等频采样方法(EFS)或基于EFS的采样方法。然而,由于物理世界通常是连续变化的,因此在实践中无法保证电磁场和基于电磁场的采样方法的准确性,并且这些方法不支持对被监测的物理世界进行重建。为了克服EFS和基于EFS的采样方法的不足,本文侧重于设计物理世界感知数据采集算法,以支持任意λ≥0时对物理世界的O(λ)近似。提出了两种基于隐式插值和样条插值的物理世界感知数据采集算法。两种算法都可以根据物理世界的变化趋势和给定的c自动调整传感频率,并对算法的性能进行了深入的分析,包括精度、输出曲线的光滑性、计算一阶导数和二阶导数的误差范围、采样次数和算法的复杂性。证明了算法的误差限为O(λ),算法的复杂度为O(1/ϵ1/4)。在新的数据采集算法的基础上,提出并分析了一种重建物理世界的算法。理论分析和实验结果表明,所提出的算法在精度和能耗方面都具有较高的性能。
{"title":"O(ε)-Approximation to physical world by sensor networks","authors":"Siyao Cheng, Jianzhong Li, Zhipeng Cai","doi":"10.1109/INFCOM.2013.6567121","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6567121","url":null,"abstract":"To observe the complicate physical world by a WSN, the sensors in the WSN senses and samples the data from the physical world. Currently, most of the existing work use equi-frequency sampling methods (EFS) or EFS based sampling methods for data acquisition in sensor networks. However, the accuracies of EFS and EFS based sampling methods cannot be guaranteed in practice since the physical world usually varies continuously, and these methods does not support reconstructing of the monitored physical world. To overcome the shortages of EFS and EFS based sampling methods, this paper focuses on designing physical-world-aware data acquisition algorithms to support O(ϵ)-approximation to the physical world for any ϵ ≥ 0. Two physical-world-aware data acquisition algorithms based on Hermit and Spline interpolation are proposed in the paper. Both algorithms can adjust the sensing frequency automatically based on the changing trend of the physical world and given c. The thorough analysis on the performance of the algorithms are also provided, including the accuracies, the smooth of the outputted curves, the error bounds for computing first and second derivatives, the number of the sampling times and complexities of the algorithms. It is proven that the error bounds of the algorithms are O(ϵ) and the complexities of the algorithms are O(1/ϵ1/4). Based on the new data acquisition algorithms, an algorithm for reconstructing physical world is also proposed and analyzed. The theoretical analysis and experimental results show that all the proposed algorithms have high performance in items of accuracy and energy consumption.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124566491","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 71
Reactive planar spanner construction in wireless ad hoc and sensor networks 无线自组织和传感器网络中的反应平面扳手构造
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6567022
M. Benter, Florentin Neumann, Hannes Frey
Within reactive topology control, a node determines its adjacent edges of a network subgraph without prior knowledge of its neighborhood. The goal is to construct a local view on a topology which provides certain desired properties such as planarity. During algorithm execution, a node, in general, is not allowed to determine all its neighbors of the network graph. There are well-known reactive algorithms for computing planar subgraphs. However, the subgraphs obtained do not have constant Euclidean spanning ratio. This means that routing along these subgraphs may result in potentially long detours. So far, it has been unknown if planar spanners can be constructed reactively. In this work, we show that at least under the unit disk network model, this is indeed possible, by proposing an algorithm for reactive construction of the partial Delaunay triangulation, which recently turned out to be a spanner. Furthermore, we show that our algorithm is message-optimal as a node will only exchange messages with nodes that are also neighbors in the spanner. The algorithm's presentation is complemented by a rigorous proof of correctness.
在被动拓扑控制中,一个节点在不知道其邻域的情况下确定其网络子图的相邻边。目标是在拓扑结构上构造一个局部视图,该视图提供某些所需的属性,如平面性。在算法执行过程中,一般不允许一个节点确定其网络图的所有邻居。有一些众所周知的用于计算平面子图的响应式算法。然而,得到的子图并不具有恒定的欧氏生成比。这意味着沿着这些子图的路由可能会导致潜在的长弯路。到目前为止,尚不清楚是否可以反应构造平面扳手。在这项工作中,我们表明,至少在单位磁盘网络模型下,这确实是可能的,通过提出部分德劳内三角测量的反应构造算法,这最近被证明是一个扳手。此外,我们证明了我们的算法是消息最优的,因为节点只会与扳手中的邻居节点交换消息。该算法的演示由一个严格的正确性证明来补充。
{"title":"Reactive planar spanner construction in wireless ad hoc and sensor networks","authors":"M. Benter, Florentin Neumann, Hannes Frey","doi":"10.1109/INFCOM.2013.6567022","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6567022","url":null,"abstract":"Within reactive topology control, a node determines its adjacent edges of a network subgraph without prior knowledge of its neighborhood. The goal is to construct a local view on a topology which provides certain desired properties such as planarity. During algorithm execution, a node, in general, is not allowed to determine all its neighbors of the network graph. There are well-known reactive algorithms for computing planar subgraphs. However, the subgraphs obtained do not have constant Euclidean spanning ratio. This means that routing along these subgraphs may result in potentially long detours. So far, it has been unknown if planar spanners can be constructed reactively. In this work, we show that at least under the unit disk network model, this is indeed possible, by proposing an algorithm for reactive construction of the partial Delaunay triangulation, which recently turned out to be a spanner. Furthermore, we show that our algorithm is message-optimal as a node will only exchange messages with nodes that are also neighbors in the spanner. The algorithm's presentation is complemented by a rigorous proof of correctness.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114335508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
QoS satisfaction games for spectrum sharing 频谱共享的QoS满意度博弈
Pub Date : 2013-04-14 DOI: 10.1109/INFCOM.2013.6566837
Richard Southwell, Xu Chen, Jianwei Huang
Today's wireless networks are facing tremendous growth and many applications have more demanding quality of service (QoS) requirements than ever before. However, there is only a finite amount of wireless resources (such as spectrum) that can be used to satisfy these demanding requirements. We present a general QoS satisfaction game framework for modeling the issue of distributed spectrum sharing to meet QoS requirements. Our study is motivated by the observation that finding globally optimal spectrum sharing solutions with QoS guarantees is NP hard. We show that the QoS satisfaction game has the finite improvement property, and the users can self-organize into a pure Nash equilibrium in polynomial time. By bounding the price of anarchy, we demonstrate that the worst case pure Nash equilibrium can be close to the global optimal solution when users' QoS demands are not too diverse.
当今的无线网络正面临着巨大的增长,许多应用程序对服务质量(QoS)的要求比以往任何时候都要高。然而,只有有限的无线资源(如频谱)可以用来满足这些苛刻的要求。我们提出了一个通用的QoS满意度博弈框架,用于对分布式频谱共享问题进行建模,以满足QoS需求。我们研究的动机是观察到寻找具有QoS保证的全局最优频谱共享解决方案是NP困难的。证明了QoS满意度博弈具有有限改进性质,用户可以在多项式时间内自组织成纯纳什均衡。通过限制无政府状态的价格,我们证明了当用户的QoS需求不太多样化时,最坏情况下的纯纳什均衡可以接近全局最优解。
{"title":"QoS satisfaction games for spectrum sharing","authors":"Richard Southwell, Xu Chen, Jianwei Huang","doi":"10.1109/INFCOM.2013.6566837","DOIUrl":"https://doi.org/10.1109/INFCOM.2013.6566837","url":null,"abstract":"Today's wireless networks are facing tremendous growth and many applications have more demanding quality of service (QoS) requirements than ever before. However, there is only a finite amount of wireless resources (such as spectrum) that can be used to satisfy these demanding requirements. We present a general QoS satisfaction game framework for modeling the issue of distributed spectrum sharing to meet QoS requirements. Our study is motivated by the observation that finding globally optimal spectrum sharing solutions with QoS guarantees is NP hard. We show that the QoS satisfaction game has the finite improvement property, and the users can self-organize into a pure Nash equilibrium in polynomial time. By bounding the price of anarchy, we demonstrate that the worst case pure Nash equilibrium can be close to the global optimal solution when users' QoS demands are not too diverse.","PeriodicalId":206346,"journal":{"name":"2013 Proceedings IEEE INFOCOM","volume":"58 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114732072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 8
期刊
2013 Proceedings IEEE INFOCOM
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1