There is a growing need for high radix switches in data centres and high performance computing. Current computing systems are interconnected using large numbers of relatively low radix (32--48 port) switches that restrict scalability and performance, while increasing cost and management complexity. In parallel, there is a growing interest in dense rack scale computing in which a single rack can contain several thousand network nodes. To meet these demands, we recently demonstrated a flexible optical switch architecture using fast tuneable lasers and coherent receivers which scales to over 1000 ports. However, using traditional clock data recovery circuits in this or any optical packet switch results in large latency and throughput penalties due to resynchronisation on each new connection. In this talk, we will address the challenges of building a fully synchronous optical switch network, of rack-scale or greater, in which a reference clock is distributed to every node to reduce resynchronisation overhead. We will firstly present results from preliminary FPGA-based experiments demonstrating the viability of synchronising a rack scale network. We will then discuss the limitations on port count, range and bit rate which would limit the ability to build larger synchronous systems in this way.
{"title":"Enabling high performance rack-scale optical switching through global synchronisation","authors":"Kari A. Clark, Phillip Watt","doi":"10.1145/3073763.3073773","DOIUrl":"https://doi.org/10.1145/3073763.3073773","url":null,"abstract":"There is a growing need for high radix switches in data centres and high performance computing. Current computing systems are interconnected using large numbers of relatively low radix (32--48 port) switches that restrict scalability and performance, while increasing cost and management complexity. In parallel, there is a growing interest in dense rack scale computing in which a single rack can contain several thousand network nodes. To meet these demands, we recently demonstrated a flexible optical switch architecture using fast tuneable lasers and coherent receivers which scales to over 1000 ports. However, using traditional clock data recovery circuits in this or any optical packet switch results in large latency and throughput penalties due to resynchronisation on each new connection. In this talk, we will address the challenges of building a fully synchronous optical switch network, of rack-scale or greater, in which a reference clock is distributed to every node to reduce resynchronisation overhead. We will firstly present results from preliminary FPGA-based experiments demonstrating the viability of synchronising a rack scale network. We will then discuss the limitations on port count, range and bit rate which would limit the ability to build larger synchronous systems in this way.","PeriodicalId":20560,"journal":{"name":"Proceedings of the 2nd International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems","volume":"101 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"85813182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many computational workloads from commercial and scientific fields have high demands in total throughput, and energy efficiency. For example the largest radio telescope, to be built in South Africa and Australia combines cost, performance and power targets that cannot be met by the technological development until its installation. In processor architecture a design tradeoff between cost and power efficiency against single-thread performance is observed. Hence, to achieve a high system power efficiency, large-scale parallelism has to be employed. In order to maintain wire length, and hence network delays, energy losses, and cost, the volume of compute nodes and network switches has to be reduced to a minimum âĂŞ hence the term microserver. Our DOME microserver compute card measures 130 by 7.5 by 65 mm3. The presented switch module is confined to the same area (140 by 55 mm2), yet is deeper (40mm) because of the 630-pin high-speed connector. For 64 ports of 10Gbit Ethernet (10Gbase-KR) our switch consumes about 150W maximal. In addition to the switch ASIC (Intel FM6000 series), the power converters, clock generation, configuration memory and management processor is integrated on a second PCB. The switch management (âĂIJControl PointâĂİ) is implemented in a separate compute node. In the talk options to integrate the management into the switch (same volume as now) will be discussed. Another topic covered is the cooling of the microserver, and of the switch in particular, using (warm) water in the infrastructure and heat pipes on the module.
{"title":"Microserver + micro-switch = micro-datacenter","authors":"F. Abel, A. Doering","doi":"10.1145/3073763.3073772","DOIUrl":"https://doi.org/10.1145/3073763.3073772","url":null,"abstract":"Many computational workloads from commercial and scientific fields have high demands in total throughput, and energy efficiency. For example the largest radio telescope, to be built in South Africa and Australia combines cost, performance and power targets that cannot be met by the technological development until its installation. In processor architecture a design tradeoff between cost and power efficiency against single-thread performance is observed. Hence, to achieve a high system power efficiency, large-scale parallelism has to be employed. In order to maintain wire length, and hence network delays, energy losses, and cost, the volume of compute nodes and network switches has to be reduced to a minimum âĂŞ hence the term microserver. Our DOME microserver compute card measures 130 by 7.5 by 65 mm3. The presented switch module is confined to the same area (140 by 55 mm2), yet is deeper (40mm) because of the 630-pin high-speed connector. For 64 ports of 10Gbit Ethernet (10Gbase-KR) our switch consumes about 150W maximal. In addition to the switch ASIC (Intel FM6000 series), the power converters, clock generation, configuration memory and management processor is integrated on a second PCB. The switch management (âĂIJControl PointâĂİ) is implemented in a separate compute node. In the talk options to integrate the management into the switch (same volume as now) will be discussed. Another topic covered is the cooling of the microserver, and of the switch in particular, using (warm) water in the infrastructure and heat pipes on the module.","PeriodicalId":20560,"journal":{"name":"Proceedings of the 2nd International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems","volume":"94 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79613038","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Monobrata Debnath, Dimitris Konstantinou, C. Nicopoulos, G. Dimitrakopoulos, Wei-Ming Lin, Junghee Lee
Implementing cost effective congestion control within the Network-on-Chip (NoC) is a major design challenge. Whenever congestion awareness and/or mitigation is desired, architects typically rely on the use of adaptive routing algorithms, which aim to (intelligently) balance the traffic load throughout the NoC. Nevertheless, the hardware cost incurred by such solutions is quite considerable, since it entails the collection/propagation of traffic-related information and the provisioning of deadlock freedom guarantees. In this paper, we explore the potential of simultaneous edge and in-network traffic throttling, as a low-cost alternative to adaptive routing techniques. Without any reliance on adaptivity by the routing algorithm, combined throttling is demonstrated to yield better (in most cases) throughput improvements than state-of-the-art adaptive routing algorithms, but at a significantly lower cost.
{"title":"Low-cost congestion management in networks-on-chip using edge and in-network traffic throttling","authors":"Monobrata Debnath, Dimitris Konstantinou, C. Nicopoulos, G. Dimitrakopoulos, Wei-Ming Lin, Junghee Lee","doi":"10.1145/3073763.3073764","DOIUrl":"https://doi.org/10.1145/3073763.3073764","url":null,"abstract":"Implementing cost effective congestion control within the Network-on-Chip (NoC) is a major design challenge. Whenever congestion awareness and/or mitigation is desired, architects typically rely on the use of adaptive routing algorithms, which aim to (intelligently) balance the traffic load throughout the NoC. Nevertheless, the hardware cost incurred by such solutions is quite considerable, since it entails the collection/propagation of traffic-related information and the provisioning of deadlock freedom guarantees. In this paper, we explore the potential of simultaneous edge and in-network traffic throttling, as a low-cost alternative to adaptive routing techniques. Without any reliance on adaptivity by the routing algorithm, combined throttling is demonstrated to yield better (in most cases) throughput improvements than state-of-the-art adaptive routing algorithms, but at a significantly lower cost.","PeriodicalId":20560,"journal":{"name":"Proceedings of the 2nd International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems","volume":"131 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-01-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74798889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 2nd International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems","authors":"","doi":"10.1145/3073763","DOIUrl":"https://doi.org/10.1145/3073763","url":null,"abstract":"","PeriodicalId":20560,"journal":{"name":"Proceedings of the 2nd International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems","volume":"42 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2017-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76329148","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}