Pub Date : 2023-12-28DOI: 10.1007/s11432-022-3815-4
Zehao Wu, Huacheng Zhou, Baozhu Guo, Feiqi Deng
This paper investigates the convergence, noise-tolerance, and filtering performance of a tracking differentiator in the presence of multiple stochastic disturbances for the first time. We consider a general case wherein the input signal is corrupted by additive colored noise, and the tracking differentiator is disturbed by additive colored noise and white noise. The tracking differentiator is shown to track the input signal and its generalized derivatives in the mean square sense. Further, the almost sure convergence can be achieved when the stochastic noise affecting the input signal is vanishing. Herein, numerical simulations are performed to validate the theoretical results.
{"title":"On the convergence of tracking differentiator with multiple stochastic disturbances","authors":"Zehao Wu, Huacheng Zhou, Baozhu Guo, Feiqi Deng","doi":"10.1007/s11432-022-3815-4","DOIUrl":"https://doi.org/10.1007/s11432-022-3815-4","url":null,"abstract":"<p>This paper investigates the convergence, noise-tolerance, and filtering performance of a tracking differentiator in the presence of multiple stochastic disturbances for the first time. We consider a general case wherein the input signal is corrupted by additive colored noise, and the tracking differentiator is disturbed by additive colored noise and white noise. The tracking differentiator is shown to track the input signal and its generalized derivatives in the mean square sense. Further, the almost sure convergence can be achieved when the stochastic noise affecting the input signal is vanishing. Herein, numerical simulations are performed to validate the theoretical results.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"22 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139398523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper proposes a novel modulated symbols-based one-time pad (SOTP) secure transmission scheme using physical layer keys. Unlike classical physical layer key generation and exclusive OR (XOR) encryption in the discrete binary space, we design a framework for modulated symbols-based one-time pad (OTP) encryption, where the cryptographic primitive and mathematical model of SOTP is given to build a practical cryptographic protocol. Compared with existing physical layer encryption (PLE) schemes, we provide rigorous proof that the framework can meet perfect secrecy and correctness requirements. In addition, we provide a specific scheme of physical layer OTP secure transmission for quadrature amplitude modulation (QAM) and phase-shift keying (PSK) symbols based on physical layer keys. This scheme realizes the unification of bit encryption and symbol encryption, which can adaptively select the quantization level according to the signal-to-noise ratio (SNR) to minimize the symbol error rate (SER). Further, we analyze the performance quantitatively and derive the closed-form expressions of SER, which indicates that the proposed scheme has a lower SER. Finally, simulation results verify that the proposed symbol-wise OTP secure transmission scheme can achieve perfect secrecy and high reliability.
{"title":"Modulated symbol-based one-time pad secure transmission scheme using physical layer keys","authors":"Xiaoyan Hu, Zheng Wan, Kaizhi Huang, Liang Jin, Mengyao Yan, Yajun Chen, Jinmei Yang","doi":"10.1007/s11432-022-3768-8","DOIUrl":"https://doi.org/10.1007/s11432-022-3768-8","url":null,"abstract":"<p>This paper proposes a novel modulated symbols-based one-time pad (SOTP) secure transmission scheme using physical layer keys. Unlike classical physical layer key generation and exclusive OR (XOR) encryption in the discrete binary space, we design a framework for modulated symbols-based one-time pad (OTP) encryption, where the cryptographic primitive and mathematical model of SOTP is given to build a practical cryptographic protocol. Compared with existing physical layer encryption (PLE) schemes, we provide rigorous proof that the framework can meet perfect secrecy and correctness requirements. In addition, we provide a specific scheme of physical layer OTP secure transmission for quadrature amplitude modulation (QAM) and phase-shift keying (PSK) symbols based on physical layer keys. This scheme realizes the unification of bit encryption and symbol encryption, which can adaptively select the quantization level according to the signal-to-noise ratio (SNR) to minimize the symbol error rate (SER). Further, we analyze the performance quantitatively and derive the closed-form expressions of SER, which indicates that the proposed scheme has a lower SER. Finally, simulation results verify that the proposed symbol-wise OTP secure transmission scheme can achieve perfect secrecy and high reliability.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"78 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Code recommendation systems have been widely used in helping developers implement unfamiliar programming tasks. Many existing code recommenders or code search engines can retrieve relevant code rapidly with high accuracy, however, they cannot recommend any code outside similar ones. We propose an approach to predict the functionality of incomplete programming code by using syntactical information, and providing a list of potential functionalities to guess what the developers want, in order to increase the diversity of recommendations. In this paper, we propose a deep learning model called ASTSDL, which uses a sequence-based representation of source code to predict functionality. We extract syntactical information from the abstract syntax tree (AST) of the source code, apply a deep learning model to capture the syntactic and sequential information, and predict the functionality of the source code fragments. The experimental results demonstrate that ASTSDL can effectively predict the functionality of incomplete code with an accuracy above 84% in the top-10 list, even if there is only half of the complete code.
{"title":"ASTSDL: predicting the functionality of incomplete programming code via an AST-sequence-based deep learning model","authors":"Yaoshen Yu, Zhiqiu Huang, Guohua Shen, Weiwei Li, Yichao Shao","doi":"10.1007/s11432-021-3665-1","DOIUrl":"https://doi.org/10.1007/s11432-021-3665-1","url":null,"abstract":"<p>Code recommendation systems have been widely used in helping developers implement unfamiliar programming tasks. Many existing code recommenders or code search engines can retrieve relevant code rapidly with high accuracy, however, they cannot recommend any code outside similar ones. We propose an approach to predict the functionality of incomplete programming code by using syntactical information, and providing a list of potential functionalities to guess what the developers want, in order to increase the diversity of recommendations. In this paper, we propose a deep learning model called ASTSDL, which uses a sequence-based representation of source code to predict functionality. We extract syntactical information from the abstract syntax tree (AST) of the source code, apply a deep learning model to capture the syntactic and sequential information, and predict the functionality of the source code fragments. The experimental results demonstrate that ASTSDL can effectively predict the functionality of incomplete code with an accuracy above 84% in the top-10 list, even if there is only half of the complete code.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"78 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-27DOI: 10.1007/s11432-023-3847-x
Yan Tian, Hanshi Fu, Hao Wang, Yuqi Liu, Zhaocheng Xu, Hong Chen, Jianyuan Li, Ruili Wang
Abstract
Orthodontic treatment monitoring involves using current images and previous 3D models to estimate the relative position of individual teeth before and after orthodontic treatment. This process differs from image-based object 6D pose estimation due to the gingiva deformation and varying pose offsets for each tooth during treatment. Motivated by the fact that the poses of molars remain relatively fixed in implicit orthodontics, we design an approach that employs multiview pose evaluation and bidirectional temporal propagation for jaw pose estimation and then employs an iteration-based method for tooth alignment. To handle changes in tooth appearance or location with weak texture across frames, we also introduce an instance propagation module that leverages positional and semantic information to explore instance relations in the temporal domain. We evaluated the performance of our approach using both the Shining3D tooth pose dataset and the Aoralscan3 tooth registration dataset. Our experimental results demonstrate remarkable accuracy improvements compared with existing methods.
{"title":"RGB oralscan video-based orthodontic treatment monitoring","authors":"Yan Tian, Hanshi Fu, Hao Wang, Yuqi Liu, Zhaocheng Xu, Hong Chen, Jianyuan Li, Ruili Wang","doi":"10.1007/s11432-023-3847-x","DOIUrl":"https://doi.org/10.1007/s11432-023-3847-x","url":null,"abstract":"<h3>Abstract</h3> <p>Orthodontic treatment monitoring involves using current images and previous 3D models to estimate the relative position of individual teeth before and after orthodontic treatment. This process differs from image-based object 6D pose estimation due to the gingiva deformation and varying pose offsets for each tooth during treatment. Motivated by the fact that the poses of molars remain relatively fixed in implicit orthodontics, we design an approach that employs multiview pose evaluation and bidirectional temporal propagation for jaw pose estimation and then employs an iteration-based method for tooth alignment. To handle changes in tooth appearance or location with weak texture across frames, we also introduce an instance propagation module that leverages positional and semantic information to explore instance relations in the temporal domain. We evaluated the performance of our approach using both the Shining3D tooth pose dataset and the Aoralscan3 tooth registration dataset. Our experimental results demonstrate remarkable accuracy improvements compared with existing methods.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"29 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065975","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-27DOI: 10.1007/s11432-022-3697-x
Jie Wang, Jiamin Li, Pengcheng Zhu, Dongming Wang, Hongbiao Zhang, Yue Hao, Bin Sheng
Network-assisted full-duplex (NAFD) cell-free distributed massive multiple-input multiple-output (MIMO) systems enable uplink (UL) and downlink (DL) communications within the same time-frequency resources, which potentially reduce latency by avoiding the overhead of switching UL/DL modes. However, how to choose UL/DL modes remains an important factor affecting system performance. With the dramatic increase in the number of users and access points (APs), massive access brings significant overhead in the mode selection. Additionally, the different quality of service (QoS) among users also makes the effective utilization of resources difficult. As one of the most promising technologies in sixth-generation (6G), network slicing enables the adaptive configuration of limited UL/DL resources through the resource isolation assisted NAFD technique. Therefore, we propose a slicing capacity-centered scheme. Under this scheme, APs are motivated by slicing requirements and associated slices to form different subsystems. Collaborative mode selection and resource allocation are performed within each subsystem to reduce overhead and improve resource utilization. To implement this scheme efficiently, a double-layer deep reinforcement learning (DRL) mechanism is used to realize the joint optimization of mode selection and resource allocation. Simulation results show that the slicing capacity-centered scheme can effectively improve resource utilization and reduce overhead.
{"title":"Slicing capacity-centered mode selection and resource optimization for network-assisted full-duplex cell-free distributed massive MIMO systems","authors":"Jie Wang, Jiamin Li, Pengcheng Zhu, Dongming Wang, Hongbiao Zhang, Yue Hao, Bin Sheng","doi":"10.1007/s11432-022-3697-x","DOIUrl":"https://doi.org/10.1007/s11432-022-3697-x","url":null,"abstract":"<p>Network-assisted full-duplex (NAFD) cell-free distributed massive multiple-input multiple-output (MIMO) systems enable uplink (UL) and downlink (DL) communications within the same time-frequency resources, which potentially reduce latency by avoiding the overhead of switching UL/DL modes. However, how to choose UL/DL modes remains an important factor affecting system performance. With the dramatic increase in the number of users and access points (APs), massive access brings significant overhead in the mode selection. Additionally, the different quality of service (QoS) among users also makes the effective utilization of resources difficult. As one of the most promising technologies in sixth-generation (6G), network slicing enables the adaptive configuration of limited UL/DL resources through the resource isolation assisted NAFD technique. Therefore, we propose a slicing capacity-centered scheme. Under this scheme, APs are motivated by slicing requirements and associated slices to form different subsystems. Collaborative mode selection and resource allocation are performed within each subsystem to reduce overhead and improve resource utilization. To implement this scheme efficiently, a double-layer deep reinforcement learning (DRL) mechanism is used to realize the joint optimization of mode selection and resource allocation. Simulation results show that the slicing capacity-centered scheme can effectively improve resource utilization and reduce overhead.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"206 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065187","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, the increased high-rate wireless services (5G and 6G) and reliable sensing capabilities (automotive radar, air traffic control, geophysical monitoring), have led to more serious spectrum congestion. Conventionally, radars and communication systems are often seen to be “competing” for the same resources. In this scenario, an integrated platform for radar and communication systems seems to be an optimal solution to address the problem. In this paper, we propose a preamble waveform design based on zero-correlation zone (ZCZ) and zero odd-correlation zone (ZOCZ) sequence sets, which are Doppler resilient in multiple-input, multiple-output (MIMO) radar. The preamble waveforms can be simultaneously used for channel estimation and radar sensing in RadCom systems. The ambiguity function of the proposed waveforms displays low/zero sidelobes within a time-delay correlation zone. Finally, we give some numerical results to show the efficiency of the proposed waveforms in radar sensing and channel estimation.
{"title":"Doppler-resilient waveform design in integrated MIMO radar-communication systems","authors":"Zhengchun Zhou, Bing Liu, Bingsheng Shen, Avik Ranjan Adhikary, Pingzhi Fan","doi":"10.1007/s11432-022-3733-2","DOIUrl":"https://doi.org/10.1007/s11432-022-3733-2","url":null,"abstract":"<p>In recent years, the increased high-rate wireless services (5G and 6G) and reliable sensing capabilities (automotive radar, air traffic control, geophysical monitoring), have led to more serious spectrum congestion. Conventionally, radars and communication systems are often seen to be “competing” for the same resources. In this scenario, an integrated platform for radar and communication systems seems to be an optimal solution to address the problem. In this paper, we propose a preamble waveform design based on zero-correlation zone (ZCZ) and zero odd-correlation zone (ZOCZ) sequence sets, which are Doppler resilient in multiple-input, multiple-output (MIMO) radar. The preamble waveforms can be simultaneously used for channel estimation and radar sensing in RadCom systems. The ambiguity function of the proposed waveforms displays low/zero sidelobes within a time-delay correlation zone. Finally, we give some numerical results to show the efficiency of the proposed waveforms in radar sensing and channel estimation.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"9 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
High-performance computing and deep learning domains have been motivating the design of domain-specific processors. Although these processors can provide promising computation capability, they are notorious for exotic programming paradigms. To improve programming productivity and fully exploit the performance potential of these processors, domain-specific compilers (DSCs) have been proposed. However, building DSCs for emerging processors requires tremendous engineering efforts because the commonly used compilation stack is difficult to be reused. Owing to the advent of multilevel intermediate representation (MLIR), DSC developers can leverage reusable infrastructure to extend their customized functionalities without rebuilding the entire compilation stack. In this paper, we further demonstrate the effectiveness of MLIR by extending its reusable infrastructure to embrace a heterogeneous many-core processor (Sunway processor). In particular, we design a new Sunway dialect and corresponding backend for the Sunway processor, fully exploiting its architectural advantage and hiding its programming complexity. To show the ease of building a DSC, we leverage the Sunway dialect and existing MLIR dialects to build a stencil compiler for the Sunway processor. The experimental results show that our stencil compiler, built with a reusable approach, can even perform better than state-of-the-art stencil compilers.
{"title":"Building a domain-specific compiler for emerging processors with a reusable approach","authors":"Mingzhen Li, Yi Liu, Bangduo Chen, Hailong Yang, Zhongzhi Luan, Depei Qian","doi":"10.1007/s11432-022-3727-6","DOIUrl":"https://doi.org/10.1007/s11432-022-3727-6","url":null,"abstract":"<p>High-performance computing and deep learning domains have been motivating the design of domain-specific processors. Although these processors can provide promising computation capability, they are notorious for exotic programming paradigms. To improve programming productivity and fully exploit the performance potential of these processors, domain-specific compilers (DSCs) have been proposed. However, building DSCs for emerging processors requires tremendous engineering efforts because the commonly used compilation stack is difficult to be reused. Owing to the advent of multilevel intermediate representation (MLIR), DSC developers can leverage reusable infrastructure to extend their customized functionalities without rebuilding the entire compilation stack. In this paper, we further demonstrate the effectiveness of MLIR by extending its reusable infrastructure to embrace a heterogeneous many-core processor (Sunway processor). In particular, we design a new Sunway dialect and corresponding backend for the Sunway processor, fully exploiting its architectural advantage and hiding its programming complexity. To show the ease of building a DSC, we leverage the Sunway dialect and existing MLIR dialects to build a stencil compiler for the Sunway processor. The experimental results show that our stencil compiler, built with a reusable approach, can even perform better than state-of-the-art stencil compilers.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"18 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139105126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-27DOI: 10.1007/s11432-022-3736-y
Weihua Li, Huaguang Zhang, Juan Zhang, Rui Wang
This paper is dedicated to addressing the time-varying output formation-containment tracking (TVOFCT) problem for heterogeneous linear multiagent systems (MASs) with multiple types of disturbances under directed communication topology. The agents are divided into tracking leaders, formation leaders, and followers. In the output space, the formation leaders are required to form an expected geometric formation while tracking the tracking leader, and the followers are required to enter the convex hull spanned by formation leaders. First, a disturbance observer is designed to observe the disturbance with deterministic dynamics. Then, by combining adaptive technology with event-triggered technology, an independent fully distributed dynamic event-triggered (FDDET) compensator is designed to estimate the state of the tracking leader. The main advantages of this compensator lie in the following three aspects: (1) the adaptive weight does not increase unboundedly; (2) the interval time between events is expanded; (3) the design does not rely on any global information. Similar to the design of this compensator, an independent FDDET convex hull observer is further designed to observe the convex hull spanned by formation leaders. Subsequently, based on the designed disturbance observer, compensator, and convex hull observer, the adaptive disturbance rejection control input which can make the MASs achieve TVOFCT is formulated. Finally, a numerical simulation is provided to clearly verify the validity of the theoretical results.
本文致力于在定向通信拓扑结构下,解决具有多种扰动的异构线性多代理系统(MAS)的时变输出编队-控制跟踪(TVOFCT)问题。代理分为跟踪领导者、编队领导者和跟随者。在输出空间中,编队领导者需要在跟踪领导者的同时形成预期的几何编队,跟随者则需要进入编队领导者所跨过的凸壳。首先,设计一个扰动观测器来观测具有确定性动态的扰动。然后,通过将自适应技术与事件触发技术相结合,设计出一个独立的全分布式动态事件触发(FDDET)补偿器来估计跟踪领导者的状态。这种补偿器的主要优势在于以下三个方面:(1)自适应权重不会无限制地增加;(2)事件之间的间隔时间得到了扩展;(3)设计不依赖于任何全局信息。与该补偿器的设计类似,还进一步设计了一个独立的 FDDET 凸壳观测器,用于观测编队首领所跨过的凸壳。随后,基于所设计的扰动观测器、补偿器和凸壳观测器,制定了可使 MAS 实现 TVOFCT 的自适应扰动抑制控制输入。最后,通过数值模拟清楚地验证了理论结果的正确性。
{"title":"Fully distributed dynamic event-triggered formation-containment tracking for multiagent systems with multiple types of disturbances","authors":"Weihua Li, Huaguang Zhang, Juan Zhang, Rui Wang","doi":"10.1007/s11432-022-3736-y","DOIUrl":"https://doi.org/10.1007/s11432-022-3736-y","url":null,"abstract":"<p>This paper is dedicated to addressing the time-varying output formation-containment tracking (TVOFCT) problem for heterogeneous linear multiagent systems (MASs) with multiple types of disturbances under directed communication topology. The agents are divided into tracking leaders, formation leaders, and followers. In the output space, the formation leaders are required to form an expected geometric formation while tracking the tracking leader, and the followers are required to enter the convex hull spanned by formation leaders. First, a disturbance observer is designed to observe the disturbance with deterministic dynamics. Then, by combining adaptive technology with event-triggered technology, an independent fully distributed dynamic event-triggered (FDDET) compensator is designed to estimate the state of the tracking leader. The main advantages of this compensator lie in the following three aspects: (1) the adaptive weight does not increase unboundedly; (2) the interval time between events is expanded; (3) the design does not rely on any global information. Similar to the design of this compensator, an independent FDDET convex hull observer is further designed to observe the convex hull spanned by formation leaders. Subsequently, based on the designed disturbance observer, compensator, and convex hull observer, the adaptive disturbance rejection control input which can make the MASs achieve TVOFCT is formulated. Finally, a numerical simulation is provided to clearly verify the validity of the theoretical results.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"98 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139065190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-22DOI: 10.1007/s11432-022-3657-3
Jiazhi Jiang, Dan Huang, Hu Chen, Yutong Lu, Xiangke Liao
High-throughput computing (HTC) is a computing paradigm that aims to accomplish jobs by easily breaking them into smaller, independent components. However, it requires a large amount of computing power for a long time. Most existing HTC frameworks are job-oriented without support for coscheduling with hardware architecture and task-level execution. Also, most of the frameworks reach a limited scale, and their usability needs further improvement. Herein, we present HTDcr, a job execution framework for the HTC on supercomputers. This study aims to improve the throughput, task dispatching, and usability of the framework. In detail, the throughput optimizations include a sophisticated designed task management system, a hierarchical scheduler, and the co-optimization of the task-scheduling strategy with the application and hardware characteristics. The optimizations for usability include a programable execution workflow, mechanisms for more robust and reliable service qualities, and a fine-grained resource allocation system for the colocation of multiple jobs. According to our evaluations, HTDcr can achieve outstanding scalability and high throughput on large-scale clusters for the HTC workload. We evaluate HTDcr with several microbenchmarks and real-world applications on Tianhe-2 and Sunway TaihuLight to demonstrate its effects on existing design mechanisms. For instance, the task scheduling for two real-world applications integrated with the application and hardware characteristics achieves 1.7× and 1.9× speedups over the basic task-scheduling strategy.
{"title":"HTDcr: a job execution framework for high-throughput computing on supercomputers","authors":"Jiazhi Jiang, Dan Huang, Hu Chen, Yutong Lu, Xiangke Liao","doi":"10.1007/s11432-022-3657-3","DOIUrl":"https://doi.org/10.1007/s11432-022-3657-3","url":null,"abstract":"<p>High-throughput computing (HTC) is a computing paradigm that aims to accomplish jobs by easily breaking them into smaller, independent components. However, it requires a large amount of computing power for a long time. Most existing HTC frameworks are job-oriented without support for coscheduling with hardware architecture and task-level execution. Also, most of the frameworks reach a limited scale, and their usability needs further improvement. Herein, we present HTDcr, a job execution framework for the HTC on supercomputers. This study aims to improve the throughput, task dispatching, and usability of the framework. In detail, the throughput optimizations include a sophisticated designed task management system, a hierarchical scheduler, and the co-optimization of the task-scheduling strategy with the application and hardware characteristics. The optimizations for usability include a programable execution workflow, mechanisms for more robust and reliable service qualities, and a fine-grained resource allocation system for the colocation of multiple jobs. According to our evaluations, HTDcr can achieve outstanding scalability and high throughput on large-scale clusters for the HTC workload. We evaluate HTDcr with several microbenchmarks and real-world applications on Tianhe-2 and Sunway TaihuLight to demonstrate its effects on existing design mechanisms. For instance, the task scheduling for two real-world applications integrated with the application and hardware characteristics achieves 1.7× and 1.9× speedups over the basic task-scheduling strategy.</p>","PeriodicalId":21618,"journal":{"name":"Science China Information Sciences","volume":"10 1","pages":""},"PeriodicalIF":8.8,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139056118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}