Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129397
Lennart Weingarten, Alireza Mahzoon, Mehran Goli, R. Drechsler
Formal verification is an important task to ensure the correctness of a circuit. In the last 30 years, several formal methods have been proposed to verify various architectures. However, the space and time complexities of these methods are usually unknown, particularly, when it comes to complex designs, e.g., processors. As a result, there is always unpredictability in the performance of the verification tool. If we prove that a formal method has polynomial space and time complexities, we can successfully resolve the unpredictability problem and ensure the scalability of the method.In this paper, we propose a Polynomial Formal Verification (PFV) method based on Binary Decision Diagrams (BDDs) to fully verify a RISC-V processor. We take advantage of partial simulation to extract the hardware related to each instruction. Then, we create the reference BDD for each instruction with respect to its size and function. Finally, we run a symbolic simulation for each hardware instruction and compare it with the reference model. We prove that the whole verification task can be carried out in polynomial space and time. The experiments demonstrate that the PFV of a RISC-V RV32I processor can be performed in less than one second.
{"title":"Polynomial Formal Verification of a Processor: A RISC-V Case Study","authors":"Lennart Weingarten, Alireza Mahzoon, Mehran Goli, R. Drechsler","doi":"10.1109/ISQED57927.2023.10129397","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129397","url":null,"abstract":"Formal verification is an important task to ensure the correctness of a circuit. In the last 30 years, several formal methods have been proposed to verify various architectures. However, the space and time complexities of these methods are usually unknown, particularly, when it comes to complex designs, e.g., processors. As a result, there is always unpredictability in the performance of the verification tool. If we prove that a formal method has polynomial space and time complexities, we can successfully resolve the unpredictability problem and ensure the scalability of the method.In this paper, we propose a Polynomial Formal Verification (PFV) method based on Binary Decision Diagrams (BDDs) to fully verify a RISC-V processor. We take advantage of partial simulation to extract the hardware related to each instruction. Then, we create the reference BDD for each instruction with respect to its size and function. Finally, we run a symbolic simulation for each hardware instruction and compare it with the reference model. We prove that the whole verification task can be carried out in polynomial space and time. The experiments demonstrate that the PFV of a RISC-V RV32I processor can be performed in less than one second.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114546068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129374
James Geist, Travis Meade, Shaojie Zhang, Yier Jin
Algorithmic analysis of gate level netlists has become an important technique in hardware security. Algorithms can help detect malicious hardware injected into a design, or lock a design against reverse engineering or malicious modification. Many analysis tools have come from the research and commercial communities; however, it is currently the job of the analyst to make these tools work together and interpret the results. Typically tools are text-based, and require error-prone editing of input files in different formats. The analyst must interpret textual results, and sometimes transform them into other formats for use in third party visualization tools. These tasks are repetitive overhead that take time and effort that could better be spent on investigating the netlist. In this paper we introduce NetViz, a visual hardware security environment. NetViz is a meta-tool which combines other analysis tools, automates the task of transferring data between them, and helps with interpretation of results by providing graphical representations of the data.
{"title":"NetViz: A Tool for Netlist Security Visualization","authors":"James Geist, Travis Meade, Shaojie Zhang, Yier Jin","doi":"10.1109/ISQED57927.2023.10129374","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129374","url":null,"abstract":"Algorithmic analysis of gate level netlists has become an important technique in hardware security. Algorithms can help detect malicious hardware injected into a design, or lock a design against reverse engineering or malicious modification. Many analysis tools have come from the research and commercial communities; however, it is currently the job of the analyst to make these tools work together and interpret the results. Typically tools are text-based, and require error-prone editing of input files in different formats. The analyst must interpret textual results, and sometimes transform them into other formats for use in third party visualization tools. These tasks are repetitive overhead that take time and effort that could better be spent on investigating the netlist. In this paper we introduce NetViz, a visual hardware security environment. NetViz is a meta-tool which combines other analysis tools, automates the task of transferring data between them, and helps with interpretation of results by providing graphical representations of the data.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124168425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129309
Bikrant Das Sharma, Abdul Rahman Ismail, Chris Meyers
USB has been the dominant external I/O in computing systems over the past two decades. With the increased adoption of USB-C with high data rates, USB hubs are becoming more popular. Existing power-saving mechanisms do not save much power in USB hubs when there is a steady bandwidth demand from devices. In this paper, we demonstrate significant power savings with a proactive scheduling policy for hubs. Our approach includes the introduction of a shallow U1/CL1 low-power state, resulting in better overall power savings due to the reduced entry and exit times to U1/CL1. Our results demonstrate power savings of tens of watts by increasing the scheduling interval up to the minimum latency tolerance across all devices connected to that hub. As USB moves to USB4 and hubs are used to connect to higher bandwidth devices, these power savings will become even more pronounced.
{"title":"Power Savings in USB Hubs Through A Proactive Scheduling Strategy","authors":"Bikrant Das Sharma, Abdul Rahman Ismail, Chris Meyers","doi":"10.1109/ISQED57927.2023.10129309","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129309","url":null,"abstract":"USB has been the dominant external I/O in computing systems over the past two decades. With the increased adoption of USB-C with high data rates, USB hubs are becoming more popular. Existing power-saving mechanisms do not save much power in USB hubs when there is a steady bandwidth demand from devices. In this paper, we demonstrate significant power savings with a proactive scheduling policy for hubs. Our approach includes the introduction of a shallow U1/CL1 low-power state, resulting in better overall power savings due to the reduced entry and exit times to U1/CL1. Our results demonstrate power savings of tens of watts by increasing the scheduling interval up to the minimum latency tolerance across all devices connected to that hub. As USB moves to USB4 and hubs are used to connect to higher bandwidth devices, these power savings will become even more pronounced.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127204392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129356
Andrea Guerrieri, Gabriel Da Silva Marques, F. Regazzoni, A. Upegui
With the performance promises of quantum computers, standard encryption algorithms can be defeated. For this reason, a set of new quantum-resistant algorithms have been proposed and submitted for a standardization contest initiated by NIST. While the submission requirement was ANSI C for the reference implementation, NIST encouraged providing software implementations optimized for different target platforms, such as high-performance CPUs, embedded microcontrollers, and FPGAs. Yet, none of the algorithms submitted any FPGA-optimized code, due to the large and expensive development time required for coding at RTL. High-Level synthesis (HLS) covers the gap by creating automatically hardware code for FPGA out of C/C++. However, the quality of results is suboptimal due to the limitation imposed by the inadequacy of source code for HLS. In this paper, we propose a version of Saber’s code optimized for FPGA targets. We show how we detected and improved the performance of the reference code, achieving competitive results compared to the hand-made RTL-based designs.
{"title":"H-Saber: An FPGA-Optimized Version for Designing Fast and Efficient Post-Quantum Cryptography Hardware Accelerators","authors":"Andrea Guerrieri, Gabriel Da Silva Marques, F. Regazzoni, A. Upegui","doi":"10.1109/ISQED57927.2023.10129356","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129356","url":null,"abstract":"With the performance promises of quantum computers, standard encryption algorithms can be defeated. For this reason, a set of new quantum-resistant algorithms have been proposed and submitted for a standardization contest initiated by NIST. While the submission requirement was ANSI C for the reference implementation, NIST encouraged providing software implementations optimized for different target platforms, such as high-performance CPUs, embedded microcontrollers, and FPGAs. Yet, none of the algorithms submitted any FPGA-optimized code, due to the large and expensive development time required for coding at RTL. High-Level synthesis (HLS) covers the gap by creating automatically hardware code for FPGA out of C/C++. However, the quality of results is suboptimal due to the limitation imposed by the inadequacy of source code for HLS. In this paper, we propose a version of Saber’s code optimized for FPGA targets. We show how we detected and improved the performance of the reference code, achieving competitive results compared to the hand-made RTL-based designs.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"68 12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129879871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
With the demand for high performance and density, silicon interposer-based three-dimensional integrated circuit (3DIC) can be one of promising solutions for these requirements. However, simultaneously switching noise (SSN) will cause voltage fluctuation and hence performance degradation and logic failure might occur. Our work proposes an efficient Simulated Annealing (SA) based algorithm to perform decap placement automatically on the interposer. In our solution, target impedance can be achieved within certain frequency range. Results show that number of decaps as well as impedance of PDN are minimized to meet the requirement.
{"title":"On-Interposer Decoupling Capacitors Placement for Interposer-based 3DIC","authors":"Po-Yang Chen, Chang-Yun Liu, Hung-Ming Chen, Po-Tsang Huang","doi":"10.1109/ISQED57927.2023.10129288","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129288","url":null,"abstract":"With the demand for high performance and density, silicon interposer-based three-dimensional integrated circuit (3DIC) can be one of promising solutions for these requirements. However, simultaneously switching noise (SSN) will cause voltage fluctuation and hence performance degradation and logic failure might occur. Our work proposes an efficient Simulated Annealing (SA) based algorithm to perform decap placement automatically on the interposer. In our solution, target impedance can be achieved within certain frequency range. Results show that number of decaps as well as impedance of PDN are minimized to meet the requirement.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129965299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129396
P. Ghosh, V. N. D. Mai, Aditya Chopra, Baljinder Sood
Modern SoCs are designed by integrating several IPs using various interconnect layers(NoC). Although the exact functionality of the device is of the highest importance, the correct behavior in terms of performance is a crucial factor. To gain a competitive edge in the market, safety-critical devices (such as automotive devices) must meet various performance-related requirements. In this paper, we propose a methodology that includes the automatic addition of expected performance numbers of each performance test in the testbench. The definition of two proposed performance metrics, developing a proposed performance scoreboard to implement a self-check mechanism in UVM testbench, and regression management of several hundred performance verification test cases run on SoC RTL. The proposed methodology has been applied to multiple commercial SoCs at the chip and sub-system levels and has detected several performance design flaws during the initial design phase. It has been improved the productivity of the design team also. In complicated SoCs, it has been proven helpful in the absence of any established standard technique for performance verification at the SoC level.
{"title":"Self-Checking Performance Verification Methodology for Complex SoCs","authors":"P. Ghosh, V. N. D. Mai, Aditya Chopra, Baljinder Sood","doi":"10.1109/ISQED57927.2023.10129396","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129396","url":null,"abstract":"Modern SoCs are designed by integrating several IPs using various interconnect layers(NoC). Although the exact functionality of the device is of the highest importance, the correct behavior in terms of performance is a crucial factor. To gain a competitive edge in the market, safety-critical devices (such as automotive devices) must meet various performance-related requirements. In this paper, we propose a methodology that includes the automatic addition of expected performance numbers of each performance test in the testbench. The definition of two proposed performance metrics, developing a proposed performance scoreboard to implement a self-check mechanism in UVM testbench, and regression management of several hundred performance verification test cases run on SoC RTL. The proposed methodology has been applied to multiple commercial SoCs at the chip and sub-system levels and has detected several performance design flaws during the initial design phase. It has been improved the productivity of the design team also. In complicated SoCs, it has been proven helpful in the absence of any established standard technique for performance verification at the SoC level.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133076828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129340
Mi-Sung Han, Jiwan Kim, Donggeon Kim, Hyunuk Jeong, Gilho Jung, Myeongwon Oh, Hyundong Lee, Yunjeong Go, Hyunwoo Kim, Jongbeom Kim, Taigon Song
graph convolutional network (GCN) is a type of neural network that inference new nodes based on the connectivity of the graphs. GCN requires high-calculation volume for processing, similar to other neural networks requiring significant calculation. In this paper, we propose a new hardware architecture for GCN that tackles the problem of wasted cycles during processing. We propose a new scheduler module that reduces memory access through aggregation and an optimized systolic array with improved delay. We compare our study with the state-of-the-art GCN accelerator and show outperforming results.
{"title":"HFGCN: High-speed and Fully-optimized GCN Accelerator","authors":"Mi-Sung Han, Jiwan Kim, Donggeon Kim, Hyunuk Jeong, Gilho Jung, Myeongwon Oh, Hyundong Lee, Yunjeong Go, Hyunwoo Kim, Jongbeom Kim, Taigon Song","doi":"10.1109/ISQED57927.2023.10129340","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129340","url":null,"abstract":"graph convolutional network (GCN) is a type of neural network that inference new nodes based on the connectivity of the graphs. GCN requires high-calculation volume for processing, similar to other neural networks requiring significant calculation. In this paper, we propose a new hardware architecture for GCN that tackles the problem of wasted cycles during processing. We propose a new scheduler module that reduces memory access through aggregation and an optimized systolic array with improved delay. We compare our study with the state-of-the-art GCN accelerator and show outperforming results.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131031733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129347
Ruben Dominguez, Wen Zhang, Hongzhi Xu, Pablo Rangel, Chen Pan
Recent advancements in Internet of Things (IoT) technology draws attention to Energy Harvesting (EH) systems as a promising energy-efficient solution to the limited sustainability in IoT edge devices. However, due to the weak and unstable nature of the ambient energy source, EH nodes are vulnerable to frequent power outages. Consequently, such outages will, unfortunately, reset the volatile time module onboard, which results in synchronization problems. To enable intermittent communication under energy harvesting scenarios with limited and unstable power supply, instead of merely minimizing the occurrence of a power outage, this work will also enable a smart and swift "self-healing" MAC protocol for desynchronized EH IoT devices to synchronize its timeline with the rest of the network for communication. To demonstrate the effectiveness, we will take the popular Long-range Wide Area Network (LoRaWAN) communication protocol as the backbone for upgrading, testing, and evaluation. The experiments conducted on LoRa Nodes demonstrate the effectiveness of the proposed techniques.
{"title":"ISSAC: An Self-organizing and Self-healing MAC Design for Intermittent Communication Systems","authors":"Ruben Dominguez, Wen Zhang, Hongzhi Xu, Pablo Rangel, Chen Pan","doi":"10.1109/ISQED57927.2023.10129347","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129347","url":null,"abstract":"Recent advancements in Internet of Things (IoT) technology draws attention to Energy Harvesting (EH) systems as a promising energy-efficient solution to the limited sustainability in IoT edge devices. However, due to the weak and unstable nature of the ambient energy source, EH nodes are vulnerable to frequent power outages. Consequently, such outages will, unfortunately, reset the volatile time module onboard, which results in synchronization problems. To enable intermittent communication under energy harvesting scenarios with limited and unstable power supply, instead of merely minimizing the occurrence of a power outage, this work will also enable a smart and swift \"self-healing\" MAC protocol for desynchronized EH IoT devices to synchronize its timeline with the rest of the network for communication. To demonstrate the effectiveness, we will take the popular Long-range Wide Area Network (LoRaWAN) communication protocol as the backbone for upgrading, testing, and evaluation. The experiments conducted on LoRa Nodes demonstrate the effectiveness of the proposed techniques.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132644178","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129369
N. Aswathy, H. Kapoor
Emerging non-volatile memories overcome the bottlenecks associated with traditional DRAM memories, such as low density and high energy. The high operating voltages required for such non-volatile memories make them vulnerable to Biased Temperature Instability (BTI) aging. The aging of a device can be controlled by the de-stress operation, where the stress voltage applied to the device is removed for a small duration. Performing de-stress in regular intervals helps to partially recover from age degradation. Such an interval-based de-stress can affect the service of regular requests and thus can hamper the system performance.To control the aging of PCM memories while maintaining the system performance, we propose AGRAS: age and memory request-rate aware scheduling method to schedule de-stress as well as regular requests. AGRAS schedules the de-stress operation only when the incoming request rate is not very high, thus controlling performance degradation. Additionally, it makes sure that in events of a prolonged high request rate, the de-stress gets scheduled in order to control device age degradation. The proposal helps to improve the system performance while minimizing the age degradation compared to the setup, which de-stresses at regular intervals.
{"title":"AGRAS: Aging and memory request rate aware scheduler for PCM memories","authors":"N. Aswathy, H. Kapoor","doi":"10.1109/ISQED57927.2023.10129369","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129369","url":null,"abstract":"Emerging non-volatile memories overcome the bottlenecks associated with traditional DRAM memories, such as low density and high energy. The high operating voltages required for such non-volatile memories make them vulnerable to Biased Temperature Instability (BTI) aging. The aging of a device can be controlled by the de-stress operation, where the stress voltage applied to the device is removed for a small duration. Performing de-stress in regular intervals helps to partially recover from age degradation. Such an interval-based de-stress can affect the service of regular requests and thus can hamper the system performance.To control the aging of PCM memories while maintaining the system performance, we propose AGRAS: age and memory request-rate aware scheduling method to schedule de-stress as well as regular requests. AGRAS schedules the de-stress operation only when the incoming request rate is not very high, thus controlling performance degradation. Additionally, it makes sure that in events of a prolonged high request rate, the de-stress gets scheduled in order to control device age degradation. The proposal helps to improve the system performance while minimizing the age degradation compared to the setup, which de-stresses at regular intervals.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115680230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-05DOI: 10.1109/ISQED57927.2023.10129362
Joseph Clemmons, Yu-Fang Jin
Reinforcement learning (RL) has attracted significant research efforts to guide an autonomous vehicle (AV) for a collision-free path due to its advantages in investigating interactions among multiple vehicles and dynamic environments. This study deploys a Deep Q-Network (DQN) based RL algorithm with reward shaping to control an ego AV in an environment with multiple vehicles. Specifically, the state space of the RL algorithm depends on the desired destination, the ego vehicle’s location and orientation, and the location of other vehicles in the system. The training time of the proposed RL algorithm is much shorter than most current image-based algorithms. The RL algorithm also provides an extendable framework to include a varying number of vehicles in the environment and can be easily adapted to different maps without changing the setup of the RL algorithm. Three scenarios were simulated in the Cars Learn to Act (CARLA) simulator to examine the effects of the proposed RL algorithm on guiding the ego AV interacting with multiple vehicles on straight and curvy roads. Our results showed that the ego AV could learn to reach its destination within 5000 episodes for all scenarios tested.
强化学习(RL)由于其在研究多车辆和动态环境之间的相互作用方面的优势，已经吸引了大量的研究工作来指导自动驾驶汽车(AV)寻找无碰撞路径。本研究部署了一种基于深度q网络(DQN)的RL算法，该算法带有奖励整形，用于在多车辆环境中控制自我自动驾驶汽车。具体来说，RL算法的状态空间取决于期望的目的地、自我车辆的位置和方向以及系统中其他车辆的位置。所提出的强化学习算法的训练时间比目前大多数基于图像的算法要短得多。RL算法还提供了一个可扩展的框架，以包括环境中不同数量的车辆，并且可以很容易地适应不同的地图，而无需改变RL算法的设置。在Cars Learn to Act (CARLA)模拟器中模拟了三种场景，以检验RL算法在引导自动驾驶汽车在直线和弯曲道路上与多辆车辆交互时的效果。我们的研究结果表明，在所有测试场景中，自我自动驾驶汽车都可以在5000集内学会到达目的地。
{"title":"Reinforcement Learning-Based Guidance of Autonomous Vehicles","authors":"Joseph Clemmons, Yu-Fang Jin","doi":"10.1109/ISQED57927.2023.10129362","DOIUrl":"https://doi.org/10.1109/ISQED57927.2023.10129362","url":null,"abstract":"Reinforcement learning (RL) has attracted significant research efforts to guide an autonomous vehicle (AV) for a collision-free path due to its advantages in investigating interactions among multiple vehicles and dynamic environments. This study deploys a Deep Q-Network (DQN) based RL algorithm with reward shaping to control an ego AV in an environment with multiple vehicles. Specifically, the state space of the RL algorithm depends on the desired destination, the ego vehicle’s location and orientation, and the location of other vehicles in the system. The training time of the proposed RL algorithm is much shorter than most current image-based algorithms. The RL algorithm also provides an extendable framework to include a varying number of vehicles in the environment and can be easily adapted to different maps without changing the setup of the RL algorithm. Three scenarios were simulated in the Cars Learn to Act (CARLA) simulator to examine the effects of the proposed RL algorithm on guiding the ego AV interacting with multiple vehicles on straight and curvy roads. Our results showed that the ego AV could learn to reach its destination within 5000 episodes for all scenarios tested.","PeriodicalId":315053,"journal":{"name":"2023 24th International Symposium on Quality Electronic Design (ISQED)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115684077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：