As three-dimensional acquisition technologies like LiDAR cameras advance, the need for efficient transmission of 3D point clouds is becoming increasingly important. In this paper, we present a novel semantic communication (SemCom) approach for efficient 3D point cloud transmission. Different from existing methods that rely on downsampling and feature extraction for compression, our approach utilizes a parallel structure to separately extract both global and local information from point clouds. This system is composed of five key components: local semantic encoder, global semantic encoder, channel encoder, channel decoder, and semantic decoder. Our numerical results indicate that this approach surpasses both the traditional Octree compression methodology and alternative deep learning-based strategies in terms of reconstruction quality. Moreover, our system is capable of achieving high-quality point cloud reconstruction under adverse channel conditions, specifically maintaining a reconstruction quality of over 37dB even with severe channel noise.
{"title":"Semantic Communication for Efficient Point Cloud Transmission","authors":"Shangzhuo Xie, Qianqian Yang, Yuyi Sun, Tianxiao Han, Zhaohui Yang, Zhiguo Shi","doi":"arxiv-2409.03319","DOIUrl":"https://doi.org/arxiv-2409.03319","url":null,"abstract":"As three-dimensional acquisition technologies like LiDAR cameras advance, the\u0000need for efficient transmission of 3D point clouds is becoming increasingly\u0000important. In this paper, we present a novel semantic communication (SemCom)\u0000approach for efficient 3D point cloud transmission. Different from existing\u0000methods that rely on downsampling and feature extraction for compression, our\u0000approach utilizes a parallel structure to separately extract both global and\u0000local information from point clouds. This system is composed of five key\u0000components: local semantic encoder, global semantic encoder, channel encoder,\u0000channel decoder, and semantic decoder. Our numerical results indicate that this\u0000approach surpasses both the traditional Octree compression methodology and\u0000alternative deep learning-based strategies in terms of reconstruction quality.\u0000Moreover, our system is capable of achieving high-quality point cloud\u0000reconstruction under adverse channel conditions, specifically maintaining a\u0000reconstruction quality of over 37dB even with severe channel noise.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142214101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Alessio Carpegna, Chiara De Luca, Federico Emanuele Pozzi, Alessandro Savino, Stefano Di Carlo, Giacomo Indiveri, Elisa Donati
Detecting monotonic changes in heart rate (HR) is crucial for early identification of cardiac conditions and health management. This is particularly important for dementia patients, where HR trends can signal stress or agitation. Developing wearable technologies that can perform always-on monitoring of HRs is essential to effectively detect slow changes over extended periods of time. However, designing compact electronic circuits that can monitor and process bio-signals continuously, and that can operate in a low-power regime to ensure long-lasting performance, is still an open challenge. Neuromorphic technology offers an energy-efficient solution for real-time health monitoring. We propose a neuromorphic implementation of a Neural State Machine (NSM) network to encode different health states and switch between them based on the input stimuli. Our focus is on detecting monotonic state switches in electrocardiogram data to identify progressive HR increases. This innovative approach promises significant advancements in continuous health monitoring and management.
{"title":"Neuromorphic Heart Rate Monitors: Neural State Machines for Monotonic Change Detection","authors":"Alessio Carpegna, Chiara De Luca, Federico Emanuele Pozzi, Alessandro Savino, Stefano Di Carlo, Giacomo Indiveri, Elisa Donati","doi":"arxiv-2409.02618","DOIUrl":"https://doi.org/arxiv-2409.02618","url":null,"abstract":"Detecting monotonic changes in heart rate (HR) is crucial for early\u0000identification of cardiac conditions and health management. This is\u0000particularly important for dementia patients, where HR trends can signal stress\u0000or agitation. Developing wearable technologies that can perform always-on\u0000monitoring of HRs is essential to effectively detect slow changes over extended\u0000periods of time. However, designing compact electronic circuits that can\u0000monitor and process bio-signals continuously, and that can operate in a\u0000low-power regime to ensure long-lasting performance, is still an open\u0000challenge. Neuromorphic technology offers an energy-efficient solution for\u0000real-time health monitoring. We propose a neuromorphic implementation of a\u0000Neural State Machine (NSM) network to encode different health states and switch\u0000between them based on the input stimuli. Our focus is on detecting monotonic\u0000state switches in electrocardiogram data to identify progressive HR increases.\u0000This innovative approach promises significant advancements in continuous health\u0000monitoring and management.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"18 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142214103","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Timeline Analysis (TA) is a key part of Timeline Forensics (TF) in Digital Forensics (DF), focusing primarily on examining and analysing temporal digital artefacts such as timestamps, derived from event logs, file metadata, and other related data to correlate events resulting from cyber incidents and reconstruct their chronological timeline. Traditional tools often struggle to efficiently process the vast volume and variety of data acquired during DF investigations and Incident Response (IR) processes. This paper presents a novel framework, GenDFIR, that combines Rule-Based Artificial Intelligence (R-BAI) algorithms with Large Language Models (LLMs) to advance and automate the TA process. Our approach consists of two main stages (1) We use R-BAI to identify and select anomalous digital artefacts based on predefined rules. (2) The selected artefacts are then converted into embeddings for processing by an LLM with the help of a Retrieval-Augmented Generation (RAG) agent. The LLM consequently leverages its capabilities to perform automated TA on the artefacts and predict potential incident scenarios. To validate our framework, we evaluate GenDFIR performance, efficiency, and reliability using various metrics across synthetic cyber incident simulation scenarios. This paper presents a proof of concept, where the findings demonstrate the significant potential of integrating R-BAI and LLMs for TA. This novel approach highlights the power of Generative AI (GenAI), specifically LLMs, and opens new avenues for advanced threat detection and incident reconstruction, representing a significant step forward in the field.
{"title":"Advancing Cyber Incident Timeline Analysis Through Rule Based AI and Large Language Models","authors":"Fatma Yasmine Loumachi, Mohamed Chahine Ghanem","doi":"arxiv-2409.02572","DOIUrl":"https://doi.org/arxiv-2409.02572","url":null,"abstract":"Timeline Analysis (TA) is a key part of Timeline Forensics (TF) in Digital\u0000Forensics (DF), focusing primarily on examining and analysing temporal digital\u0000artefacts such as timestamps, derived from event logs, file metadata, and other\u0000related data to correlate events resulting from cyber incidents and reconstruct\u0000their chronological timeline. Traditional tools often struggle to efficiently\u0000process the vast volume and variety of data acquired during DF investigations\u0000and Incident Response (IR) processes. This paper presents a novel framework,\u0000GenDFIR, that combines Rule-Based Artificial Intelligence (R-BAI) algorithms\u0000with Large Language Models (LLMs) to advance and automate the TA process. Our\u0000approach consists of two main stages (1) We use R-BAI to identify and select\u0000anomalous digital artefacts based on predefined rules. (2) The selected\u0000artefacts are then converted into embeddings for processing by an LLM with the\u0000help of a Retrieval-Augmented Generation (RAG) agent. The LLM consequently\u0000leverages its capabilities to perform automated TA on the artefacts and predict\u0000potential incident scenarios. To validate our framework, we evaluate GenDFIR\u0000performance, efficiency, and reliability using various metrics across synthetic\u0000cyber incident simulation scenarios. This paper presents a proof of concept,\u0000where the findings demonstrate the significant potential of integrating R-BAI\u0000and LLMs for TA. This novel approach highlights the power of Generative AI\u0000(GenAI), specifically LLMs, and opens new avenues for advanced threat detection\u0000and incident reconstruction, representing a significant step forward in the\u0000field.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"10 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142214105","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Minfeng Qi, Qin Wang, Zhipeng Wang, Manvir Schneider, Tianqing Zhu, Shiping Chen, William Knottenbelt, Thomas Hardjono
We present the first Systematization of Knowledge (SoK) on constructing Layer Two (L2) solutions for Bitcoin. We carefully examine a representative subset of ongoing Bitcoin L2 solutions (40 out of 335 extensively investigated cases) and provide a concise yet impactful identification of six classic design patterns through two approaches (i.e., modifying transactions & creating proofs). Notably, we are the first to incorporate the inscription technology (emerged in mid-2023), along with a series of related innovations. We further establish a reference framework that serves as a baseline criterion ideally suited for evaluating the security aspects of Bitcoin L2 solutions, and which can also be extended to broader L2 applications. We apply this framework to evaluate each of the projects we investigated. We find that the inscription-based approaches introduce new functionality (i.e., programability) to Bitcoin systems, whereas existing proof-based solutions primarily address scalability challenges. Our security analysis reveals new attack vectors targeting data/state (availability, verification), assets (withdrawal, recovery), and users (disputes, censorship).
{"title":"SoK: Bitcoin Layer Two (L2)","authors":"Minfeng Qi, Qin Wang, Zhipeng Wang, Manvir Schneider, Tianqing Zhu, Shiping Chen, William Knottenbelt, Thomas Hardjono","doi":"arxiv-2409.02650","DOIUrl":"https://doi.org/arxiv-2409.02650","url":null,"abstract":"We present the first Systematization of Knowledge (SoK) on constructing Layer\u0000Two (L2) solutions for Bitcoin. We carefully examine a representative subset of ongoing Bitcoin L2 solutions\u0000(40 out of 335 extensively investigated cases) and provide a concise yet\u0000impactful identification of six classic design patterns through two approaches\u0000(i.e., modifying transactions & creating proofs). Notably, we are the first to\u0000incorporate the inscription technology (emerged in mid-2023), along with a\u0000series of related innovations. We further establish a reference framework that\u0000serves as a baseline criterion ideally suited for evaluating the security\u0000aspects of Bitcoin L2 solutions, and which can also be extended to broader L2\u0000applications. We apply this framework to evaluate each of the projects we\u0000investigated. We find that the inscription-based approaches introduce new functionality\u0000(i.e., programability) to Bitcoin systems, whereas existing proof-based\u0000solutions primarily address scalability challenges. Our security analysis\u0000reveals new attack vectors targeting data/state (availability, verification),\u0000assets (withdrawal, recovery), and users (disputes, censorship).","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142214104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Disaggregating memory from compute offers the opportunity to better utilize stranded memory in data centers. It is important to cache data in the compute nodes and maintain cache coherence across multiple compute nodes to save on round-trip communication cost between the disaggregated memory and the compute nodes. However, the limited computing power on the disaggregated memory servers makes it challenging to maintain cache coherence among multiple compute-side caches over disaggregated shared memory. This paper introduces SELCC; a Shared-Exclusive Latch Cache Coherence protocol that maintains cache coherence without imposing any computational burden on the remote memory side. SELCC builds on a one-sided shared-exclusive latch protocol by introducing lazy latch release and invalidation messages among the compute nodes so that it can guarantee both data access atomicity and cache coherence. SELCC minimizes communication round-trips by embedding the current cache copy holder IDs into RDMA latch words and prioritizes local concurrency control over global concurrency control. We instantiate the SELCC protocol onto compute-sided cache, forming an abstraction layer over disaggregated memory. This abstraction layer provides main-memory-like APIs to upper-level applications, and thus enabling existing data structures and algorithms to function over disaggregated memory with minimal code change. To demonstrate the usability of SELCC, we implement a B-tree and three transaction concurrency control algorithms over SELCC's APIs. Micro-benchmark results show that the SELCC protocol achieves better performance compared to RPC-based cache-coherence protocols. Additionally, YCSB and TPC-C benchmarks indicate that applications over SELCC can achieve comparable or superior performance against competitors over disaggregated memory.
{"title":"SELCC: Coherent Caching over Compute-Limited Disaggregated Memory","authors":"Ruihong Wang, Jianguo Wang, Walid G. Aref","doi":"arxiv-2409.02088","DOIUrl":"https://doi.org/arxiv-2409.02088","url":null,"abstract":"Disaggregating memory from compute offers the opportunity to better utilize\u0000stranded memory in data centers. It is important to cache data in the compute\u0000nodes and maintain cache coherence across multiple compute nodes to save on\u0000round-trip communication cost between the disaggregated memory and the compute\u0000nodes. However, the limited computing power on the disaggregated memory servers\u0000makes it challenging to maintain cache coherence among multiple compute-side\u0000caches over disaggregated shared memory. This paper introduces SELCC; a\u0000Shared-Exclusive Latch Cache Coherence protocol that maintains cache coherence\u0000without imposing any computational burden on the remote memory side. SELCC\u0000builds on a one-sided shared-exclusive latch protocol by introducing lazy latch\u0000release and invalidation messages among the compute nodes so that it can\u0000guarantee both data access atomicity and cache coherence. SELCC minimizes\u0000communication round-trips by embedding the current cache copy holder IDs into\u0000RDMA latch words and prioritizes local concurrency control over global\u0000concurrency control. We instantiate the SELCC protocol onto compute-sided\u0000cache, forming an abstraction layer over disaggregated memory. This abstraction\u0000layer provides main-memory-like APIs to upper-level applications, and thus\u0000enabling existing data structures and algorithms to function over disaggregated\u0000memory with minimal code change. To demonstrate the usability of SELCC, we\u0000implement a B-tree and three transaction concurrency control algorithms over\u0000SELCC's APIs. Micro-benchmark results show that the SELCC protocol achieves\u0000better performance compared to RPC-based cache-coherence protocols.\u0000Additionally, YCSB and TPC-C benchmarks indicate that applications over SELCC\u0000can achieve comparable or superior performance against competitors over\u0000disaggregated memory.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"72 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213907","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
L Rochit, Nithish Kumar N, Devi Priya V S, Sibi Chakkaravarthy Sethuraman, Anitha Subramanian
The ocean ecology is badly impacted by large-scale oil spills, plastic waste, and chemical pollution, which destroy ecosystems and endanger marine life. Acknowledging the detrimental effects of oil spills on ecosystems, our research aims to establish the foundation for creative methods to lessen their impact. With an emphasis on the containment and prediction of oil spills, this research investigates the potential of acoustic levitation as a cutting-edge technique for environmental cleanup. Effectively separating and eliminating pollutants without causing additional ecological harm is a major issue for traditional oil spill cleanup techniques. Acoustic levitation provides a non-invasive, accurate, and effective alternative by using sound waves to precisely and subtly separate oil droplets from water in controlled environments. This proposed approach can reduce the negative effects on the environment and increase the efficacy of cleanup efforts. The findings have been examined and assessed by proof of concept experiments with oil droplets, identifying the relationship between the intensity of ultrasonic pressure and the proportion of oil droplets collected.
{"title":"Acoustic Levitation for Environmental Remediation: An Effective Approach for Containment and Forecasting of Oil Spills","authors":"L Rochit, Nithish Kumar N, Devi Priya V S, Sibi Chakkaravarthy Sethuraman, Anitha Subramanian","doi":"arxiv-2409.01642","DOIUrl":"https://doi.org/arxiv-2409.01642","url":null,"abstract":"The ocean ecology is badly impacted by large-scale oil spills, plastic waste,\u0000and chemical pollution, which destroy ecosystems and endanger marine life.\u0000Acknowledging the detrimental effects of oil spills on ecosystems, our research\u0000aims to establish the foundation for creative methods to lessen their impact.\u0000With an emphasis on the containment and prediction of oil spills, this research\u0000investigates the potential of acoustic levitation as a cutting-edge technique\u0000for environmental cleanup. Effectively separating and eliminating pollutants\u0000without causing additional ecological harm is a major issue for traditional oil\u0000spill cleanup techniques. Acoustic levitation provides a non-invasive,\u0000accurate, and effective alternative by using sound waves to precisely and\u0000subtly separate oil droplets from water in controlled environments. This\u0000proposed approach can reduce the negative effects on the environment and\u0000increase the efficacy of cleanup efforts. The findings have been examined and\u0000assessed by proof of concept experiments with oil droplets, identifying the\u0000relationship between the intensity of ultrasonic pressure and the proportion of\u0000oil droplets collected.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"43 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142214107","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anas Skalli, Mirko Goldmann, Nasibeh Haghighi, Stephan Reitzenstein, James A. Lott, Daniel Brunner
Artificial neural networks (ANNs) represent a fundamentally connectionnist and distributed approach to computing, and as such they differ from classical computers that utilize the von Neumann architecture. This has revived research interest in new unconventional hardware to enable more efficient implementations of ANNs rather than emulating them on traditional machines. In order to fully leverage the capabilities of this new generation of ANNs, optimization algorithms that take into account hardware limitations and imperfections are necessary. Photonics represents a particularly promising platform, offering scalability, high speed, energy efficiency, and the capability for parallel information processing. Yet, fully fledged implementations of autonomous optical neural networks (ONNs) with in-situ learning remain scarce. In this work, we propose a ternary weight architecture high-dimensional semiconductor laser-based ONN. We introduce a simple method for achieving ternary weights with Boolean hardware, significantly increasing the ONN's information processing capabilities. Furthermore, we design a novel in-situ optimization algorithm that is compatible with, both, Boolean and ternary weights, and provide a detailed hyperparameter study of said algorithm for two different tasks. Our novel algorithm results in benefits, both in terms of convergence speed and performance. Finally, we experimentally characterize the long-term inference stability of our ONN and find that it is extremely stable with a consistency above 99% over a period of more than 10 hours, addressing one of the main concerns in the field. Our work is of particular relevance in the context of in-situ learning under restricted hardware resources, especially since minimizing the power consumption of auxiliary hardware is crucial to preserving efficiency gains achieved by non-von Neumann ANN implementations.
{"title":"Annealing-inspired training of an optical neural network with ternary weights","authors":"Anas Skalli, Mirko Goldmann, Nasibeh Haghighi, Stephan Reitzenstein, James A. Lott, Daniel Brunner","doi":"arxiv-2409.01042","DOIUrl":"https://doi.org/arxiv-2409.01042","url":null,"abstract":"Artificial neural networks (ANNs) represent a fundamentally connectionnist\u0000and distributed approach to computing, and as such they differ from classical\u0000computers that utilize the von Neumann architecture. This has revived research\u0000interest in new unconventional hardware to enable more efficient\u0000implementations of ANNs rather than emulating them on traditional machines. In\u0000order to fully leverage the capabilities of this new generation of ANNs,\u0000optimization algorithms that take into account hardware limitations and\u0000imperfections are necessary. Photonics represents a particularly promising\u0000platform, offering scalability, high speed, energy efficiency, and the\u0000capability for parallel information processing. Yet, fully fledged\u0000implementations of autonomous optical neural networks (ONNs) with in-situ\u0000learning remain scarce. In this work, we propose a ternary weight architecture\u0000high-dimensional semiconductor laser-based ONN. We introduce a simple method\u0000for achieving ternary weights with Boolean hardware, significantly increasing\u0000the ONN's information processing capabilities. Furthermore, we design a novel\u0000in-situ optimization algorithm that is compatible with, both, Boolean and\u0000ternary weights, and provide a detailed hyperparameter study of said algorithm\u0000for two different tasks. Our novel algorithm results in benefits, both in terms\u0000of convergence speed and performance. Finally, we experimentally characterize\u0000the long-term inference stability of our ONN and find that it is extremely\u0000stable with a consistency above 99% over a period of more than 10 hours,\u0000addressing one of the main concerns in the field. Our work is of particular\u0000relevance in the context of in-situ learning under restricted hardware\u0000resources, especially since minimizing the power consumption of auxiliary\u0000hardware is crucial to preserving efficiency gains achieved by non-von Neumann\u0000ANN implementations.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"95 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The rapid growth of large models' size has far outpaced that of GPU memory. To bridge this gap, inspired by the succinct relationship between genotype and phenotype, we turn the model compression problem into the issue of parameter representation to propose the so-called hyper-compression. The hyper-compression uses a hyperfunction to represent the parameters of the target network, and notably, here the hyperfunction is designed per ergodic theory that relates to a problem: if a low-dimensional dynamic system can fill the high-dimensional space eventually. Empirically, the proposed hyper-compression enjoys the following merits: 1) textbf{P}referable compression ratio; 2) textbf{N}o post-hoc retraining; 3) textbf{A}ffordable inference time; and 4) textbf{S}hort compression time. It compresses LLaMA2-7B in an hour and achieves close-to-int4-quantization performance, without retraining and with a performance drop of less than 1%. Our work has the potential to invigorate the field of model compression, towards a harmony between the scaling law and the stagnation of hardware upgradation.
{"title":"Hyper-Compression: Model Compression via Hyperfunction","authors":"Fenglei Fan, Juntong Fan, Dayang Wang, Jingbo Zhang, Zelin Dong, Shijun Zhang, Ge Wang, Tieyong Zeng","doi":"arxiv-2409.00592","DOIUrl":"https://doi.org/arxiv-2409.00592","url":null,"abstract":"The rapid growth of large models' size has far outpaced that of GPU memory.\u0000To bridge this gap, inspired by the succinct relationship between genotype and\u0000phenotype, we turn the model compression problem into the issue of parameter\u0000representation to propose the so-called hyper-compression. The\u0000hyper-compression uses a hyperfunction to represent the parameters of the\u0000target network, and notably, here the hyperfunction is designed per ergodic\u0000theory that relates to a problem: if a low-dimensional dynamic system can fill\u0000the high-dimensional space eventually. Empirically, the proposed\u0000hyper-compression enjoys the following merits: 1) textbf{P}referable\u0000compression ratio; 2) textbf{N}o post-hoc retraining; 3) textbf{A}ffordable\u0000inference time; and 4) textbf{S}hort compression time. It compresses LLaMA2-7B\u0000in an hour and achieves close-to-int4-quantization performance, without\u0000retraining and with a performance drop of less than 1%. Our work has the\u0000potential to invigorate the field of model compression, towards a harmony\u0000between the scaling law and the stagnation of hardware upgradation.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"34 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
3D integration offers key advantages in improving system performance and efficiency for the End-of-Scaling era. It enables the incorporation of heterogeneous system components and disparate technologies, eliminates off-chip communication constraints, reduces on-chip latency and total power dissipation. Moreover, AIs demand for increased computational power, larger GPU cache capacity, energy efficiency and low power custom AI hardware integration all serve as drivers for 3D integration. Although 3D advantages such as enhanced interconnectivity and increased performance have been demonstrated through numerous technology sites, heterogeneous 3D system design raises numerous unanswered questions. Among the primary challenges are the temperature and lifetime reliability issues caused by the complex interaction patterns among system components. Such interactions are harder to model with current modeling tools and require detailed hardware characterization. This study presents the latest drivers for 3D integration and the resulting need for hardware emulation frameworks. It then presents a design to profile power, temperature, noise, inter-layer bandwidth and lifetime reliability characterization that can emulate a wide range of stacking alternatives. This framework allows for controlling activity levels at the macro-level, along with customized sensor infrastructure to characterize heat propagation, inter-layer noise, power delivery, reliability and inter-connectivity as well as the interactions among critical design objectives.
三维集成在提高系统性能和效率方面具有关键优势，可满足 "规模末期 "时代的需求。此外，人工智能对更高计算能力、更大 GPU 缓存容量、能效和低功耗定制人工智能硬件集成的需求都是三维集成的驱动力。尽管 3D 的优势（如增强的互联性和更高的性能）已通过众多技术网站得到证实，但异构 3D 系统设计仍存在许多未解之谜。其中最主要的挑战是系统组件之间复杂的交互模式所导致的温度和寿命可靠性问题。目前的建模工具难以对这种相互作用进行建模，需要进行详细的硬件表征。本研究介绍了三维集成的最新驱动力以及由此产生的对硬件仿真框架的需求。然后，它介绍了一种用于功率、温度、噪声、层间带宽和寿命可靠性特征描述的设计，可以模拟各种堆叠替代方案。该框架允许在宏观层面控制活动水平，同时采用定制的传感器基础设施来描述热传播、层间噪声、功率传输、可靠性和互连性，以及关键设计目标之间的相互作用。
{"title":"Towards 3D AI Hardware: Fine-Grain Hardware Characterization of 3D Stacks for Heterogeneous System Integration & AI Systems","authors":"Eren Kurshan, Paul Franzon","doi":"arxiv-2409.10539","DOIUrl":"https://doi.org/arxiv-2409.10539","url":null,"abstract":"3D integration offers key advantages in improving system performance and\u0000efficiency for the End-of-Scaling era. It enables the incorporation of\u0000heterogeneous system components and disparate technologies, eliminates off-chip\u0000communication constraints, reduces on-chip latency and total power dissipation.\u0000Moreover, AIs demand for increased computational power, larger GPU cache\u0000capacity, energy efficiency and low power custom AI hardware integration all\u0000serve as drivers for 3D integration. Although 3D advantages such as enhanced\u0000interconnectivity and increased performance have been demonstrated through\u0000numerous technology sites, heterogeneous 3D system design raises numerous\u0000unanswered questions. Among the primary challenges are the temperature and\u0000lifetime reliability issues caused by the complex interaction patterns among\u0000system components. Such interactions are harder to model with current modeling\u0000tools and require detailed hardware characterization. This study presents the\u0000latest drivers for 3D integration and the resulting need for hardware emulation\u0000frameworks. It then presents a design to profile power, temperature, noise,\u0000inter-layer bandwidth and lifetime reliability characterization that can\u0000emulate a wide range of stacking alternatives. This framework allows for\u0000controlling activity levels at the macro-level, along with customized sensor\u0000infrastructure to characterize heat propagation, inter-layer noise, power\u0000delivery, reliability and inter-connectivity as well as the interactions among\u0000critical design objectives.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"30 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142263119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nazmus Sakib, Md Yeasin Ali, Nuran Mubashshira Momo, Marzia Islam Mumu, Masum Al Nahid, Fairuz Rahaman Chowdhury, Md Sadek Ferdous
The popularity of the Internet of Things (IoT) has driven its usage in our homes and industries over the past 10-12 years. However, there have been some major issues related to identity management and ownership transfer involving IoT devices, particularly for consumer IoT devices, e. g. smart appliances such as smart TVs, smart refrigerators, and so on. There have been a few attempts to address this issue; however, user-centric and effective ownership and identity management of IoT devices have not been very successful so far. Recently, blockchain technology has been used to address these issues with limited success. This article presents a Self-sovereign Identity (SSI) based system that facilitates a secure and user-centric ownership management and transfer of consumer IoT devices. The system leverages a number of emerging technologies, such as blockchain and decentralized identifiers (DID), verifiable credentials (VC), under the umbrella of SSI. We present the architecture of the system based on a threat model and requirement analysis, discuss the implementation of a Proof-of-Concept based on the proposed system and illustrate a number of use-cases with their detailed protocol flows. Furthermore, we analyse its security using ProVerif, a state-of-the art protocol verification tool and examine its performance.
{"title":"Secure Ownership Management and Transfer of Consumer Internet of Things Devices with Self-sovereign Identity","authors":"Nazmus Sakib, Md Yeasin Ali, Nuran Mubashshira Momo, Marzia Islam Mumu, Masum Al Nahid, Fairuz Rahaman Chowdhury, Md Sadek Ferdous","doi":"arxiv-2408.17184","DOIUrl":"https://doi.org/arxiv-2408.17184","url":null,"abstract":"The popularity of the Internet of Things (IoT) has driven its usage in our\u0000homes and industries over the past 10-12 years. However, there have been some\u0000major issues related to identity management and ownership transfer involving\u0000IoT devices, particularly for consumer IoT devices, e. g. smart appliances such\u0000as smart TVs, smart refrigerators, and so on. There have been a few attempts to\u0000address this issue; however, user-centric and effective ownership and identity\u0000management of IoT devices have not been very successful so far. Recently,\u0000blockchain technology has been used to address these issues with limited\u0000success. This article presents a Self-sovereign Identity (SSI) based system\u0000that facilitates a secure and user-centric ownership management and transfer of\u0000consumer IoT devices. The system leverages a number of emerging technologies,\u0000such as blockchain and decentralized identifiers (DID), verifiable credentials\u0000(VC), under the umbrella of SSI. We present the architecture of the system\u0000based on a threat model and requirement analysis, discuss the implementation of\u0000a Proof-of-Concept based on the proposed system and illustrate a number of\u0000use-cases with their detailed protocol flows. Furthermore, we analyse its\u0000security using ProVerif, a state-of-the art protocol verification tool and\u0000examine its performance.","PeriodicalId":501168,"journal":{"name":"arXiv - CS - Emerging Technologies","volume":"3 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-08-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：