Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.041479
Xi Zhao, Haizheng Yu, Hong Bian
In recent years, Pix2Pix, a model within the domain of GANs, has found widespread application in the field of image-to-image translation. However, traditional Pix2Pix models suffer from significant drawbacks in image generation, such as the loss of important information features during the encoding and decoding processes, as well as a lack of constraints during the training process. To address these issues and improve the quality of Pix2Pix-generated images, this paper introduces two key enhancements. Firstly, to reduce information loss during encoding and decoding, we utilize the U-Net++ network as the generator for the Pix2Pix model, incorporating denser skip-connection to minimize information loss. Secondly, to enhance constraints during image generation, we introduce a specialized discriminator designed to distinguish differential images, further enhancing the quality of the generated images. We conducted experiments on the facades dataset and the sketch portrait dataset from the Chinese University of Hong Kong to validate our proposed model. The experimental results demonstrate that our improved Pix2Pix model significantly enhances image quality and outperforms other models in the selected metrics. Notably, the Pix2Pix model incorporating the differential image discriminator exhibits the most substantial improvements across all metrics. An analysis of the experimental results reveals that the use of the U-Net++ generator effectively reduces information feature loss, while the Pix2Pix model incorporating the differential image discriminator enhances the supervision of the generator during training. Both of these enhancements collectively improve the quality of Pix2Pix-generated images.
{"title":"Image to Image Translation Based on Differential Image Pix2Pix Model","authors":"Xi Zhao, Haizheng Yu, Hong Bian","doi":"10.32604/cmc.2023.041479","DOIUrl":"https://doi.org/10.32604/cmc.2023.041479","url":null,"abstract":"In recent years, Pix2Pix, a model within the domain of GANs, has found widespread application in the field of image-to-image translation. However, traditional Pix2Pix models suffer from significant drawbacks in image generation, such as the loss of important information features during the encoding and decoding processes, as well as a lack of constraints during the training process. To address these issues and improve the quality of Pix2Pix-generated images, this paper introduces two key enhancements. Firstly, to reduce information loss during encoding and decoding, we utilize the U-Net++ network as the generator for the Pix2Pix model, incorporating denser skip-connection to minimize information loss. Secondly, to enhance constraints during image generation, we introduce a specialized discriminator designed to distinguish differential images, further enhancing the quality of the generated images. We conducted experiments on the facades dataset and the sketch portrait dataset from the Chinese University of Hong Kong to validate our proposed model. The experimental results demonstrate that our improved Pix2Pix model significantly enhances image quality and outperforms other models in the selected metrics. Notably, the Pix2Pix model incorporating the differential image discriminator exhibits the most substantial improvements across all metrics. An analysis of the experimental results reveals that the use of the U-Net++ generator effectively reduces information feature loss, while the Pix2Pix model incorporating the differential image discriminator enhances the supervision of the generator during training. Both of these enhancements collectively improve the quality of Pix2Pix-generated images.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"46 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.041332
Somin Park, Mpabulungi Mark, Bogyung Park, Hyunki Hong
Speech emotion recognition is essential for frictionless human-machine interaction, where machines respond to human instructions with context-aware actions. The properties of individuals’ voices vary with culture, language, gender, and personality. These variations in speaker-specific properties may hamper the performance of standard representations in downstream tasks such as speech emotion recognition (SER). This study demonstrates the significance of speaker-specific speech characteristics and how considering them can be leveraged to improve the performance of SER models. In the proposed approach, two wav2vec-based modules (a speaker-identification network and an emotion classification network) are trained with the Arcface loss. The speaker-identification network has a single attention block to encode an input audio waveform into a speaker-specific representation. The emotion classification network uses a wav2vec 2.0-backbone as well as four attention blocks to encode the same input audio waveform into an emotion representation. These two representations are then fused into a single vector representation containing emotion and speaker-specific information. Experimental results showed that the use of speaker-specific characteristics improves SER performance. Additionally, combining these with an angular marginal loss such as the Arcface loss improves intra-class compactness while increasing inter-class separability, as demonstrated by the plots of t-distributed stochastic neighbor embeddings (t-SNE). The proposed approach outperforms previous methods using similar training strategies, with a weighted accuracy (WA) of 72.14% and unweighted accuracy (UA) of 72.97% on the Interactive Emotional Dynamic Motion Capture (IEMOCAP) dataset. This demonstrates its effectiveness and potential to enhance human-machine interaction through more accurate emotion recognition in speech.
{"title":"Using Speaker-Specific Emotion Representations in Wav2vec 2.0-Based Modules for Speech Emotion Recognition","authors":"Somin Park, Mpabulungi Mark, Bogyung Park, Hyunki Hong","doi":"10.32604/cmc.2023.041332","DOIUrl":"https://doi.org/10.32604/cmc.2023.041332","url":null,"abstract":"Speech emotion recognition is essential for frictionless human-machine interaction, where machines respond to human instructions with context-aware actions. The properties of individuals’ voices vary with culture, language, gender, and personality. These variations in speaker-specific properties may hamper the performance of standard representations in downstream tasks such as speech emotion recognition (SER). This study demonstrates the significance of speaker-specific speech characteristics and how considering them can be leveraged to improve the performance of SER models. In the proposed approach, two wav2vec-based modules (a speaker-identification network and an emotion classification network) are trained with the Arcface loss. The speaker-identification network has a single attention block to encode an input audio waveform into a speaker-specific representation. The emotion classification network uses a wav2vec 2.0-backbone as well as four attention blocks to encode the same input audio waveform into an emotion representation. These two representations are then fused into a single vector representation containing emotion and speaker-specific information. Experimental results showed that the use of speaker-specific characteristics improves SER performance. Additionally, combining these with an angular marginal loss such as the Arcface loss improves intra-class compactness while increasing inter-class separability, as demonstrated by the plots of t-distributed stochastic neighbor embeddings (t-SNE). The proposed approach outperforms previous methods using similar training strategies, with a weighted accuracy (WA) of 72.14% and unweighted accuracy (UA) of 72.97% on the Interactive Emotional Dynamic Motion Capture (IEMOCAP) dataset. This demonstrates its effectiveness and potential to enhance human-machine interaction through more accurate emotion recognition in speech.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.040756
Muhammad Ali Khan, Rao Naveed Bin Rais, Osman Khalid
The Internet of Things (IoT) comprises numerous resource-constrained devices that generate large volumes of data. The inherent vulnerabilities in IoT infrastructure, such as easily spoofed IP and MAC addresses, pose significant security challenges. Traditional routing protocols designed for wired or wireless networks may not be suitable for IoT networks due to their limitations. Therefore, the Routing Protocol for Low-Power and Lossy Networks (RPL) is widely used in IoT systems. However, the built-in security mechanism of RPL is inadequate in defending against sophisticated routing attacks, including Sybil attacks. To address these issues, this paper proposes a centralized and collaborative approach for securing RPL-based IoT against Sybil attacks. The proposed approach consists of detection and prevention algorithms based on the Random Password Generation and comparison methodology (RPG). The detection algorithm verifies the passwords of communicating nodes before comparing their keys and constant IDs, while the prevention algorithm utilizes a delivery delay ratio to restrict the participation of sensor nodes in communication. Through simulations, it is demonstrated that the proposed approach achieves better results compared to distributed defense mechanisms in terms of throughput, average delivery delay and detection rate. Moreover, the proposed countermeasure effectively mitigates brute-force and side-channel attacks in addition to Sybil attacks. The findings suggest that implementing the RPG-based detection and prevention algorithms can provide robust security for RPL-based IoT networks.
{"title":"Collaborative Detection and Prevention of Sybil Attacks against RPL-Based Internet of Things","authors":"Muhammad Ali Khan, Rao Naveed Bin Rais, Osman Khalid","doi":"10.32604/cmc.2023.040756","DOIUrl":"https://doi.org/10.32604/cmc.2023.040756","url":null,"abstract":"The Internet of Things (IoT) comprises numerous resource-constrained devices that generate large volumes of data. The inherent vulnerabilities in IoT infrastructure, such as easily spoofed IP and MAC addresses, pose significant security challenges. Traditional routing protocols designed for wired or wireless networks may not be suitable for IoT networks due to their limitations. Therefore, the Routing Protocol for Low-Power and Lossy Networks (RPL) is widely used in IoT systems. However, the built-in security mechanism of RPL is inadequate in defending against sophisticated routing attacks, including Sybil attacks. To address these issues, this paper proposes a centralized and collaborative approach for securing RPL-based IoT against Sybil attacks. The proposed approach consists of detection and prevention algorithms based on the Random Password Generation and comparison methodology (RPG). The detection algorithm verifies the passwords of communicating nodes before comparing their keys and constant IDs, while the prevention algorithm utilizes a delivery delay ratio to restrict the participation of sensor nodes in communication. Through simulations, it is demonstrated that the proposed approach achieves better results compared to distributed defense mechanisms in terms of throughput, average delivery delay and detection rate. Moreover, the proposed countermeasure effectively mitigates brute-force and side-channel attacks in addition to Sybil attacks. The findings suggest that implementing the RPG-based detection and prevention algorithms can provide robust security for RPL-based IoT networks.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317509","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.036074
Fiaz Majeed, Afzaal Ahmad, Muhammad Awais Hassan, Muhammad Shafiq, Jin-Ghoo Choi, Habib Hamam
Every day, the media reports tons of crimes that are considered by a large number of users and accumulate on a regular basis. Crime news exists on the Internet in unstructured formats such as books, websites, documents, and journals. From such homogeneous data, it is very challenging to extract relevant information which is a time-consuming and critical task for the public and law enforcement agencies. Keyword-based Information Retrieval (IR) systems rely on statistics to retrieve results, making it difficult to obtain relevant results. They are unable to understand the user's query and thus face word mismatches due to context changes and the inevitable semantics of a given word. Therefore, such datasets need to be organized in a structured configuration, with the goal of efficiently manipulating the data while respecting the semantics of the data. An ontological semantic IR system is needed that can find the right investigative information and find important clues to solve criminal cases. The semantic system retrieves information in view of the similarity of the semantics among indexed data and user queries. In this paper, we develop an ontology-based semantic IR system that leverages the latest semantic technologies including resource description framework (RDF), semantic protocol and RDF query language (SPARQL), semantic web rule language (SWRL), and web ontology language (OWL). We have conducted two experiments. In the first experiment, we implemented a keyword-based textual IR system using Apache Lucene. In the second experiment, we implemented a semantic system that uses ontology to store the data and retrieve precise results with high accuracy using SPARQL queries. The keyword-based system has filtered results with 51% accuracy, while the semantic system has filtered results with 95% accuracy, leading to significant improvements in the field and opening up new horizons for researchers.
{"title":"Ontology-Based Crime News Semantic Retrieval System","authors":"Fiaz Majeed, Afzaal Ahmad, Muhammad Awais Hassan, Muhammad Shafiq, Jin-Ghoo Choi, Habib Hamam","doi":"10.32604/cmc.2023.036074","DOIUrl":"https://doi.org/10.32604/cmc.2023.036074","url":null,"abstract":"Every day, the media reports tons of crimes that are considered by a large number of users and accumulate on a regular basis. Crime news exists on the Internet in unstructured formats such as books, websites, documents, and journals. From such homogeneous data, it is very challenging to extract relevant information which is a time-consuming and critical task for the public and law enforcement agencies. Keyword-based Information Retrieval (IR) systems rely on statistics to retrieve results, making it difficult to obtain relevant results. They are unable to understand the user's query and thus face word mismatches due to context changes and the inevitable semantics of a given word. Therefore, such datasets need to be organized in a structured configuration, with the goal of efficiently manipulating the data while respecting the semantics of the data. An ontological semantic IR system is needed that can find the right investigative information and find important clues to solve criminal cases. The semantic system retrieves information in view of the similarity of the semantics among indexed data and user queries. In this paper, we develop an ontology-based semantic IR system that leverages the latest semantic technologies including resource description framework (RDF), semantic protocol and RDF query language (SPARQL), semantic web rule language (SWRL), and web ontology language (OWL). We have conducted two experiments. In the first experiment, we implemented a keyword-based textual IR system using Apache Lucene. In the second experiment, we implemented a semantic system that uses ontology to store the data and retrieve precise results with high accuracy using SPARQL queries. The keyword-based system has filtered results with 51% accuracy, while the semantic system has filtered results with 95% accuracy, leading to significant improvements in the field and opening up new horizons for researchers.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"101 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.038810
Allam Abumwais, Mujahed Eleyat
Recent architectures of multi-core systems may have a relatively large number of cores that typically ranges from tens to hundreds; therefore called many-core systems. Such systems require an efficient interconnection network that tries to address two major problems. First, the overhead of power and area cost and its effect on scalability. Second, high access latency is caused by multiple cores’ simultaneous accesses of the same shared module. This paper presents an interconnection scheme called N-conjugate Shuffle Clusters (NCSC) based on multi-core multi-cluster architecture to reduce the overhead of the just mentioned problems. NCSC eliminated the need for router devices and their complexity and hence reduced the power and area costs. It also resigned and distributed the shared caches across the interconnection network to increase the ability for simultaneous access and hence reduce the access latency. For intra-cluster communication, Multi-port Content Addressable Memory (MPCAM) is used. The experimental results using four clusters and four cores each indicated that the average access latency for a write process is 1.14785 ± 0.04532 ns which is nearly equal to the latency of a write operation in MPCAM. Moreover, it was demonstrated that the average read latency within a cluster is 1.26226 ± 0.090591 ns and around 1.92738 ± 0.139588 ns for read access between cores from different clusters.
{"title":"A Scalable Interconnection Scheme in Many-Core Systems","authors":"Allam Abumwais, Mujahed Eleyat","doi":"10.32604/cmc.2023.038810","DOIUrl":"https://doi.org/10.32604/cmc.2023.038810","url":null,"abstract":"Recent architectures of multi-core systems may have a relatively large number of cores that typically ranges from tens to hundreds; therefore called many-core systems. Such systems require an efficient interconnection network that tries to address two major problems. First, the overhead of power and area cost and its effect on scalability. Second, high access latency is caused by multiple cores’ simultaneous accesses of the same shared module. This paper presents an interconnection scheme called N-conjugate Shuffle Clusters (NCSC) based on multi-core multi-cluster architecture to reduce the overhead of the just mentioned problems. NCSC eliminated the need for router devices and their complexity and hence reduced the power and area costs. It also resigned and distributed the shared caches across the interconnection network to increase the ability for simultaneous access and hence reduce the access latency. For intra-cluster communication, Multi-port Content Addressable Memory (MPCAM) is used. The experimental results using four clusters and four cores each indicated that the average access latency for a write process is 1.14785 ± 0.04532 ns which is nearly equal to the latency of a write operation in MPCAM. Moreover, it was demonstrated that the average read latency within a cluster is 1.26226 ± 0.090591 ns and around 1.92738 ± 0.139588 ns for read access between cores from different clusters.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.041108
Shadi Al-Sarawi, Mohammed Anbar, Basim Ahmad Alabsi, Mohammad Adnan Aladaileh, Shaza Dawood Ahmed Rihan
The Internet of Things (IoT) consists of interconnected smart devices communicating and collecting data. The Routing Protocol for Low-Power and Lossy Networks (RPL) is the standard protocol for Internet Protocol Version 6 (IPv6) in the IoT. However, RPL is vulnerable to various attacks, including the sinkhole attack, which disrupts the network by manipulating routing information. This paper proposes the Unweighted Voting Method (UVM) for sinkhole node identification, utilizing three key behavioral indicators: DODAG Information Object (DIO) Transaction Frequency, Rank Harmony, and Power Consumption. These indicators have been carefully selected based on their contribution to sinkhole attack detection and other relevant features used in previous research. The UVM method employs an unweighted voting mechanism, where each voter or rule holds equal weight in detecting the presence of a sinkhole attack based on the proposed indicators. The effectiveness of the UVM method is evaluated using the COOJA simulator and compared with existing approaches. Notably, the proposed approach fulfills power consumption requirements for constrained nodes without increasing consumption due to the deployment design. In terms of detection accuracy, simulation results demonstrate a high detection rate ranging from 90% to 100%, with a low false-positive rate of 0% to 0.2%. Consequently, the proposed approach surpasses Ensemble Learning Intrusion Detection Systems by leveraging three indicators and three supporting rules.
{"title":"Unweighted Voting Method to Detect Sinkhole Attack in RPL-Based Internet of Things Networks","authors":"Shadi Al-Sarawi, Mohammed Anbar, Basim Ahmad Alabsi, Mohammad Adnan Aladaileh, Shaza Dawood Ahmed Rihan","doi":"10.32604/cmc.2023.041108","DOIUrl":"https://doi.org/10.32604/cmc.2023.041108","url":null,"abstract":"The Internet of Things (IoT) consists of interconnected smart devices communicating and collecting data. The Routing Protocol for Low-Power and Lossy Networks (RPL) is the standard protocol for Internet Protocol Version 6 (IPv6) in the IoT. However, RPL is vulnerable to various attacks, including the sinkhole attack, which disrupts the network by manipulating routing information. This paper proposes the Unweighted Voting Method (UVM) for sinkhole node identification, utilizing three key behavioral indicators: DODAG Information Object (DIO) Transaction Frequency, Rank Harmony, and Power Consumption. These indicators have been carefully selected based on their contribution to sinkhole attack detection and other relevant features used in previous research. The UVM method employs an unweighted voting mechanism, where each voter or rule holds equal weight in detecting the presence of a sinkhole attack based on the proposed indicators. The effectiveness of the UVM method is evaluated using the COOJA simulator and compared with existing approaches. Notably, the proposed approach fulfills power consumption requirements for constrained nodes without increasing consumption due to the deployment design. In terms of detection accuracy, simulation results demonstrate a high detection rate ranging from 90% to 100%, with a low false-positive rate of 0% to 0.2%. Consequently, the proposed approach surpasses Ensemble Learning Intrusion Detection Systems by leveraging three indicators and three supporting rules.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.039582
Jiwei Fan, Xiaogang Yang, Ruitao Lu, Qingge Li, Siyu Wang
In recent years, many visual positioning algorithms have been proposed based on computer vision and they have achieved good results. However, these algorithms have a single function, cannot perceive the environment, and have poor versatility, and there is a certain mismatch phenomenon, which affects the positioning accuracy. Therefore, this paper proposes a location algorithm that combines a target recognition algorithm with a depth feature matching algorithm to solve the problem of unmanned aerial vehicle (UAV) environment perception and multi-modal image-matching fusion location. This algorithm was based on the single-shot object detector based on multi-level feature pyramid network (M2Det) algorithm and replaced the original visual geometry group (VGG) feature extraction network with the ResNet-101 network to improve the feature extraction capability of the network model. By introducing a depth feature matching algorithm, the algorithm shares neural network weights and realizes the design of UAV target recognition and a multi-modal image-matching fusion positioning algorithm. When the reference image and the real-time image were mismatched, the dynamic adaptive proportional constraint and the random sample consensus consistency algorithm (DAPC-RANSAC) were used to optimize the matching results to improve the correct matching efficiency of the target. Using the multi-modal registration data set, the proposed algorithm was compared and analyzed to verify its superiority and feasibility. The results show that the algorithm proposed in this paper can effectively deal with the matching between multi-modal images (visible image–infrared image, infrared image–satellite image, visible image–satellite image), and the contrast, scale, brightness, ambiguity deformation, and other changes had good stability and robustness. Finally, the effectiveness and practicability of the algorithm proposed in this paper were verified in an aerial test scene of an S1000 six-rotor UAV.
{"title":"Multi-Modal Scene Matching Location Algorithm Based on M2Det","authors":"Jiwei Fan, Xiaogang Yang, Ruitao Lu, Qingge Li, Siyu Wang","doi":"10.32604/cmc.2023.039582","DOIUrl":"https://doi.org/10.32604/cmc.2023.039582","url":null,"abstract":"In recent years, many visual positioning algorithms have been proposed based on computer vision and they have achieved good results. However, these algorithms have a single function, cannot perceive the environment, and have poor versatility, and there is a certain mismatch phenomenon, which affects the positioning accuracy. Therefore, this paper proposes a location algorithm that combines a target recognition algorithm with a depth feature matching algorithm to solve the problem of unmanned aerial vehicle (UAV) environment perception and multi-modal image-matching fusion location. This algorithm was based on the single-shot object detector based on multi-level feature pyramid network (M2Det) algorithm and replaced the original visual geometry group (VGG) feature extraction network with the ResNet-101 network to improve the feature extraction capability of the network model. By introducing a depth feature matching algorithm, the algorithm shares neural network weights and realizes the design of UAV target recognition and a multi-modal image-matching fusion positioning algorithm. When the reference image and the real-time image were mismatched, the dynamic adaptive proportional constraint and the random sample consensus consistency algorithm (DAPC-RANSAC) were used to optimize the matching results to improve the correct matching efficiency of the target. Using the multi-modal registration data set, the proposed algorithm was compared and analyzed to verify its superiority and feasibility. The results show that the algorithm proposed in this paper can effectively deal with the matching between multi-modal images (visible image–infrared image, infrared image–satellite image, visible image–satellite image), and the contrast, scale, brightness, ambiguity deformation, and other changes had good stability and robustness. Finally, the effectiveness and practicability of the algorithm proposed in this paper were verified in an aerial test scene of an S1000 six-rotor UAV.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"2015 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Event extraction stands as a significant endeavor within the realm of information extraction, aspiring to automatically extract structured event information from vast volumes of unstructured text. Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data. Although researchers have proposed various methods to accomplish this task, most existing event extraction models cannot address these challenges because they are only applicable to text scenarios. To solve the above issues, this paper proposes a multi-modal event extraction method based on knowledge fusion. Specifically, for event-type recognition, we use a meticulous pipeline approach that integrates multiple pre-trained models. This approach enables a more comprehensive capture of the multidimensional event semantic features present in military texts, thereby enhancing the interconnectedness of information between trigger words and events. For event element extraction, we propose a method for constructing a priori templates that combine event types with corresponding trigger words. This approach facilitates the acquisition of fine-grained input samples containing event trigger words, thus enabling the model to understand the semantic relationships between elements in greater depth. Furthermore, a fusion method for spatial mapping of textual event elements and image elements is proposed to reduce the category number overload and effectively achieve multi-modal knowledge fusion. The experimental results based on the CCKS 2022 dataset show that our method has achieved competitive results, with a comprehensive evaluation value F1-score of 53.4% for the model. These results validate the effectiveness of our method in extracting event elements from multi-modal data.
{"title":"Multi-Modal Military Event Extraction Based on Knowledge Fusion","authors":"Yuyuan Xiang, Yangli Jia, Xiangliang Zhang, Zhenling Zhang","doi":"10.32604/cmc.2023.040751","DOIUrl":"https://doi.org/10.32604/cmc.2023.040751","url":null,"abstract":"Event extraction stands as a significant endeavor within the realm of information extraction, aspiring to automatically extract structured event information from vast volumes of unstructured text. Extracting event elements from multi-modal data remains a challenging task due to the presence of a large number of images and overlapping event elements in the data. Although researchers have proposed various methods to accomplish this task, most existing event extraction models cannot address these challenges because they are only applicable to text scenarios. To solve the above issues, this paper proposes a multi-modal event extraction method based on knowledge fusion. Specifically, for event-type recognition, we use a meticulous pipeline approach that integrates multiple pre-trained models. This approach enables a more comprehensive capture of the multidimensional event semantic features present in military texts, thereby enhancing the interconnectedness of information between trigger words and events. For event element extraction, we propose a method for constructing a priori templates that combine event types with corresponding trigger words. This approach facilitates the acquisition of fine-grained input samples containing event trigger words, thus enabling the model to understand the semantic relationships between elements in greater depth. Furthermore, a fusion method for spatial mapping of textual event elements and image elements is proposed to reduce the category number overload and effectively achieve multi-modal knowledge fusion. The experimental results based on the CCKS 2022 dataset show that our method has achieved competitive results, with a comprehensive evaluation value F1-score of 53.4% for the model. These results validate the effectiveness of our method in extracting event elements from multi-modal data.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135317115","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.32604/cmc.2023.032069
Ramiz Gorkem Birdal, Ahmet Sertbas
Biometric gait recognition is a lesser-known but emerging and effective biometric recognition method which enables subjects’ walking patterns to be recognized. Existing research in this area has primarily focused on feature analysis through the extraction of individual features, which captures most of the information but fails to capture subtle variations in gait dynamics. Therefore, a novel feature taxonomy and an approach for deriving a relationship between a function of one set of gait features with another set are introduced. The gait features extracted from body halves divided by anatomical planes on vertical, horizontal, and diagonal axes are grouped to form canonical gait covariates. Canonical Correlation Analysis is utilized to measure the strength of association between the canonical covariates of gait. Thus, gait assessment and identification are enhanced when more semantic information is available through CCA-based multi-feature fusion. Hence, Carnegie Mellon University’s 3D gait database, which contains 32 gait samples taken at different paces, is utilized in analyzing gait characteristics. The performance of Linear Discriminant Analysis, K-Nearest Neighbors, Naive Bayes, Artificial Neural Networks, and Support Vector Machines was improved by a 4% average when the CCA-utilized gait identification approach was used. A significant maximum accuracy rate of 97.8% was achieved through CCA-based gait identification. Beyond that, the rate of false identifications and unrecognized gaits went down to half, demonstrating state-of-the-art for gait identification.
{"title":"3-D Gait Identification Utilizing Latent Canonical Covariates Consisting of Gait Features","authors":"Ramiz Gorkem Birdal, Ahmet Sertbas","doi":"10.32604/cmc.2023.032069","DOIUrl":"https://doi.org/10.32604/cmc.2023.032069","url":null,"abstract":"Biometric gait recognition is a lesser-known but emerging and effective biometric recognition method which enables subjects’ walking patterns to be recognized. Existing research in this area has primarily focused on feature analysis through the extraction of individual features, which captures most of the information but fails to capture subtle variations in gait dynamics. Therefore, a novel feature taxonomy and an approach for deriving a relationship between a function of one set of gait features with another set are introduced. The gait features extracted from body halves divided by anatomical planes on vertical, horizontal, and diagonal axes are grouped to form canonical gait covariates. Canonical Correlation Analysis is utilized to measure the strength of association between the canonical covariates of gait. Thus, gait assessment and identification are enhanced when more semantic information is available through CCA-based multi-feature fusion. Hence, Carnegie Mellon University’s 3D gait database, which contains 32 gait samples taken at different paces, is utilized in analyzing gait characteristics. The performance of Linear Discriminant Analysis, K-Nearest Neighbors, Naive Bayes, Artificial Neural Networks, and Support Vector Machines was improved by a 4% average when the CCA-utilized gait identification approach was used. A significant maximum accuracy rate of 97.8% was achieved through CCA-based gait identification. Beyond that, the rate of false identifications and unrecognized gaits went down to half, demonstrating state-of-the-art for gait identification.","PeriodicalId":93535,"journal":{"name":"Computers, materials & continua","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136052701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：