Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892637
Tao Zhang, Shiliang Sun, Jing Zhao
Cross-modal retrieval is usually implemented based on cross-modal representation learning, which is used to extract semantic information from cross-modal data. Recent work shows that cross-modal representation learning is vulnerable to adversarial attacks, even using large-scale pre-trained networks. By attacking the representation, it can be simple to attack the downstream tasks, especially for cross-modal retrieval tasks. Adversarial attacks on any modality will easily lead to obvious retrieval errors, which brings the challenge to improve the adversarial robustness of cross-modal retrieval. In this paper, we propose a robust cross-modal retrieval method (RoCMR), which generates adversarial examples for both the query modality and candidate modality and performs adversarial training for cross-modal retrieval. Specifically, we generate adversarial examples for both image and text modalities and train the model with benign and adversarial examples in the framework of contrastive learning. We evaluate the proposed RoCMR on two datasets and show its effectiveness in defending against gradient-based attacks.
{"title":"Robust Cross-Modal Retrieval by Adversarial Training","authors":"Tao Zhang, Shiliang Sun, Jing Zhao","doi":"10.1109/IJCNN55064.2022.9892637","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892637","url":null,"abstract":"Cross-modal retrieval is usually implemented based on cross-modal representation learning, which is used to extract semantic information from cross-modal data. Recent work shows that cross-modal representation learning is vulnerable to adversarial attacks, even using large-scale pre-trained networks. By attacking the representation, it can be simple to attack the downstream tasks, especially for cross-modal retrieval tasks. Adversarial attacks on any modality will easily lead to obvious retrieval errors, which brings the challenge to improve the adversarial robustness of cross-modal retrieval. In this paper, we propose a robust cross-modal retrieval method (RoCMR), which generates adversarial examples for both the query modality and candidate modality and performs adversarial training for cross-modal retrieval. Specifically, we generate adversarial examples for both image and text modalities and train the model with benign and adversarial examples in the framework of contrastive learning. We evaluate the proposed RoCMR on two datasets and show its effectiveness in defending against gradient-based attacks.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134460170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892132
Zhaocheng Zhang, Gang Yang
With the purpose of addressing the scarcity of attribute diversity in Zero-shot Learning (ZSL), we propose to search for additional attributes in embedding space to extend the class embedding, providing a more discriminative representation of the class prototype. Meanwhile, to tackle the inherent noise behind manually annotated attributes, we apply multi-layer convolutional processing on semantic features rather than conventional linear transformation for filtering. Moreover, we employ Center Loss to assist the training stage, which helps the learned mapping be more accurate and consistent with the corresponding class's prototype. Combining these modules mentioned above, extensive experiments on several public datasets show that our method could yield decent improvements. This proposed way of extending attributes can also be migrated to other models or tasks and obtain better results.
{"title":"Exploring Attribute Space with Word Embedding for Zero-shot Learning","authors":"Zhaocheng Zhang, Gang Yang","doi":"10.1109/IJCNN55064.2022.9892132","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892132","url":null,"abstract":"With the purpose of addressing the scarcity of attribute diversity in Zero-shot Learning (ZSL), we propose to search for additional attributes in embedding space to extend the class embedding, providing a more discriminative representation of the class prototype. Meanwhile, to tackle the inherent noise behind manually annotated attributes, we apply multi-layer convolutional processing on semantic features rather than conventional linear transformation for filtering. Moreover, we employ Center Loss to assist the training stage, which helps the learned mapping be more accurate and consistent with the corresponding class's prototype. Combining these modules mentioned above, extensive experiments on several public datasets show that our method could yield decent improvements. This proposed way of extending attributes can also be migrated to other models or tasks and obtain better results.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134490477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892401
Adedapo Alabi, D. Vanderelst, A. Minai
The hippocampus in rodents encodes physical space using place cells that show maximal firing in specific regions of space - their place fields. These place cells are reused across different contexts and environments with uncorrelated place fields. Though place fields are known to depend on distal sensory cues, even identical environments can have completely different place fields if the contexts are different. We propose a novel place cell network model for this feature using two frequently overlooked aspects of neural computation - dendritic morphology and the spatial co-location of spatiotemporally co-active afferent synapses - and show that these enable the reuse of place cells to encode different maps for environments with identical sensory cues.
{"title":"Context-Dependent Spatial Representations in the Hippocampus using Place Cell Dendritic Computation","authors":"Adedapo Alabi, D. Vanderelst, A. Minai","doi":"10.1109/IJCNN55064.2022.9892401","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892401","url":null,"abstract":"The hippocampus in rodents encodes physical space using place cells that show maximal firing in specific regions of space - their place fields. These place cells are reused across different contexts and environments with uncorrelated place fields. Though place fields are known to depend on distal sensory cues, even identical environments can have completely different place fields if the contexts are different. We propose a novel place cell network model for this feature using two frequently overlooked aspects of neural computation - dendritic morphology and the spatial co-location of spatiotemporally co-active afferent synapses - and show that these enable the reuse of place cells to encode different maps for environments with identical sensory cues.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131675731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892326
Senwen Li, Liang Ge, Yongquan Lin, Bo Zeng
Traffic flow forecasting is a significant issue in the field of transportation. Early works model temporal dependencies and spatial correlations, respectively. Recently, some models are proposed to capture spatial-temporal dependencies simultaneously. However, these models have three defects. Firstly, they only use the information of road network structure to construct graph structure. It may not accurately reflect the spatial-temporal correlations among nodes. Secondly, only the correlations among nodes adjacent in time or space are considered in each graph convolutional layer. Finally, it's challenging for them to describe that future traffic flow is influenced by different scale spatial-temporal information. In this paper, we propose a model called Adaptive Spatial-Temporal Fusion Graph Convolutional Networks to address these problems. Firstly, the model can find cross-time, cross-space correlations among nodes to adjust spatial-temporal graph structure by a learnable adaptive matrix. Secondly, it can help nodes attain a larger spatiotemporal receptive field through constructing spatial-temporal graphs of different time spans. At last, the results of various spatial-temporal scale graph convolutional layers are fused to produce node embedding for prediction. It helps find the different spatial-temporal ranges' influence for various nodes. Experiments are conducted on real-world traffic datasets, and results show that our model outperforms the state-of-the-art baselines.
{"title":"Adaptive Spatial-Temporal Fusion Graph Convolutional Networks for Traffic Flow Forecasting","authors":"Senwen Li, Liang Ge, Yongquan Lin, Bo Zeng","doi":"10.1109/IJCNN55064.2022.9892326","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892326","url":null,"abstract":"Traffic flow forecasting is a significant issue in the field of transportation. Early works model temporal dependencies and spatial correlations, respectively. Recently, some models are proposed to capture spatial-temporal dependencies simultaneously. However, these models have three defects. Firstly, they only use the information of road network structure to construct graph structure. It may not accurately reflect the spatial-temporal correlations among nodes. Secondly, only the correlations among nodes adjacent in time or space are considered in each graph convolutional layer. Finally, it's challenging for them to describe that future traffic flow is influenced by different scale spatial-temporal information. In this paper, we propose a model called Adaptive Spatial-Temporal Fusion Graph Convolutional Networks to address these problems. Firstly, the model can find cross-time, cross-space correlations among nodes to adjust spatial-temporal graph structure by a learnable adaptive matrix. Secondly, it can help nodes attain a larger spatiotemporal receptive field through constructing spatial-temporal graphs of different time spans. At last, the results of various spatial-temporal scale graph convolutional layers are fused to produce node embedding for prediction. It helps find the different spatial-temporal ranges' influence for various nodes. Experiments are conducted on real-world traffic datasets, and results show that our model outperforms the state-of-the-art baselines.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131675966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892650
Zekang Qiu, J. Zhao, Chudong Shan, Jianyong Huang, Zhiyong Yuan
Performing automatic retinal vessel segmentation on fundus image can obtain clear retinal vessel structure quickly, which will assist doctors to improve the efficiency and reliability of diagnosis. In fundus image, there are many small vessels and some areas with low contrast, and there may be abnormal areas. Therefore, achieving automatic retinal vessel segmentation with high performance is still challenging. The retinal vessel in the image is a topological structure, so the distribution of retinal vessel pixels in each pixel row (or column) should have some relationship to other rows (or columns). Motivated by this observation, we propose Pixel Rows and Columns Relationship Modeling Network (PRCRM-Net) to achieve high-performance retinal vessel segmentation. PRCRM-Net separately models the relationship between different pixel rows and pixel columns of fundus image, and achieves retinal vessel segmentation by classifying the pixels in units of pixel row and pixel column. The input of PRCRM-Net is the feature map extracted by U-Net. PRCRM-Net firstly processes the input feature map into row feature sequence and column feature sequence respectively. Secondly, it models the relationship between the elements in the row feature sequence and column feature sequence respectively based on Transformer. Finally, the updated row feature sequence and column feature sequence are used to obtain row-based segmentation result and column-based segmentation result respectively. And the final segmentation result is the combination of these two types of results. To evaluate the performance of PRCRM-Net, we conduct comprehensive experiments on three representative datasets, DRIVE, STARE and CHASE_DB1. The experiment results show that the proposed PRCRM-Net achieves state-of-the-art performance.
{"title":"Pixel Rows and Columns Relationship Modeling Network based on Transformer for Retinal Vessel Segmentation","authors":"Zekang Qiu, J. Zhao, Chudong Shan, Jianyong Huang, Zhiyong Yuan","doi":"10.1109/IJCNN55064.2022.9892650","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892650","url":null,"abstract":"Performing automatic retinal vessel segmentation on fundus image can obtain clear retinal vessel structure quickly, which will assist doctors to improve the efficiency and reliability of diagnosis. In fundus image, there are many small vessels and some areas with low contrast, and there may be abnormal areas. Therefore, achieving automatic retinal vessel segmentation with high performance is still challenging. The retinal vessel in the image is a topological structure, so the distribution of retinal vessel pixels in each pixel row (or column) should have some relationship to other rows (or columns). Motivated by this observation, we propose Pixel Rows and Columns Relationship Modeling Network (PRCRM-Net) to achieve high-performance retinal vessel segmentation. PRCRM-Net separately models the relationship between different pixel rows and pixel columns of fundus image, and achieves retinal vessel segmentation by classifying the pixels in units of pixel row and pixel column. The input of PRCRM-Net is the feature map extracted by U-Net. PRCRM-Net firstly processes the input feature map into row feature sequence and column feature sequence respectively. Secondly, it models the relationship between the elements in the row feature sequence and column feature sequence respectively based on Transformer. Finally, the updated row feature sequence and column feature sequence are used to obtain row-based segmentation result and column-based segmentation result respectively. And the final segmentation result is the combination of these two types of results. To evaluate the performance of PRCRM-Net, we conduct comprehensive experiments on three representative datasets, DRIVE, STARE and CHASE_DB1. The experiment results show that the proposed PRCRM-Net achieves state-of-the-art performance.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124252526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892289
Kunyuan Pang, Shasha Li, Jintao Tang, Ting Wang
Entity annotation in Wikipedia (officially named wikilinks) greatly benefits human end-users. Human editors are required to select all mentions that are most helpful to human end-users and link each mention to a Wikipedia page. We aim to design an automatic system to generate Wikipedia-style entity annotation for any plain text. However, existing research either rely heavily on mention-entity map or are restricted to named entities only. Besides, they neglect to select the appropriate mentions as Wikipedia requires. As a result, they leave out some necessary annotation and introduce excessive distracting annotation. Existing benchmarks also skirt around the coverage and selection issues. We propose a new task called Mention Detection and Se-lection for entity annotation, along with a new benchmark, WikiC, to better reflect annotation quality. The task is coined centering mentions specific to each position in high-quality human-annotated examples. We also proposed a new framework, DrWiki, to fulfill the task. We adopt a deep pre-trained span selection model inferring directly from plain text via tokens' context embedding. It can cover all possible spans and avoid limiting to mention-entity maps. In addition, information of both inarguable mention-entity pairs, and mention repeat has been introduced as token-wise representation enhancement by FLAT attention and repeat embedding respectively. Empirical results on WikiC show that, compared with often adopted and state-of-the-art Entity Linking and Entity Recognition methods, our method achieves improvement to previous methods in overall performance. Additional experiments show that DrWiki gains improvement even with a low-coverage mention-entity map.
{"title":"Multi-source Representation Enhancement for Wikipedia-style Entity Annotation","authors":"Kunyuan Pang, Shasha Li, Jintao Tang, Ting Wang","doi":"10.1109/IJCNN55064.2022.9892289","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892289","url":null,"abstract":"Entity annotation in Wikipedia (officially named wikilinks) greatly benefits human end-users. Human editors are required to select all mentions that are most helpful to human end-users and link each mention to a Wikipedia page. We aim to design an automatic system to generate Wikipedia-style entity annotation for any plain text. However, existing research either rely heavily on mention-entity map or are restricted to named entities only. Besides, they neglect to select the appropriate mentions as Wikipedia requires. As a result, they leave out some necessary annotation and introduce excessive distracting annotation. Existing benchmarks also skirt around the coverage and selection issues. We propose a new task called Mention Detection and Se-lection for entity annotation, along with a new benchmark, WikiC, to better reflect annotation quality. The task is coined centering mentions specific to each position in high-quality human-annotated examples. We also proposed a new framework, DrWiki, to fulfill the task. We adopt a deep pre-trained span selection model inferring directly from plain text via tokens' context embedding. It can cover all possible spans and avoid limiting to mention-entity maps. In addition, information of both inarguable mention-entity pairs, and mention repeat has been introduced as token-wise representation enhancement by FLAT attention and repeat embedding respectively. Empirical results on WikiC show that, compared with often adopted and state-of-the-art Entity Linking and Entity Recognition methods, our method achieves improvement to previous methods in overall performance. Additional experiments show that DrWiki gains improvement even with a low-coverage mention-entity map.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114832372","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892939
Fabian Gerz, Tolga Renan Bastürk, Julian Kirchhoff, Joachim Denker, L. Al-Shrouf, M. Jelali
The occurrence of anomalies and unexpected, process-related faults is a major problem for manufacturing systems, which has a significant impact on product quality. Early detection of anomalies is therefore of central importance in order to create sufficient room for maneuver to take countermeasures and ensure product quality. This paper investigates the performance of machine learning (ML) algorithms for anomaly detection in sensor data streams. For this purpose, the performance of six ML algorithms (K-means, DBSCAN, Isolation Forest, OCSVM, LSTM-Network, and DeepAnt) is evaluated based on defined performance metrics. These methods are benchmarked on publicly available datasets, own synthetic datasets, and novel industrial datasets. The latter include radar sensor datasets from a hot rolling mill. Research results show a high detection performance of K-means algorithm, DBSCAN algorithm and LSTM network for punctual, collective and contextual anomalies. A decentralized strategy for (real-time) anomaly detection using sensor data streams is proposed and an industrial (Cloud-Edge Computing) platform is developed and implemented for this purpose.
{"title":"A Comparative Study and a New Industrial Platform for Decentralized Anomaly Detection Using Machine Learning Algorithms","authors":"Fabian Gerz, Tolga Renan Bastürk, Julian Kirchhoff, Joachim Denker, L. Al-Shrouf, M. Jelali","doi":"10.1109/IJCNN55064.2022.9892939","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892939","url":null,"abstract":"The occurrence of anomalies and unexpected, process-related faults is a major problem for manufacturing systems, which has a significant impact on product quality. Early detection of anomalies is therefore of central importance in order to create sufficient room for maneuver to take countermeasures and ensure product quality. This paper investigates the performance of machine learning (ML) algorithms for anomaly detection in sensor data streams. For this purpose, the performance of six ML algorithms (K-means, DBSCAN, Isolation Forest, OCSVM, LSTM-Network, and DeepAnt) is evaluated based on defined performance metrics. These methods are benchmarked on publicly available datasets, own synthetic datasets, and novel industrial datasets. The latter include radar sensor datasets from a hot rolling mill. Research results show a high detection performance of K-means algorithm, DBSCAN algorithm and LSTM network for punctual, collective and contextual anomalies. A decentralized strategy for (real-time) anomaly detection using sensor data streams is proposed and an industrial (Cloud-Edge Computing) platform is developed and implemented for this purpose.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114931615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892073
Andrea Falanti, Eugenio Lomurno, Stefano Samele, D. Ardagna, Matteo Matteucci
Automating the research for the best neural network model is a task that has gained more and more relevance in the last few years. In this context, Neural Architecture Search (NAS) represents the most effective technique whose results rival the state of the art hand-crafted architectures. However, this approach requires a lot of computational capabilities as well as research time, which make prohibitive its usage in many real-world scenarios. With its sequential model-based optimization strategy, Progressive Neural Architecture Search (PNAS) represents a possible step forward to face this resources issue. Despite the quality of the found network architectures, this technique is still limited in research time. A significant step in this direction has been done by Pareto-Optimal Progressive Neural Architecture Search (POPNAS), which expand PNAS with a time predictor to enable a trade-off between search time and accuracy, considering a multi-objective optimization problem. This paper proposes a new version of the Pareto-Optimal Progressive Neural Architecture Search, called POPNASv2. Our approach enhances its first version and improves its performance. We expanded the search space by adding new operators and improved the quality of both predictors to build more accurate Pareto fronts. Moreover, we introduced cell equivalence checks and enriched the search strategy with an adaptive greedy exploration step. Our efforts allow POPNASv2 to achieve PNAS-like performance with an average 4x factor search time speed-up. Code: https://doi.org/10.5281/zenodo.6574040
{"title":"POPNASv2: An Efficient Multi-Objective Neural Architecture Search Technique","authors":"Andrea Falanti, Eugenio Lomurno, Stefano Samele, D. Ardagna, Matteo Matteucci","doi":"10.1109/IJCNN55064.2022.9892073","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892073","url":null,"abstract":"Automating the research for the best neural network model is a task that has gained more and more relevance in the last few years. In this context, Neural Architecture Search (NAS) represents the most effective technique whose results rival the state of the art hand-crafted architectures. However, this approach requires a lot of computational capabilities as well as research time, which make prohibitive its usage in many real-world scenarios. With its sequential model-based optimization strategy, Progressive Neural Architecture Search (PNAS) represents a possible step forward to face this resources issue. Despite the quality of the found network architectures, this technique is still limited in research time. A significant step in this direction has been done by Pareto-Optimal Progressive Neural Architecture Search (POPNAS), which expand PNAS with a time predictor to enable a trade-off between search time and accuracy, considering a multi-objective optimization problem. This paper proposes a new version of the Pareto-Optimal Progressive Neural Architecture Search, called POPNASv2. Our approach enhances its first version and improves its performance. We expanded the search space by adding new operators and improved the quality of both predictors to build more accurate Pareto fronts. Moreover, we introduced cell equivalence checks and enriched the search strategy with an adaptive greedy exploration step. Our efforts allow POPNASv2 to achieve PNAS-like performance with an average 4x factor search time speed-up. Code: https://doi.org/10.5281/zenodo.6574040","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115027760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-18DOI: 10.1109/IJCNN55064.2022.9892540
Hao Luan, Mu Hua, Jigen Peng, Shigang Yue, Shengyong Chen, Qinbing Fu
In nature, crabs have a panoramic vision for the localization and perception of accelerating motion from local segments to global view in order to guide reactive behaviours including escape. The visual neuronal ensemble in crab plays crucial roles in such capability, however, has never been investigated and modelled as an artificial vision system. To bridge this gap, we propose an accelerating motion perception model (AMPM) mimicking the visual neuronal ensemble in crab. The AMPM includes two main parts, wherein the pre-synaptic network from the previous modelling work simulates 16 MLGI neurons covering the entire view to localize moving objects. The emphasis herein is laid on the original modelling of MLGIs' post-synaptic network to perceive accelerating motions from a global view, which employs a novel spatial-temporal difference encoder (STDE), and an adaptive spiking threshold temporal difference encoder (AT-TDE). Specifically, the STDE transforms “time-to-travel” between activations of two successive segments of MLG1 into excitatory post-synaptic current (EPSC), which decays with the elapse of time. The AT-TDE in two directional, i.e., counter-clockwise and clockwise accelerating detectors guarantees “non-firing” to constant movements. Accordingly, the accelerating motion can be effectively localized and perceived by the whole network. The systematic experiments verified the feasibility and robustness of the proposed method. The model responses to translational accelerating motion also fit many of the explored physiological features of direction selective neurons in the lobula complex of crab (i.e. lobula complex direction cells, LCDCs). This modelling study not only provides a reasonable hypothesis for such biological neural pathways, but is also critical for developing a new neuromorphic sensor strategy.
在自然界中，螃蟹具有从局部到全局加速运动的定位和感知全景视野，以指导包括逃跑在内的反应性行为。然而，螃蟹的视觉神经元集合在这种能力中起着至关重要的作用，但从未作为人工视觉系统进行过研究和建模。为了弥补这一差距，我们提出了一种模拟螃蟹视觉神经元集合的加速运动感知模型(AMPM)。AMPM包括两个主要部分，其中来自先前建模工作的突触前网络模拟了覆盖整个视图的16个MLGI神经元来定位运动物体。本文重点介绍了mlgi突触后网络的原始建模，从全局角度感知加速运动，该模型采用了一种新的时空差分编码器(STDE)和自适应尖峰阈值时间差分编码器(AT-TDE)。具体来说，STDE将MLG1两个连续片段激活之间的“旅行时间”转化为兴奋性突触后电流(EPSC)， EPSC随着时间的流逝而衰减。AT-TDE的两个方向，即逆时针和顺时针加速探测器，保证了恒定运动的“不着火”。因此，加速运动可以被整个网络有效地定位和感知。系统实验验证了该方法的可行性和鲁棒性。该模型对平移加速运动的响应也符合螃蟹小叶复合体(即小叶复合体方向细胞，lobula complex direction cells, LCDCs)中方向选择神经元的许多生理特征。该模型研究不仅为这种生物神经通路提供了合理的假设，而且对于开发新的神经形态传感器策略也至关重要。
{"title":"Accelerating Motion Perception Model Mimics the Visual Neuronal Ensemble of Crab","authors":"Hao Luan, Mu Hua, Jigen Peng, Shigang Yue, Shengyong Chen, Qinbing Fu","doi":"10.1109/IJCNN55064.2022.9892540","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892540","url":null,"abstract":"In nature, crabs have a panoramic vision for the localization and perception of accelerating motion from local segments to global view in order to guide reactive behaviours including escape. The visual neuronal ensemble in crab plays crucial roles in such capability, however, has never been investigated and modelled as an artificial vision system. To bridge this gap, we propose an accelerating motion perception model (AMPM) mimicking the visual neuronal ensemble in crab. The AMPM includes two main parts, wherein the pre-synaptic network from the previous modelling work simulates 16 MLGI neurons covering the entire view to localize moving objects. The emphasis herein is laid on the original modelling of MLGIs' post-synaptic network to perceive accelerating motions from a global view, which employs a novel spatial-temporal difference encoder (STDE), and an adaptive spiking threshold temporal difference encoder (AT-TDE). Specifically, the STDE transforms “time-to-travel” between activations of two successive segments of MLG1 into excitatory post-synaptic current (EPSC), which decays with the elapse of time. The AT-TDE in two directional, i.e., counter-clockwise and clockwise accelerating detectors guarantees “non-firing” to constant movements. Accordingly, the accelerating motion can be effectively localized and perceived by the whole network. The systematic experiments verified the feasibility and robustness of the proposed method. The model responses to translational accelerating motion also fit many of the explored physiological features of direction selective neurons in the lobula complex of crab (i.e. lobula complex direction cells, LCDCs). This modelling study not only provides a reasonable hypothesis for such biological neural pathways, but is also critical for developing a new neuromorphic sensor strategy.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117030058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We present a new neighbor sampling method on temporal graphs. In a temporal graph, predicting different nodes' time-varying properties can require the receptive neigh-borhood of various temporal scales. In this work, we propose the TNS (Time-aware Neighbor Sampling) method: TNS learns from temporal information to provide an adaptive receptive neighborhood for every node at any time. Learning how to sample neighbors is non-trivial, since the neighbor indices in time order are discrete and not differentiable. To address this challenge, we transform neighbor indices from discrete values to continuous ones by interpolating the neighbors' messages. TNS can be flexibly incorporated into popular temporal graph networks to improve their effectiveness without increasing their time complexity. TNS can be trained in an end-to-end manner. It requires no extra supervision and is automatically and implicitly guided to sample the neighbors that are most beneficial for prediction. Empirical results on multiple standard datasets show that TNS yields significant gains on edge prediction and node classification.
{"title":"Time-Aware Neighbor Sampling on Temporal Graphs","authors":"Yiwei Wang, Yujun Cai, Yuxuan Liang, Henghui Ding, Changhu Wang, Bryan Hooi","doi":"10.1109/IJCNN55064.2022.9892942","DOIUrl":"https://doi.org/10.1109/IJCNN55064.2022.9892942","url":null,"abstract":"We present a new neighbor sampling method on temporal graphs. In a temporal graph, predicting different nodes' time-varying properties can require the receptive neigh-borhood of various temporal scales. In this work, we propose the TNS (Time-aware Neighbor Sampling) method: TNS learns from temporal information to provide an adaptive receptive neighborhood for every node at any time. Learning how to sample neighbors is non-trivial, since the neighbor indices in time order are discrete and not differentiable. To address this challenge, we transform neighbor indices from discrete values to continuous ones by interpolating the neighbors' messages. TNS can be flexibly incorporated into popular temporal graph networks to improve their effectiveness without increasing their time complexity. TNS can be trained in an end-to-end manner. It requires no extra supervision and is automatically and implicitly guided to sample the neighbors that are most beneficial for prediction. Empirical results on multiple standard datasets show that TNS yields significant gains on edge prediction and node classification.","PeriodicalId":106974,"journal":{"name":"2022 International Joint Conference on Neural Networks (IJCNN)","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2022-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117043023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：