Pub Date : 2022-11-30DOI: 10.1109/DICTA56598.2022.10034570
Deep learning methods have demonstrated stateof-the-art performance for the segmentation of the retina and choroid in optical coherence tomography (OCT) images. These methods are automatic and fast, yielding high accuracy and precision, thus reducing the load of manual analysis. However, deep learning usually requires large amounts of diverse, labelled data for training which can be difficult or infeasible to obtain, especially for medical images. For example, privacy concerns and lack of confidentiality agreements are common and are an obstacle to the sharing of useful training data. Additionally, some data can be significantly more difficult to obtain in the first place such as that of rare pathologies. Even in cases where sufficient data is available, the cost and time to perform image labelling can be significant. In many cases, data augmentation is employed to enhance the size of the training set. Similarly, semisupervised learning (SSL) can be used to exploit potentially large amounts of unlabeled data which would otherwise be unused. Motivated by this, in this study, we propose an enhanced StyleGAN2-based data augmentation method for OCT images by employing SSL through a novel crosslocalisation technique. For OCT image patches, the proposed method significantly improved the classification accuracy over the previous GAN data augmentation approach which uses labelled data only. The technique works by automatically learning, mixing, and injecting unlabelled styles into the labelled data to further increase the diversity of the synthetic data. The proposed method can be trained using differing quantities of both labelled and unlabelled data simultaneously. The method is simple, effective, generalizable and can be easily applied and used to extend StyleGAN2. Hence, there is also significant potential for the proposed method to be applied to other domains and imaging modalities for data augmentation purposes where unlabelled data exists.
{"title":"Semi-supervised learning with cross-localisation in shared GAN latent space for enhanced OCT data augmentation","authors":"","doi":"10.1109/DICTA56598.2022.10034570","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034570","url":null,"abstract":"Deep learning methods have demonstrated stateof-the-art performance for the segmentation of the retina and choroid in optical coherence tomography (OCT) images. These methods are automatic and fast, yielding high accuracy and precision, thus reducing the load of manual analysis. However, deep learning usually requires large amounts of diverse, labelled data for training which can be difficult or infeasible to obtain, especially for medical images. For example, privacy concerns and lack of confidentiality agreements are common and are an obstacle to the sharing of useful training data. Additionally, some data can be significantly more difficult to obtain in the first place such as that of rare pathologies. Even in cases where sufficient data is available, the cost and time to perform image labelling can be significant. In many cases, data augmentation is employed to enhance the size of the training set. Similarly, semisupervised learning (SSL) can be used to exploit potentially large amounts of unlabeled data which would otherwise be unused. Motivated by this, in this study, we propose an enhanced StyleGAN2-based data augmentation method for OCT images by employing SSL through a novel crosslocalisation technique. For OCT image patches, the proposed method significantly improved the classification accuracy over the previous GAN data augmentation approach which uses labelled data only. The technique works by automatically learning, mixing, and injecting unlabelled styles into the labelled data to further increase the diversity of the synthetic data. The proposed method can be trained using differing quantities of both labelled and unlabelled data simultaneously. The method is simple, effective, generalizable and can be easily applied and used to extend StyleGAN2. Hence, there is also significant potential for the proposed method to be applied to other domains and imaging modalities for data augmentation purposes where unlabelled data exists.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128998040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-11-30DOI: 10.1109/DICTA56598.2022.10034628
Developing a reliable high-level perception system that can work stably in different environments is highly useful, especially in autonomous driving tasks. Many previous studies have investigated extreme cases such as dark, rainy and foggy environments and proposed various datasets for these different tasks. In this work, we explore another extreme case: destructive over-exposure which may result in different degrees of content loss due to the limitations of dynamic range. These over-exposure cases can be found in most outdoor datasets with structured or unstructured environments but are usually neglected as they are mixed with other well-exposed images. To analyse the influence imposed by this kind of corruption, we generate realistic over-exposed images based on existing outdoor datasets using a simple but controllable formula proposed in a photographer's view. Our simulation is realistic, indicated by similar illumination distributions to other real over-exposed images. We also conduct several experiments on our over-exposed datasets and discover performance drops using state-of-the-art segmentation models. Subsequently, to address the over-exposure problem, we compare several image restoration approaches for over-exposure recovery and demonstrate their potential effectiveness as a preprocessing step in scene parsing tasks.
{"title":"Analysis of the Over-Exposure Problem for Robust Scene Parsing","authors":"","doi":"10.1109/DICTA56598.2022.10034628","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034628","url":null,"abstract":"Developing a reliable high-level perception system that can work stably in different environments is highly useful, especially in autonomous driving tasks. Many previous studies have investigated extreme cases such as dark, rainy and foggy environments and proposed various datasets for these different tasks. In this work, we explore another extreme case: destructive over-exposure which may result in different degrees of content loss due to the limitations of dynamic range. These over-exposure cases can be found in most outdoor datasets with structured or unstructured environments but are usually neglected as they are mixed with other well-exposed images. To analyse the influence imposed by this kind of corruption, we generate realistic over-exposed images based on existing outdoor datasets using a simple but controllable formula proposed in a photographer's view. Our simulation is realistic, indicated by similar illumination distributions to other real over-exposed images. We also conduct several experiments on our over-exposed datasets and discover performance drops using state-of-the-art segmentation models. Subsequently, to address the over-exposure problem, we compare several image restoration approaches for over-exposure recovery and demonstrate their potential effectiveness as a preprocessing step in scene parsing tasks.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129489080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-11-30DOI: 10.1109/DICTA56598.2022.10034569
Advancements in deep learning make it increasingly easy to produce highly realistic fake images and videos (also known as deepfakes), which could undermine trust in public discourse and pose threats to national and economic security. Despite diligent efforts that have been made to develop deepfake detection techniques, existing approaches often generalize poorly when the characteristics of new data and tasks differ significantly from the ones involved in their initial training phase. The detectors' limited generalizability hinders their widespread adoption if they cannot handle unseen manipulations in an open set. One solution to this issue is to endow the detectors with the capability of lifelong learning from the new data to improve themselves. However, it is not uncommon in real-world scenarios that the amount of training data associated with a certain deepfake algorithm is limited. Therefore, the effectiveness and agility of a continual learning scheme depend heavily on its ability to learn from limited new data. In this work, we propose a deepfake detection approach that combines spectral analysis and continual learning methods to pave the way towards generalized deepfake detection with limited new data. We demonstrate the generalization capability of the proposed approach through experiments using five datasets of deepfakes. The experiment results show that our proposed approach is effective in addressing catastrophic forgetting despite being updated with limited new data, decreasing the average forgetting rate by 35.04% and increasing the average accuracy by 22.45% compared without continual learning.
{"title":"Towards Generalized Deepfake Detection With Continual Learning On Limited New Data: Anonymous Authors","authors":"","doi":"10.1109/DICTA56598.2022.10034569","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034569","url":null,"abstract":"Advancements in deep learning make it increasingly easy to produce highly realistic fake images and videos (also known as deepfakes), which could undermine trust in public discourse and pose threats to national and economic security. Despite diligent efforts that have been made to develop deepfake detection techniques, existing approaches often generalize poorly when the characteristics of new data and tasks differ significantly from the ones involved in their initial training phase. The detectors' limited generalizability hinders their widespread adoption if they cannot handle unseen manipulations in an open set. One solution to this issue is to endow the detectors with the capability of lifelong learning from the new data to improve themselves. However, it is not uncommon in real-world scenarios that the amount of training data associated with a certain deepfake algorithm is limited. Therefore, the effectiveness and agility of a continual learning scheme depend heavily on its ability to learn from limited new data. In this work, we propose a deepfake detection approach that combines spectral analysis and continual learning methods to pave the way towards generalized deepfake detection with limited new data. We demonstrate the generalization capability of the proposed approach through experiments using five datasets of deepfakes. The experiment results show that our proposed approach is effective in addressing catastrophic forgetting despite being updated with limited new data, decreasing the average forgetting rate by 35.04% and increasing the average accuracy by 22.45% compared without continual learning.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115174462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-11-29DOI: 10.1109/DICTA56598.2022.10034566
Matthew Howe, Boris Repasky, Timothy Payne
Utilising a single point cloud segmentation model can be desirable in situations where point cloud source, quality, and content is unknown. In these situations the segmentation model must be able to handle these variations with predictable and consistent results. Although deep learning can segment point clouds accurately it often suffers with generalisation, adapting poorly to data which is different than the data it was trained on. To address this issue, we propose to utilise multiple available open source fully annotated datasets to train and test models that are better able to generalise. The open-source datasets we utilise are DublinCity, DALES, ISPRS, Swiss3DCities, SensatUrban, SUM, and H3D [5], [11], [10], [1], [3], [2], [6]. In this paper we discuss the combination of these datasets into a simple training set and challenging test set which evaluates multiple aspects of the generalisation task. We show that a naive combination and training produces improved results as expected. We also show that an improved sampling strategy which decreases sampling variations increases the generalisation performance substantially on top of this. Experiments to find the contributing factor of which variables give this performance boost found that none individually boost performance and rather it is the consistency of samples the model is evaluated on which yields this improvement.
{"title":"Effective Utilisation of Multiple Open-Source Datasets to Improve Generalisation Performance of Point Cloud Segmentation Models","authors":"Matthew Howe, Boris Repasky, Timothy Payne","doi":"10.1109/DICTA56598.2022.10034566","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034566","url":null,"abstract":"Utilising a single point cloud segmentation model can be desirable in situations where point cloud source, quality, and content is unknown. In these situations the segmentation model must be able to handle these variations with predictable and consistent results. Although deep learning can segment point clouds accurately it often suffers with generalisation, adapting poorly to data which is different than the data it was trained on. To address this issue, we propose to utilise multiple available open source fully annotated datasets to train and test models that are better able to generalise. The open-source datasets we utilise are DublinCity, DALES, ISPRS, Swiss3DCities, SensatUrban, SUM, and H3D [5], [11], [10], [1], [3], [2], [6]. In this paper we discuss the combination of these datasets into a simple training set and challenging test set which evaluates multiple aspects of the generalisation task. We show that a naive combination and training produces improved results as expected. We also show that an improved sampling strategy which decreases sampling variations increases the generalisation performance substantially on top of this. Experiments to find the contributing factor of which variables give this performance boost found that none individually boost performance and rather it is the consistency of samples the model is evaluated on which yields this improvement.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125226494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-31DOI: 10.1109/DICTA56598.2022.10034593
Zihao Tang, Xinyi Wang, Lihaowen Zhu, M. Cabezas, Dongnan Liu, Michael H Barnett, Weidong (Tom) Cai, Chengyu Wang
Diffusion Weighted Imaging (DWI) is an advanced imaging technique commonly used in neuroscience and neurological clinical research through a Diffusion Tensor Imaging (DTI) model. Volumetric scalar metrics including fractional anisotropy, mean diffusivity, and axial diffusivity can be derived from the DTI model to summarise water diffusivity and other quantitative microstructural information for clinical studies. However, clinical practice constraints can lead to sub-optimal DWI acquisitions with missing slices (either due to a limited field of view or the acquisition of disrupted slices). To avoid discarding valuable subjects for group-wise studies, we propose a novel 3D Tensor-Wise Brain-Aware Gate network (TW-BAG) for inpainting disrupted DTIs. The proposed method is tailored to the problem with a dynamic gate mechanism and independent tensor-wise decoders. We evaluated the proposed method on the publicly available Human Connectome Project (HCP) dataset using common image similarity metrics derived from the predicted tensors and scalar DTI metrics. Our experimental results show that the proposed approach can reconstruct the original brain DTI volume and recover relevant clinical imaging information.
{"title":"TW-BAG: Tensor-wise Brain-aware Gate Network for Inpainting Disrupted Diffusion Tensor Imaging","authors":"Zihao Tang, Xinyi Wang, Lihaowen Zhu, M. Cabezas, Dongnan Liu, Michael H Barnett, Weidong (Tom) Cai, Chengyu Wang","doi":"10.1109/DICTA56598.2022.10034593","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034593","url":null,"abstract":"Diffusion Weighted Imaging (DWI) is an advanced imaging technique commonly used in neuroscience and neurological clinical research through a Diffusion Tensor Imaging (DTI) model. Volumetric scalar metrics including fractional anisotropy, mean diffusivity, and axial diffusivity can be derived from the DTI model to summarise water diffusivity and other quantitative microstructural information for clinical studies. However, clinical practice constraints can lead to sub-optimal DWI acquisitions with missing slices (either due to a limited field of view or the acquisition of disrupted slices). To avoid discarding valuable subjects for group-wise studies, we propose a novel 3D Tensor-Wise Brain-Aware Gate network (TW-BAG) for inpainting disrupted DTIs. The proposed method is tailored to the problem with a dynamic gate mechanism and independent tensor-wise decoders. We evaluated the proposed method on the publicly available Human Connectome Project (HCP) dataset using common image similarity metrics derived from the predicted tensors and scalar DTI metrics. Our experimental results show that the proposed approach can reconstruct the original brain DTI volume and recover relevant clinical imaging information.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130601913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-21DOI: 10.1109/DICTA56598.2022.10034585
Rabindra Dulal, Lihong Zheng, M. A. Kabir, S. McGrath, J. Medway, D. Swain, Will Swain
You Only Look Once (YOLO) is a single-stage object detection model popular for real-time object detection, accuracy, and speed. This paper investigates the YOLOv5 model to identify cattle in the yards. The current solution to cattle identification includes radio-frequency identification (RFID) tags. The problem occurs when the RFID tag is lost or damaged. A biometric solution identifies the cattle and helps to assign the lost or damaged tag or replace the RFID-based system. Muzzle patterns in cattle are unique biometric solutions like a fingerprint in humans. This paper aims to present our recent research in utilizing five popular object detection models, looking at the architecture of YOLOv5, investigating the performance of eight backbones with the YOLOv5 model, and the influence of mosaic augmentation in YOLOv5 by experimental results on the available cattle muzzle images. Finally, we concluded with the excellent potential of using YOLOv5 in automatic cattle identification. Our experiments show YOLOv5 with transformer performed best with mean Average Precision (mAP)_0.5 (the average of AP when the IoU is greater than 50%) of 0.995, and mAP_0.5:0.95 (the average of AP from 50% to 95% IoU with an interval of 5%) of 0.9366. In addition, our experiments show the increase in accuracy of the model by using mosaic augmentation in all backbones used in our experiments. Moreover, we can also detect cattle with partial muzzle images.
You Only Look Once (YOLO)是一种单阶段对象检测模型，用于实时对象检测，准确性和速度。本文研究了YOLOv5模型在畜牧场牛群识别中的应用。目前的牛识别解决方案包括射频识别(RFID)标签。当RFID标签丢失或损坏时，就会出现问题。生物识别解决方案识别牛，并帮助分配丢失或损坏的标签或替换基于rfid的系统。牛的口吻图案是独特的生物识别解决方案，就像人类的指纹一样。本文介绍了我们利用5种流行的目标检测模型的最新研究成果，研究了YOLOv5的结构，研究了YOLOv5模型下8个骨干的性能，并通过实验结果研究了YOLOv5中马赛克增强对现有牛口枪口图像的影响。最后，我们认为YOLOv5在牛的自动识别中具有良好的应用潜力。我们的实验表明，带变压器的YOLOv5的平均平均精度(mAP)_0.5 (IoU大于50%时AP的平均值)为0.995,mAP_0.5:0.95 (IoU从50%到95%，间隔为5%)的平均值为0.9366。此外，我们的实验表明，通过在实验中使用的所有骨干网中使用马赛克增强，模型的精度得到了提高。此外，我们还可以用部分口吻图像来检测牛。
{"title":"Automatic Cattle Identification using YOLOv5 and Moasic Augmentation: A Comparative Analysis","authors":"Rabindra Dulal, Lihong Zheng, M. A. Kabir, S. McGrath, J. Medway, D. Swain, Will Swain","doi":"10.1109/DICTA56598.2022.10034585","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034585","url":null,"abstract":"You Only Look Once (YOLO) is a single-stage object detection model popular for real-time object detection, accuracy, and speed. This paper investigates the YOLOv5 model to identify cattle in the yards. The current solution to cattle identification includes radio-frequency identification (RFID) tags. The problem occurs when the RFID tag is lost or damaged. A biometric solution identifies the cattle and helps to assign the lost or damaged tag or replace the RFID-based system. Muzzle patterns in cattle are unique biometric solutions like a fingerprint in humans. This paper aims to present our recent research in utilizing five popular object detection models, looking at the architecture of YOLOv5, investigating the performance of eight backbones with the YOLOv5 model, and the influence of mosaic augmentation in YOLOv5 by experimental results on the available cattle muzzle images. Finally, we concluded with the excellent potential of using YOLOv5 in automatic cattle identification. Our experiments show YOLOv5 with transformer performed best with mean Average Precision (mAP)_0.5 (the average of AP when the IoU is greater than 50%) of 0.995, and mAP_0.5:0.95 (the average of AP from 50% to 95% IoU with an interval of 5%) of 0.9366. In addition, our experiments show the increase in accuracy of the model by using mosaic augmentation in all backbones used in our experiments. Moreover, we can also detect cattle with partial muzzle images.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134284302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-10-15DOI: 10.1109/DICTA56598.2022.10034573
T. M. Khan, Muhammad Arsalan, A. Robles-Kelly, E. Meijering
Image segmentation is an important task in medical imaging. It constitutes the backbone of a wide variety of clinical diagnostic methods, treatments, and computer-aided surgeries. In this paper, we propose a multi-kernel image segmentation net (MKIS-Net), which uses multiple kernels to create an efficient receptive field and enhance segmentation performance. As a result of its multi-kernel design, MKIS-Net is a light-weight architecture with a small number of trainable parameters. Moreover, these multi-kernel receptive fields also contribute to better segmentation results. We demonstrate the efficacy of MKIS-Net on several tasks including segmentation of retinal vessels, skin lesion segmentation, and chest X-ray segmentation. The performance of the proposed network is quite competitive, and often superior, in comparison to state-of-the-art methods. Moreover, in some cases MKIS-Net has more than an order of magnitude fewer trainable parameters than existing medical image segmentation alternatives and is at least four times smaller than other light-weight architectures.
{"title":"MKIS-Net: A Light-Weight Multi-Kernel Network for Medical Image Segmentation","authors":"T. M. Khan, Muhammad Arsalan, A. Robles-Kelly, E. Meijering","doi":"10.1109/DICTA56598.2022.10034573","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034573","url":null,"abstract":"Image segmentation is an important task in medical imaging. It constitutes the backbone of a wide variety of clinical diagnostic methods, treatments, and computer-aided surgeries. In this paper, we propose a multi-kernel image segmentation net (MKIS-Net), which uses multiple kernels to create an efficient receptive field and enhance segmentation performance. As a result of its multi-kernel design, MKIS-Net is a light-weight architecture with a small number of trainable parameters. Moreover, these multi-kernel receptive fields also contribute to better segmentation results. We demonstrate the efficacy of MKIS-Net on several tasks including segmentation of retinal vessels, skin lesion segmentation, and chest X-ray segmentation. The performance of the proposed network is quite competitive, and often superior, in comparison to state-of-the-art methods. Moreover, in some cases MKIS-Net has more than an order of magnitude fewer trainable parameters than existing medical image segmentation alternatives and is at least four times smaller than other light-weight architectures.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129729404","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-29DOI: 10.1109/DICTA56598.2022.10034623
Majid Nasiri, A. Cheraghian, T. Chowdhury, Sahar Ahmadi, Morteza Saberi, Shafin Rahman
Zero-shot learning on 3D point cloud data is a related underexplored problem compared to its 2D image counterpart. 3D data brings new challenges for ZSL due to the unavailability of robust pre-trained feature extraction models. To address this problem, we propose a prompt-guided 3D scene generation and supervision method that augments 3D data to learn the network better, exploring the complex interplay of seen and unseen objects. First, we merge point clouds of two 3D models in certain ways described by a prompt. The prompt acts like the annotation describing each 3D scene. Later, we perform contrastive learning to train our proposed architecture in an end-to-end manner. We argue that 3D scenes can relate objects more efficiently than single objects because popular language models (like BERT) can achieve high performance when objects appear in a context. Our proposed prompt-guided scene generation method encapsulates data augmentation and prompt-based annotation/captioning to improve 3D ZSL performance. We have achieved state-of-the-art ZSL and generalized ZSL performance on synthetic (ModelNet40, ModelNet10) and real-scanned (ScanOjbectNN) 3D object datasets.
{"title":"Prompt-guided Scene Generation for 3D Zero-Shot Learning","authors":"Majid Nasiri, A. Cheraghian, T. Chowdhury, Sahar Ahmadi, Morteza Saberi, Shafin Rahman","doi":"10.1109/DICTA56598.2022.10034623","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034623","url":null,"abstract":"Zero-shot learning on 3D point cloud data is a related underexplored problem compared to its 2D image counterpart. 3D data brings new challenges for ZSL due to the unavailability of robust pre-trained feature extraction models. To address this problem, we propose a prompt-guided 3D scene generation and supervision method that augments 3D data to learn the network better, exploring the complex interplay of seen and unseen objects. First, we merge point clouds of two 3D models in certain ways described by a prompt. The prompt acts like the annotation describing each 3D scene. Later, we perform contrastive learning to train our proposed architecture in an end-to-end manner. We argue that 3D scenes can relate objects more efficiently than single objects because popular language models (like BERT) can achieve high performance when objects appear in a context. Our proposed prompt-guided scene generation method encapsulates data augmentation and prompt-based annotation/captioning to improve 3D ZSL performance. We have achieved state-of-the-art ZSL and generalized ZSL performance on synthetic (ModelNet40, ModelNet10) and real-scanned (ScanOjbectNN) 3D object datasets.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127519777","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-09-29DOI: 10.1109/DICTA56598.2022.10034562
Avraham Chapman, Lingqiao Liu
It is well-known that a deep neural network has a strong fitting capability and can easily achieve a low training error even with randomly assigned class labels. When the number of training samples is small, or the class labels are noisy, networks tend to memorize patterns specific to individual instances to minimize the training error. This leads to the issue of overfitting and poor generalisation performance. This paper explores a remedy by suppressing the network's tendency to rely on instance-specific patterns for empirical error minimisation. The proposed method is based on an adversarial training framework. It suppresses features that can be utilized to identify individual instances among samples within each class. This leads to classifiers only using features that are both discriminative across classes and common within each class. We call our method Adversarial Suppression of Identity Features (ASIF), and demonstrate the usefulness of this technique in boosting generalisation accuracy when faced with small datasets or noisy labels. Our source code is available.
{"title":"Regularizing Neural Network Training via Identity-wise Discriminative Feature Suppression","authors":"Avraham Chapman, Lingqiao Liu","doi":"10.1109/DICTA56598.2022.10034562","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034562","url":null,"abstract":"It is well-known that a deep neural network has a strong fitting capability and can easily achieve a low training error even with randomly assigned class labels. When the number of training samples is small, or the class labels are noisy, networks tend to memorize patterns specific to individual instances to minimize the training error. This leads to the issue of overfitting and poor generalisation performance. This paper explores a remedy by suppressing the network's tendency to rely on instance-specific patterns for empirical error minimisation. The proposed method is based on an adversarial training framework. It suppresses features that can be utilized to identify individual instances among samples within each class. This leads to classifiers only using features that are both discriminative across classes and common within each class. We call our method Adversarial Suppression of Identity Features (ASIF), and demonstrate the usefulness of this technique in boosting generalisation accuracy when faced with small datasets or noisy labels. Our source code is available.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130724104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-04-13DOI: 10.1109/DICTA56598.2022.10034605
Zhixi Cai, Kalin Stefanov, Abhinav Dhall, Munawar Hayat
Due to its high societal impact, deepfake detection is getting active attention in the computer vision community. Most deepfake detection methods rely on identity, facial attributes, and adversarial perturbation-based spatio-temporal modifications at the whole video or random locations while keeping the meaning of the content intact. However, a sophisticated deepfake may contain only a small segment of video/audio manipulation, through which the meaning of the content can be, for example, completely inverted from a sentiment perspective. We introduce a content-driven audio-visual deepfake dataset, termed Localized Audio Visual DeepFake (LAV-DF), explicitly designed for the task of learning temporal forgery localization. Specifically, the content-driven audio-visual manipulations are performed strategically to change the sentiment polarity of the whole video. Our baseline method for benchmarking the proposed dataset is a 3DCNN model, termed as Boundary Aware Temporal Forgery Detection (BA-TFD), which is guided via contrastive, boundary matching, and frame classification loss functions. Our extensive quantitative and qualitative analysis demonstrates the proposed method's strong performance for temporal forgery localization and deepfake detection tasks.
{"title":"Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization: Anonymous submission Paper ID 73","authors":"Zhixi Cai, Kalin Stefanov, Abhinav Dhall, Munawar Hayat","doi":"10.1109/DICTA56598.2022.10034605","DOIUrl":"https://doi.org/10.1109/DICTA56598.2022.10034605","url":null,"abstract":"Due to its high societal impact, deepfake detection is getting active attention in the computer vision community. Most deepfake detection methods rely on identity, facial attributes, and adversarial perturbation-based spatio-temporal modifications at the whole video or random locations while keeping the meaning of the content intact. However, a sophisticated deepfake may contain only a small segment of video/audio manipulation, through which the meaning of the content can be, for example, completely inverted from a sentiment perspective. We introduce a content-driven audio-visual deepfake dataset, termed Localized Audio Visual DeepFake (LAV-DF), explicitly designed for the task of learning temporal forgery localization. Specifically, the content-driven audio-visual manipulations are performed strategically to change the sentiment polarity of the whole video. Our baseline method for benchmarking the proposed dataset is a 3DCNN model, termed as Boundary Aware Temporal Forgery Detection (BA-TFD), which is guided via contrastive, boundary matching, and frame classification loss functions. Our extensive quantitative and qualitative analysis demonstrates the proposed method's strong performance for temporal forgery localization and deepfake detection tasks.","PeriodicalId":159377,"journal":{"name":"2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114462972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：