Pub Date : 2022-09-02DOI: 10.3389/frsip.2022.984169
M. Allouche, M. Mitrea
The last decades have seen video production and consumption rise significantly: TV/cinematography, social networking, digital marketing, and video surveillance incrementally and cumulatively turned video content into the predilection type of data to be exchanged, stored, and processed. Belonging to video processing realm, video fingerprinting (also referred to as content-based copy detection or near duplicate detection) regroups research efforts devoted to identifying duplicated and/or replicated versions of a given video sequence (query) in a reference video dataset. The present paper reports on a state-of-the-art study on the past and present of video fingerprinting, while attempting to identify trends for its development. First, the conceptual basis and evaluation frameworks are set. This way, the methodological approaches (situated at the cross-roads of image processing, machine learning, and neural networks) can be structured and discussed. Finally, fingerprinting is confronted to the challenges raised by the emerging video applications (e.g., unmanned vehicles or fake news) and to the constraints they set in terms of content traceability and computational complexity. The relationship with other technologies for content tracking (e.g., DLT - Distributed Ledger Technologies) are also presented and discussed.
{"title":"Video fingerprinting: Past, present, and future","authors":"M. Allouche, M. Mitrea","doi":"10.3389/frsip.2022.984169","DOIUrl":"https://doi.org/10.3389/frsip.2022.984169","url":null,"abstract":"The last decades have seen video production and consumption rise significantly: TV/cinematography, social networking, digital marketing, and video surveillance incrementally and cumulatively turned video content into the predilection type of data to be exchanged, stored, and processed. Belonging to video processing realm, video fingerprinting (also referred to as content-based copy detection or near duplicate detection) regroups research efforts devoted to identifying duplicated and/or replicated versions of a given video sequence (query) in a reference video dataset. The present paper reports on a state-of-the-art study on the past and present of video fingerprinting, while attempting to identify trends for its development. First, the conceptual basis and evaluation frameworks are set. This way, the methodological approaches (situated at the cross-roads of image processing, machine learning, and neural networks) can be structured and discussed. Finally, fingerprinting is confronted to the challenges raised by the emerging video applications (e.g., unmanned vehicles or fake news) and to the constraints they set in terms of content traceability and computational complexity. The relationship with other technologies for content tracking (e.g., DLT - Distributed Ledger Technologies) are also presented and discussed.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"11 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82057395","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-26DOI: 10.3389/frsip.2022.815058
Sana Alamgeer, Mylène C. Q. Farias
Light Field (LF) cameras capture spatial and angular information of a scene, generating a high-dimensional data that brings several challenges to compression, transmission, and reconstruction algorithms. One research area that has been attracting a lot of attention is the design of Light Field images quality assessment (LF-IQA) methods. In this paper, we propose a No-Reference (NR) LF-IQA method that is based on reference-free distortion maps. With this goal, we first generate a synthetically distorted dataset of 2D images. Then, we compute SSIM distortion maps of these images and use these maps as ground error maps. We train a GAN architecture using these SSIM distortion maps as quality labels. This trained model is used to generate reference-free distortion maps of sub-aperture images of LF contents. Finally, the quality prediction is obtained performing the following steps: 1) perform a non-linear dimensionality reduction with a isometric mapping of the generated distortion maps to obtain the LFI feature vectors and 2) perform a regression using a Random Forest Regressor (RFR) algorithm to obtain the LF quality estimates. Results show that the proposed method is robust and accurate, outperforming several state-of-the-art LF-IQA methods.
{"title":"Blind visual quality assessment of light field images based on distortion maps","authors":"Sana Alamgeer, Mylène C. Q. Farias","doi":"10.3389/frsip.2022.815058","DOIUrl":"https://doi.org/10.3389/frsip.2022.815058","url":null,"abstract":"Light Field (LF) cameras capture spatial and angular information of a scene, generating a high-dimensional data that brings several challenges to compression, transmission, and reconstruction algorithms. One research area that has been attracting a lot of attention is the design of Light Field images quality assessment (LF-IQA) methods. In this paper, we propose a No-Reference (NR) LF-IQA method that is based on reference-free distortion maps. With this goal, we first generate a synthetically distorted dataset of 2D images. Then, we compute SSIM distortion maps of these images and use these maps as ground error maps. We train a GAN architecture using these SSIM distortion maps as quality labels. This trained model is used to generate reference-free distortion maps of sub-aperture images of LF contents. Finally, the quality prediction is obtained performing the following steps: 1) perform a non-linear dimensionality reduction with a isometric mapping of the generated distortion maps to obtain the LFI feature vectors and 2) perform a regression using a Random Forest Regressor (RFR) algorithm to obtain the LF quality estimates. Results show that the proposed method is robust and accurate, outperforming several state-of-the-art LF-IQA methods.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"194 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72459259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-23DOI: 10.3389/frsip.2022.904398
Pongsakorn Siripornpitak, Isaac Engel, Isaac Squires, Samuel J. Cooper, L. Picinali
Headphones-based spatial audio simulations rely on Head-related Transfer Functions (HRTFs) in order to reconstruct the sound field at the entrance of the listener’s ears. A HRTF is strongly dependent on the listener’s specific anatomical structures, and it has been shown that virtual sounds recreated with someone else’s HRTF result in worse localisation accuracy, as well as altering other subjective measures such as externalisation and realism. Acoustic measurements of the filtering effects generated by ears, head and torso has proven to be one of the most reliable ways to obtain a personalised HRTF. However this requires a dedicated and expensive setup, and is time-intensive. In order to simplify the measurement setup, thereby improving the scalability of the process, we are exploring strategies to reduce the number of acoustic measurements without degrading the spatial resolution of the HRTF. Traditionally, spatial up-sampling of HRTF sets is achieved through barycentric interpolation or by employing the spherical harmonics framework. However, such methods often perform poorly when the provided HRTF data is spatially very sparse. This work investigates the use of generative adversarial networks (GANs) to tackle the up-sampling problem, offering an initial insight about the suitability of this technique. Numerical evaluations based on spectral magnitude error and perceptual model outputs are presented on single spatial dimensions, therefore considering sources positioned only in one of the three main planes: Horizontal, median, and frontal. Results suggest that traditional HRTF interpolation methods perform better than the proposed GAN-based one when the distance between measurements is smaller than 90°, but for the sparsest conditions (i.e., one measurement every 120°–180°), the proposed approach outperforms the others.
{"title":"Spatial up-sampling of HRTF sets using generative adversarial networks: A pilot study","authors":"Pongsakorn Siripornpitak, Isaac Engel, Isaac Squires, Samuel J. Cooper, L. Picinali","doi":"10.3389/frsip.2022.904398","DOIUrl":"https://doi.org/10.3389/frsip.2022.904398","url":null,"abstract":"Headphones-based spatial audio simulations rely on Head-related Transfer Functions (HRTFs) in order to reconstruct the sound field at the entrance of the listener’s ears. A HRTF is strongly dependent on the listener’s specific anatomical structures, and it has been shown that virtual sounds recreated with someone else’s HRTF result in worse localisation accuracy, as well as altering other subjective measures such as externalisation and realism. Acoustic measurements of the filtering effects generated by ears, head and torso has proven to be one of the most reliable ways to obtain a personalised HRTF. However this requires a dedicated and expensive setup, and is time-intensive. In order to simplify the measurement setup, thereby improving the scalability of the process, we are exploring strategies to reduce the number of acoustic measurements without degrading the spatial resolution of the HRTF. Traditionally, spatial up-sampling of HRTF sets is achieved through barycentric interpolation or by employing the spherical harmonics framework. However, such methods often perform poorly when the provided HRTF data is spatially very sparse. This work investigates the use of generative adversarial networks (GANs) to tackle the up-sampling problem, offering an initial insight about the suitability of this technique. Numerical evaluations based on spectral magnitude error and perceptual model outputs are presented on single spatial dimensions, therefore considering sources positioned only in one of the three main planes: Horizontal, median, and frontal. Results suggest that traditional HRTF interpolation methods perform better than the proposed GAN-based one when the distance between measurements is smaller than 90°, but for the sparsest conditions (i.e., one measurement every 120°–180°), the proposed approach outperforms the others.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"76 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77188883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-17DOI: 10.3389/frsip.2022.877336
H. Messer, A. Eshel, H. Habi, S. Sagiv, X. Zheng
Rain gauges (RGs) have been utilized as sensors for local rain monitoring dating back to ancient Greece. The use of a network of RGs for 2D rain mapping is based on spatial interpolation that, while presenting good results in limited experimental areas, has limited scalability because of the unrealistic need to install and maintain a large quantity of sensors. Alternatively, commercial microwave links (CMLs), widely spread around the globe, have proven effective as near-ground opportunistic rain sensors. In this study, we study 2D rain field mapping using CMLs and/or RGs from a practical and a theoretical point of view, aiming to understand their inherent performance differences. We study sensor networks of either CMLs or RGs, and also a mixed network of CMLs and RGs. We show that with proper preprocessing, the rain field retrieval performance of the CML network is better than that of RGs. However, depending on the characteristics of the rain field, this performance gain can be negligible, especially when the rain field is smooth (relative to the topology of the sensor network). In other words, for a given network, the advantage of rain retrieval using a network of CMLs is more significant when the rain field is spotty.
{"title":"Rain Field Retrieval by Ground-Level Sensors of Various Types","authors":"H. Messer, A. Eshel, H. Habi, S. Sagiv, X. Zheng","doi":"10.3389/frsip.2022.877336","DOIUrl":"https://doi.org/10.3389/frsip.2022.877336","url":null,"abstract":"Rain gauges (RGs) have been utilized as sensors for local rain monitoring dating back to ancient Greece. The use of a network of RGs for 2D rain mapping is based on spatial interpolation that, while presenting good results in limited experimental areas, has limited scalability because of the unrealistic need to install and maintain a large quantity of sensors. Alternatively, commercial microwave links (CMLs), widely spread around the globe, have proven effective as near-ground opportunistic rain sensors. In this study, we study 2D rain field mapping using CMLs and/or RGs from a practical and a theoretical point of view, aiming to understand their inherent performance differences. We study sensor networks of either CMLs or RGs, and also a mixed network of CMLs and RGs. We show that with proper preprocessing, the rain field retrieval performance of the CML network is better than that of RGs. However, depending on the characteristics of the rain field, this performance gain can be negligible, especially when the rain field is smooth (relative to the topology of the sensor network). In other words, for a given network, the advantage of rain retrieval using a network of CMLs is more significant when the rain field is spotty.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89567970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-16DOI: 10.3389/frsip.2022.965551
Dongsheng Zheng, Yuli Yang
In this work, a novel scheme is proposed to enhance the self-interference (SI) cancellation in full-duplex communications. Beyond conventional SI cancellation schemes that rely on the SI suppression, our proposed scheme exploits periodic antenna switching to generate the pseudo-Doppler effect, thus completely removing the SI at the fundamental frequency. In this way, the desired signal is readily obtained through a low-pass filter. For the purpose of performance evaluation, the SI cancellation capability is defined as the difference between the output signal-to-interference-plus-noise ratio (SINR) and the input SINR. Theoretical formulations and numerical results validate that our pseudo-Doppler aided scheme has higher SI cancellation capability than the conventional SI suppression schemes. Moreover, the impact of the SI suppression achieved by conventional schemes and the influence of antenna switching timing difference on the practical implementation of the proposed scheme are investigated, to further substantiate the validity of our pseudo-Doppler aided SI cancellation.
{"title":"Pseudo-doppler aided cancellation of self-interference in full-duplex communications","authors":"Dongsheng Zheng, Yuli Yang","doi":"10.3389/frsip.2022.965551","DOIUrl":"https://doi.org/10.3389/frsip.2022.965551","url":null,"abstract":"In this work, a novel scheme is proposed to enhance the self-interference (SI) cancellation in full-duplex communications. Beyond conventional SI cancellation schemes that rely on the SI suppression, our proposed scheme exploits periodic antenna switching to generate the pseudo-Doppler effect, thus completely removing the SI at the fundamental frequency. In this way, the desired signal is readily obtained through a low-pass filter. For the purpose of performance evaluation, the SI cancellation capability is defined as the difference between the output signal-to-interference-plus-noise ratio (SINR) and the input SINR. Theoretical formulations and numerical results validate that our pseudo-Doppler aided scheme has higher SI cancellation capability than the conventional SI suppression schemes. Moreover, the impact of the SI suppression achieved by conventional schemes and the influence of antenna switching timing difference on the practical implementation of the proposed scheme are investigated, to further substantiate the validity of our pseudo-Doppler aided SI cancellation.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"14 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84394723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-16DOI: 10.3389/frsip.2022.977475
H. Messer
One of the turn-points in my life was in the mid-90th, during the yearly major conference of the Signal Processing community, the IEEE international conference on acoustic, speech and signal processing (ICASSP). Women were always minorities in these meetings, and if one of them joined a chat in a social gathering, she were naturally considered as the wife of one of the men around. Being young and naïve then, I never saw it as an issue. However, at that specific meeting on 1995 I decided to join, for the first time, a social event, entitled “lunch for women in signal processing.” I found there a small but very diverse group of about 50 women from all around the world, and when each introduced herself, I had a very strong emotional reaction of a sisterhood. For the first time I felt at home in my professional community, and at that very specific moment I became active in the advancement of women in science and engineering, and in particular in my field, i.e., signal processing. An essential question rises is about the quantity and the visibility of women in signal processing today. Such data is hard to trace, but fortunately, the IEEE keeps and publishes statistical records. These records show that while the overall share of women in the IEEE (including students) is still around 10%, in the signal processing society it is a bit but not much better, about 2,300 out of 19,000 (~12%). However, Figure 1 shows a promising trend over the last decade: while the total number of women (non-students) in the IEEE signal processing society has increased by 45%, the number of women in higher-level grades (senior member and fellow) has doubled. Moreover, women take leadership positions in the IEEE signal processing society with the current president Athina P. Petropulu and 11 out of its 23 board members being women. OPEN ACCESS
我生命中的一个转折点是在90年代中期，在信号处理领域的年度主要会议期间，IEEE国际声学，语音和信号处理会议(ICASSP)。在这些会议中，女性总是少数，如果她们中的一个参加社交聚会聊天，她自然会被认为是周围某个男人的妻子。那时我还年轻，还在naïve，我从来没把它当成一个问题。然而，在1995年的那次会议上，我第一次决定参加一个名为“信号处理领域女性午餐”的社会活动。我在那里发现了一个人数不多但非常多样化的团体，大约有50名来自世界各地的女性，当她们自我介绍时，我有一种非常强烈的姐妹情谊的情感反应。那是我第一次在我的专业团体中感到自在，在那个非常特殊的时刻，我开始积极参与推动女性在科学和工程领域的发展，特别是在我的领域，即信号处理领域。一个重要的问题是，今天女性在信号处理领域的数量和知名度。这些数据很难追踪，但幸运的是，IEEE保存并发布了统计记录。这些记录表明，虽然IEEE中女性的总体比例(包括学生)仍在10%左右，但在信号处理领域，这一比例有所提高，在19,000名成员中约有2,300名(约12%)。然而，图1显示了过去十年中一个有希望的趋势:虽然IEEE信号处理协会的女性(非学生)总数增加了45%，但更高级别(高级会员和研究员)的女性人数增加了一倍。此外，女性在IEEE信号处理协会担任领导职务，现任主席Athina P. Petropulu和23名董事会成员中的11名是女性。开放获取
{"title":"Editorial: Women in signal processing","authors":"H. Messer","doi":"10.3389/frsip.2022.977475","DOIUrl":"https://doi.org/10.3389/frsip.2022.977475","url":null,"abstract":"One of the turn-points in my life was in the mid-90th, during the yearly major conference of the Signal Processing community, the IEEE international conference on acoustic, speech and signal processing (ICASSP). Women were always minorities in these meetings, and if one of them joined a chat in a social gathering, she were naturally considered as the wife of one of the men around. Being young and naïve then, I never saw it as an issue. However, at that specific meeting on 1995 I decided to join, for the first time, a social event, entitled “lunch for women in signal processing.” I found there a small but very diverse group of about 50 women from all around the world, and when each introduced herself, I had a very strong emotional reaction of a sisterhood. For the first time I felt at home in my professional community, and at that very specific moment I became active in the advancement of women in science and engineering, and in particular in my field, i.e., signal processing. An essential question rises is about the quantity and the visibility of women in signal processing today. Such data is hard to trace, but fortunately, the IEEE keeps and publishes statistical records. These records show that while the overall share of women in the IEEE (including students) is still around 10%, in the signal processing society it is a bit but not much better, about 2,300 out of 19,000 (~12%). However, Figure 1 shows a promising trend over the last decade: while the total number of women (non-students) in the IEEE signal processing society has increased by 45%, the number of women in higher-level grades (senior member and fellow) has doubled. Moreover, women take leadership positions in the IEEE signal processing society with the current president Athina P. Petropulu and 11 out of its 23 board members being women. OPEN ACCESS","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72998794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-08-15DOI: 10.3389/frsip.2022.883696
Tim Lübeck, Sebastià V. Amengual Garí, P. Calamia, D. Alon, Jeff Crukley, Z. Ben-Hur
Microphone arrays consisting of sensors mounted on the surface of a rigid, spherical scatterer are popular tools for the capture and binaural reproduction of spatial sound scenes. However, microphone arrays with a perfectly spherical body and uniformly distributed microphones are often impractical for the consumer sector, in which microphone arrays are generally mounted on mobile and wearable devices of arbitrary geometries. Therefore, the binaural reproduction of sound fields captured with arbitrarily shaped microphone arrays has become an important field of research. In this work, we present a comparison of methods for the binaural reproduction of sound fields captured with non-spherical microphone arrays. First, we evaluated equatorial microphone arrays (EMAs), where the microphones are distributed on an equatorial contour of a rigid, spherical 1 . Second, we evaluated a microphone array with six microphones mounted on a pair of glasses. Using these two arrays, we conducted two listening experiments comparing four rendering methods based on acoustic scenes captured in different rooms2. The evaluation includes a microphone-based stereo approach (sAB stereo), a beamforming-based stereo approach (sXY stereo), beamforming-based binaural reproduction (BFBR), and BFBR with binaural signal matching (BSM). Additionally, the perceptual evaluation included binaural Ambisonics renderings, which were based on measurements with spherical microphone arrays. In the EMA experiment we included a fourth-order Ambisonics rendering, while in the glasses array experiment we included a second-order Ambisonics rendering. In both listening experiments in which participants compared all approaches with a dummy head recording we applied non-head-tracked binaural synthesis, with sound sources only in the horizontal plane. The perceived differences were rated separately for the attributes timbre and spaciousness. Results suggest that most approaches perform similarly to the Ambisonics rendering. Overall, BSM, and microphone-based stereo were rated the best for EMAs, and BFBR and microphone-based stereo for the glasses array.
{"title":"Perceptual evaluation of approaches for binaural reproduction of non-spherical microphone array signals","authors":"Tim Lübeck, Sebastià V. Amengual Garí, P. Calamia, D. Alon, Jeff Crukley, Z. Ben-Hur","doi":"10.3389/frsip.2022.883696","DOIUrl":"https://doi.org/10.3389/frsip.2022.883696","url":null,"abstract":"Microphone arrays consisting of sensors mounted on the surface of a rigid, spherical scatterer are popular tools for the capture and binaural reproduction of spatial sound scenes. However, microphone arrays with a perfectly spherical body and uniformly distributed microphones are often impractical for the consumer sector, in which microphone arrays are generally mounted on mobile and wearable devices of arbitrary geometries. Therefore, the binaural reproduction of sound fields captured with arbitrarily shaped microphone arrays has become an important field of research. In this work, we present a comparison of methods for the binaural reproduction of sound fields captured with non-spherical microphone arrays. First, we evaluated equatorial microphone arrays (EMAs), where the microphones are distributed on an equatorial contour of a rigid, spherical 1 . Second, we evaluated a microphone array with six microphones mounted on a pair of glasses. Using these two arrays, we conducted two listening experiments comparing four rendering methods based on acoustic scenes captured in different rooms2. The evaluation includes a microphone-based stereo approach (sAB stereo), a beamforming-based stereo approach (sXY stereo), beamforming-based binaural reproduction (BFBR), and BFBR with binaural signal matching (BSM). Additionally, the perceptual evaluation included binaural Ambisonics renderings, which were based on measurements with spherical microphone arrays. In the EMA experiment we included a fourth-order Ambisonics rendering, while in the glasses array experiment we included a second-order Ambisonics rendering. In both listening experiments in which participants compared all approaches with a dummy head recording we applied non-head-tracked binaural synthesis, with sound sources only in the horizontal plane. The perceived differences were rated separately for the attributes timbre and spaciousness. Results suggest that most approaches perform similarly to the Ambisonics rendering. Overall, BSM, and microphone-based stereo were rated the best for EMAs, and BFBR and microphone-based stereo for the glasses array.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"106 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88121800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-25DOI: 10.3389/frsip.2022.936790
Darshil Shah, G. Gopan K, N. Sinha
Electroencephalographic (EEG) signals are electrical signals generated in the brain due to cognitive activities. They are non-invasive and are widely used to assess neurodegenerative conditions, mental load, and sleep patterns. In this work, we explore the utility of representing the inherently single dimensional time-series in different dimensions such as 1D-feature vector, 2D-feature maps, and 3D-videos. The proposed methodology is applied to four diverse datasets: 1) EEG baseline, 2) mental arithmetic, 3) Parkinson’s disease, and 4) emotion dataset. For a 1D analysis, popular 1D features hand-crafted from the time-series are utilized for classification. This performance is compared against the data-driven approach of using raw time-series as the input to the deep learning framework. To assess the efficacy of 2D representation, 2D feature maps that utilize a combination of the Feature Pyramid Network (FPN) and Atrous Spatial Pyramid Pooling (ASPP) is proposed. This is compared against an approach utilizing a composite feature set consisting of 2D feature maps and 1D features. However, these approaches do not exploit spatial, spectral, and temporal characteristics simultaneously. To address this, 3D EEG videos are created by stacking spectral feature maps obtained from each sub-band per time frame in a temporal domain. The EEG videos are the input to a combination of the Convolution Neural Network (CNN) and Long–Short Term Memory (LSTM) for classification. Performances obtained using the proposed methodologies have surpassed the state-of-the-art for three of the classification scenarios considered in this work, namely, EEG baselines, mental arithmetic, and Parkinson’s disease. The video analysis resulted in 92.5% and 98.81% peak mean accuracies for the EEG baseline and EEG mental arithmetic, respectively. On the other hand, for distinguishing Parkinson’s disease from controls, a peak mean accuracy of 88.51% is achieved using traditional methods on 1D feature vectors. This illustrates that 3D and 2D feature representations are effective for those EEG data where topographical changes in brain activation regions are observed. However, in scenarios where topographical changes are not consistent across subjects of the same class, these methodologies fail. On the other hand, the 1D analysis proves to be significantly effective in the case involving changes in the overall activation of the brain due to varying degrees of deterioration.
{"title":"An investigation of the multi-dimensional (1D vs. 2D vs. 3D) analyses of EEG signals using traditional methods and deep learning-based methods","authors":"Darshil Shah, G. Gopan K, N. Sinha","doi":"10.3389/frsip.2022.936790","DOIUrl":"https://doi.org/10.3389/frsip.2022.936790","url":null,"abstract":"Electroencephalographic (EEG) signals are electrical signals generated in the brain due to cognitive activities. They are non-invasive and are widely used to assess neurodegenerative conditions, mental load, and sleep patterns. In this work, we explore the utility of representing the inherently single dimensional time-series in different dimensions such as 1D-feature vector, 2D-feature maps, and 3D-videos. The proposed methodology is applied to four diverse datasets: 1) EEG baseline, 2) mental arithmetic, 3) Parkinson’s disease, and 4) emotion dataset. For a 1D analysis, popular 1D features hand-crafted from the time-series are utilized for classification. This performance is compared against the data-driven approach of using raw time-series as the input to the deep learning framework. To assess the efficacy of 2D representation, 2D feature maps that utilize a combination of the Feature Pyramid Network (FPN) and Atrous Spatial Pyramid Pooling (ASPP) is proposed. This is compared against an approach utilizing a composite feature set consisting of 2D feature maps and 1D features. However, these approaches do not exploit spatial, spectral, and temporal characteristics simultaneously. To address this, 3D EEG videos are created by stacking spectral feature maps obtained from each sub-band per time frame in a temporal domain. The EEG videos are the input to a combination of the Convolution Neural Network (CNN) and Long–Short Term Memory (LSTM) for classification. Performances obtained using the proposed methodologies have surpassed the state-of-the-art for three of the classification scenarios considered in this work, namely, EEG baselines, mental arithmetic, and Parkinson’s disease. The video analysis resulted in 92.5% and 98.81% peak mean accuracies for the EEG baseline and EEG mental arithmetic, respectively. On the other hand, for distinguishing Parkinson’s disease from controls, a peak mean accuracy of 88.51% is achieved using traditional methods on 1D feature vectors. This illustrates that 3D and 2D feature representations are effective for those EEG data where topographical changes in brain activation regions are observed. However, in scenarios where topographical changes are not consistent across subjects of the same class, these methodologies fail. On the other hand, the 1D analysis proves to be significantly effective in the case involving changes in the overall activation of the brain due to varying degrees of deterioration.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"5 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-07-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83877733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-15DOI: 10.3389/frsip.2022.975838
M. Naccari, Fan Zhang, Saverio G. Blasi, T. Guionnet
{"title":"Editorial: Video Content Production and Delivery Over IP Networks and Distributed Computing Facilities","authors":"M. Naccari, Fan Zhang, Saverio G. Blasi, T. Guionnet","doi":"10.3389/frsip.2022.975838","DOIUrl":"https://doi.org/10.3389/frsip.2022.975838","url":null,"abstract":"","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"3 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-07-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83367049","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-07-08DOI: 10.3389/frsip.2022.918725
Jun Wang , B. Tondi, M. Barni
Generative Adversarial Network (GAN) models are nowadays able to generate synthetic images which are visually indistinguishable from the real ones, thus raising serious concerns about the spread of fake news and the need to develop tools to distinguish fake and real images in order to preserve the trustworthiness of digital images. The most powerful current detection methods are based on Deep Learning (DL) technology. While these methods get excellent performance when tested under conditions similar to those considered for training, they often suffer from a lack of robustness and generalization ability, as they fail to detect fake images that are generated by “unseen” GAN models. A possibility to overcome this problem is to develop tools that rely on the semantic attributes of the image. In this paper, we propose a semantic-based method for distinguishing GAN-generated from real faces, that relies on the analysis of inter-eye symmetries and inconsistencies. The method resorts to the superior capabilities of similarity learning of extracting representative and robust features. More specifically, a Siamese Neural Network (SNN) is utilized to extract high-level features characterizing the inter-eye similarity, that can be used to discriminate between real and synthetic pairs of eyes. We carried out extensive experiments to assess the performance of the proposed method in both matched and mismatched conditions pertaining to the GAN type used to generate the synthetic images and the robustness of the method in presence of post-processing. The results we got are comparable, and in some cases superior, to those achieved by the best performing state-of-the-art method leveraging on the analysis of the entire face image.
{"title":"An Eyes-Based Siamese Neural Network for the Detection of GAN-Generated Face Images","authors":"Jun Wang , B. Tondi, M. Barni","doi":"10.3389/frsip.2022.918725","DOIUrl":"https://doi.org/10.3389/frsip.2022.918725","url":null,"abstract":"Generative Adversarial Network (GAN) models are nowadays able to generate synthetic images which are visually indistinguishable from the real ones, thus raising serious concerns about the spread of fake news and the need to develop tools to distinguish fake and real images in order to preserve the trustworthiness of digital images. The most powerful current detection methods are based on Deep Learning (DL) technology. While these methods get excellent performance when tested under conditions similar to those considered for training, they often suffer from a lack of robustness and generalization ability, as they fail to detect fake images that are generated by “unseen” GAN models. A possibility to overcome this problem is to develop tools that rely on the semantic attributes of the image. In this paper, we propose a semantic-based method for distinguishing GAN-generated from real faces, that relies on the analysis of inter-eye symmetries and inconsistencies. The method resorts to the superior capabilities of similarity learning of extracting representative and robust features. More specifically, a Siamese Neural Network (SNN) is utilized to extract high-level features characterizing the inter-eye similarity, that can be used to discriminate between real and synthetic pairs of eyes. We carried out extensive experiments to assess the performance of the proposed method in both matched and mismatched conditions pertaining to the GAN type used to generate the synthetic images and the robustness of the method in presence of post-processing. The results we got are comparable, and in some cases superior, to those achieved by the best performing state-of-the-art method leveraging on the analysis of the entire face image.","PeriodicalId":93557,"journal":{"name":"Frontiers in signal processing","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82019829","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：