I see artifacts: ICA-based EEG artifact removal does not improve deep network decoding across three BCI tasks.

\textit{Objective.} In this paper, we conduct a detailed investigation on the effect of IC-based noise rejection methods in neural network classifier-based decoding of electroencephalography (EEG) data in different task datasets. \textit{Approach.} We apply a pipeline matrix of two popular different Independent Component (IC) decomposition methods (Infomax, AMICA) with three different component rejection strategies (none, ICLabel, and MARA) on three different EEG datasets (Motor imagery, long-term memory formation, and visual memory). We cross-validate processed data from each pipeline with three architectures commonly used for EEG classification (two convolutional neural networks (CNN) and one long short term memory (LSTM) based model. We compare decoding performances on within-participant and within-dataset levels. \textit{Main Results.} Our results show that the benefit from using IC-based noise rejection for decoding analyses is at best minor, as component-rejected data did not show consistently better performance than data without rejections---especially given the significant computational resources required for ICA computations. \textit{Significance.} With ever growing emphasis on transparency and reproducibility, as well as the obvious benefits arising from streamlined processing of large-scale datasets, there has been an increased interest in automated methods for pre-processing EEG data. One prominent part of such pre-processing pipelines consists of identifying and potentially removing artifacts arising from extraneous sources. This is typically done via Independent Component (IC) based correction for which numerous methods have been proposed, differing not only in the decomposition of the raw data into ICs, but also in how they reject the computed ICs. While the benefits of these methods are well established in univariate statistical analyses, it is unclear whether they help in multivariate scenarios, and specifically in neural network based decoding studies. As computational costs for pre-processing large-scale datasets are considerable, it is important to consider whether the tradeoff between model performance and available resources is worth the effort.