Joint Transmit Antenna Selection and Beamforming Design for Millimeter Wave Massive MIMO NOMA Secure Networks

In this letter, we investigate the joint beamforming design and antenna selection (AS) of a millimeter wave (mmWave) massive multiple-input-multiple-output (m-MIMO) secure network. Our objective is to maximize the system secrecy sum rate while considering the successful successive interference cancellation (SIC) constraint, base station (BS) maximum transmit power and the transmit AS constraint. Considering the sparsity of the antenna selection matrix in mmWave m-MIMO network, we propose a sparse optimization approach coupled with the weighted mean squared error minimization (WMMSE) method for the joint beamforming design and AS problem, in which an $l_{0}$ norm constraint involving the coupling of AS and transmit beamforming is introduced. To address the challenges posed by the $l_{0}$ norm constraint, we employ bisection search to transform the optimization problem into a sequence of feasibility problems. In each feasibility problem, we replace the $l_{0}$ norm with continuous surrogate functions based on the majorization-minimization (MM) method where the valid quadratic functions with smooth optimization are utilized. Monte-Carlo simulation results show the effectiveness of the proposed scheme.