Performance of OSTBC transmission in dual-hop amplify-and-forward MIMO relaying systems over spatially correlated Nakagami-m fading channels

The performance of orthogonal space-time block code (OSTBC) transmission in dual-hop amplify-and-forward (AF) multiple-input multiple-output (MIMO) relaying networks over spatially correlated Nakagami-m fading channel is investigated, where the source, relay, and destination terminals are all equipped with multiple antennas. We provide the compact closed-form expression for cumulative distribution function (CDF) of the end-to-end SNR and then exact analytical expressions are derived for the outage probability (OP), probability density function (PDF), and moment generating function (MGF) of the end-to-end SNR based on its CDF. Further, we present the asymptotic expression for CDF in the high SNR regime, which gives an insight of the system performance and the achievable diversity order. The analytical expressions are validated by Monte-Carlo simulations. It's shown that the correlation is detrimental to the MIMO relaying performance.