Since crystal plasticity is the result of moving and interacting dislocations, it seems self-evident that continuum plasticity should in principle be derivable as a statistical continuum theory of dislocations, though in practice we are still far from doing so. One key to any statistical continuum theory of interacting particles is the consideration of spatial correlations. However, because dislocations are extended one-dimensional defects, the classical definition of correlations for point particles is not readily applicable to dislocation systems: the line-like nature of dislocations entails that a scalar pair correlation function does not suffice for characterizing spatial correlations and a hierarchy of two-point tensors is required in general. The extended nature of dislocations as closed curves leads to strong self-correlations along the dislocation line. In the current contribution, we thoroughly introduce the concept of pair correlations for general averaged dislocation systems and illustrate self-correlations as well as the content of low order correlation tensors using a simple model system. We furthermore detail how pair correlation information may be obtained from three-dimensional discrete dislocation simulations and provide a first analysis of correlations from such simulations. We briefly discuss how the pair correlation information may be employed to improve existing continuum dislocation theories and why we think it is important for analyzing discrete dislocation data.