Reading and spelling skills are important to communicate in today's literate society, however, the underlying processes of spelling skills are under-researched compared to reading skills. Our goals were to (a) study how the component skills of phonological, orthographic and morphological awareness are different in adults with and without reading difficulties, and (b) characterize the relationship between the component skills and reading and spelling performance in both skilled and poor readers. Participants (N = 37, N = 15 with reading impairments and N = 22 skilled readers) took part in the study where they completed several literacy-based measures. We performed a series of mixed ANOVAs to study the between-group differences in performance and the relationship between different literacy outcomes, respectively. We found evidence for poor phonological and morphological awareness in the poor readers compared to the skilled readers. We also found differential relationships between the component skills and reading and spelling behavior. Specifically, sound awareness was significantly related to reading and spelling measures in the skilled readers, whereas morphological and sound awareness played an important role in the same skills in the poor readers. We discuss these findings in the context of potential remediation strategies for adults with persistent literacy impairments.