Olfactory dysfunction (OD) has been identified as a potential indicator closely related to cognitive decline (CD). This review explores the complex connection between olfactory and cognitive function, with a focus on the nose–brain axis. We summarize recent evidence linking OD of various etiologies to CD and discuss how this relationship depends on the specific disease. The review also addresses neuroanatomical changes in OD and CD and potential mechanisms underpinning the OD–CD relationship, including inflammatory response, oxidative stress, and microbiota–gut/nose–brain communication. Despite progress in elucidating these pathological mechanisms, challenges persist in applying these findings to effective clinical applications. Additionally, this review assesses new therapeutic interventions, particularly olfactory training (OT), which demonstrates potential in improving both olfactory and cognitive functions. By synthesizing this evidence, this article seeks to provide a comprehensive overview of the bidirectional relationships between olfaction and cognition and advance diagnostic and rehabilitative strategies for patients with OD and CD.