Stress exposure during the sensitive period of early development has been shown to program the brain and increases the risk to develop cognitive deficits later in life. We have shown earlier that early-life stress (ES) leads to cognitive decline at an adult age, associated with changes in adult hippocampal neurogenesis and neuroinflammation. In particular, ES has been shown to affect neurogenesis rate and the survival of newborn cells later in life as well as microglia, modulating their response to immune or metabolic challenges later in life. Both of these processes possibly contribute to the ES-induced cognitive deficits. Emerging evidence by us and others indicates that early nutritional interventions can protect against these ES-induced effects through nutritional programming. Based on human metabolomics studies, we identified various coffee-related metabolites to be part of a protective molecular signature against cognitive decline in humans. Caffeic and chlorogenic acids are coffee-polyphenols and have been described to have potent anti-oxidant and anti-inflammatory actions. Therefore, we here aimed to test whether supplementing caffeic and chlorogenic acids to the early diet could also protect against ES-induced cognitive deficits. We induced ES via the limited nesting and bedding paradigm in mice from postnatal(P) day 2–9. On P2, mice received a diet to which 0.02% chlorogenic acid (5-O-caffeoylquinic acid) + 0.02% caffeic acid (3′,4′-dihydroxycinnamic acid) were added, or a control diet up until P42. At 4 months of age, all mice were subjected to a behavioral test battery and their brains were stained for markers for microglia and neurogenesis. We found that coffee polyphenols supplemented early in life protected against ES-induced cognitive deficits, potentially this is mediated by the survival of neurons or microglia, but possibly other mechanisms not studied here are mediating the effects. This study provides additional support for the potential of early nutritional interventions and highlights polyphenols as nutrients that can protect against cognitive decline, in particular for vulnerable populations exposed to ES.