People’s behaviors are influenced by past experiences. Against the background of the famine that occurred in China from 1959 to 1961 and based on China Health and Nutrition Survey (CHNS) data from 2004 to 2011, we take advantage of both temporal and geographic variations in famine intensity to explore the long-term effects of famine experience on food consumption and examine the heterogeneous effecs across age cohorts by constructing a continuous difference-in-differences (DID) estimator. We find that famine experience affects people’s food consumption behavior in the long term. Compared with the group that did not experience the famine, the consumption of staple foods increased while the consumption of meat and vegetables decreased for the group with famine experience, and this effect was heterogeneous across age cohorts and was most significant for the group that experienced famine in their 30 s. The effect of famine experience on food consumption was significantly heterogeneous across regions, household sizes, and income groups. The findings of this paper contribute to a better understanding of the long-term effects of famine experience on food consumption and provide a basis for improving the dietary quality of older adults.