Conservation agriculture (CA) is the key agricultural soil management approach for Mediterranean rainfed systems facing extreme droughts and soil degradation. Yet, CA uptake and applicability is still marginal and disputed in the Mediterranean region, where smallholder farmers are most representative. Lack of widespread adoption of CA in the Mediterranean region despite international efforts is perplexing. In order to investigate this paradox and provide solutions, we set out to examine the perceived constraints to CA implementation among farmers and stakeholders. Our approach is based on systems analysis of Mediterranean grain production systems, considering plant and livestock production, as well as sustainability and social-ecological interactions. CA promotion efforts are rarely adapted to the context of the Mediterranean region. We argue for adopting a more pragmatic and flexible approach to CA. Such an approach should be based on site-specific bio-physical and sociocultural considerations and augmented with principles of agroecology. Our review of perceived constraints allows us to suggest five pathways that could promote CA adoption in the Mediterranean across two main areas: (i) introduction of flexible, context-specific technical solutions and (ii) change of social perceptions and literacy on soil. Our five pathways aim to enhance farmers’ resilience to challenges of climate and market shocks, while integrating agroecological principles that enhance ecosystem multifunctionality. We advocate using agroecological principles to enable a more pragmatic application of CA with respect to its strict application—such as continuous no-till—to rehabilitate degraded lands, to increase water use efficiency, and to improve food security and economic well-being of communities in the Mediterranean region.