We present a synthesis of 3ie’s Thematic Window on Agricultural Innovations − 13 evaluations of interventions to improve smallholder innovation and technology adoption, largely through improved access to and information about inputs, techniques, and markets. We ask 1) under what conditions does information about a technology improve understanding, adoption, and further downstream impacts; and 2) how do conditions of the evaluation (as a project) constrain our ability to observe impacts? Most of the 13 interventions involved information treatments (SMS text messaging, demonstration plots, farmer field days, and farmer field schools). While most information interventions lead to greater reported awareness of the focal technique or input, we observe fewer impacts further down the impact pathways – on adoption, yield gains, or improvements to wellbeing, with several possible explanations. First, more complicated techniques like integrated soil fertility management (ISFM) require deeper engagement, so that (for example) farmer field schools have greater effect than farmer field days. This highlights the tension between scalability and effective learning in improving extension. Second, many of the interventions were short (1–2 years), whereas diffusion of knowledge around the intervention (as well as benefits from adoption) can take longer to be realized. Third, all agricultural interventions are at the whims of a variable climate, where improved weather conditions from baseline to endline can mask possible benefits of an encouraged practice. We consider these three findings jointly to discuss how to better fold climate variation and information diffusion into consideration of validity and appropriate time scales for evaluation.