SynthoMinds: Bridging human programming intuition with retrieval, analogy, and reasoning in program synthesis

Abstract

Program synthesis revolutionizes software development by automatically generating executable programs based on given specifications. An emerging trend is to augment generative models with external memory before generating programs. Better memory, in general, leads to better results. However, existing models tend to devolve into a copy mechanism, where retrieved memories are copied directly into the generative model, leading to misinformation or confusion. A sharp performance decline is caused when the retrieved memories are irrelevant or incorrect.

Inspired by the human programming process—sketching a solution before programming, we propose SynthoMinds. A novel framework that decomposes program synthesis tasks into retrieval, analogy, and reasoning, enabling the generation of programs by leveraging knowledge learned from previously solved solutions. Specifically, given a natural language (NL) description, SynthoMinds first retrieves similar programs via a retrieval module, and then mines the retrieved memories for some insightful revelations via an analogy module. The revelation acts as a bird’s-eye view of a program without delving into implementation details. The reasoning module harnesses the power of insightful revelations and NL to generate programs. Experimental results demonstrate that mining revelations from retrieved memories significantly outperforms existing baselines.