{"title":"SynthoMinds: Bridging human programming intuition with retrieval, analogy, and reasoning in program synthesis","authors":"Qianwen Gou , Yunwei Dong , Qiao Ke","doi":"10.1016/j.jss.2024.112140","DOIUrl":null,"url":null,"abstract":"<div><p>Program synthesis revolutionizes software development by automatically generating executable programs based on given specifications. An emerging trend is to augment generative models with external memory before generating programs. Better memory, in general, leads to better results. However, existing models tend to devolve into a copy mechanism, where retrieved memories are copied directly into the generative model, leading to misinformation or confusion. A sharp performance decline is caused when the retrieved memories are irrelevant or incorrect.</p><p>Inspired by the human programming process—sketching a solution before programming, we propose SynthoMinds. A novel framework that decomposes program synthesis tasks into retrieval, analogy, and reasoning, enabling the generation of programs by leveraging knowledge learned from previously solved solutions. Specifically, given a natural language (NL) description, SynthoMinds first retrieves similar programs via a retrieval module, and then mines the retrieved memories for some insightful revelations via an analogy module. The revelation acts as a bird’s-eye view of a program without delving into implementation details. The reasoning module harnesses the power of insightful revelations and NL to generate programs. Experimental results demonstrate that mining revelations from retrieved memories significantly outperforms existing baselines.</p></div>","PeriodicalId":51099,"journal":{"name":"Journal of Systems and Software","volume":null,"pages":null},"PeriodicalIF":3.7000,"publicationDate":"2024-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Systems and Software","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0164121224001857","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0
Abstract
Program synthesis revolutionizes software development by automatically generating executable programs based on given specifications. An emerging trend is to augment generative models with external memory before generating programs. Better memory, in general, leads to better results. However, existing models tend to devolve into a copy mechanism, where retrieved memories are copied directly into the generative model, leading to misinformation or confusion. A sharp performance decline is caused when the retrieved memories are irrelevant or incorrect.
Inspired by the human programming process—sketching a solution before programming, we propose SynthoMinds. A novel framework that decomposes program synthesis tasks into retrieval, analogy, and reasoning, enabling the generation of programs by leveraging knowledge learned from previously solved solutions. Specifically, given a natural language (NL) description, SynthoMinds first retrieves similar programs via a retrieval module, and then mines the retrieved memories for some insightful revelations via an analogy module. The revelation acts as a bird’s-eye view of a program without delving into implementation details. The reasoning module harnesses the power of insightful revelations and NL to generate programs. Experimental results demonstrate that mining revelations from retrieved memories significantly outperforms existing baselines.
期刊介绍:
The Journal of Systems and Software publishes papers covering all aspects of software engineering and related hardware-software-systems issues. All articles should include a validation of the idea presented, e.g. through case studies, experiments, or systematic comparisons with other approaches already in practice. Topics of interest include, but are not limited to:
• Methods and tools for, and empirical studies on, software requirements, design, architecture, verification and validation, maintenance and evolution
• Agile, model-driven, service-oriented, open source and global software development
• Approaches for mobile, multiprocessing, real-time, distributed, cloud-based, dependable and virtualized systems
• Human factors and management concerns of software development
• Data management and big data issues of software systems
• Metrics and evaluation, data mining of software development resources
• Business and economic aspects of software development processes
The journal welcomes state-of-the-art surveys and reports of practical experience for all of these topics.