A validated method is presented for computation of Lipschitz constant of recurrent neural networks. Lipschitz estimation of neural networks has gained prominence due to its close links with robustness analysis, a central concern in modern machine learning, especially in safety-critical applications. In recent years, several methods for validated Lipschitz estimation of feed-forward networks have been proposed, yet there are relatively fewer methods available for recurrent networks. In the current article, based on interval enclosure of Clarke’s generalized gradient, a method is proposed for Lipschitz estimation of recurrent networks which is applicable to both differentiable and non-differentiable networks. The method has a firm foundation in domain theory, and the algorithms can be proven to be correct by construction. A maximization algorithm is devised based on bisection with which a certified estimate of the Lipschitz constant can be obtained, and the region of least robustness can be located in the input domain. The method is implemented using interval arithmetic, and some experiments on vanilla recurrent networks are reported.
{"title":"Validated Computation of Lipschitz Constant of Recurrent Neural Networks","authors":"Yuhua Guo, Yiran Li, Amin Farjudian","doi":"10.1145/3583788.3583795","DOIUrl":"https://doi.org/10.1145/3583788.3583795","url":null,"abstract":"A validated method is presented for computation of Lipschitz constant of recurrent neural networks. Lipschitz estimation of neural networks has gained prominence due to its close links with robustness analysis, a central concern in modern machine learning, especially in safety-critical applications. In recent years, several methods for validated Lipschitz estimation of feed-forward networks have been proposed, yet there are relatively fewer methods available for recurrent networks. In the current article, based on interval enclosure of Clarke’s generalized gradient, a method is proposed for Lipschitz estimation of recurrent networks which is applicable to both differentiable and non-differentiable networks. The method has a firm foundation in domain theory, and the algorithms can be proven to be correct by construction. A maximization algorithm is devised based on bisection with which a certified estimate of the Lipschitz constant can be obtained, and the region of least robustness can be located in the input domain. The method is implemented using interval arithmetic, and some experiments on vanilla recurrent networks are reported.","PeriodicalId":292167,"journal":{"name":"Proceedings of the 2023 7th International Conference on Machine Learning and Soft Computing","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125793662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 2023 7th International Conference on Machine Learning and Soft Computing","authors":"","doi":"10.1145/3583788","DOIUrl":"https://doi.org/10.1145/3583788","url":null,"abstract":"","PeriodicalId":292167,"journal":{"name":"Proceedings of the 2023 7th International Conference on Machine Learning and Soft Computing","volume":"347 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122171444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：