{"title":"Large language models and unsupervised feature learning: implications for log analysis","authors":"","doi":"10.1007/s12243-024-01028-2","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Log file analysis is increasingly being addressed through the use of large language models (LLM). LLM provides the mechanism for discovering embeddings for distinguishing between different behaviors present in log files. In this work, we are interested in discriminating between normal and anomalous behaviors via an unsupervised learning approach. To this end, firstly five recent LLM architectures are evaluated over six different log files. Then, further research is conducted to explicitly quantify the significance of performing self-supervised fine-tuning on the LLMs. Moreover, we show that the quality of an (unsupervised) feature map used to make the overall (normal/anomalous) predictions may also benefit from an AutoEncoder stage between LLM and feature map. Such an AutoEncoder provides significant reductions in the cost of training the feature map and typically improves the quality of the resulting predictions.</p>","PeriodicalId":50761,"journal":{"name":"Annals of Telecommunications","volume":"93 1","pages":""},"PeriodicalIF":1.8000,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Telecommunications","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12243-024-01028-2","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"TELECOMMUNICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Log file analysis is increasingly being addressed through the use of large language models (LLM). LLM provides the mechanism for discovering embeddings for distinguishing between different behaviors present in log files. In this work, we are interested in discriminating between normal and anomalous behaviors via an unsupervised learning approach. To this end, firstly five recent LLM architectures are evaluated over six different log files. Then, further research is conducted to explicitly quantify the significance of performing self-supervised fine-tuning on the LLMs. Moreover, we show that the quality of an (unsupervised) feature map used to make the overall (normal/anomalous) predictions may also benefit from an AutoEncoder stage between LLM and feature map. Such an AutoEncoder provides significant reductions in the cost of training the feature map and typically improves the quality of the resulting predictions.
期刊介绍：
Annals of Telecommunications is an international journal publishing original peer-reviewed papers in the field of telecommunications. It covers all the essential branches of modern telecommunications, ranging from digital communications to communication networks and the internet, to software, protocols and services, uses and economics. This large spectrum of topics accounts for the rapid convergence through telecommunications of the underlying technologies in computers, communications, content management towards the emergence of the information and knowledge society. As a consequence, the Journal provides a medium for exchanging research results and technological achievements accomplished by the European and international scientific community from academia and industry.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
