Pub Date : 2024-09-18DOI: 10.1007/s11280-024-01303-1
Xuqi Mao, Zhenyi Chen, Zhenying He, Yinan Jing, Kai Zhang, X. Sean Wang
Numerous real-world information networks form Heterogeneous Information Networks (HINs) with diverse objects and relations represented as nodes and edges in heterogeneous graphs. Similarity between nodes quantifies how closely two nodes resemble each other, mainly depending on the similarity of the nodes they are connected to, recursively. Users may be interested in only specific types of connections in the similarity definition, represented as meta-paths, i.e., a sequence of node and edge types. Existing Heterogeneous Graph Neural Network (HGNN)-based similarity search methods may accommodate meta-paths, but require retraining for different meta-paths. Conversely, existing path-based similarity search methods may switch flexibly between meta-paths but often suffer from lower accuracy, as they rely solely on path information. This paper proposes HetFS, a Fast Similarity method for ad-hoc queries with user-given meta-paths on Heterogeneous information networks. HetFS provides similarity results based on path information that satisfies the meta-path restriction, as well as node content. Extensive experiments demonstrate the effectiveness and efficiency of HetFS in addressing ad-hoc queries, outperforming state-of-the-art HGNNs and path-based approaches, and showing strong performance in downstream applications, including link prediction, node classification, and clustering.
{"title":"HetFS: a method for fast similarity search with ad-hoc meta-paths on heterogeneous information networks","authors":"Xuqi Mao, Zhenyi Chen, Zhenying He, Yinan Jing, Kai Zhang, X. Sean Wang","doi":"10.1007/s11280-024-01303-1","DOIUrl":"https://doi.org/10.1007/s11280-024-01303-1","url":null,"abstract":"<p>Numerous real-world information networks form <b>H</b>eterogeneous <b>I</b>nformation <b>N</b>etworks (HINs) with diverse objects and relations represented as nodes and edges in heterogeneous graphs. Similarity between nodes quantifies how closely two nodes resemble each other, mainly depending on the similarity of the nodes they are connected to, recursively. Users may be interested in only specific types of connections in the similarity definition, represented as meta-paths, i.e., a sequence of node and edge types. Existing <b>H</b>eterogeneous <b>G</b>raph <b>N</b>eural <b>N</b>etwork (HGNN)-based similarity search methods may accommodate meta-paths, but require retraining for different meta-paths. Conversely, existing path-based similarity search methods may switch flexibly between meta-paths but often suffer from lower accuracy, as they rely solely on path information. This paper proposes HetFS, a <b>F</b>ast <b>S</b>imilarity method for ad-hoc queries with user-given meta-paths on <b>Het</b>erogeneous information networks. HetFS provides similarity results based on path information that satisfies the meta-path restriction, as well as node content. Extensive experiments demonstrate the effectiveness and efficiency of HetFS in addressing ad-hoc queries, outperforming state-of-the-art HGNNs and path-based approaches, and showing strong performance in downstream applications, including link prediction, node classification, and clustering.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142260570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Controversy encompasses content that draws diverse perspectives, along with positive and negative feedback on a specific event, resulting in the formation of distinct user communities. we explore the explainability of controversy through the lens of SHAP (SHapley Additive exPlanations) method, aiming to provide a fair assessment of the individual contributions of different text features of tweets to controversy detection. We conduct an analysis of topic discussions on Twitter from a community perspective, investigating the role of text in accurately classifying tweets into their respective communities. To achieve this, we introduce a SHAP-based pipeline designed to quantify the influence of impactful text features on the predictions of three tweet classifiers. Text content alone offers interesting controversy detection accuracy. It can contain predictive features for controversy detection. For instance, negative connotations, pejorative tendencies and positive qualifying adjectives tend to impact the controversy model detection.
{"title":"A SHAP-based controversy analysis through communities on Twitter","authors":"Samy Benslimane, Thomas Papastergiou, Jérôme Azé, Sandra Bringay, Maximilien Servajean, Caroline Mollevi","doi":"10.1007/s11280-024-01278-z","DOIUrl":"https://doi.org/10.1007/s11280-024-01278-z","url":null,"abstract":"<p>Controversy encompasses content that draws diverse perspectives, along with positive and negative feedback on a specific event, resulting in the formation of distinct user communities. we explore the explainability of controversy through the lens of SHAP (SHapley Additive exPlanations) method, aiming to provide a fair assessment of the individual contributions of different text features of tweets to controversy detection. We conduct an analysis of topic discussions on Twitter from a community perspective, investigating the role of text in accurately classifying tweets into their respective communities. To achieve this, we introduce a SHAP-based pipeline designed to quantify the influence of impactful text features on the predictions of three tweet classifiers. Text content alone offers interesting controversy detection accuracy. It can contain predictive features for controversy detection. For instance, negative connotations, pejorative tendencies and positive qualifying adjectives tend to impact the controversy model detection.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142260571","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Web 3.0 makes crowdsensing services more popular, because of its decentralisation and interoperability. Lost Object Finding (LOF) in vehicular crowdsensing is an emerging paradigm in which vehicles act as detectors to find lost objects for their owners. To enjoy LOF services, object owners need to submit the tag ID of his lost object, and then detectors need to update their detecting results together with their locations. But the identity and location information are usually sensitive, which can be used to infer the locations of lost objects, or track participant detectors. This raises serious privacy concerns. In this paper, we study the privacy leakages associated with object finding, and propose a privacy-preserving scheme, named pFind, for locating lost objects. This scheme allows owners to retrieve the locations of their lost objects and provides strong privacy protection for the object owners, lost objects, and detectors. In pFind, we design an oblivious object detection protocol by using RBS cryptosystem, which simultaneously provides confidentiality, authentication and integrity for lost objects detection. Meanwhile, we propose a private location retrieval protocol to compute the approximate location of a lost object over encrypted data. We further propose two optimizations for pFind to enhance functionality and performance. Theoretical analysis and experimental evaluations show that pFind is secure, accurate and efficient.
{"title":"pFind: Privacy-preserving lost object finding in vehicular crowdsensing","authors":"Yinggang Sun, Haining Yu, Xiang Li, Yizheng Yang, Xiangzhan Yu","doi":"10.1007/s11280-024-01300-4","DOIUrl":"https://doi.org/10.1007/s11280-024-01300-4","url":null,"abstract":"<p>Web 3.0 makes crowdsensing services more popular, because of its decentralisation and interoperability. Lost Object Finding (LOF) in vehicular crowdsensing is an emerging paradigm in which vehicles act as detectors to find lost objects for their owners. To enjoy LOF services, object owners need to submit the tag ID of his lost object, and then detectors need to update their detecting results together with their locations. But the identity and location information are usually sensitive, which can be used to infer the locations of lost objects, or track participant detectors. This raises serious privacy concerns. In this paper, we study the privacy leakages associated with object finding, and propose a privacy-preserving scheme, named pFind, for locating lost objects. This scheme allows owners to retrieve the locations of their lost objects and provides strong privacy protection for the object owners, lost objects, and detectors. In pFind, we design an oblivious object detection protocol by using RBS cryptosystem, which simultaneously provides confidentiality, authentication and integrity for lost objects detection. Meanwhile, we propose a private location retrieval protocol to compute the approximate location of a lost object over encrypted data. We further propose two optimizations for pFind to enhance functionality and performance. Theoretical analysis and experimental evaluations show that pFind is secure, accurate and efficient.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204977","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-09DOI: 10.1007/s11280-024-01302-2
Pasindu Udawatta, Indunil Udayangana, Chathulanka Gamage, Ravi Shekhar, Surangika Ranathunga
Code-mixing and code-switching (CMCS) are prevalent phenomena observed in social media conversations and various other modes of communication. When developing applications such as sentiment analysers and hate-speech detectors that operate on this social media data, CMCS text poses challenges. Recent studies have demonstrated that prompt-based learning of pre-trained language models outperforms full fine-tuning across various tasks. Despite the growing interest in classifying CMCS text, the effectiveness of prompt-based learning for the task remains unexplored. This paper presents an extensive exploration of prompt-based learning for CMCS text classification and the first comprehensive analysis of the impact of the script on classifying CMCS text. Our study reveals that the performance in classifying CMCS text is significantly influenced by the inclusion of multiple scripts and the intensity of code-mixing. In response, we introduce a novel method, Dynamic+AdapterPrompt, which employs distinct models for each script, integrated with adapters. While DynamicPrompt captures the script-specific representation of the text, AdapterPrompt emphasizes capturing the task-oriented functionality. Our experiments on Sinhala-English, Kannada-English, and Hindi-English datasets for sentiment classification, hate-speech detection, and humour detection tasks show that our method outperforms strong fine-tuning baselines and basic prompting strategies.
{"title":"Use of prompt-based learning for code-mixed and code-switched text classification","authors":"Pasindu Udawatta, Indunil Udayangana, Chathulanka Gamage, Ravi Shekhar, Surangika Ranathunga","doi":"10.1007/s11280-024-01302-2","DOIUrl":"https://doi.org/10.1007/s11280-024-01302-2","url":null,"abstract":"<p>Code-mixing and code-switching (CMCS) are prevalent phenomena observed in social media conversations and various other modes of communication. When developing applications such as sentiment analysers and hate-speech detectors that operate on this social media data, CMCS text poses challenges. Recent studies have demonstrated that prompt-based learning of pre-trained language models outperforms full fine-tuning across various tasks. Despite the growing interest in classifying CMCS text, the effectiveness of prompt-based learning for the task remains unexplored. This paper presents an extensive exploration of prompt-based learning for CMCS text classification and the first comprehensive analysis of the impact of the script on classifying CMCS text. Our study reveals that the performance in classifying CMCS text is significantly influenced by the inclusion of multiple scripts and the intensity of code-mixing. In response, we introduce a novel method, <i>Dynamic+AdapterPrompt</i>, which employs distinct models for each script, integrated with adapters. While DynamicPrompt captures the script-specific representation of the text, AdapterPrompt emphasizes capturing the task-oriented functionality. Our experiments on Sinhala-English, Kannada-English, and Hindi-English datasets for sentiment classification, hate-speech detection, and humour detection tasks show that our method outperforms strong fine-tuning baselines and basic prompting strategies.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-06DOI: 10.1007/s11280-024-01301-3
Petar Kochovski, Maroua Masmoudi, Redouane Bouhamoum, Vlado Stankovski, Hajer Baazaoui, Chirine Ghedira, Dan Vodislav, Thamer Mecharnia
Drug traceability is a critical process involving monitoring and validation of the origin, quality, and safety of pharmaceutical products throughout their supply chain to prevent the distribution of counterfeit, substandard, or expired drugs that could harm patients. Traditional centralized solutions for drug traceability, relying on intermediaries and central authorities, introduce risks of data manipulation, corruption, and single points of failure. This work presents the design and implementation of a novel solution for decentralized drug traceability based on blockchain technology and on a reputation mechanism that operates on top of a trustworthy decentralized knowledge base, thus integrating three core technologies: blockchain, semantic, and reputation methods. Blockchain technologies ensure transparent and secure supply chain processes while providing a trustworthy estimation of the reputation of supply chain participants. Semantic technologies address drug data heterogeneity by ensuring interoperability and creating mappings between various data sources, including verifying the identities of the various users. Additionally, the reputation mechanism promotes transparency and accountability, as stakeholders contribute feedback on drug quality, authenticity, and reliability. This fosters a culture of trust and reliability, offering the drug supply chain an effective tool for continuous improvement and informed decision-making based on aggregated feedback, ultimately enhancing overall quality and safety throughout the distribution network. The design and implementation of the system, along with several evaluations, show the feasibility of the new semantic blockchain system in real-world scenarios and the improvement of the entities with a high reputation score. Our solution is more trustworthy, discouraging fraudulent activities as security is based on the various properties included in the semantic model.
{"title":"Drug traceability system based on semantic blockchain and on a reputation method","authors":"Petar Kochovski, Maroua Masmoudi, Redouane Bouhamoum, Vlado Stankovski, Hajer Baazaoui, Chirine Ghedira, Dan Vodislav, Thamer Mecharnia","doi":"10.1007/s11280-024-01301-3","DOIUrl":"https://doi.org/10.1007/s11280-024-01301-3","url":null,"abstract":"<p>Drug traceability is a critical process involving monitoring and validation of the origin, quality, and safety of pharmaceutical products throughout their supply chain to prevent the distribution of counterfeit, substandard, or expired drugs that could harm patients. Traditional centralized solutions for drug traceability, relying on intermediaries and central authorities, introduce risks of data manipulation, corruption, and single points of failure. This work presents the design and implementation of a novel solution for decentralized drug traceability based on blockchain technology and on a reputation mechanism that operates on top of a trustworthy decentralized knowledge base, thus integrating three core technologies: blockchain, semantic, and reputation methods. Blockchain technologies ensure transparent and secure supply chain processes while providing a trustworthy estimation of the reputation of supply chain participants. Semantic technologies address drug data heterogeneity by ensuring interoperability and creating mappings between various data sources, including verifying the identities of the various users. Additionally, the reputation mechanism promotes transparency and accountability, as stakeholders contribute feedback on drug quality, authenticity, and reliability. This fosters a culture of trust and reliability, offering the drug supply chain an effective tool for continuous improvement and informed decision-making based on aggregated feedback, ultimately enhancing overall quality and safety throughout the distribution network. The design and implementation of the system, along with several evaluations, show the feasibility of the new semantic blockchain system in real-world scenarios and the improvement of the entities with a high reputation score. Our solution is more trustworthy, discouraging fraudulent activities as security is based on the various properties included in the semantic model.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204978","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-02DOI: 10.1007/s11280-024-01299-8
Dongjing Wang, Ruijie Du, Qimeng Yang, Dongjin Yu, Feng Wan, Xiaojun Gong, Guandong Xu, Shuiguang Deng
Session-based recommendation which focuses on predicting the next behavior according to anonymous sessions of behavior records plays an important role in real-world applications. Most previous session-based recommendation approaches capture the preferences of users by modeling the behavior records between users and items within current session. However, items’ category information is not fully exploited, while existing works are still suffering from the severe issue of data sparsity. In this work, we propose a novel session-based recommendation model, namely Category-aware Self-supervised Graph Neural Network (namely CSGNN), which adopts a pre-training layer for capturing the features of items and categories, as well as the correlations among them. Especially, we build a category-aware heterogeneous hypergraph composed of item nodes and category nodes, which enhances the information learning in the current session. Then we design item-level and category-level self-attention models to represent the information of item and category, respectively, and integrate global and local preference of user for session-based recommendation. Finally, we combine self-supervised learning by constructing a category-aware session graph to further enhance the performance CSGNN and alleviate the data sparsity problem. Comprehensive experiments are conducted on three real-world datasets, Nowplaying, Diginetica, and Tmall, and the results show that the proposed model CSGNN achieves better performance than session-based recommendation baselines with several state-of-the-art approaches.
{"title":"Category-aware self-supervised graph neural network for session-based recommendation","authors":"Dongjing Wang, Ruijie Du, Qimeng Yang, Dongjin Yu, Feng Wan, Xiaojun Gong, Guandong Xu, Shuiguang Deng","doi":"10.1007/s11280-024-01299-8","DOIUrl":"https://doi.org/10.1007/s11280-024-01299-8","url":null,"abstract":"<p>Session-based recommendation which focuses on predicting the next behavior according to anonymous sessions of behavior records plays an important role in real-world applications. Most previous session-based recommendation approaches capture the preferences of users by modeling the behavior records between users and items within current session. However, items’ category information is not fully exploited, while existing works are still suffering from the severe issue of data sparsity. In this work, we propose a novel session-based recommendation model, namely <b>C</b>ategory-aware <b>S</b>elf-supervised <b>G</b>raph <b>N</b>eural <b>N</b>etwork (namely CSGNN), which adopts a pre-training layer for capturing the features of items and categories, as well as the correlations among them. Especially, we build a category-aware heterogeneous hypergraph composed of item nodes and category nodes, which enhances the information learning in the current session. Then we design item-level and category-level self-attention models to represent the information of item and category, respectively, and integrate global and local preference of user for session-based recommendation. Finally, we combine self-supervised learning by constructing a category-aware session graph to further enhance the performance CSGNN and alleviate the data sparsity problem. Comprehensive experiments are conducted on three real-world datasets, Nowplaying, Diginetica, and Tmall, and the results show that the proposed model CSGNN achieves better performance than session-based recommendation baselines with several state-of-the-art approaches.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204998","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Person re-identification (re-id) aims to recognize pedestrians across different camera views, which enjoys popularity in computer vision area recently. Notwithstanding the progress achieved by existing methods in rising matching rate, the prevailing solutions still suffer from two impossible-to-ignore issues: (I) multi-grained view-consistent discriminative feature learning across multiple views has barely been explored, and (II) latent non-linear correlation between multiple views is insufficient captured. To this end, this article proposes a novel end-to-end unsupervised framework for person re-id task, dubbed Multi-view Hierarchical Attention Adversarial Network (MvHAAN). This framework enjoys two merits: First, a hierarchical attention mechanism in multi-view networks is present to learn multi-grained view-consistent discriminative features; Second, a multi-view adversarial correlation learning strategy is involved to excavate complex non-linear correlation from all views simultaneously. To the best of our knowledge, it is the early attempt of marrying multi-view deep correlation learning with adversarial learning to further reduce multi-view heterogeneity. Extensive evaluations on three person re-id benchmark datasets verify that the proposed method delivers superior performance of unsupervised person re-id.
{"title":"MvHAAN: multi-view hierarchical attention adversarial network for person re-identification","authors":"Lei Zhu, Weiren Yu, Xinghui Zhu, Chengyuan Zhang, Yangding Li, Shichao Zhang","doi":"10.1007/s11280-024-01298-9","DOIUrl":"https://doi.org/10.1007/s11280-024-01298-9","url":null,"abstract":"<p>Person re-identification (re-id) aims to recognize pedestrians across different camera views, which enjoys popularity in computer vision area recently. Notwithstanding the progress achieved by existing methods in rising matching rate, the prevailing solutions still suffer from two impossible-to-ignore issues: (I) multi-grained view-consistent discriminative feature learning across multiple views has barely been explored, and (II) latent non-linear correlation between multiple views is insufficient captured. To this end, this article proposes a novel end-to-end unsupervised framework for person re-id task, dubbed Multi-view Hierarchical Attention Adversarial Network (MvHAAN). This framework enjoys two merits: First, a hierarchical attention mechanism in multi-view networks is present to learn multi-grained view-consistent discriminative features; Second, a multi-view adversarial correlation learning strategy is involved to excavate complex non-linear correlation from all views simultaneously. To the best of our knowledge, it is the early attempt of marrying multi-view deep correlation learning with adversarial learning to further reduce multi-view heterogeneity. Extensive evaluations on three person re-id benchmark datasets verify that the proposed method delivers superior performance of unsupervised person re-id.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) and have recently gained significant attention in the domain of Recommendation Systems (RS). These models, trained on massive amounts of data using self-supervised learning, have demonstrated remarkable success in learning universal representations and have the potential to enhance various aspects of recommendation systems by some effective transfer techniques such as fine-tuning, prompt tuning, etc. The crucial aspect of harnessing the power of language models in enhancing recommendation quality is the utilization of their high-quality representations of textual features and their extensive coverage of external knowledge to establish correlations between items and users. To provide a comprehensive understanding of the existing LLM-based recommendation systems, this survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec), with the latter being systematically sorted out for the first time. Furthermore, we systematically review and analyze existing LLM-based recommendation systems within each paradigm, providing insights into their methodologies, techniques, and performance. Additionally, we identify key challenges and several valuable findings to provide researchers and practitioners with inspiration. We have also created a GitHub repository to index relevant papers and resources on LLMs for recommendation (https://github.com/WLiK/LLM4Rec-Awesome-Papers).
{"title":"A survey on large language models for recommendation","authors":"Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, Enhong Chen","doi":"10.1007/s11280-024-01291-2","DOIUrl":"https://doi.org/10.1007/s11280-024-01291-2","url":null,"abstract":"<p>Large Language Models (LLMs) have emerged as powerful tools in the field of Natural Language Processing (NLP) and have recently gained significant attention in the domain of Recommendation Systems (RS). These models, trained on massive amounts of data using self-supervised learning, have demonstrated remarkable success in learning universal representations and have the potential to enhance various aspects of recommendation systems by some effective transfer techniques such as fine-tuning, prompt tuning, etc. The crucial aspect of harnessing the power of language models in enhancing recommendation quality is the utilization of their high-quality representations of textual features and their extensive coverage of external knowledge to establish correlations between items and users. To provide a comprehensive understanding of the existing LLM-based recommendation systems, this survey presents a taxonomy that categorizes these models into two major paradigms, respectively Discriminative LLM for Recommendation (DLLM4Rec) and Generative LLM for Recommendation (GLLM4Rec), with the latter being systematically sorted out for the first time. Furthermore, we systematically review and analyze existing LLM-based recommendation systems within each paradigm, providing insights into their methodologies, techniques, and performance. Additionally, we identify key challenges and several valuable findings to provide researchers and practitioners with inspiration. We have also created a GitHub repository to index relevant papers and resources on LLMs for recommendation (https://github.com/WLiK/LLM4Rec-Awesome-Papers).</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-21DOI: 10.1007/s11280-024-01296-x
Hai Liu, Xiaozhi Zhu, Yong Tang, Chaobo He, Tianyong Hao
The large-size language model is able to implicitly extract informative semantic features from queries and candidate documents to achieve impressive reranking performance. However, the large model relies on its own large number of parameters to achieve it and it is not known exactly what semantic information has been learned. In this paper, we propose a multi-stage enhanced representation learning method based on Query-View (MERL) with Intra-query stage and Inter-query stage to guide the model to explicitly learn the semantic relationship between the query and documents. In the Intra-query training stage, a content-based contrastive learning module without considering the special token [CLS] of BERT is utilized to optimize the semantic similarity of query and relevant documents. In the Inter-query training stage, an entity-oriented masked query prediction for establish a semantic relation of query-document pairs and an Inter-query contrastive learning module for extracting similar matching pattern of query-relevant documents are employed. Extensive experiments on MS MARCO passage ranking and TREC DL datasets show that the MERL method obtain significant improvements with a low number of parameters compared to the baseline models.
大型语言模型能够从查询和候选文档中隐含地提取信息丰富的语义特征,从而实现令人印象深刻的重新排序性能。然而,大模型是依靠自身的大量参数来实现的,而且不知道到底学到了哪些语义信息。在本文中,我们提出了一种基于查询视图(MERL)的多阶段增强表示学习方法,包括查询内阶段(Intra-query stage)和查询间阶段(Inter-query stage),以引导模型明确学习查询与文档之间的语义关系。在查询内训练阶段,利用基于内容的对比学习模块(不考虑 BERT 的特殊标记 [CLS])来优化查询和相关文档的语义相似性。在查询间训练阶段,利用面向实体的屏蔽查询预测建立查询-文档对的语义关系,并利用查询间对比学习模块提取查询-相关文档的相似匹配模式。在 MS MARCO 段落排序和 TREC DL 数据集上进行的大量实验表明,与基线模型相比,MERL 方法在参数数量较少的情况下就能获得显著的改进。
{"title":"Multi-stage enhanced representation learning for document reranking based on query view","authors":"Hai Liu, Xiaozhi Zhu, Yong Tang, Chaobo He, Tianyong Hao","doi":"10.1007/s11280-024-01296-x","DOIUrl":"https://doi.org/10.1007/s11280-024-01296-x","url":null,"abstract":"<p>The large-size language model is able to implicitly extract informative semantic features from queries and candidate documents to achieve impressive reranking performance. However, the large model relies on its own large number of parameters to achieve it and it is not known exactly what semantic information has been learned. In this paper, we propose a multi-stage enhanced representation learning method based on Query-View (MERL) with Intra-query stage and Inter-query stage to guide the model to explicitly learn the semantic relationship between the query and documents. In the Intra-query training stage, a content-based contrastive learning module without considering the special token [CLS] of BERT is utilized to optimize the semantic similarity of query and relevant documents. In the Inter-query training stage, an entity-oriented masked query prediction for establish a semantic relation of query-document pairs and an Inter-query contrastive learning module for extracting similar matching pattern of query-relevant documents are employed. Extensive experiments on MS MARCO passage ranking and TREC DL datasets show that the MERL method obtain significant improvements with a low number of parameters compared to the baseline models.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs’ performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs.
本文对用于知识图谱(KG)构建和推理的大型语言模型(LLM)进行了详尽的定量和定性评估。我们在八个不同的数据集上进行了实验,重点关注四个具有代表性的任务,包括实体和关系提取、事件提取、链接预测和问题解答,从而全面探索 LLM 在构建和推理领域的性能。实证研究结果表明,以 GPT-4 为代表的 LLM 更适合作为推理助手,而不是少量信息提取器。具体来说,虽然 GPT-4 在与 KG 构建相关的任务中表现出色,但在推理任务中却更胜一筹,在某些情况下甚至超过了微调模型。此外,我们的研究还扩展到了 LLM 在信息提取方面的潜在泛化能力,从而提出了虚拟知识提取任务,并开发了相应的 VINE 数据集。基于这些实证研究结果,我们进一步提出了 AutoKG,这是一种基于多机器人的方法,利用 LLMs 和外部资源进行 KG 构建和推理。我们期待这项研究能为知识图谱领域未来的工作提供宝贵的见解。
{"title":"LLMs for knowledge graph construction and reasoning: recent capabilities and future opportunities","authors":"Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, Ningyu Zhang","doi":"10.1007/s11280-024-01297-w","DOIUrl":"https://doi.org/10.1007/s11280-024-01297-w","url":null,"abstract":"<p>This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs’ performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose <b>AutoKG</b>, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs.</p>","PeriodicalId":501180,"journal":{"name":"World Wide Web","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142204996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}