首页 > 最新文献

ACM Transactions on Information Systems最新文献

英文 中文
Understanding and Predicting User Satisfaction with Conversational Recommender Systems 理解和预测会话推荐系统的用户满意度
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3624989
Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke
User satisfaction depicts the effectiveness of a system from the user’s perspective. Understanding and predicting user satisfaction is vital for the design of user-oriented evaluation methods for conversational recommender systems (CRSs) . Current approaches rely on turn-level satisfaction ratings to predict a user’s overall satisfaction with CRS. These methods assume that all users perceive satisfaction similarly, failing to capture the broader dialogue aspects that influence overall user satisfaction. We investigate the effect of several dialogue aspects on user satisfaction when interacting with a CRS. To this end, we annotate dialogues based on six aspects (i.e., relevance , interestingness , understanding , task-completion , interest-arousal , and efficiency ) at the turn and dialogue levels. We find that the concept of satisfaction varies per user. At the turn level, a system’s ability to make relevant recommendations is a significant factor in satisfaction. We adopt these aspects as features for predicting response quality and user satisfaction. We achieve an F1-score of 0.80 in classifying dissatisfactory dialogues, and a Pearson’s r of 0.73 for turn-level response quality estimation, demonstrating the effectiveness of the proposed dialogue aspects in predicting user satisfaction and being able to identify dialogues where the system is failing. With this article, we release our annotated data. 1
用户满意度从用户的角度描述了系统的有效性。理解和预测用户满意度对于设计会话推荐系统中以用户为导向的评价方法至关重要。目前的方法依赖于回合满意度评级来预测用户对CRS的总体满意度。这些方法假设所有用户对满意度的感知都是相似的,未能捕捉到影响总体用户满意度的更广泛的对话方面。我们研究了几个对话方面对用户满意度的影响,当与CRS交互时。为此,我们在回合和对话层面对对话进行了六个方面的注释(即相关性、趣味性、理解性、任务完成性、兴趣激发性和效率)。我们发现满意度的概念因用户而异。在回合级别,系统提出相关建议的能力是满意度的重要因素。我们采用这些方面作为预测响应质量和用户满意度的特征。我们在分类不满意的对话方面获得了0.80的f1分数,在回合级响应质量估计方面获得了0.73的Pearson’s r,证明了所提出的对话方面在预测用户满意度方面的有效性,并能够识别系统失败的对话。在本文中,我们发布了带注释的数据。1
{"title":"Understanding and Predicting User Satisfaction with Conversational Recommender Systems","authors":"Clemencia Siro, Mohammad Aliannejadi, Maarten de Rijke","doi":"10.1145/3624989","DOIUrl":"https://doi.org/10.1145/3624989","url":null,"abstract":"User satisfaction depicts the effectiveness of a system from the user’s perspective. Understanding and predicting user satisfaction is vital for the design of user-oriented evaluation methods for conversational recommender systems (CRSs) . Current approaches rely on turn-level satisfaction ratings to predict a user’s overall satisfaction with CRS. These methods assume that all users perceive satisfaction similarly, failing to capture the broader dialogue aspects that influence overall user satisfaction. We investigate the effect of several dialogue aspects on user satisfaction when interacting with a CRS. To this end, we annotate dialogues based on six aspects (i.e., relevance , interestingness , understanding , task-completion , interest-arousal , and efficiency ) at the turn and dialogue levels. We find that the concept of satisfaction varies per user. At the turn level, a system’s ability to make relevant recommendations is a significant factor in satisfaction. We adopt these aspects as features for predicting response quality and user satisfaction. We achieve an F1-score of 0.80 in classifying dissatisfactory dialogues, and a Pearson’s r of 0.73 for turn-level response quality estimation, demonstrating the effectiveness of the proposed dialogue aspects in predicting user satisfaction and being able to identify dialogues where the system is failing. With this article, we release our annotated data. 1","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
NIR-Prompt: A Multi-task Generalized Neural Information Retrieval Training Framework NIR-Prompt:一个多任务广义神经信息检索训练框架
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3626092
Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng
Information retrieval aims to find information that meets users’ needs from the corpus. Different needs correspond to different IR tasks such as document retrieval, open-domain question answering, retrieval-based dialogue, and so on, while they share the same schema to estimate the relationship between texts. It indicates that a good IR model can generalize to different tasks and domains. However, previous studies indicate that state-of-the-art neural information retrieval (NIR) models, e.g., pre-trained language models (PLMs) are hard to generalize. It is mainly because the end-to-end fine-tuning paradigm makes the model overemphasize task-specific signals and domain biases but loses the ability to capture generalized essential signals. To address this problem, we propose a novel NIR training framework named NIR-Prompt for retrieval and reranking stages based on the idea of decoupling signal capturing and combination. NIR-Prompt exploits Essential Matching Module (EMM) to capture the essential matching signals and gets the description of tasks by Matching Description Module (MDM). The description is used as task-adaptation information to combine the essential matching signals to adapt to different tasks. Experiments under in-domain multi-task, out-of-domain multi-task, and new task adaptation settings show that NIR-Prompt can improve the generalization of PLMs in NIR for both retrieval and reranking stages compared with baselines.
信息检索的目的是从语料库中找到符合用户需要的信息。不同的需求对应于不同的IR任务,如文档检索、开放域问答、基于检索的对话等,而它们共享相同的模式来估计文本之间的关系。这表明一个好的IR模型可以泛化到不同的任务和领域。然而,以往的研究表明,最先进的神经信息检索(NIR)模型,如预训练语言模型(PLMs)很难推广。这主要是因为端到端微调范式使模型过分强调特定于任务的信号和领域偏差,而失去了捕获广义基本信号的能力。为了解决这个问题,我们提出了一个新的NIR训练框架,名为NIR- prompt,用于检索和重新排序阶段,该框架基于解耦信号捕获和组合的思想。NIR-Prompt利用基本匹配模块(EMM)捕获基本匹配信号,并通过匹配描述模块(MDM)获取任务的描述。将描述信息作为任务自适应信息,将必要的匹配信号组合起来,以适应不同的任务。在域内多任务、域外多任务和新的任务自适应设置下的实验表明,与基线相比,NIR- prompt在检索和重新排序阶段都能提高plm在NIR中的泛化程度。
{"title":"NIR-Prompt: A Multi-task Generalized Neural Information Retrieval Training Framework","authors":"Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng","doi":"10.1145/3626092","DOIUrl":"https://doi.org/10.1145/3626092","url":null,"abstract":"Information retrieval aims to find information that meets users’ needs from the corpus. Different needs correspond to different IR tasks such as document retrieval, open-domain question answering, retrieval-based dialogue, and so on, while they share the same schema to estimate the relationship between texts. It indicates that a good IR model can generalize to different tasks and domains. However, previous studies indicate that state-of-the-art neural information retrieval (NIR) models, e.g., pre-trained language models (PLMs) are hard to generalize. It is mainly because the end-to-end fine-tuning paradigm makes the model overemphasize task-specific signals and domain biases but loses the ability to capture generalized essential signals. To address this problem, we propose a novel NIR training framework named NIR-Prompt for retrieval and reranking stages based on the idea of decoupling signal capturing and combination. NIR-Prompt exploits Essential Matching Module (EMM) to capture the essential matching signals and gets the description of tasks by Matching Description Module (MDM). The description is used as task-adaptation information to combine the essential matching signals to adapt to different tasks. Experiments under in-domain multi-task, out-of-domain multi-task, and new task adaptation settings show that NIR-Prompt can improve the generalization of PLMs in NIR for both retrieval and reranking stages compared with baselines.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293550","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation 基于会话推荐的时空对比学习增强gnn
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3626091
Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang
Session-based recommendation (SBR) systems aim to utilize the user’s short-term behavior sequence to predict the next item without the detailed user profile. Most recent works try to model the user preference by treating the sessions as between-item transition graphs and utilize various graph neural networks (GNNs) to encode the representations of pair-wise relations among items and their neighbors. Some of the existing GNN-based models mainly focus on aggregating information from the view of spatial graph structure, which ignores the temporal relations within neighbors of an item during message passing and the information loss results in a sub-optimal problem. Other works embrace this challenge by incorporating additional temporal information but lack sufficient interaction between the spatial and temporal patterns. To address this issue, inspired by the uniformity and alignment properties of contrastive learning techniques, we propose a novel framework called Session-based Recommendation with Spatio-temporal Contrastive Learning-enhanced GNNs (RESTC). The idea is to supplement the GNN-based main supervised recommendation task with the temporal representation via an auxiliary cross-view contrastive learning mechanism. Furthermore, a novel global collaborative filtering graph embedding is leveraged to enhance the spatial view in the main task. Extensive experiments demonstrate the significant performance of RESTC compared with the state-of-the-art baselines. We release our source code at https://github.com/SUSTechBruce/RESTC-Source-code .
基于会话的推荐(SBR)系统旨在利用用户的短期行为序列来预测下一个项目,而无需详细的用户配置文件。最近的研究尝试通过将会话视为项目之间的过渡图来建模用户偏好,并利用各种图神经网络(gnn)对项目及其邻居之间的成对关系表示进行编码。现有的一些基于gnn的模型主要是从空间图结构的角度对信息进行聚合,忽略了信息传递过程中相邻节点间的时间关系,导致信息丢失,存在次优问题。其他作品通过纳入额外的时间信息来迎接这一挑战,但在空间和时间模式之间缺乏足够的相互作用。为了解决这个问题,受对比学习技术的一致性和一致性的启发,我们提出了一个新的框架,称为基于会话的推荐与时空对比学习增强的gnn (RESTC)。其思想是通过辅助的跨视图对比学习机制,用时间表征来补充基于gnn的主监督推荐任务。此外,利用一种新颖的全局协同过滤图嵌入来增强主任务的空间视图。大量的实验表明,与最先进的基线相比,RESTC具有显著的性能。我们在https://github.com/SUSTechBruce/RESTC-Source-code上发布了源代码。
{"title":"Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation","authors":"Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang","doi":"10.1145/3626091","DOIUrl":"https://doi.org/10.1145/3626091","url":null,"abstract":"Session-based recommendation (SBR) systems aim to utilize the user’s short-term behavior sequence to predict the next item without the detailed user profile. Most recent works try to model the user preference by treating the sessions as between-item transition graphs and utilize various graph neural networks (GNNs) to encode the representations of pair-wise relations among items and their neighbors. Some of the existing GNN-based models mainly focus on aggregating information from the view of spatial graph structure, which ignores the temporal relations within neighbors of an item during message passing and the information loss results in a sub-optimal problem. Other works embrace this challenge by incorporating additional temporal information but lack sufficient interaction between the spatial and temporal patterns. To address this issue, inspired by the uniformity and alignment properties of contrastive learning techniques, we propose a novel framework called Session-based Recommendation with Spatio-temporal Contrastive Learning-enhanced GNNs (RESTC). The idea is to supplement the GNN-based main supervised recommendation task with the temporal representation via an auxiliary cross-view contrastive learning mechanism. Furthermore, a novel global collaborative filtering graph embedding is leveraged to enhance the spatial view in the main task. Extensive experiments demonstrate the significant performance of RESTC compared with the state-of-the-art baselines. We release our source code at https://github.com/SUSTechBruce/RESTC-Source-code .","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences 基于公理偏好模型的信息检索评价方法
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3632171
Fernando Giner
Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics, and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Some works have modelled the effectiveness of retrieval measures with axioms, heuristics or desirable properties, leading to order relationships on the set where they are defined. Each of these ordering structures constitute an axiomatic model of preferences (AMP), which can be considered as an ’ideal’ scenario of retrieval. Based on lattice theory and on the representational theory of measurement, this work formally explores numeric, metric and scale properties of some effectiveness measures defined on AMPs. In some of these scenarios, retrieval measures are completely determined from the scores of a subset of document rankings: join-irreducible elements. All the possible metrics and pseudometrics, defined on these structures are expressed in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F -measure, RBP , DCG and AP confirm some recent results in the IR field.
信息检索评价指标是获取文档与主题的相关性,决定检索系统任务执行效率的关键。通过形式属性对IR评价措施进行研究,可以更好地理解它们对特定任务的适用性。一些作品用公理、启发式或理想属性来模拟检索措施的有效性,从而在定义检索措施的集合上建立有序关系。这些排序结构中的每一个都构成了一个公理偏好模型(AMP),这可以被认为是检索的“理想”场景。基于晶格理论和测量的表征理论,本文正式探讨了在amp上定义的一些有效性度量的数值、度量和尺度性质。在其中一些场景中,检索方法完全由文档排名子集的分数决定:连接不可约元素。在这些结构上定义的所有可能度量和伪度量都用连接不可约元素表示。通过对精度、召回率、F -测度、RBP、DCG和AP等指标的推导,证实了近年来红外领域的一些研究成果。
{"title":"Information Retrieval Evaluation Measures Defined on Some Axiomatic Models of Preferences","authors":"Fernando Giner","doi":"10.1145/3632171","DOIUrl":"https://doi.org/10.1145/3632171","url":null,"abstract":"Information retrieval (IR) evaluation measures are essential for capturing the relevance of documents to topics, and determining the task performance efficiency of retrieval systems. The study of IR evaluation measures through their formal properties enables a better understanding of their suitability for a specific task. Some works have modelled the effectiveness of retrieval measures with axioms, heuristics or desirable properties, leading to order relationships on the set where they are defined. Each of these ordering structures constitute an axiomatic model of preferences (AMP), which can be considered as an ’ideal’ scenario of retrieval. Based on lattice theory and on the representational theory of measurement, this work formally explores numeric, metric and scale properties of some effectiveness measures defined on AMPs. In some of these scenarios, retrieval measures are completely determined from the scores of a subset of document rankings: join-irreducible elements. All the possible metrics and pseudometrics, defined on these structures are expressed in terms of the join-irreducible elements. The deduced scale properties of the precision, recall, F -measure, RBP , DCG and AP confirm some recent results in the IR field.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135341766","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Diffusion model for POI recommendation POI推荐的扩散模型
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3624475
Yifang Qin, Hongjun Wu, Wei Ju, Xiao Luo, Ming Zhang
Next Point-of-Interest (POI) recommendation is a critical task in location-based services that aim to provide personalized suggestions for the user’s next destination. Previous works on POI recommendation have laid focus on modeling the user’s spatial preference. However, existing works that leverage spatial information are only based on the aggregation of users’ previous visited positions, which discourages the model from recommending POIs in novel areas. This trait of position-based methods will harm the model’s performance in many situations. Additionally, incorporating sequential information into the user’s spatial preference remains a challenge. In this article, we propose Diff-POI : a Diffu sion-based model that samples the user’s spatial preference for the next POI recommendation. Inspired by the wide application of diffusion algorithm in sampling from distributions, Diff-POI encodes the user’s visiting sequence and spatial character with two tailor-designed graph encoding modules, followed by a diffusion-based sampling strategy to explore the user’s spatial visiting trends. We leverage the diffusion process and its reverse form to sample from the posterior distribution and optimized the corresponding score function. We design a joint training and inference framework to optimize and evaluate the proposed Diff-POI. Extensive experiments on four real-world POI recommendation datasets demonstrate the superiority of our Diff-POI over state-of-the-art baseline methods. Further ablation and parameter studies on Diff-POI reveal the functionality and effectiveness of the proposed diffusion-based sampling strategy for addressing the limitations of existing methods.
下一个兴趣点(POI)推荐是基于位置的服务中的一项关键任务,旨在为用户的下一个目的地提供个性化建议。先前关于POI推荐的研究主要集中在用户空间偏好的建模上。然而,现有的利用空间信息的工作仅基于用户以前访问过的位置的汇总,这阻碍了模型在新区域推荐poi。在许多情况下,基于位置的方法的这种特性会损害模型的性能。此外,将顺序信息整合到用户的空间偏好中仍然是一个挑战。在本文中,我们提出了diffi -POI:一个基于Diffu的模型,它对用户的空间偏好进行采样,以推荐下一个POI。受扩散算法在分布采样中广泛应用的启发,diffi - poi采用两个定制的图编码模块对用户的访问序列和空间特征进行编码,然后采用基于扩散的采样策略来探索用户的空间访问趋势。我们利用扩散过程及其反向形式从后验分布中抽样,并优化相应的分数函数。我们设计了一个联合训练和推理框架来优化和评估所提出的Diff-POI。在四个真实世界POI推荐数据集上进行的大量实验表明,我们的diffi -POI优于最先进的基线方法。对Diff-POI的进一步消融和参数研究揭示了所提出的基于扩散的采样策略的功能性和有效性,以解决现有方法的局限性。
{"title":"A Diffusion model for POI recommendation","authors":"Yifang Qin, Hongjun Wu, Wei Ju, Xiao Luo, Ming Zhang","doi":"10.1145/3624475","DOIUrl":"https://doi.org/10.1145/3624475","url":null,"abstract":"Next Point-of-Interest (POI) recommendation is a critical task in location-based services that aim to provide personalized suggestions for the user’s next destination. Previous works on POI recommendation have laid focus on modeling the user’s spatial preference. However, existing works that leverage spatial information are only based on the aggregation of users’ previous visited positions, which discourages the model from recommending POIs in novel areas. This trait of position-based methods will harm the model’s performance in many situations. Additionally, incorporating sequential information into the user’s spatial preference remains a challenge. In this article, we propose Diff-POI : a Diffu sion-based model that samples the user’s spatial preference for the next POI recommendation. Inspired by the wide application of diffusion algorithm in sampling from distributions, Diff-POI encodes the user’s visiting sequence and spatial character with two tailor-designed graph encoding modules, followed by a diffusion-based sampling strategy to explore the user’s spatial visiting trends. We leverage the diffusion process and its reverse form to sample from the posterior distribution and optimized the corresponding score function. We design a joint training and inference framework to optimize and evaluate the proposed Diff-POI. Extensive experiments on four real-world POI recommendation datasets demonstrate the superiority of our Diff-POI over state-of-the-art baseline methods. Further ablation and parameter studies on Diff-POI reveal the functionality and effectiveness of the proposed diffusion-based sampling strategy for addressing the limitations of existing methods.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293660","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Efficient Neural Ranking using Forward Indexes and Lightweight Encoders 使用前向索引和轻量级编码器的高效神经排序
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3631939
Jurek Leonhardt, Henrik Müller, Koustav Rudra, Megha Khosla, Abhijit Anand, Avishek Anand
Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency. We propose Fast-Forward indexes—vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially. We perform evaluation to show the effectiveness and efficiency of Fast-Forward indexes—our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.
基于双编码器的密集检索模型已经成为红外领域的标准。它们采用大型的基于transformer的语言模型,这在资源和延迟方面是出了名的低效。我们提出了快速前向索引——矢量前向索引,利用双编码器模型的语义匹配能力进行高效的重新排序。我们的框架能够在非常高的检索深度上重新排序,并通过分数插值结合了词汇和语义匹配的优点。此外,为了减轻双编码器的局限性,我们解决了两个主要挑战:首先,我们通过预计算表示来提高计算效率,避免不必要的计算,或者降低编码器的复杂性。这使我们能够大大提高排名效率和延迟。其次,优化索引的内存占用和维护成本;我们提出了两种互补的技术来减少索引大小,并表明,通过动态删除不相关的文档令牌,可以大大提高索引维护效率。我们执行评估以显示Fast-Forward索引的有效性和效率-我们的方法具有低延迟,并且无需硬件加速(如gpu)即可获得具有竞争力的结果。
{"title":"Efficient Neural Ranking using Forward Indexes and Lightweight Encoders","authors":"Jurek Leonhardt, Henrik Müller, Koustav Rudra, Megha Khosla, Abhijit Anand, Avishek Anand","doi":"10.1145/3631939","DOIUrl":"https://doi.org/10.1145/3631939","url":null,"abstract":"Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency. We propose Fast-Forward indexes—vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially. We perform evaluation to show the effectiveness and efficiency of Fast-Forward indexes—our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135342067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Contrastive Self-supervised Learning in Recommender Systems: A Survey 推荐系统中的对比自监督学习研究综述
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-08 DOI: 10.1145/3627158
Mengyuan Jing, Yanmin Zhu, Tianzi Zang, Ke Wang
Deep learning-based recommender systems have achieved remarkable success in recent years. However, these methods usually heavily rely on labeled data (i.e., user-item interactions), suffering from problems such as data sparsity and cold-start. Self-supervised learning, an emerging paradigm that extracts information from unlabeled data, provides insights into addressing these problems. Specifically, contrastive self-supervised learning, due to its flexibility and promising performance, has attracted considerable interest and recently become a dominant branch in self-supervised learning-based recommendation methods. In this survey, we provide an up-to-date and comprehensive review of current contrastive self-supervised learning-based recommendation methods. Firstly, we propose a unified framework for these methods. We then introduce a taxonomy based on the key components of the framework, including view generation strategy, contrastive task, and contrastive objective. For each component, we provide detailed descriptions and discussions to guide the choice of the appropriate method. Finally, we outline open issues and promising directions for future research.
近年来,基于深度学习的推荐系统取得了显著的成功。然而,这些方法通常严重依赖于标记数据(即用户-项目交互),因此存在数据稀疏性和冷启动等问题。自监督学习是一种新兴的范例,它从未标记的数据中提取信息,为解决这些问题提供了见解。具体来说,对比自监督学习由于其灵活性和良好的性能,引起了人们的广泛关注,最近成为基于自监督学习的推荐方法的一个主导分支。在这项调查中,我们提供了一个最新的和全面的审查,目前对比自监督学习为基础的推荐方法。首先,我们为这些方法提出了一个统一的框架。然后,我们介绍了一个基于框架关键组件的分类法,包括视图生成策略、对比任务和对比目标。对于每个组件,我们提供了详细的描述和讨论,以指导选择合适的方法。最后,我们概述了未来研究的开放问题和有希望的方向。
{"title":"Contrastive Self-supervised Learning in Recommender Systems: A Survey","authors":"Mengyuan Jing, Yanmin Zhu, Tianzi Zang, Ke Wang","doi":"10.1145/3627158","DOIUrl":"https://doi.org/10.1145/3627158","url":null,"abstract":"Deep learning-based recommender systems have achieved remarkable success in recent years. However, these methods usually heavily rely on labeled data (i.e., user-item interactions), suffering from problems such as data sparsity and cold-start. Self-supervised learning, an emerging paradigm that extracts information from unlabeled data, provides insights into addressing these problems. Specifically, contrastive self-supervised learning, due to its flexibility and promising performance, has attracted considerable interest and recently become a dominant branch in self-supervised learning-based recommendation methods. In this survey, we provide an up-to-date and comprehensive review of current contrastive self-supervised learning-based recommendation methods. Firstly, we propose a unified framework for these methods. We then introduce a taxonomy based on the key components of the framework, including view generation strategy, contrastive task, and contrastive objective. For each component, we provide detailed descriptions and discussions to guide the choice of the appropriate method. Finally, we outline open issues and promising directions for future research.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Improving First-stage Retrieval of Point-of-Interest Search by Pre-training Models 利用预训练模型改进兴趣点搜索的第一阶段检索
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-07 DOI: 10.1145/3631937
Lang Mei, Jiaxin Mao, Juan Hu, Naiqiang Tan, Hua Chai, Ji-rong Wen
Point-of-interest (POI) search is important for location-based services, such as navigation and online ride-hailing service. The goal of POI search is to find the most relevant destinations from a large-scale POI database given a text query. To improve the effectiveness and efficiency of POI search, most existing approaches are based on a multi-stage pipeline that consists of an efficiency-oriented retrieval stage and one or more effectiveness-oriented re-rank stages. In this paper, we focus on the first efficiency-oriented retrieval stage of the POI search. We first identify the limitations of existing first-stage POI retrieval models in capturing the semantic-geography relationship and modeling the fine-grained geographical context information. Then, we propose a Geo-Enhanced Dense Retrieval framework for POI search to alleviate the above problems. Specifically, the proposed framework leverages the capacity of pre-trained language models (e.g., BERT) and designs a pre-training approach to better model the semantic match between the query prefix and POIs. With the POI collection, we first perform a token-level pre-training task based on a geographical-sensitive masked language prediction, and design two retrieval-oriented pre-training tasks that link the address of each POI to its name and geo-location. With the user behavior logs collected from an online POI search system, we design two additional pre-training tasks based on users’ query reformulation behavior and the transitions between POIs. We also utilize a late-interaction network structure to model the fine-grained interactions between the text and geographical context information within an acceptable query latency. Extensive experiments on the real-world datasets collected from the Didichuxing application demonstrate that the proposed framework can achieve superior retrieval performance over existing first-stage POI retrieval methods.
兴趣点(POI)搜索对于基于位置的服务很重要,比如导航和在线叫车服务。POI搜索的目标是从给定文本查询的大规模POI数据库中找到最相关的目的地。为了提高POI搜索的有效性和效率,大多数现有方法都基于多级管道,该管道由一个面向效率的检索阶段和一个或多个面向效率的重新排序阶段组成。在本文中,我们关注POI搜索的第一个面向效率的检索阶段。本文首先指出了现有第一阶段POI检索模型在捕获语义-地理关系和建模细粒度地理上下文信息方面的局限性。针对上述问题,提出了一种基于地理增强的密集检索框架。具体来说,所提出的框架利用了预训练语言模型(例如BERT)的能力,并设计了一种预训练方法来更好地建模查询前缀和poi之间的语义匹配。对于POI集合,我们首先基于地理敏感的掩码语言预测执行令牌级预训练任务,并设计两个面向检索的预训练任务,将每个POI的地址与其名称和地理位置联系起来。利用从在线POI搜索系统中收集的用户行为日志,我们基于用户的查询重新表述行为和POI之间的转换设计了两个额外的预训练任务。我们还利用后期交互网络结构在可接受的查询延迟内对文本和地理上下文信息之间的细粒度交互进行建模。在didihuxing应用中收集的真实数据集上进行的大量实验表明,所提出的框架比现有的第一阶段POI检索方法具有更好的检索性能。
{"title":"Improving First-stage Retrieval of Point-of-Interest Search by Pre-training Models","authors":"Lang Mei, Jiaxin Mao, Juan Hu, Naiqiang Tan, Hua Chai, Ji-rong Wen","doi":"10.1145/3631937","DOIUrl":"https://doi.org/10.1145/3631937","url":null,"abstract":"Point-of-interest (POI) search is important for location-based services, such as navigation and online ride-hailing service. The goal of POI search is to find the most relevant destinations from a large-scale POI database given a text query. To improve the effectiveness and efficiency of POI search, most existing approaches are based on a multi-stage pipeline that consists of an efficiency-oriented retrieval stage and one or more effectiveness-oriented re-rank stages. In this paper, we focus on the first efficiency-oriented retrieval stage of the POI search. We first identify the limitations of existing first-stage POI retrieval models in capturing the semantic-geography relationship and modeling the fine-grained geographical context information. Then, we propose a Geo-Enhanced Dense Retrieval framework for POI search to alleviate the above problems. Specifically, the proposed framework leverages the capacity of pre-trained language models (e.g., BERT) and designs a pre-training approach to better model the semantic match between the query prefix and POIs. With the POI collection, we first perform a token-level pre-training task based on a geographical-sensitive masked language prediction, and design two retrieval-oriented pre-training tasks that link the address of each POI to its name and geo-location. With the user behavior logs collected from an online POI search system, we design two additional pre-training tasks based on users’ query reformulation behavior and the transitions between POIs. We also utilize a late-interaction network structure to model the fine-grained interactions between the text and geographical context information within an acceptable query latency. Extensive experiments on the real-world datasets collected from the Didichuxing application demonstrate that the proposed framework can achieve superior retrieval performance over existing first-stage POI retrieval methods.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135476078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
SetRank: A Setwise Bayesian Approach for Collaborative Ranking in Recommender System SetRank:推荐系统协同排名的一种Setwise贝叶斯方法
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-07 DOI: 10.1145/3626194
Chao Wang, Hengshu Zhu, Chen Zhu, Chuan Qin, Enhong Chen, Hui Xiong
The recent development of recommender systems has a focus on collaborative ranking, which provides users with a sorted list rather than rating prediction. The sorted item lists can more directly reflect the preferences for users and usually perform better than rating prediction in practice. While considerable efforts have been made in this direction, the well-known pairwise and listwise approaches have still been limited by various challenges. Specifically, for the pairwise approaches, the assumption of independent pairwise preference is not always held in practice. Also, the listwise approaches cannot efficiently accommodate “ties” and unobserved data due to the precondition of the entire list permutation. To this end, in this article, we propose a novel setwise Bayesian approach for collaborative ranking, namely, SetRank, to inherently accommodate the characteristics of user feedback in recommender systems. SetRank aims to maximize the posterior probability of novel setwise preference structures and three implementations for SetRank are presented. We also theoretically prove that the bound of excess risk in SetRank can be proportional to (sqrt {M/N}) , where M and N are the numbers of items and users, respectively. Finally, extensive experiments on four real-world datasets clearly validate the superiority of SetRank compared with various state-of-the-art baselines.
推荐系统的最新发展侧重于协作排名,它为用户提供排序列表而不是评级预测。排序后的物品列表可以更直接地反映用户的偏好,在实践中通常比评级预测效果更好。虽然在这方面已经作出了相当大的努力,但众所周知的成对和列表方法仍然受到各种挑战的限制。具体而言,对于两两方法,在实践中并不总是坚持独立的两两偏好假设。此外,由于整个列表排列的先决条件,列表方法不能有效地容纳“关系”和未观察到的数据。为此,在本文中,我们提出了一种新的setwise贝叶斯协同排名方法,即SetRank,以内在地适应推荐系统中用户反馈的特征。SetRank的目的是最大化新的集合偏好结构的后验概率,并提出了三种SetRank的实现方法。我们还从理论上证明了SetRank的超额风险界可以与(sqrt {M/N})成正比,其中M和N分别为商品数量和用户数量。最后,在四个真实数据集上进行的大量实验清楚地验证了SetRank与各种最先进基线相比的优越性。
{"title":"SetRank: A Setwise Bayesian Approach for Collaborative Ranking in Recommender System","authors":"Chao Wang, Hengshu Zhu, Chen Zhu, Chuan Qin, Enhong Chen, Hui Xiong","doi":"10.1145/3626194","DOIUrl":"https://doi.org/10.1145/3626194","url":null,"abstract":"The recent development of recommender systems has a focus on collaborative ranking, which provides users with a sorted list rather than rating prediction. The sorted item lists can more directly reflect the preferences for users and usually perform better than rating prediction in practice. While considerable efforts have been made in this direction, the well-known pairwise and listwise approaches have still been limited by various challenges. Specifically, for the pairwise approaches, the assumption of independent pairwise preference is not always held in practice. Also, the listwise approaches cannot efficiently accommodate “ties” and unobserved data due to the precondition of the entire list permutation. To this end, in this article, we propose a novel setwise Bayesian approach for collaborative ranking, namely, SetRank, to inherently accommodate the characteristics of user feedback in recommender systems. SetRank aims to maximize the posterior probability of novel setwise preference structures and three implementations for SetRank are presented. We also theoretically prove that the bound of excess risk in SetRank can be proportional to (sqrt {M/N}) , where M and N are the numbers of items and users, respectively. Finally, extensive experiments on four real-world datasets clearly validate the superiority of SetRank compared with various state-of-the-art baselines.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135432011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Bi-Preference Learning Heterogeneous Hypergraph Networks for Session-based Recommendation 基于会话推荐的双偏好学习异构超图网络
2区 计算机科学 Q1 Business, Management and Accounting Pub Date : 2023-11-07 DOI: 10.1145/3631940
Xiaokun Zhang, Bo Xu, Fenglong Ma, Chenliang Li, Yuan Lin, Hongfei Lin
Session-based recommendation intends to predict next purchased items based on anonymous behavior sequences. Numerous economic studies have revealed that item price is a key factor influencing user purchase decisions. Unfortunately, existing methods for session-based recommendation only aim at capturing user interest preference, while ignoring user price preference. Actually, there are primarily two challenges preventing us from accessing price preference. Firstly, the price preference is highly associated to various item features ( i.e., category and brand), which asks us to mine price preference from heterogeneous information. Secondly, price preference and interest preference are interdependent and collectively determine user choice, necessitating that we jointly consider both price and interest preference for intent modeling. To handle above challenges, we propose a novel approach Bi-Preference Learning Heterogeneous Hypergraph Networks (BiPNet) for session-based recommendation. Specifically, the customized heterogeneous hypergraph networks with a triple-level convolution are devised to capture user price and interest preference from heterogeneous features of items. Besides, we develop a Bi-Preference Learning schema to explore mutual relations between price and interest preference and collectively learn these two preferences under the multi-task learning architecture. Extensive experiments on multiple public datasets confirm the superiority of BiPNet over competitive baselines. Additional research also supports the notion that the price is crucial for the task.
基于会话的推荐旨在根据匿名行为序列预测下一个购买的物品。大量经济学研究表明,商品价格是影响用户购买决策的关键因素。遗憾的是,现有的基于会话的推荐方法只关注用户的兴趣偏好,而忽略了用户的价格偏好。实际上,阻碍我们获得价格优惠的主要有两个挑战。首先,价格偏好与各种商品特征(即品类和品牌)高度相关,这就要求我们从异构信息中挖掘价格偏好。其次,价格偏好和兴趣偏好是相互依赖的,共同决定了用户的选择,因此我们需要在意向建模中同时考虑价格偏好和兴趣偏好。为了应对上述挑战,我们提出了一种新的基于会话的推荐方法——双偏好学习异构超图网络(BiPNet)。具体而言,设计了具有三层卷积的定制异构超图网络,以从物品的异构特征中捕获用户价格和兴趣偏好。此外,我们开发了一个双偏好学习模式来探索价格偏好和兴趣偏好之间的相互关系,并在多任务学习架构下集体学习这两种偏好。在多个公共数据集上进行的大量实验证实了BiPNet优于竞争基线的优势。额外的研究也支持了价格对任务至关重要的观点。
{"title":"Bi-Preference Learning Heterogeneous Hypergraph Networks for Session-based Recommendation","authors":"Xiaokun Zhang, Bo Xu, Fenglong Ma, Chenliang Li, Yuan Lin, Hongfei Lin","doi":"10.1145/3631940","DOIUrl":"https://doi.org/10.1145/3631940","url":null,"abstract":"Session-based recommendation intends to predict next purchased items based on anonymous behavior sequences. Numerous economic studies have revealed that item price is a key factor influencing user purchase decisions. Unfortunately, existing methods for session-based recommendation only aim at capturing user interest preference, while ignoring user price preference. Actually, there are primarily two challenges preventing us from accessing price preference. Firstly, the price preference is highly associated to various item features ( i.e., category and brand), which asks us to mine price preference from heterogeneous information. Secondly, price preference and interest preference are interdependent and collectively determine user choice, necessitating that we jointly consider both price and interest preference for intent modeling. To handle above challenges, we propose a novel approach Bi-Preference Learning Heterogeneous Hypergraph Networks (BiPNet) for session-based recommendation. Specifically, the customized heterogeneous hypergraph networks with a triple-level convolution are devised to capture user price and interest preference from heterogeneous features of items. Besides, we develop a Bi-Preference Learning schema to explore mutual relations between price and interest preference and collectively learn these two preferences under the multi-task learning architecture. Extensive experiments on multiple public datasets confirm the superiority of BiPNet over competitive baselines. Additional research also supports the notion that the price is crucial for the task.","PeriodicalId":50936,"journal":{"name":"ACM Transactions on Information Systems","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135475334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
ACM Transactions on Information Systems
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1