Computer Speech and Language最新文献

Sentence embedding, which aims to learn an effective representation of a sentence, is a significant part for downstream tasks. Recently, using contrastive learning and pre-trained model, most methods of sentence embedding achieve encouraging results. However, on the one hand, these methods utilize discrete data augmentation to obtain positive samples performing contrastive learning, which could distort the original semantic of sentences. On the other hand, most methods directly employ the contrastive frameworks of computer vision to perform contrastive learning, which could confine the contrastive training due to the discrete and sparse text data compared with image data. To solve the issues above, we design a novel contrastive framework based on generation model with multi-task learning by supervised contrastive training on the dataset of natural language inference (NLI) to obtain meaningful sentence embedding (SEBGM). SEBGM makes use of multi-task learning to enhance the usage of word-level and sentence-level semantic information of samples. In this way, the positive samples of SEBGM are from NLI rather than data augmentation. Extensive experiments show that our proposed SEBGM can advance the state-of-the-art sentence embedding on the semantic textual similarity (STS) tasks by utilizing multi-task learning.

Fine-tuning and inference on Large Language Models like BERT have become increasingly expensive regarding memory cost and computation resources. The recently proposed computation-flexible BERT models facilitate their deployment in varied computational environments. Training such flexible BERT models involves jointly optimizing multiple BERT subnets, which will unavoidably interfere with one another. Besides, the performance of large subnets is limited by the performance gap between the smallest subnet and the supernet, despite efforts to enhance the smaller subnets. In this regard, we propose layer-wise Neural grafting to boost BERT subnets, especially the larger ones. The proposed method improves the average performance of the subnets on six GLUE tasks and boosts the supernets on all GLUE tasks and the SQuAD data set. Based on the boosted subnets, we further build an inference framework enabling practical width- and depth-dynamic inference regarding different inputs by combining width-dynamic gating modules and early exit off-ramps in the depth dimension. Experimental results show that the proposed framework achieves a better dynamic inference range than other methods in terms of trading off performance and computational complexity on four GLUE tasks and SQuAD. In particular, our best-tradeoff inference result outperforms other fixed-size models with similar amount of computations. Compared to BERT-Base, the proposed inference framework yields a 1.3-point improvement in the average GLUE score and a 2.2-point increase in the F1 score on SQuAD, while reducing computations by around 45%.

Currently, the majority of retrieval-based question-answering systems depend on supervised training using question pairs. However, there is still a significant need for further exploration of how to employ unsupervised methods to improve the accuracy of retrieval-based question-answering systems. From the perspective of question topic keywords, this paper presents TFCSG, an unsupervised question-retrieval approach based on topic keyword filtering and multi-task learning. Firstly, we design the topic keyword filtering algorithm, which, unlike the topic model, can sequentially filter out the keywords of the question and can provide a training corpus for subsequent unsupervised learning. Then, three tasks are designed in this paper to complete the training of the question-retrieval model. The first task is a question contrastive learning task based on topic keywords repetition strategy, the second is questions and its corresponding sequential topic keywords similarity distribution task, and the third is a sequential topic keywords generation task using questions. These three tasks are trained in parallel in order to obtain quality question representations and thus improve the accuracy of question-retrieval task. Finally, our experimental results on the four publicly available datasets demonstrate the effectiveness of the TFCSG, with an average improvement of 7.1%, 4.4%, and 3.5% in the P@1, MAP, and MRR metrics when using the BERT model compared to the baseline model. The corresponding metrics improved by 5.7%, 3.5% and 3.0% on average when using the RoBERTa model. The accuracy of unsupervised similar question-retrieval task is effectively improved. In particular, the values of P@1, P@5, and P@10 are close, the retrieved similar questions are ranked more advance.

Relational triple extraction is a critical step in knowledge graph construction. Compared to pipeline-based extraction, joint extraction is gaining more attention because it can better utilize entity and relation information without causing error propagation issues. Yet, the challenge with joint extraction lies in handling overlapping triples. Existing approaches adopt sequential steps or multiple modules, which often accumulate errors and interfere with redundant data. In this study, we propose an innovative joint extraction model with cross-attention mechanism and global pointers with context shield window. Specifically, our methodology begins by inputting text data into a pre-trained RoBERTa model to generate word vector representations. Subsequently, these embeddings are passed through a modified cross-attention layer along with entity type embeddings to address missing entity type information. Next, we employ the global pointer to transform the extraction problem into a quintuple extraction problem, which skillfully solves the issue of overlapping triples. It is worth mentioning that we design a context shield window on the global pointer, which facilitates the identification of correct entities within a limited range during the entity extraction process. Finally, the capability of our model against malicious samples is improved by adding adversarial training during the training process. Demonstrating superiority over mainstream models, our approach achieves impressive results on three publicly available datasets.

Orthogonal linear mapping is a commonly used approach for generating cross-lingual embedding between two monolingual corpora that uses a word frequency-based seed dictionary alignment approach. While this approach is found to be effective for isomorphic language pairs, they do not perform well for distant language pairs with different sentence structures and morphological properties. For a distance language pair, the existing frequency-aligned orthogonal mapping methods suffer from two problems - (i)the frequency of source and target word are not comparable, and (ii)different word pairs in the seed dictionary may have different contribution. Motivated by the above two concerns, this paper proposes a novel centrality-aligned ridge regression-based orthogonal mapping. The proposed method uses centrality-based alignment for seed dictionary selection and ridge regression framework for incorporating influential weights of different word pairs in the seed dictionary. From various experimental observations over five language pairs (both isomorphic and distant languages), it is evident that the proposed method outperforms baseline methods in the Bilingual Dictionary Induction(BDI) task, Sentence Retrieval Task(SRT), and Machine Translation. Further, several analyses are also included to support the proposed method.

Reinforcement learning (RL) has demonstrated effectiveness in improving model performance and robustness for automatic speech recognition (ASR) tasks. Researchers have employed RL-based training strategies to enhance performance beyond conventional supervised or semi-supervised learning. However, existing approaches treat RL as a supplementary tool, leaving the untapped potential of RL training largely unexplored. In this paper, we formulate a novel pure RL setting where an ASR model is trained exclusively through RL via human feedback metrics, e.g., word error rate (WER) or binary reward. This approach promises to significantly simplify the annotation process if we could replace the conventional onerous annotation with a single numeric value in a human–computer interaction (HCI) way. Our experiments demonstrate the feasibility of this new setting and also identify two main inherent issues in conventional RL-based ASR training that may lead to performance degradation: (1) the mismatch issue between the action and reward has been commonly overlooked in Connectionist Temporal Classification (CTC) based models, which is attributed to the inherent CTC alignment mapping issue; (2) the classic exploration–exploitation trade-off still exists in the sampling stage of RL-based ASR, and finding the balance between them becomes a challenge. To address these issues, we first propose a new RL-based approach named CTC-aligned Policy Gradient (CTC-PG), which provides a unified formulation for different sampling strategies and alleviates the mismatch issue of action and reward in CTC-based models. Moreover, we propose Focal Sampling to balance the trade-off between exploration and exploitation with a flexible temperature parameter. Experiment results on LibriSpeech dataset showcase the effectiveness and robustness of our methods by harnessing the full potential of RL in training ASR models.

Higher order statistics (HOS), can be effectively employed for noise suppression, provided the noise follows a Gaussian distribution. Since most of the noises are distributed normally, HOS can be effectively used for speech enhancement in noisy environments. In the current work, HOS-based parametric modelling for magnitude spectrum estimation is proposed to improve the SNR under noisy conditions. To establish this, a non-Gaussian reduced ARMA model formulated using third order cumulant sequences (Giannakis, 1990) is used. Here, the AR and MA model orders, $p$ and $q$, are dynamically estimated by the well-established periodicity estimation technique under noisy conditions namely the Ramanujan Filter Bank (RFB) approach. The AR coefficients estimated from the reduced ARMA model are used to obtain the partially enhanced speech output, whose magnitude spectrum is then subjected to second-level enhancement using log MMSE with modified speech presence uncertainty (SPU) estimation technique. The refined magnitude spectrum, is combined with the phase spectrum extracted using proposed bicoherence-based phase compensation (BPC) technique, to estimate the enhanced speech output. The HOS-driven speech enhancement technique proposed in the current work is observed to be efficient for white, pink, babble and buccaneer noises. The objective measures, PESQ and STOI, indicate that the proposed method works well under all the noise conditions considered for evaluation.

With the explosion of information and users’ changing interest, program sequential recommendation becomes increasingly important for TV program platforms to help their users find interesting programs. Existing sequential recommendation methods mainly focus on modeling user preferences from users’ historical interaction behaviors directly, with insufficient learning about the dynamics of programs and users, while ignoring the rich semantic information from the heterogeneous graph. To address these issues, we propose the multipath-guided heterogeneous graph neural networks for TV program sequential recommendation (MHG-PSR), which can enhance the representations of programs and users through multiple paths in heterogeneous graphs. In our method, the auxiliary information is fused to supplement the semantics of program and user to obtain initial representations. Then, we explore the interactive behaviors of programs and users with temporal and auxiliary information to model the collaborative signals in the heterogeneous graph and extract the users’ dynamic preferences of programs. Extensive experiments on real-world datasets verify the proposed method can effectively improve the performance of TV program sequential recommendation.

Text matching is a computational task that involves comparing and establishing the semantic relationship between two textual inputs. The prevailing approach in text matching entails the computation of textual representations or employing attention mechanisms to facilitate interaction with the text. These techniques have demonstrated notable efficacy in various text-matching scenarios. However, these methods primarily focus on modeling the sentence pairs themselves and rarely incorporate additional information to enrich the models. In this study, we address the challenge of text matching in natural language processing by proposing a novel approach that leverages external knowledge sources, namely Wiktionary for word definitions and a knowledge graph for text triplet information. Unlike conventional methods that primarily rely on textual representations and attention mechanisms, our approach enhances semantic understanding by integrating relevant external information. We introduce a fusion module to amalgamate the semantic insights derived from the text and the external knowledge. Our methodology’s efficacy is evidenced through comprehensive experiments conducted on diverse datasets, encompassing natural language inference, text classification, and medical natural language inference. The results unequivocally indicate a significant enhancement in model performance, underscoring the effectiveness of incorporating external knowledge into text-matching tasks.

To alleviate the shortage of dialogue datasets for Cantonese, one of the low-resource languages, and facilitate the development of customized task-oriented dialogue systems, we propose KddRES, the first Cantonese Knowledge-driven dialogue dataset for REStaurants. It contains 834 multi-turn dialogues, 8000 utterances, and 26 distinct slots. The slots are hierarchical, and beneath the 26 coarse-grained slots are the additional 16 fine-grained slots. Annotations of dialogue states and dialogue actions at both the user and system sides are provided to suit multiple downstream tasks such as natural language understanding and dialogue state tracking. To effectively detect hierarchical slots, we propose a framework HierBERT by modelling label semantics and relationships between different slots. Experimental results demonstrate that KddRES is more challenging compared with existing datasets due to the introduction of hierarchical slots and our framework is particularly effective in detecting secondary slots and achieving a new state-of-the-art. Given the rich annotation and hierarchical slot structure of KddRES, we hope it will promote research on the development of customized dialogue systems in Cantonese and other conversational AI tasks, such as dialogue state tracking and policy learning.