How Effectively Can Machines Defend Against Machine-Generated Fake News? An Empirical Study

First Workshop on Insights from Negative Results in NLP Pub Date : 2020-11-01 DOI:10.18653/v1/2020.insights-1.7

Meghana Moorthy Bhat, S. Parthasarathy

引用次数: 12

Abstract

We empirically study the effectiveness of machine-generated fake news detectors by understanding the model’s sensitivity to different synthetic perturbations during test time. The current machine-generated fake news detectors rely on provenance to determine the veracity of news. Our experiments find that the success of these detectors can be limited since they are rarely sensitive to semantic perturbations and are very sensitive to syntactic perturbations. Also, we would like to open-source our code and believe it could be a useful diagnostic tool for evaluating models aimed at fighting machine-generated fake news.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

机器如何有效防御机器生成的假新闻?实证研究

通过了解模型在测试期间对不同合成扰动的敏感性，我们对机器生成假新闻检测器的有效性进行了实证研究。目前机器生成的假新闻检测器依赖于来源来确定新闻的真实性。我们的实验发现，这些检测器的成功是有限的，因为它们很少对语义扰动敏感，而对句法扰动非常敏感。此外，我们希望开源我们的代码，并相信它可能是一个有用的诊断工具，用于评估旨在打击机器生成假新闻的模型。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

First Workshop on Insights from Negative Results in NLP

自引率

0.00%

发文量

期刊最新文献

What GPT Knows About Who is Who Pathologies of Pre-trained Language Models in Few-shot Fine-tuning Can Question Rewriting Help Conversational Question Answering? Extending the Scope of Out-of-Domain: Examining QA models in multiple subdomains Do Data-based Curricula Work?