Analyzing the impact of conversation structure on predicting persuasive comments online

Abstract

The topic of persuasion in online conversations has social, political and security implications; as a consequence, the problem of predicting persuasive comments in online discussions is receiving increasing attention in the literature. Following recent advancements in graph neural networks, we analyze the impact of conversation structure in predicting persuasive comments in online discussions. We evaluate the performance of artificial intelligence models receiving as input graphs constructed on the top of online conversations sourced from the “Change My View” Reddit channel. We experiment with different graph architectures and compare the performance on graph neural networks, as structure-based models, and dense neural networks as baseline models. Experiments are conducted on two tasks: (1) persuasive comment detection, aiming to predict which comments are persuasive, and (2) influence prediction, aiming to predict which users are persuasive. The experimental results show that the role of the conversation structure in predicting persuasiveness is strongly dependent on its graph representation given as input to the graph neural network. In particular, a graph structure linking only comments belonging to the same speaker in the conversation achieves the best performance in both tasks. This structure outperforms both the baseline model, which does not consider any structural information, and structures linking different speakers’ comments with each other. Specifically, the F1 score of the best performing model is 0.58, which represents an improvement of 5.45% over the baseline model (F1 score of 0.55) and 7.41% over the model linking different speakers’ comments (F1 score of 0.54).