An Emotional Spatial Handwriting System

Abstract

According to graphology, people's emotional states can be detected from their handwriting. Unlike writing on paper, which can be analysed through its on-surface properties, spatial interaction-based handwriting is entirely in-air. Consequently, the techniques used in graphology to reveal the emotions of the writer are not directly transferable to spatial interaction. The purpose of our research is to propose a 3D handwriting system with emotional capabilities. For our study, we retained height basic emotions represented by a large spectrum of coordinates in the Russell's valence-arousal model: afraid, angry, disgusted, happy, sad, surprised, amorous and serious. We used the Leap Motion sensor (https://www.leapmotion.com) to capture hand motion; C# and the Unity 3D game engine (https://unity3d.com) for the 3D rendering of the handwritten characters. With our system, users can write freely with their fingers in the air and immerse themselves in their handwriting by wearing a virtual reality headset. We aim to create a rendering model that can be universally applied to any handwriting and any alphabet: our choice of parameters is inspired by both Latin typography and Chinese calligraphy, characterised by its four elementary writing instruments: the brush, the ink, the brush-stand and the ink-stone. The final parameter selection process was carried out by immersing ourselves in our own in-air handwriting and through numerous trials. The five rendering parameters we chose are: (1) weight determined by the radius of the rendered stroke; (2) smoothness determined by the minimum length of one stroke segment; (3) tip of stroke determined by the ratio of the radius to the writing speed; (4) ink density determined by the opacity of the rendering material; and (5) ink dryness determined by the texture of the rendering material, which can be coarse or smooth. Having implemented the 3D handwriting system and empirically determined five rendering parameters, we designed a survey to gather opinions on which rendering parameters' values are most effective at conveying the intended emotions. For each parameter, we created three handwriting samples by varying the value of the parameter, and to avoid the combinatorial explosion of the number of samples, each parameter was made to vary independently of the others. The formula we used to calculate the optimal value of a parameter is as follows: Where i = 1, 2, 3 refers to the value of the parameter used in the sample; Q is the total number of respondents (64 in average); qi is the number of people who chose that sample; and Pi denotes the parameter. Applying the R values to the 3D handwriting system in Unity, we obtain the eight emotional styles illustrated below. We calculated the Euclidean distances between each pair of emotions using their 2D coordinates (x, y) in the Russell's valence-arousal emotion model and their 5-dimensional vectors of normalised parameters' values. Across all pairs of emotions, there is a positive correlation (R=0.41) between the two distances. This is an interesting result, which seems to support the choice of parameters' values that was made in the model. We then conducted another survey (42 respondents) to evaluate the emotional capabilities of our rendering model. Handwriting samples in both Chinese and English were produced for each of the 8 emotions, making a total of 16 samples. The four most notably recognised emotions are: afraid, sad, serious and angry. Binomial tests with a 95% confidence interval showed that for these four emotions the respondents' choices were significantly different from random chance. We note that these four emotions have all negative or neutral valence in the Russell's valence-arousal model. The emotion afraid was particularly well recognised. The emotion happy was well recognised but was also often confused for serious. The least correctly identified emotions are disgusted, amorous and surprised. Selecting one emotion among eight by observing a single word sample is a difficult exercise, but the results are encouraging.