Mental Health Computing via Harvesting Social Media Data

1 Introduction

With the rapid pace of life, mental health has received widespread attention nowadays. Mental, neurological, and substance use disorders contribute to over 10 percent of global burden of disease [WHO, 2013]. Common symptoms like stress, or clinical disorders like depression, excessive and chronic undesirable mental states are quite harmful, and thus it is of vital significance to detect mental health problems before they lead to severe consequences. Authoritative mental criteria like DSM and ICD-10 [Segal, 2000; WHO, 1992] have defined the distinguishing behaviors in daily lives for disorder diagnosis. However, traditional interventions based on face-to-face interviews or self-report questionnaires are expensive and hysteretic, and the potential antipathy towards consulting psychiatrists exacerbate the conditions [WHO, 2013].

Nowadays, social media platforms like Twitter 1 and Weibo 2 have become increasingly prevalent for users to express themselves and interact with friends. The usergenerated content (UGC) may indicate their real-life states and emotions in a timely manner, which makes the analysis of users' mental wellness feasible [Zhang et al., 2011; Sadilek et al., 2012; Park et al., 2012; 2013; Daine et al., 2013]. Underlying the discoveries, research efforts have also been devoted to detecting mental problems. For example, Choudhury et al. [2013] and Wang et al. [2013a; 2013b] managed to detect depressive disorders while Saleem et al. [2012] and Xue et al. [2013; 2014] focused on psychological stress detection via harvesting social media.

1 https://twitter.com/.

These initial successes demonstrate the potential of providing proactive care for users with mental problems via harvesting social media. However, the aforementioned research efforts gather datasets by questionnaires, which is reliable, but expensive and inefficient. Consequently, experiments are typically conducted on rather small datasets, the robustness and generality of whose results may not be sufficiently convincing. Besides, they mainly leverage the textual content in social media, while ignoring other important information like images and social behavior.

In this paper, we focus on the timely detection of mental wellness, with two typical mental problems, stress and depression, as specific examples. Initializing with binary userlevel detection, we expand our research by considering the trigger and level of mental problems, and involving different social media platforms of different cultures. We present our recent progress from three aspects: 1) Through self-reported sentence pattern matching, we construct a series of largescale well-labeled datasets in the field of online mental health analysis. (2) Based on previous psychological research, we extract multiple groups of discriminating features for detection and present several multi-modal models targeting at different contexts. We conduct extensive experiments and our models demonstrate significantly better performance as compared to the state-of-the-art methods . (3) We deeply investigate the feature contributions, online behaviors and even cultural distinctions in different contexts, managing to reveal the behaviors not covered in traditional psychological criteria, and provide new perspectives and insights for current and future research.

2 https://weibo.com/.

2 Mental Health Detection via Social Media

In this section, we describe our detection work on two typical mental health problems, stress and depression. We initiate each study with binary user-level detection. We expand the research towards detecting the trigger and level of mental problems with stress as an example. We also explore cultural differences by enhancing detection performance with crossdomain datasets and validating our method in depression detection.

2.1 Stress Detection

We first devote efforts to binary user-level psychological stress detection. Since mental disorders are usually continuous and chronic [Cohen et al., 1983], we study each user by gathering a period of one week of tweet data. Inspired by [Coppersmith et al., 2014], we construct several datasets out of large-scale randomly crawled tweets by self-report sentence pattern matching (i.e., labeling user as stressed / nonstressed when matching "I feel stressed / relaxed" in UGC), denoted as DB1, DB3, and DB4, with summarized statistics in Table 1. We also construct dataset DB2 based on the shared score of a psychological stress scale 3 to validate our labeling method.

We extract features at different levels of granularity to comprehensively describe each user. For tweet-level content attributes, we analyze each single tweet and extract three categories of features: 1) Linguistic features. We employ the LIWC dictionary [Pennebaker et al., 2001; Gao et al., 2013] for language psychological analysis, considering emotion words, adverbs, punctuation marks, etc. 2) Visual features. Based on previous work on affective image classification [Wang et al., 2014], we perform image processing and colorrelated attributes computation, including saturation, brightness, warm / clear color and five-color theme. 3) Social features. We study several typical social interaction behaviors, such as comments, retweets and likes. For user-level statistical attributes, we extract two categories of features from all tweets in the sampling period: 1) Posting behavior features, e.g., tweeting type and tweeting time. 2) Social interaction features, e.g., social influence and social structure distribution.

3 http://types.yuzeli.com/survey/pstr50.

To leverage both tweet-level and user-level features, we propose a hybrid detection model. For cross-media tweets with probable missing modality, we propose a Cross AutoEncoder (CAE) to learn the modality-invariant joint representation, which, compared with auto-encoder, encodes with incomplete data and reconstructs all modalities [Lin et al., 2014a]. We further apply a one-dimension convolutional neural network (CNN), with CAE units listed in the attribute maps, to aggregate the user's single tweets in the time-series and acquire user-level content attributes [Lin et al., 2014b]. The user-level attributes are subsequently combined with dynamic temporal factors and social correlation factors within a partially-labeled factor graph (PFG) for final stress detection [Lin et al., 2017]. Extensive experiments verify the effectiveness of our hybrid model, which achieves 93.40% in terms of F1-measure on dataset DB1, far better than other baselines. Results on other datasets are also desirable, indicating that our method is universally applicable.

Beyond binary detection, we work towards more wellrounded stress detection by measuring two key factors, stressor and stress level, since the trigger and level of mental problems may be more significant for successive care [Lin et al., 2016]. We devise a comprehensive scheme to automatically detect the stressor events and subjects, and therefore estimate the stressor and stress level by looking up an expertdefined stress table. Specifically, by referring to psychological researches [Holmes and Rahe, 1967; Haslam, 1994], we take into consideration 12 stressor event and 6 stressor subject categories, and define a set of seed words using LIWC [Gao et al., 2013], with which we filter one billion crawled Weibo tweets. We invite 30 volunteers to manually label the filtered data and eventually collect a dataset containing nearly 2,000 tweets with well labeled stressor events and subjects. This dataset is denoted as DB5 in Table 1.

To perform stressor and stress level detection, we further construct a stressor event and subject dictionary for each predefined category by expanding the seed words based on the word embeddings [Mikolov et al., 2013] learned over the one billion tweets. Therefore, we comprehensively describe each tweet by extracting the word distribution of stressor events and subjects, and combining them with high-level features learned by CNN. Since the stressor events and subjects are closely connected, we propose a multi-task learning (MTL) model for detection, which can adaptively capture the relatedness among tasks, and leverage the shared patterns to strengthen the performance. We further enhance the MTL model with stacked auto-encoders and propagate the gradient of the final multi-task learning layer to fine-tune the convolution layer and improve the learnt features of the CNN. With stressor event and stressor subject detected, we employ the Social Readjustment Rating Scale (SRRS) [Holmes and Rahe, 1967], a widely-accepted stress metric, to estimate the stressor and the corresponding stress level. In experiments, we evaluate the performance of stressor event and subject detection, together with stress level measurement. In all the tasks, our hybrid model significantly outperforms the compared methods, including other multi-learning models, which demonstrates that our scheme achieves promising results on measuring stress.

2.2 Depression Detection

We also work towards depression detection via harvesting social media. While depression and stress are both common mental problems, depression is much more severe, defined as a clinical disease. Following the previous scheme of stress detection, we conduct our preliminary work of binary depression detection on Twitter [Shen et al., 2017]. Several modifications are pointedly made:

• According to ICD-10 [WHO, 1992], a period of four weeks of tweet data is gathered to study each user's depression state.
• For dataset construction, we employ the strict self-report pattern "I am diagnosed depression" for ground truth labeling and find 1,402 depressed users. Users are labeled as non-depressed if no tweets containing "depress" were published in the sampling period. By loosely matching the word "depress", we also construct a large depression candidate dataset for online behavior analysis, as summarized by DB6 in Table 1.
• For feature extraction, inspired by [Park et al., 2012; Resnik et al., 2015], we additionally extract user profile features with bBridge [Farseev et al., 2016] and employ the unsupervised Latent Dirichlet Allocation (LDA) model to extract the topic-level features. To capture more depression-oriented content, we also define domainspecific features of antidepressant and depression symptoms according to [Segal, 2000].
• We propose a multi-modal depressive dictionary learning (MDL) model for detection. We learn the latent and sparse representation of users by dictionary learning, and thus extend it to multimodal, so as to model cross modality relatedness, capture the common patterns, and learn the joint sparse representation. Finally, we train a binary classifier with the learned features based on cumulative loss to detect depressed users. We conduct experiments on the labeled dataset and our MDL model achieves 85% in terms of F1measure, outperforming the baselines by +3% to +10%.

While achievements of depression detection have been made in Twitter, we have to admit that the online depression dataset is an order of magnitude smaller than the stress dataset. It is the sufficient labeled training data that paves the way for effective mental problem detection via social media. However, due to cultural differences, including the distinctive attitudes towards mental diseases and disparate online discussion environment, replicating the self-report sentence pattern labeling method to different social media platforms may encounter difficulties. Specifically, with the same pattern, dramatically fewer depressed users can be obtained in Weibo than in Twitter with the same quantity of randomly crawled tweets [Shen et al., 2018]. This leads us to a novel but challenging idea: can we utilize the multi-source datasets to improve depression detection performance for a specific platform?

We therefore systematically study the cross-domain mental health detection problem and as a particular example, we focus on depression detection with Twitter and Weibo as the source and target domain respectively [Shen et al., 2018]. We first construct the benchmark datasets. For Twitter, we employ the aforementioned labeled dataset and for Weibo, we access 580 depressed and 580 non-depressed users via selfreported sentence pattern matching out of 400 million tweets (Dataset DB7 in Table 1). In particular, we design a sophisticated Chinese regular expression that both excludes noisy content and takes into account the highly flexible Chinese expressions. The scale contrast of Twitter and Weibo dataset is consistent with the aforesaid problem of cultural differences.

Due to the intrinsic distinctions of social media platforms, features extracted in Weibo are partially different from those in Twitter. We consider 60 shared features and 18 exclusive features in the target domain (Weibo), inspired by [Piccinelli and Wilkinson, 2000; Li et al., 2015]. We investigate the distributions of features and reveal two major detection challenges regarding cross-domain feature patterns: 1) isomerism, which means that one feature may follow distinctive integral distributions in different domains. 2) divergency, which means that one feature may have distinctive, or even opposite implications on detection in different domains. These challenges are quite different from those in traditional transfer learning problems of visual and textual tasks, which can be solved by common existing approaches via mapping numerous unimodal low-level features [Weiss et al., 2016].

Leveraging the discoveries, we propose a cross-domain Deep Neural Network model with Feature Adaptive Transformation & Combination strategy (DNN-FATC) that utilizes the source domain data to enhance mental problem detection in the target domain. For the shared features, we first perform transformations of feature normalization & alignment (FNA) to fill the distributional gap of features, or in other words, to reduce isomerism. We then train a deep neural network (DNN) classifier on the transformed source domain data. To cope with divergency, we devise a divergent feature conversion (DFC) approach to recognize the divergent features and thereby conduct a targeted transformation. From this way, we take full advantage of the rich source domain data, and the model trained in the source domain can be well adapted to the target domain. We ultimately combine the exclusive features in the target domain into the deep framework with feature combination (FC) method, where we train another DNN based on the previous one to incorporate all the features. We conduct a series of experiments, by considering all the combinations of different processing approaches regarding normalization and utilization of the two domains, so that each part of our model can be validated. The results show remarkable improvement is achieved as compared to both the existing heterogeneous transfer learning methods (+3.4% to +4.8% in F1-measure) and directly training using the sparsely labeled source domain dataset (+5.2% to +14.3% in terms of F1-measure). All these results verify the importance of considering both isomerism and divergency, and corroborate that mental problem detection can be enhanced with multi-source learning, and in particular, our DNN-FATC model is desirably effective.

3 Online Behavior Analysis

Apart from experimental results of mental wellness detection models, we also conduct data analysis and case studies to reveal the online behaviors that are not covered in the traditional psychological criteria, and aim to offer more insights on how social media contribute to mental health problems.

3.1 Content and Posting Behavior

From individual user's point of view, we look into the content and posting behavior, and discover some notable characteristics of stressed or depressed users:

Emotion. Stressed and depressed users show similar emotional tendency from the linguistic perspective, expressing apparently more negative emotion words. They also use more words from the categories like death, biology, anger and anxiety, showing more concern about health issues. It is also worth mentioning that, in Weibo, depressed users express positive emotions a little more than the non-depressed, which is different from the case in Twitter. Still, this is understandable since their usage of negative emotion words is far more than average, and they may just be more emotional online.

Pronoun. We analyze the morphology distribution and discover that, depressed users tend to use more first person singulars, manifesting stronger senses of self-awareness.

Tweeting time. We study the temporal distribution of tweeting and find that, in both Twitter and Weibo, depressed users are more likely to post tweets between 23:00 and 6:00, which reflects that they may be susceptible to insomnia.

3.2 Social Interaction

Interpersonal interaction is an important component of social media. Therefore, we analyze the correlation between users' mental health states and several fundamental social concepts, so as to examine how and why a user's mental wellness state is developed and affected by other users.

Social engagement. We discover that in both Weibo and Twitter, the frequency of being retweeted is much lower for the depressed users, indicating their lack in social engagement and attention from others.

Social structure. We use the social networks of users in dataset DB1 to study the structural pattern in friends' connection. Based on the following link, the top four users with the most frequent interactions in the sampling period are selected, which may concisely produce 10 different structural combinations. We compare the proportion of different social structures of interacting users to measure the structural diversity, and see notable differences between the stressed and nonstressed users. The number of sparse connection (i.e. with no delta connections) in stressed users' social structures is 14% higher than that of non-stressed users', indicating that the social structure of stressed users tends to be less connected and less complicated.

Social influence. The principle of social influence [Kelman, 1958] suggests that users tend to change their behaviors to match those of their friends. We calculate on dataset DB1 the probability variation of one user's stress state when the user has different types of relationship with other stressed users. According to our statistical result, the chance that a user becoming stressed grows sharply with the increasing number of stressed neighbors in his / her social network . We also investigate the social influences by classifying the social relationships into strong and weak ties according to interaction frequency. We find that, strong ties have strong influence on users' stress states, while the influence of weak ties is relatively weak. These insights verify that mental wellness is mutually correlated and point towards the need for further study over the online contagion of mental health problems.

4 Conclusion

This paper offers a broad view of mental health computing via harvesting social media. We systematically studied the issues with two typical mental problems, stress and depression, as specific examples. In particular, we constructed a set of benchmark real-world datasets, defined several discriminating feature groups, and proposed a series of effective multimodal detection models. Beyond binary user-level detection, we further explored the roles of trigger and level of mental health, and made efforts towards enhancing detection performance with multi-source datasets of different cultures. We also made in-depth data analysis to reveal the underlying online behaviors beneath the mental health problems.

In the future, we plan to further improve online mental health computing by incorporating offline research techniques, to perform more accurate and fine-grained mental health measurement, and engage in timely and personalized online psychological intervention for social media users. We expect our work to offer more perspectives and insights to traditional psychological research, provide proactive care for those who suffer from mental health problems, and contribute to the well-being of the vast social media users.

Acknowledgments

The author thanks Tat-Seng Chua of National University of Singapore and Wendy Hall of University of Southampton for their valuable instructions. This work is done by the author together with her students, Tiancheng Shen, Guangyao Shen and Huijie Lin.

This work is supported by National Key Research and Development Plan (2016YFB1001200), the Innovation Method Fund of China (2016IM010200), Tiangong Institute for Intelligent Computing, Tsinghua University and the Royal Society-Newton Advanced Fellowship Award. This research is also part of the NExT research, supported by the National Research Foundation, Prime Minister's Office, Singapore under its IRC@SG Funding Initiative.