AI-powered Posture Training: Application of Machine Learning in Sitting Posture Recognition Using the LifeChair Smart Cushion

1 Introduction

The last few decades have seen a surge in health consciousness around the world [Plank and Gould, 1990]. With recent advances in medical and health sciences and the accessibility of information about healthy living, more people are aware of the importance of investing in personal well-being such as through the adoption of better dietary choices and exercising. However, the stresses and commitments associated with modern living can hinder efforts towards achieving better health and well-being on a daily basis. Particularly, the increase in desk-bound work and the use of hand-held devices such as smart-phones and tablets has exacerbated the problems of sedentary lifestyles and poor sitting posture [Jung et al., 2016]. Concurrently, advances in information and communication technologies and ubiquitous sensing have highlighted the practicality and effectiveness of collecting and mining human health-related data in real-time for the assessment and improvement of human health and well-being. Better sensing technologies and the large quantities of data they generate has called for the application of Artificial Intelligence (AI) and Machine Learning (ML) to address various problems related to health and well-being such as poor sitting posture.

∗ katia.bmn@kde.cs.tsukuba.ac.jp

A vast body of research has shown that poor sitting posture leads to a wide range of physical and mental health issues. With the rise of desk-bound work and the use of handheld devices, it is estimated than we spend more than half to eighty percent of our day sitting down [Biswas et al., 2015]. Sitting poorly leads to multiple physical health issues such as lower back pain, neck pain, headaches, respiratory and cardiovascular issues, digestive issues and an overall higher risk of disease and death [Lis et al., 2007; Veerman et al., 2012; Biswas et al., 2015; Dainese et al., 2003]. It also contributes to multiple mental health issues such as poor mood, fatigue, unproductivity, and depression [Owen et al., 2010; Matthews et al., 2012]. Sitting upright has many benefits on human health and well-being in preventing disease and injury, increasing productivity at work, and improving mood and confidence [Nair et al., 2015; Karakolis and Callaghan, 2014]. To mitigate the problems associated with poor sitting posture, various solutions have been proposed with both passive approaches (ergonomics, materials and fabrics) and active approaches (IoT and sensors). Passive solutions include ergonomic chairs, cushions, elastic bands and foot rests. Active solutions track sitting posture and include smart cushions, wearable point trackers and smartphone applications. However, passive solutions do not guarantee that users adopts a good posture, they might still slouch while using them or sit for too long unaware of their poor posture. The active solutions available today come with multiple shortcomings such as limited sensing capabilities and inadequate feedback schemes. Additionally to these solutions, to mitigate the problems associated with prolonged sitting, occupational health awareness programs often include incentives for workers to stand up, take small and frequent breaks and perform regular stretching. Frequent postural transitions and regular stretching are important aspects of good posture awareness.

Recent studies have shown that incorporating stretching exercises in the training programs of office workers is effective in preventing Musculoskeletal Disorders (MSD) in the long term and reducing pain and discomfort [Shariat et al., 2018; Van Eerd et al., 2016].

Real-time sitting posture recognition and prolonged sitting monitoring is a challenging task that requires accurate tracking of sitting posture and seated behaviour. Sitting is a dynamic task that comes with a wide range of inter-individual variability in body characteristics, differences in working environments, sitting habits and various other user-specific parameters which current active posture tracking solution have yet to address. Furthermore, the lack of a standard source of sitting posture and seated behaviour data hinders progress in research into achieving active and accurate sitting posture monitoring. Accurate posture tracking leads to effective feedback for active posture correction. Continuous posture tracking and correction covers many domains including the workplace, personal fitness, driver assistance and entertainment. In this study we apply a smart posture training cushion for the front of the chair called LifeChair [Ishac and Suzuki, 2018] and supervised machine learning (ML) in sitting posture and seated stretch recognition for improving human well-being. The LifeChair uses its own novel textile sensing technology and human biomechanics model to actively train a user's sitting posture. In this paper we designed an experimental setup to build a sitting posture and seated stretch database using the LifeChair system. Using supervised machine learning, we achieved an accuracy of 98.93% in detecting over 13 different sitting postures using an easily-deployable ML algorithm. We also showed that user Body Mass Index (BMI) is an important parameter in sitting posture recognition. We also achieved 97.99% in detecting 6 common chair bound stretches.

In Section 2, we will give an overview of the related work in machine learning and sitting posture detection. In Section 3, we will present our proposed framework for sitting posture recognition and correction with a focus on the application of a bio-mechanical posture model and machine learning. In Section 4, we will outline our experimental setup for building a database of sitting posture and seated stretch data, and the development of the ML-agent. In Section 5, we will present our results relating to the performance of various ML models, the importance of individual body mass index in sitting posture recognition and the first implementation to our knowledge of pressure sensing and ML for seated stretch pose recognition. We will conclude with some remarks on our future work and the challenges to be expected in the deployment of an AIbased posture training system.

2 RelatedWork

Previous studies have explored the application of ML algorithms in human sitting posture recognition. Zemp et al. (2016) have trained several ML classifiers on data obtained from 20 pressure sensors mounted on the chair and on the arm rests in addition to accelerometers, gyroscopes, and magnetometers attached to the rear of the backrest [Zemp et al., 2016]. They have trained several ML classifiers on manuallylabeled sensor data obtained from guided experiments to detect 7 sitting postures. Roh et al. (2018) used a low-cost load cell system made of four load cells mounted on the bottom rest of a the chair [Roh et al., 2018]. They used Support Vector Machines (SVM) with a RBF kernel to classify 6 sitting postures. A study by Ma et al. (2017) used 12 textile pressure sensors mounted on the bottom rest and backrest of a wheelchair and J48 trees to classify 5 wheelchairspecific sitting postures [Ma et al., 2017]. Zhu et al. (2003) used 42-by-48 pressure sensing and Slide Inverse Regression to classify 10 sitting postures [Zhu et al., 2003]. Previous studies have achieved fair accuracies using pressure sensing, pressure sensor imaging using PCA, Hidden Markov Models, and Naive Bayes [Pizarro et al., 2018; Tan et al., 2001; Mota and Picard, 2003]. The advantage of the LifeChair system as a platform is that it is not a fixed interface like previous works, it is portable, lightweight and can be fitted to various chairs for active posture recognition. Furthermore, the LifeChair's sensor structure is flexible and based on human biomechanics modelling so it covers all areas of the user's back including the shoulders, lumbar regions, centre of the back and bottom of the neck. While the mentioned previous studies compared various machine learning algorithms for sitting posture recognition and achieved satisfactory accuracies, they come with a range of shortcomings that hinder their application in real-life scenarios and thus their contribution to the improvement of human well-being. First, they require complex, unintuitive and experimental systems fixed on chairs and are proof of concept systems for the development of better posture tracking systems. Second, they propose alternative sensor deployment using outdated sensing technologies and off-the-shelf components which restrict accurate human posture coverage. Third, they do not perform the posture classification in real-time and use computationally intensive classification models that are not suitable for real-life implementation. Poor sitting posture is a real-world problem that requires a real-time, accurate and user-friendly solution. This ensures its continuous use and its contribution to the improvement of well-being. The LifeChair system is designed to operate in real time and provides corrective feedback to the user since it is not only a posture monitoring device but also a posture training device. As such, we optimize for posture recognition for real time use.

3 Proposed System for Active Posture Monitoring

In this study we have implemented a smart cushion called LifeChair to collect data related to sitting posture and stretch poses in real-time. The LifeChair is a smart cushion for the front of the chair that uses a novel pressure sensing technology specifically developed for human posture detection as detailed by [Ishac and Suzuki, 2018]. The LifeChair uses nine pressure sensors (1-9) distributed on a cushion interface made of leather and mesh in anthropomorphically-defined locations as shown in Figure 1.

The LifeChair also uses four vibration actuators (A-D) to provide haptic feedback for posture correction (Figure 1). The LifeChair smart cushion aims to solve the sedentary problem by actively correcting the user's posture to improve their health, mood and productivity. By using different vibration patterns, it communicates to the user spatial information, such as how to correct their posture [Ishac and Suzuki, 2018]. The type of data generated by the LifeChair system is characteristically and temporally relevant for the application of machine learning for the automation and enhancement of performance in tasks such as posture recognition, stretch pose classification, and habit-based optimization. The LifeChair uses bidirectional communication between the cushion device and a dedicated smartphone application. The data work-flow protocol of the LifeChair system is shown in Figure 2.

Before tracking sitting posture for a given user, the system first calibrates the subject's upright posture through an app-guided calibration routine. The LifeChair app then scans the calibration snapshot for approximate balance in pressure distribution from the centre-line and warns the user if an imbalanced calibration is detected. The calibration sensor data pressure readings are then stored as a reference frame to which deviation in pressure will be compared. The LifeChair system records in real-time the raw pressure sensor values at a frequency of 5Hz in addition to the user-input data, the timestamp and the posture labels. User-input data includes body characteristics (height and weight) and back-pain history. We aim to build a machine learning classifier that recognizes sitting posture and the seated stretch pose based on the LifeChair sensor data. To teach the machine learning models, different sitting postures and posture labels are generated through a threshold-based automated process based on a rigorous posture model that matches the data in real-time to posture templates based on deviation of errors from the calibration reference array. The posture model thus takes into account Vi (t = 0) which denotes the sensor pressure values at the time of calibration for a given user at each pressure sensor fi , where i is in reference to the sensor position in the LifeChair system shown in Figure 1, Vi (t) which denotes the instantaneous sensor pressure values at time t and epsilon which denotes the deviation errors as defined in Equation (3).

α is a constant strictness threshold defined in previous work which controls the strictness of the error deviation from the calibration reference [Ishac and Suzuki, 2018]. The subject is assumed to be upright if the error deviation is smaller than α at each location. If the error exceeds α, a posture is matched based on a state look-up table as detailed by [Ishac and Suzuki, 2018]. This system can detect more than 13 different postures and these postures are: Upright, Slouching Forward, Extreme Slouching Forward, Leaning Back, Extreme Leaning Back, Left Shoulder Slouch, Right Shoulder Slouch, Left Side Slouch, Right Side Slouch, Left Lumbar Slouch, Right Lumbar Slouch, Rounded Shoulders, Forward Head Posture, Slight Correction Needed and No User (i.e. no contact with the LifeChair).

The LifeChair system also records user Body Mass Index (BMI) which is used as an additional feature in training the machine learning models. BMI combines information on the user's weight and height and is defined as the ratio of the weight (kg) to the height squared (m2 ) as shown in Equation (4). Previous studies have pointed to a potential effect of user BMI on recognition performance when using pressure sensors to detect sitting posture [Ma et al., 2017; Kim et al., 2018]. However, none of these studies fully investigated the importance of BMI in posture recognition and some of them show conflicting results.

4 Experimental Setup and Data Set

An AI-based approach for sitting posture recognition in a sensing device allows for less rigid classification with the potential to capture dynamic changes in user behaviour or environment. It also allows for the establishment of a constantly learning platform that improves as the number of users of the system grows. In order to build an AI-based agent for sitting posture and stretch pose recognition, we trained multiple supervised machine learning classifiers using the data obtained from experiments conducted with the LifeChair cushion. To build the sitting posture database, 10 healthy subjects (5 males and 5 females) with an average height of 168.5 cm, and an average weight of 60.9 kg belonging to three different groups of BMI (High BMI, Normal BMI, Low BMI) were requested to sit in a standard office chair equipped with a LifeChair device and perform a set of common predefined postures. The chair used is the Plus Office Chair Be KD-MA61SL YG. A front facing camera and a 45 degree front facing camera were also used to capture video footage of the experiments for further visual crossreference. All experiment subjects were properly coached on how to use the LifeChair and a one-time calibration was performed for each subject prior to the experiment. All subjects were familiarized with each posture prior to the experiment and were asked to follow an automated slide-show of the postures with no further feedback or instructions in order to capture inter-individual variability in sitting posture. Two rounds were performed where each posture was held for 10 seconds and repeated three times with an interval of 10 seconds of upright posture between each posture. The final posture dataset thus consisted of user-input data, timestamps, raw sensor values and the posture labels (10 features, 184914 recordings). Furthermore, within the same experimental set-up, we have collected the stretch pose data of six common chair-bound stretches from the same 10 subjects. Similarly to sitting posture data collection, each user was instructed to perform 6 common stretches while sitting down on the chair and each stretch was held for 10 seconds and repeated 3 times with intervals of upright posture between each stretch. The stretch pose labels were obtained through manual labeling with video cross-referencing and were: both arms up (BAU), hanging arms down (HAD), left arm cross (LAC), right arm cross (RAC), left-leg-up and across (LLU) and right-leg-up and across (RLU). Ideally, a good machine learning model for this problem should be: (1) highly accurate and (2) computationally cheap and (3) efficiently deployable on smartphones/tablets in real time. We have applied Decision Trees (DT-CART), Random Forest (RF), K-Nearest Neighbours (KNN), Linear Regression (LR), Linear Discriminant Analysis (LDA), and Na¨ıve Bayes (NB).

To investigate the importance of user BMI in AI-based sitting posture detection, we conducted two sets of experiments with the posture dataset where we trained the machine learning classifiers either on (1) the raw sensor values only as input or (2) the raw sensor values and user BMI as an additional input. We also performed group-specific model training to investigate the divergence further.

5 Results

5.1 Sitting Posture Recognition

Table 1 shows the accuracy results of the supervised learning classifiers on the sitting posture recognition task when trained on the sensor values only and when trained on the sensors values and BMI. Two results are shown for each classifier according to the input data used; 'Sensor Only' indicates that the input features consisted of the raw sensor values only and 'Sensor + BMI' indicates that the input features consisted of the raw sensor values and user BMI. The scoring metric used for comparison is the overall accuracy as defined in Equation (5). This metric takes into account both the model's precision and recall where Tp is the true positives, Tn is the true negatives, Fp is the false positives and Fn is the false negatives.

The random forest classifier and the decision tree classifier achieved the best accuracies in both cases when using sensor data only and sensor data with individual BMI. Random Forest achieved the highest accuracy of 98.93% with the combination of sensor data and BMI as input and 97.76% with the sensor data only (Table 1). The increase in classification accuracy when using BMI as an additional feature is significant (Wilcoxon Signed Rank Test, p < 0.01) and shows that BMI can indeed be useful in capturing individual variability. This is investigated further in section 5.2. Zemp et al. (2016) achieved an accuracy of 90.9% using Random Forest in classifying seven postures [Zemp et al., 2016]. Roh et al. (2018) achieved an accuracy of 97.20% using RBF-kernel SVM in classifying six postures [Roh et al., 2018]. Zhu et al. (2003) achieved an accuracy of 86% using Slide inverse Regression in classifying ten postures [Zhu et al., 2003]. Previous studies have achieved fair accuracies ranging from 78% to 88% using PCA, Hidden Markov Models, and Naive Bayes [Pizarro et al., 2018; Tan et al., 2001; Mota and Picard, 2003]. Our results outperformed all these studies and achieved an accuracy of 98.93% in sitting posture recognition task using Random Forest. It is important to point out that these studies are optimized for the datasets obtained in their respective experiments, and a direct comparison of the algorithms' performance in classifying sitting posture is not the goal of this paper. In our system as well, our results are optimized for the postures generated by our threshold-based method and one of our goals was to find the best classification model for our sitting posture recognition task with the LifeChair device. Ma et al. (2017) achieved an accuracy of 99.51% using J48 decision trees [Ma et al., 2017]. However, they used more sensors than in our study and detected only five wheel-chair specific postures. Our system detected over 13 different postures with a high accuracy using a common, fast and robust machine learning classifier. The improvement in classification accuracy achieved in our study is likely due to the combination of the spatial deployment of the sensors in our interface and our biomechanics-based training of the ML models. Furthermore, we were able to detect two unique key postures related to the head and neck position that previous studies did not address. These are the Forward Head Posture (Slight) and Rounded Shoulders (No Shoulders) which are widely common in modern times due to the rise in extended interaction with hand-held devices.

Figure 3, shows the validation curves for the Random Forest classifier when using sensor data only and when using sensor data with BMI. The maximum accuracy was reached early at a depth of 30 trees. This result is important because of its suitability for implementation and deployment in mobile environments.

5.2 BMI Divergence

To investigate the relevance of BMI in posture recognition further, the dataset was grouped by user BMI into 3 categories: Low BMI, Normal BMI, High BMI (Table 2). Posture recognition using Random Forest was performed separately for each group and showed that the accuracy for Low BMI users was significantly lower than that of Normal BMI users and that of High BMI users (Wilcoxon Signed Rank Test, p < 0.01). In the study of Ma et al. (2017), BMI had no effect on the accuracy of the machine learning models in posture detection while Kim et al. (2018) noted a lower accuracy for smaller children in their child posture study [Ma et al., 2017; Kim et al., 2018]. In the case of Ma et al., the absence of a BMI effect might be due to the position of the sensors (bottom rest of a wheelchair) and the limited range of motion associated with wheelchair-specific postures. Height is an important parameter to consider in sitting posture. The majority of office chairs have height adjustment knobs to control the elevation of the bottom rest from the ground; this is because user height has a direct effect on the user's weight distribution on the chair and consequently on the user's posture and level of comfort. Weight is an important parameter in our system because pressure sensors are force-based. Thus, one of the ways to improve performance in sitting posture recognition tasks is to include BMI as a feature in addition to the sensor data. Another way to address this divergence in our system lies in the initial posture model itself. In our system, a potential source of this difference is the threshold α in Equation (3) which took a uniform value for all three groups during the calibration step. As α represents the strictness of the sensor error deviation from the calibration reference, the Low BMI group might require a less strict value than the Normal BMI group and the High BMI group. So, to improve the accuracy for the Low BMI group and thus overall accuracy of the system, we will combine the inclusion of BMI as an input feature for training the machine learning classifiers and the implementation of a lower α upon calibration for users with a BMI lower than 18.5 kg/m2 . This effectively allows for a user-based personalization and improvement of the system.

5.3 Stretch Pose Recognition

Figure 4 shows the heatmaps of the average pressure distribution for each of the six stretches across all subjects. A low pressure reading is represented in the lighter greens and a high pressure reading is represented in the darker blues. For example, in the first row of Figure 4, the heatmap for the stretch pose 'Right Arm Cross' (RAC) shows a high pressure reading on the upper left sensor. This captures accurately the pose performed by the experiment subjects where they extended the right arm across the chest and pressed the left arm on the right elbow. The pressure distribution heatmaps show that the six stretches are to some degree visually distinguishable from each other. Unlike sitting posture, a stretch is a static event that requires a conscious and dedicated effort from the user. Subjectivity in stretch interpretation and variable physical predisposition to perform the stretches result in greater inter-individual variability compared to the sitting posture recognition task. A stretch template approach using a similar threshold-based method to the posture recognition task in this case is not desirable. As mentioned in Section 4, we constructed the stretch pose database by manually labelling the data through video cross-referencing. We trained the same supervised learning classifiers used in the posture recognition task and predicted the stretch label by taking the sensor values and user BMI as input to capture as much variability as possible. Table 3 shows the classification results of multiple supervised learning algorithms when using the sensor values and BMI as input on the seated stretch recognition task. Random Forest again achieved the best results with an accuracy of 97.99%.

Figure 5 shows the normalized confusion matrix for the Random Forest classifier on the stretch pose recognition task. The confusion matrix indicates how many stretch poses were correctly classified against their true label. In our results, we noticed that the stretch pose where subjects had their right leg up and across (RLU) was the most prone to misclassifications. RLU was mostly misclassified as the stretch pose where subjects were leaning forward with their arms hanging down (HAD) or as the stretch pose where subjects had their left leg up and across (LLU). A possible explanation for this is related to the subjects' sitting behaviour around the lumbar region. As can be seen on the heatmaps, all subjects on average had a consistent reading on the 9th sensor positioned behind their left lumbar region. We noticed a similar pattern in a previous study that used the LifeChair system for the validation of its haptic feedback correction [Ishac and Suzuki, 2018]. In this study, the pressure distribution of the participants in a LifeChair feasibility experiment with and without the vibration feedback showed that when using the LifeChair without correction or vibration feedback, the innate posture distribution was on average imbalanced with a high reading on the left lumbar region and a low reading on the right shoulder region. This might be due to the propensity of right-hand dominant users to compensate for a lower pressure on the right shoulder with a higher pressure on the left side of the back in general and the left lumbar region in particular [Lee et al., 2018].

Figure 5: Normalized Confusion Matrix for Random Forest in the stretch pose recognition task

6 Conclusion and FutureWork

In this research we have used the LifeChair smart cushion for posture training and applied supervised learning to develop an active sitting posture recognition model. We have designed an experimental set-up for sitting posture and seated stretch data collection using the LifeChair system. By using sensor data and user-input body data (BMI), we have achieved 98.93% accuracy in detecting over 13 different sitting postures using an easily deployable supervised machine learning model suitable for mobile-based and cloud-based ML implementations. We showed that user BMI is an important feature to consider in sitting posture recognition systems based on pressure sensing and discussed strategies to improve the recognition performance for low BMI users. We also investigated stretching as an important aspect of well-being by building a seated stretch pose database and achieving a 97.99% accuracy in ML-based stretch pose recognition using Random Forest. A potential challenge in AI-based sitting posture recognition is the dynamicity of sitting posture and the vast interindividual variability between different users and their sitting behaviours. Beyond diversity in physical characteristics, variability in work environment, work schedule and sitting habits will be important points to address in deploying an AI-based solution for improving human well-being through sitting posture training. We currently address some of these points by relying on a combination of ML and a biomechanics posture model, including user BMI as an input feature for the ML models and a situation-based calibration of the device. However, we also envision to extend our experiments with the LifeChair system to include long-term performance analysis and behaviour prediction to capture a wider range of posturerelated parameters. We plan to further validate our AI-based approach in corporate and automotive scenarios by deploying multiple LifeChairs in offices and cars to track worker and driver sitting posture for multiple days using our AI-based approach and evaluate its effect on overall well-being.