Logo VU-AMS
Logo VU-AMS
Get in contact

Speech Detection via Respiratory Inductance Plethysmography, Thoracic Impedance, Accelerometers, and Gyroscopes: A Machine Learning-Informed Comparative Study

Speech production interferes with the measurement of changes in cardiac vagal activity during acute stress by attenuating the expected drop in heart rate variability. Speech also induces cardiac sympathetic changes similar to those induced by psychological stress. In the laboratory, confounding of physiological stress reactivity by speech may be controlled experimentally. In ambulatory assessments, however, detection of speech episodes would be necessary to separate the physiological effects of psychosocial stress from those of speech. Using machine learning (https://osf.io/bk9nf), we trained and tested speech classification models on data from 56 participants (ages 18–39) under controlled laboratory conditions. They were equipped with privacy-secure wearables measuring thoracoabdominal respiratory inductance plethysmography (RIP from a single and a dual-band set-up), thoracic impedance pneumography, and an upper sternum positioned unit with triaxial accelerometers and gyroscopes. Following an 80/20 train-test split, nested cross-validations were run with the machine learning algorithms XGBoost, gradient boosting, random forest, and logistic regression on the training set to get generalized performance estimates. Speech classification by the best model per method was then validated in the test set. Speech versus no-speech classification performance (AUC) for both nested cross-validation and test set predictions was excellent for thorax–abdomen RIP (nested cross-validation: 96.6%, test set prediction: 98.5%), thorax-only RIP (97.5%, 99.1%), impedance (97.0%, 97.8%), and accelerometry (99.3%, 99.6%). The sternal accelerometer method outperformed others. These open-access models leveraging biosignals have the potential to also work in daily life settings. This could enhance the trustworthiness of ambulatory psychophysiology, by enabling detection of speech and controlling for its confounding effects on physiology.

Accelerometer-based heart rate adjustment for ambulatory stress research

Using heart rate (HR) measurements to detect mental stress in naturalistic settings is hampered by the physiological impact of hemodynamic and metabolic demands. Correcting HR for these demands can help isolate fluctuations in HR associated with psychosocial stress responses, a concept termed additional heart rate (aHR). This study examined whether adding predictors for posture, activity type, and lagged movement intensity for the prolonged impact of physical activity (PA) improved aHR estimation across various manipulations of mental stress, posture, and PA in a controlled laboratory environment (n = 197). Accelerometer signals were used to obtain the movement intensity and to classify posture and activity type. Posture, activity type, and lagged movement intensity each led to a significant improvement in HR estimation, as measured by adjusted R2 and root mean squared error. However, HR was overestimated during quiet sitting. The aHR, computed as the difference between observed and predicted HR, generally underestimated observed task-baseline reactivity but was sensitive to individual differences in reactivity to mental stressors. Between-subject correlations of aHR with task-baseline reactivity ranged from 0.62 to 0.93 across conditions. On a within-subject level, the ability of aHR to differentiate between exposure to physical stress and mental stress was limited (recall = 0.32, precision = 0.31), but better than that of observed HR (recall = 0.02, precision = 0.02). Future research should explore the potential of this novel aHR estimation method in differentiating physical and mental demands on HR in daily life, and its predictive value for health outcomes.