Logo VU-AMS
Logo VU-AMS
Get in contact

From dyadic coping to emotional sharing and multimodal interpersonal synchrony: Protocol for a laboratory experiment

During interpersonal emotion regulation, relationship partners mutually regulate each other’s emotional states. Interpersonal emotion regulation occurs at three main timescales: phasic (from several hundred milliseconds to about 10s), tonic (from 10s to 1 hour), and chronic (from weeks to months and years). Prior research has examined interpersonal emotion regulation at only one or two timescales simultaneously. The proposed research will examine variables relating to interpersonal emotion regulation in close relationships across all three timescales. A total of 150 romantic couples will engage in an emotional sharing task, in which they will be instructed to either engage in natural sharing or co-rumination. At the phasic timescale, primary outcomes will be interpersonal synchrony in movements and cardiovascular responses throughout the sharing task. At the tonic timescale, primary outcomes will be changes in mood and emotional appraisals pre- and post-sharing. At the chronic timescale, the study will primarily assess individual differences in relationship quality and dyadic coping style prior to the task, which are expected to shape phasic and tonic patterns during emotional sharing. Our general expectation is that phasic patterns in interpersonal emotion regulation (e.g., movement synchrony) will be meaningfully related to tonic patterns (e.g., mood change), which, in turn, will be meaningfully related to chronic patterns (e.g., relationship quality). More differentiated hypotheses and exploratory analyses are detailed in the protocol. The results of this research will contribute to the integration of interpersonal emotion regulation theories across different time scales.

Machine-learning detection of stress severity expressed on a continuous scale using acoustic, verbal, visual, and physiological data: lessons learned

BackgroundEarly detection of elevated acute stress is necessary if we aim to reduce consequences associated with prolonged or recurrent stress exposure. Stress monitoring may be supported by valid and reliable machine-learning algorithms. However, investigation of algorithms detecting stress severity on a continuous scale is missing due to high demands on data quality for such analyses. Use of multimodal data, meaning data coming from multiple sources, might contribute to machine-learning stress severity detection. We aimed to detect laboratory-induced stress using multimodal data and identify challenges researchers may encounter when conducting a similar study.MethodsWe conducted a preliminary exploration of performance of a machine-learning algorithm trained on multimodal data, namely visual, acoustic, verbal, and physiological features, in its ability to detect stress severity following a partially automated online version of the Trier Social Stress Test. College students (n = 42; M age = 20.79, 69% female) completed a self-reported stress visual analogue scale at five time-points: After the initial resting period (P1), during the three stress-inducing tasks (i.e., preparation for a presentation, a presentation task, and an arithmetic task, P2-4) and after a recovery period (P5). For the whole duration of the experiment, we recorded the participants’ voice and facial expressions by a video camera and measured cardiovascular and electrodermal physiology by an ambulatory monitoring system. Then, we evaluated the performance of the algorithm in detection of stress severity using 3 combinations of visual, acoustic, verbal, and physiological data collected at each of the periods of the experiment (P1-5).ResultsParticipants reported minimal (P1, M = 21.79, SD = 17.45) to moderate stress severity (P2, M = 47.95, SD = 15.92), depending on the period at hand. We found a very weak association between the detected and observed scores (r2 = .154; p = .021). In our post-hoc analysis, we classified participants into categories of stressed and non-stressed individuals. When applying all available features (i.e., visual, acoustic, verbal, and physiological), or a combination of visual, acoustic and verbal features, performance ranged from acceptable to good, but only for the presentation task (accuracy up to.71, F1-score up to.73).ConclusionsThe complexity of input features needed for machine-learning detection of stress severity based on multimodal data requires large sample sizes with wide variability of stress reactions and inputs among participants. These are difficult to recruit for laboratory setting, due to high time and effort demands on the side of both researcher and participant. Resources needed may be decreased using automatization of experimental procedures, which may, however, lead to additional technological challenges, potentially causing other recruitment setbacks. Further investigation is necessary, with the emphasis on quality ground truth, i.e., gold standard (self-report) instruments, but also outside of laboratory experiments, mainly in general populations and mental health care patients.