Construct validity and responsiveness of clinical upper limb measures and sensor-based arm use within the first year after stroke: a longitudinal cohort study

Abstract

Background

Construct validity and responsiveness of upper limb outcome measures are essential to interpret motor recovery poststroke. Evaluating the associations between clinical upper limb measures and sensor-based arm use (AU) fosters a coherent understanding of motor recovery. Defining sensor-based AU metrics for intentional upper limb movements could be crucial in mitigating bias from walking-related activities. Here, we investigate the measurement properties of a comprehensive set of clinical measures and sensor-based AU metrics when gait and non-functional upper limb movements are excluded.

Methods

In this prospective, longitudinal cohort study, individuals with motor impairment were measured at days 3 ± 2 (D3), 10 ± 2 (D10), 28 ± 4 (D28), 90 ± 7 (D90), and 365 ± 14 (D365) after their first stroke. Using clinical measures, upper limb motor function (Fugl-Meyer Assessment), capacity (Action Research Arm Test, Box & Block Test), and perceived performance (14-item Motor Activity Log) were assessed. Additionally, individuals wore five movement sensors (trunk, wrists, and ankles) for three days. Thirteen AU metrics were computed based on functional movements during non-walking periods. Construct validity across clinical measures and AU metrics was determined by Spearman’s rank correlations for each time point. Criterion responsiveness was examined by correlating patient-reported Global Rating of Perceived Change (GRPC) scores and observed change in upper limb measures and AU metrics. Optimal cut-off values for minimal important change (MIC) were estimated by ROC curve analysis.

Results

Ninety-three individuals participated. At D3 and D10, correlations between clinical measures and AU metrics showed variability (range r_s: 0.44–0.90). All following time points showed moderate-to-high positive correlations between clinical measures and affected AU metrics (range r_s: 0.57–0.88). Unilateral nonaffected AU duration was negatively correlated with clinical measures (range r_s: -0.48 to -0.77). Responsiveness across outcomes was highest between D10-D28 within moderate to strong relations between GRPC and clinical measures (r_s: range 0.60–0.73), whereas relations were weaker for AU metrics (range r_s: 0.28–0.43) Eight MIC values were estimated for clinical measures and nine for AU metrics, showing moderate to good accuracy (66–87%).

Conclusions

We present reference data on the construct validity and responsiveness of clinical upper limb measures and specified sensor-based AU metrics within the first year after stroke. The MIC values can be used as a benchmark for clinical stroke rehabilitation.

Trial registration

This trial was registered on clinicaltrials.gov; registration number NCT03522519.

Background

Stroke survivors frequently suffer from hemiparesis, a condition that disrupts their functioning and autonomy [1, 2], leading to substantial challenges in personal care, meaningful activities, and relationships [2, 3]. Holistically assessing an individual’s health status, guided by the International Classification of Functioning, Disability and Health [4] (ICF), is crucial when evaluating poststroke recovery. Consensus-based recommendations guide the selection of outcomes and integration of measurements throughout the recovery continuum [5,6,7]. These upper limb assessments evaluate constructs within the ICF domains of body funtions and structures, activities, and participation [4]. When evaluating activities, two dimensions can be distinguished: capacity and performance. Capacity refers to the maximal voluntary action within a standardised task (i.e., what a person can do), while performance describes real-life movement behaviour (i.e., what a person does) [4, 6]. Due to similarities in standardised assessment conditions, we use capacity as an umbrella term for upper limb function and activity capacity measures. Performance can be evaluated from a subjective and an objective perspective. Subjective performance is assessed through patient-reported outcome measures (PROMs), which capture the individual’s perspective on functioning in their daily life that clinicians can use for goal-setting and decision-making [8]. However, various factors influence the data quality of PROMs through differences in response behaviour, depending on the patient’s background and cognitive, psychological, and social factors [9,10,11].

Wearable movement sensors offer a discreet and objective way to quantify a patient’s physical activities and arm use (AU) performance [12, 13], providing valuable information for clinicians [14]. Numerous sensor-based AU metrics have been established in stroke, measuring real-life performance in terms of intensity, duration, and AU symmetry [15,16,17,18,19]. Therefore, clinical assessments should be complemented with real-world monitoring data to determine how individuals use their upper limb capacity in daily life.

Measurement properties of clinical and sensor-based outcomes play a crucial role in interpreting the results, as they allow qualitative meaning to be assigned to quantitative results [20]. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) provide a taxonomy of measurement properties: reliability, validity, and responsiveness. Therein, validity refers to “the degree to which a measurement instrument purported the constructs to be measured [21]. Criterion validity is evaluated when comparing a measurement instrument to the construct’s gold standard, the best available measure of that domain [21]. Construct validity applies when a gold standard is unavailable, so construct validity involves investigating the dimensionality of a measure and relationships between other measurement constructs or populations of interest [21].

Research has shown strong relationships between upper limb capacity measured by the Fugl-Meyer Assessment upper extremity subscale (FMA-UE), the Action Research Arm Test (ARAT), and the Box and Block Test (BBT) [22,23,24]. However, motor capacity does not necessarily translate into performance. For example, perceived performance measured by the patient-reported Motor Activity Log (MAL) has shown low to moderate correlations with the ARAT (r = 0.23 to 0.51) in the subacute phase [25] and to the ARAT and BBT (r = 0.31 to 0.65) in the chronic phase after stroke [26,27,28].

Wearable movement sensors offer promise for understanding complex human movement behaviour but also present challenges to developing sound quantification methods. Associations between sensor-based AU performance and clinical measures greatly vary across studies [29, 30] due to factors such as different computation methods, small sample sizes, and differences in study designs [29].

Additionally, AU metrics are typically computed over the entire recording period without distinguishing between different physical activity types. This is especially concerning for walking activities, as upper limb movements during gait are primarily ballistic in nature, although there seem to be some effects of tonic cortical drive [31]. Gait, for instance, has been reported to inflate the AU duration (AU-d) of the affected upper limb by 31% in the subacute and 41% in the chronic phase after stroke [32]. Therefore, removing gait sequences is critical to maintaining the targeted outcome and avoiding including non-voluntary arm movements in the outcome calculation [32]. Specifying AU performance could enhance the understanding of relationships to clinical upper limb measures, which might facilitate the clinical integration of wearable sensors.

Quantifying change and interpreting its meaningfulness is essential in clinical stroke rehabilitation. Responsiveness is a vital measurement property defined as “the ability of a measurement instrument to reflect change over time” [21, 33]. Reference values for responsiveness in clinical upper limb measures are often derived from distribution-based methods (i.e. the significance of change magnitude) [25, 34,35,36,37]. These methods are criticised for neglecting clinical relevance [33, 38] considering that significant change does not necessarily have a meaningful impact on a person’s life.

In analogy to validity concepts, the COSMIN framework distinguishes between criterion responsiveness, which assesses the longitudinal association between observed and patient-perceived changes and construct responsiveness, which evaluates the relationship between changes in similar measurement constructs [21, 33]. Criterion responsiveness, considered a gold standard approach, enables estimating minimal important change (MIC) values, allowing clinicians to interpret whether a patient experienced meaningful change [38]. We refer to the MIC as a threshold for meaningful within-person changes over time [38]. Different terms, such as MID or MCID (minimal clinically important difference), are often used interchangeably. However, these terms address ‘differences’ and concern cross-sectional group differences rather than within-person change.

MIC values are widely used as benchmarks to evaluate the effectiveness of interventions in stroke research [39]. However, available criterion-based MIC values are often derived from high-dosed intervention trials in the chronic phase [40,41,42], which limits their applicability across different recovery phases and clinical settings. Additionally, it is unclear whether changes in AU performance reflect relevant changes from a patient’s perspective. For example, Lang et al. (2008) found no relationship between changes in affected AU duration and patient-perceived change in performance [40]; therefore, MIC values could not be estimated. Therefore, Based on this, criterion MIC values corresponding to motor recovery phases are still needed for clinical stroke rehabilitation.

This study aims to provide reference data on construct validity and responsiveness for clinical upper limb measures and sensor-based AU metrics within the first year post-stroke. We aimed to refine the AU metrics measurement construct by specifying functional AU performance and excluding walking activities. Given the extensive evidence on non-specified AU metrics, we aimed to provide reference data for two approaches: conventional AU metrics (AU_conv) and AU specified for non-walking bouts. We hypothesised that construct validity was higher within similar measurement constructs (i.e., perceived performance versus AU performance) but lower between different constructs (i.e., perceived performance versus capacity). Regarding criterion responsiveness, we hypothesised that observed changes would be at least moderately correlated with patient-perceived change.

Methods

Design

A prospective, longitudinal cohort study was conducted from September 2018 until April 2021. Individuals were screened at the acute stroke centre at the University Hospital of Zurich (Switzerland). Throughout the one-year observational period, participants received conventional movement therapy interventions delivered by physiotherapists and occupational therapists. Motor training regimes applied in Swiss rehabilitation facilities follow principles of repetitive task-specific training. Participation was possible upon signed informed consent when the following inclusion criteria were met: diagnosis of first stroke (ischemic and/or haemorrhagic) confirmed by diffusion-weighted magnetic resonance imaging or computer tomography, age ≥ 18 years, motor impairment of the upper and/or lower extremities (Motricity Index total score < 200), pre-stroke modified Rankin Scale ≤ 2, no neurological or other diagnosis affecting physical activities pre-stroke. Exclusion criteria were contraindications on ethical grounds (vulnerable individuals), known substance abuse, or incompliance. Ethical approval was granted before the start of recruitment by the cantonal ethics committee of Zurich, Switzerland (BASEC identifier: 2017 − 01070).

Data collection

Individuals were assessed at five time points, including days 3 ± 2 (D3), 10 ± 2 (D10), 28 ± 4 (D28), 90 ± 7 (D90), and 365 ± 14 (D365) after stroke onset. Assessments took place at the location of stay, which was the stroke centre of the acute hospital, on the first two time points or at rehabilitation centres, the patient’s home, or a care facility. A trained physiotherapist carried out a comprehensive battery of standardised clinical assessments. Study data were collected and managed using REDCap electronic data capture tools [43] hosted at the Lake Lucerne Institute, Vitznau, Switzerland.

At each measurement time point, individuals were asked to wear five movement sensors attached to wrists, ankles, and chest with elastic straps for three consecutive days. Real-world data were collected using ZurichMOVE sensors [44]. These research-grade devices contain 3-axial accelerometers and gyroscopes, a magnetometer, and a barometer. The built-in accelerometers and gyroscopes (MPU9250, InvenSense Inc. California, USA) are comparable to other research-grade devices and show absolute errors in raw acceleration [45]. To achieve comparability in AU measures, we applied open-source algorithms to derive uniformity for activity counts by down-sampling and filtering operations [46]. Activity outcomes previously showed consistency across various devices when raw data is processed uniformly [47, 48]. The modules were synchronised via radio frequency before recordings with a sampling frequency of 50 Hz were started.

Upper limb motor outcome

Upper limb functioning was assessed from three perspectives: the standardised clinical assessor’s perspective, rating clinical measures on upper limb function (FMA-UE) and activity capacity (ARAT, BBT); the patient’s perspective, rating upper limb performance using structured questionnaires (14-item version of the Motor Activity Log; MAL); and the sensor-based perspective measuring real-life AU performance (Fig. 1).

Fig. 1

Measurement design of this longitudinal cohort study. Abbreviations: ARAT, Action Research Arm Test; BBT, Box and Block Test; FMA-UE, Fugl-Meyer Assessment upper extremity subscale; GRPC, Global Rating of Perceived Change; MAL, Motor Activity Log

Fugl-Meyer Motor Assessment for the upper extremity (FMA-UE)

The FMA-UE [49] operates in the ICF domain of body functions and assesses upper limb motor impairment within an underlying framework of hierarchical recovery. In total, 33 items are scored from 0 to 2 points, totalling a maximal score of 66, indicating the normal motor function of the shoulder, elbow, wrist, hand, and coordination [50]. The FMA-UE is widely used and recommended by international expert panels [6, 7, 51]. It demonstrates good reliability [52, 53] and validity [36, 50, 53] in stroke patients. The original protocol was used for measurement procedures [49].

Action Research Arm Test (ARAT)

The ARAT [54] assesses the upper limb activity capacity domain by quantifying the movement quality and time required to execute unilateral repositioning of differently sized and shaped objects, which is requested to be fast and precise. The test contains 19 items divided into four subscales, including variations of grasp, grip, pinch, and gross movements. Each task is timed, and movement quality is rated on a 4-point scale (0 to 3), with a maximal score of 57 points indicating normal capacity. The ARAT is widely used and recommended for research and clinical rehabilitation [5, 7]. Good reliability [55, 56] and validity [55, 57] have been reported for stroke individuals. The ARAT was conducted according to the standardised protocol by Yozbatiran et al. (2008) [58].

Box and Block Test (BBT)

The BBT [59] assesses the activity capacity domain regarding timed repetitive unilateral object manipulation requiring dexterity. It contains 150 squared wooden blocks (2.5 cm) and two compartments separated by a wooden board. All blocks are placed into one compartment. The patient is asked to take the boxes unimanually one by one over the board into the opposite compartment within 60 s. The number of blocks is counted and compared to age-matched normative values [59]. The BBT demonstrates good reliability [34] and validity [27] in stroke patients.

Motor Activity Log 14-Item version (MAL)

The MAL [60] is a PROM that assesses the perceived performance of the paretic upper limb during daily activities. In a structured questionnaire, the patient reports retrospectively a set of 14 tasks regarding how often and how well activities were performed with the affected upper limb within the past seven days. Each task is rated on a 6-point scale (0 to 5), and the amount of use (MAL-AOU) and quality of movement (MAL-QOM) is averaged over the performed tasks. The MAL has good reliability [28, 61] and validity [61] in individuals with stroke.

Global Rating of Perceived Change (GRPC)

The GRPC is a patient-reported outcome considered the gold standard criterion to distinguish important change by considering the patient’s perspective [62]. Individuals are asked to rate changes on the desired outcome level within a defined period. In this study, the individuals were guided to think about the preceding study visit before answering the following questions: “How do you judge change regarding the ability to move your affected arm and to generate force (function) since the last time we met?“. The same question was asked regarding the activity’s domain, regarding how the ability to perform daily tasks, including reaching and grasping using the affected upper limb, had changed. The change was rated for each domain using a 7-point Likert scale (Table 1) previously used for similar purposes [40].

Table 1 Global Rating of Perceived Change

Sensor-based arm use (AU)

Physical activity can be quantified in four dimensions: frequency, intensity, time/duration, and type of physical activity [63]. AU performance was specified for the activity types of functional arm movements during non-gait activities (e.g., lying, sitting, standing), excluding ballistic arm movements during gait (see Data processing section). AU metrics were structured into two components: AU intensity (AU-i) in acceleration units and AU-d in minutes. Each component was further categorised into three conditions: (1) isolated unilateral activity of the affected or nonaffected side, occurring when only one side is active (the contralateral side is inactive); (2) bilateral activity, occuring when both sides are simultaneously active, and (3) the total activity of the affected or nonaffected side, including both, unilateral and bilateral activity. AU symmetry between the affected and nonaffected sides was evaluated by ratios (affected/nonaffected) separately for intensity and duration components.

Data processing

Sensor read-outs were split into 24-hour intervals per measurement time point, whereas partial recordings were discarded. We applied a gait detection algorithm [64] that was previously validated in stroke survivors with various impairment levels and achieved gait-detection accuracies of 93% using the full 5-sensor setup and 86% using one-sided wrists and ankle sensors. From wrist sensor data, acceleration time series were processed to Actigraph Counts [65] and combined to tri-axial vector magnitude $\:\sqrt{{\varvec{x}}^{2}+{\varvec{y}}^{2}+{\varvec{z}}^{2}}$ over 1-second epochs. AU-i metrics were expressed in units of activity counts per second corresponding to the three-dimensional acceleration of 0.00167 g/count [46].

Pre-processing was conducted twice: first, to create reference data, including novel activity classifications (non-gait functional upper limb movement), and second, to allow a comparison to conventional AU metrics (AU_conv). Functional activity epochs were obtained by applying previously validated thresholds of 20.1 activity counts to the affected and 38.6 activity counts to the nonaffected upper limb [66]. We applied these thresholds during non-gait activities [67], removing non-functional upper limb movements.

In a separate analysis, we computed AU_conv for the full data, including all activity types (i.e. not specified for walking, stair ascend/descend predictions). Conventional thresholds were applied to distinguish activity from inactivity using > 2 activity counts for total and unilateral affected/nonaffected movements [19, 68] and a threshold > 0 activity counts for bilateral activity [15]. These very low thresholds previously showed lower specificity since they include larger proportions of non-functional activity. To provide reference data to the existing body of evidence, we report full results on AU_conv in the supplements.

AU metrics were computed for validated thresholds and conventional thresholds [15, 16, 18, 19] in separate data sets. Median activity counts were calculated for all three conditions of the AU-i component, namely total AU intensity of the affected (total aff. AU-i), the nonaffected side (total nonaff. AU-i), unilateral (isolated) AU intensity for the affected (unilateral aff. AU-i), nonaffected side (unilateral nonaff. AU-i) and bilateral AU intensity (bilateral AU-i).

The symmetry of AU-i was calculated using the log-transformed metric magnitude ratio, which is calculated by the natural logarithm of the total affected divided by the total nonaffected activity counts for each second [19]. The range of magnitude ratio is confined to a minimum of -7 (no activity of the affected side but unilateral activity of the nonaffected side) and a maximum of 7 (unilateral activity of the affected side but no activity of the nonaffected side). A magnitude ratio of 0 indicates perfect symmetrical AU-i during an active epoch [19]. The variation ratio was calculated for intervals by dividing the standard deviations of the affected side by those of the nonaffected side.

The component AU-d was calculated by summing the duration of activity at supra-threshold intensity in minutes. AU-d metrics were calculated for the conditions of total AU duration of the affected (total aff. AU-d) and nonaffected side (total nonaff. AU-d), unilateral (isolated) AU duration of the affected (unilateral aff. AU-d) and nonaffected side (unilateral nonaff. AU-d); and bilateral AU duration (bilateral AU-d). The metric duration ratio reflecting the symmetry between the affected and nonaffected AU-d side was calculated by dividing the active use duration of the affected by the nonaffected side. The final set of AU metrics was averaged over recorded 24-hour intervals. Sensor data were processed using Python [69].

Data analysis

Distributions of all included parameters were investigated by Shapiro-Wilk tests and reported by medians (Mdn), first and third quartiles (Q1, Q3), if any of the compared parameters were non-normally distributed. Differences in participants’ characteristics across time points were tested using analysis of variance for age and McNemar’s χ² test for sex (male/female) and dominant side affected (yes/no). The level of significance was set to p < 0.05 (two-sided).

Construct validity was evaluated by Spearman rank correlation coefficients (r_s) calculated across upper limb measures and for each measurement time point. The following attributes for correlation coefficients were employed to interpret the strength of relationships: negligible (r_s = 0.0 to 0.29), low (r_s = 0.3 to 0.49), moderate (r_s = 0.5 to 0.69), high (r_s = 0.7 to 0.89), and very high (r_s > 0.9) correlation, for negative direction respectively [70]. We focused on the strength and direction of the relationships, excluding p-values due to their susceptibility to misinterpretation and sample size dependency [71].

To evaluate responsiveness, we calculated the change in upper limb outcome observed between two consecutive time points by subtracting the first value from the one following. Thus, positive change values indicate improvement, whereas negative values present deterioration.

Construct responsiveness [21] was evaluated by calculating Spearman’s rank correlation coefficients (r_s) of change across upper limb measures within four periods of consecutive measurements: D3-D10, D10-D28, D28-D90, and D90-D365.

Criterion responsiveness was evaluated through relationships between observed change and GRPC scores across measures. Upon significant correlations of at least r_s ≥0.3 between the measured change and the GRPC score, outcome variables qualified to determine MIC cut-off values [72] by receiver operating characteristic (ROC) curve analysis. The period D90-D365 was excluded from MIC estimation due to the long recall period, which can induce bias in the GRPC scores [73].

To estimate MIC values, GRPC scores were dichotomised to no change/nonimportant change (scores 3 to 5) and important change (6 to 7). Since it is recommended to separate important deterioration, scores 1 and 2 were excluded [38]. The ROC analysis was computed using observed change as the predictor and dichotomised classes of change importance as the response. The optimal MIC cut-off point was determined by applying equal weights on maximal sensitivity and specificity. The diagnostic performance of the cut-off values was reported by the area under the curve (AUC), correct classification rates (accuracy), true-positive rates (sensitivity), and true-negative rates (specificity).

Data analysis was conducted using RStudio software [74] with the additional software packages corrplot [75] and cutpointr [76].

Results

Ninety-three patients were enrolled (Fig. 2) and had a median age of 72 years (Q1-Q3: 60–79) and a median National Institutes of Health Stroke Scale score of 8 points (Q1-Q3: 5–11). Stroke characteristics are presented in Table 2, and participants’ demographics, place of stay, therapy dose and walking time by time point are summarised in Table 3. All individuals had one-sided motor deficits (mono- or hemiparesis), although some had bilateral lesions, which were mostly infratentorial.

Fig. 2

Patient flow chart. In the middle column, the number of participants with clinical measures (N) and the number of participants with additional sensor-based measures (N) are listed. Participants were enrolled until December 31, 2020, and data collection was halted after D90 measurements of the last enrolled patient (end of funding)

The distribution of age and proportions of sex were not statistically different across time points (p > 0.05). Compared to D3, the proportions of the dominant side affected (yes/no) differed at D28 and D365 (p < 0.05). Across all measurements, 17% of participants were monitored for 72 h, 66% were recorded for 48 h, and 17% for 24 h. Data from the complete 5-sensor setup was available for most recordings (94%), enabling walking detection and full metrics extraction. In the remaining recordings, gait was predicted using unilateral wrist and ankle sensors (3%) or only wrist sensors (3%). The duration of walking activities gradually increased from a median of 2 min (Q1-Q3: 0.1–11) at timepoint D3 to a median of 64.3 min (Q1-Q3: 26.2–90.9) at D90 and 67.3 min (Q1-Q3: 26.9 86.4) at D365 (Table 3). Distributions of clinical measures and AU metrics by time point are presented in Table 4 (AU_conv metrics, in supplement Table S1), and distributions of activity durations (AU vs. AU_conv) are presented in Figure S1.

Table 2 Stroke characteristics

Table 3 Participants’ characteristics and contextual factors

Table 4 Distribution of upper limb clinical measures and AU metrics by measurement time point

Construct validity

Correlation matrices showing relationships across clinical measures and AU metrics are presented in Fig. 3. The clinical capacity measures FMA-UE, ARAT, and BBT were highly correlated at D10 to D90 (range r_s 0.92–0.96) and slightly lower for the BBT at D365 (r_s 0.82). Capacity measures were likewise highly correlated with perceived performance (MAL) at D10 to D90 (range r_s 0.79–0.90) and particularly lower for the MAL-AOU subscale at D365 (range r_s 0.66–0.76, p < 0.001).

Fig. 3

Spearman’s rank correlations across measures of respective time points. Arm use metrics are specified for functional performance during non-gait bouts. Corralations including total nonaffected AU-i and AU-d, were consistently low (r_s <0.3) and thus removed. Abbreviations: aff., affected; ARAT, Action Research Arm Test; BBT, Box and Block Test; AU-d, arm use duration; AU-i, art use intensity; FMA-UE, Fugl-Meyer Assessment upper extremity subscale; MAL-AOU, Motor Activity Log Amount of Use; MAL-QOM, Motor Activity Log Quality of movement; nonaff. nonaffected

Relationships between clinical measures (function, capacity and perceived performance) and sensor-based AU of the affected side were most consistent with duration metrics. The total and unilateral aff. AU-d showed high positive correlations with the FMA-UE, ARAT, BBT, and MAL at the time points D3 to D90 (range r_s 0.71–0.88), but correlations were lower for AU-d vs. MAL at D365 (range r_s 0.57–0.67). High negative correlations were found between the unilateral nonaff. AU-d and capacity were most pronounced at D90 and D365 (range r_s -0.65 to -0.77).

The affected AU-i was considerably less correlated with clinical measures early after the stroke but corresponded to AU-d in subacute and chronic phases. The total and unilateral aff. AU-i were moderately correlated with the FMA-UE, ARAT, BBT, and MAL (range r_s 0.44–0.6, p < 0.001) at time points D3 and D10 but showed a moderate-to-high correlation at D28, D90, and D365 (range r_s 0.56–0.84). Negative correlations for unilateral non-affected AU-i were low only at D365 (range r_s -0.31 to -0.36), and coefficients were below 0.3 at other time points.

Bilateral AU-d was more strongly correlated with the FMA-UE, ARAT, and BBT than bilateral AU-i (range r_s 0.57–0.65) at early time points (D10 and D28). However, their correlation coefficients were comparable at D90 and D365 (range r_s 0.49–0.74). The symmetry AU metrics magnitude ratio and duration ratio were highly correlated to body functions/ capacity measures across all time points (range r_s 0.81–0.90). Distributions of measures across time points are presented in Table 4, indicating larger dispersions of AU-d than AU-i for the affected and nonaffected sides.

The construct validity of conventional AU metrics (AU_conv) and distributions by time point are presented in the supplement (Figure S2 and Table S1). Differences in the magnitude of correlation coefficients were found comparing AU metrics to AU_conv metrics (range r_s 0.01–0.32). Across time points, these differences were most prominent for total nonaff. AU-i and AU-d. At D3 and D10, correlations between total aff. AU-i_conv and clinical measures were higher (range of difference 0.14–0.18) and slightly lower for total aff. AU-d_conv (range of difference 0.01–0.05). At D90 and D365, correlations were lower for total aff. AU-d_conv, bilateral AU-i_conv, and bilateral AU-d_conv (range of difference: 0.1–0.21). Differences in correlations between AU and AU_conv metrics are presented in the supplemental Figure S3.

Responsiveness

Construct responsiveness, indicated by change correlations across clinical upper limb measures and AU metrics, is presented for each period in Fig. 4 and change distributions are shown in Table 5. The overall best responsiveness across all upper limb measures was found in the period D10-D28, whereas correlations were considerably lower in the remaining periods. The FMA-UE and ARAT score changes were consistently correlated in all investigated periods (range r_s 0.51–0.68). Change in AU-i and AU-d metrics was most consistently related to the change in FMA-UE and BBT scores, indicated by low to moderate positive correlations in D10-D28 (range r_s 0.35–0.60). Changes in the symmetry metrics magnitude ratio, variation ratio, and duration ratio were low to moderate correlated with changes in FMA-UE, ARAT, BBT, and MAL scores in all periods (range r_s 0.46–0.6). Distributions of outcome change for respective periods are presented in the supplement Table 5. Change rates and variability were higher for AU-d than for AU-i metrics.

Fig. 4

Spearman’s rank correlations of change across measures of respective period displyed in panels. Corralations including total nonaff. AU-i and AU-d, were consistently low (r _s<0.3) and thus removed supporting readability. Abbreviations: aff., affected; ARAT, Action Research Arm Test; BBt, Box and Block Test; AU-d, arm use duration; AU-i, arm use intensity; FMA-UE, Fugl-Meyer Assessment Upper Extremity subscale; MAL-AOU, Motor Activity Log Amount of Use; MAL-QOM, Motor Activity Log Quality of movement; nonaff, nonaffected

Criterion responsiveness was most present in the period D10-D28. Correlations between GRPC scores and the FMA-UE, ARAT, BBT or MAL ranged from r_s 0.6 to 0.73 (p < 0.001) and were lower for the MAL (r_s 0.6). Low to moderate relationships were found between clinical measures and GRPC in the remaining periods (range r_s 0.06–0.55).

Correlations for changes in AU were considerably lower in the period D10-D28, ranging from r_s 0.33–0.39 for affected AU and slightly higher for ratio metrics (range r_s 0.42–0.52). Responsiveness of AU_conv metrics differences in correlations (AU_conv) are presented in the supplemental Figures S4 and S5, respectively. Distributions of change in AU_conv metrics are displayed in Table S2.

Minimal important change

MIC cut-off values could be estimated for eight clinical measures and nine AU metrics. MIC values for the FMA-UE and ARAT were estimated for all periods that included the respective assessments. The distribution of change scores and the discriminative performance of MICs for clinical outcomes are presented in Table 6.

Nine MICs were estimated for AU metrics (Table 7), including seven values for AU-i, AU-d, and symmetry in period D10-D28. In D28-D90, MIC values could only be calculated for total aff. AU-i and bilateral AU-d. Across all MIC values, discriminative power was presented as acceptable to good (range AUC 0.62 to 0.88), distinguishing between important and nonimportant change with accuracies ranging from 66 to 87%. We found discrepancies between observed and perceived change, as illustrated in Fig. 5.

Fig. 5

Association between change in upper limb capacity and arm use performance. Patient-perceived revelence of change is indicated by subgroups experiencing nonimportant | important change. Estimated MIC values are displayed as dashed lines ilustrating discrepancies between observed and perceived change revelence. Abbreviations: AU-d, arm use duration in minutes per 24 hours refers to functional upper limb performance during non-gait activities; FMA-UE, Fugl-Meyer Assessment Upper Extremity subscale; MIC, minimal imporatant change

Eight MICs could be estimated for AU_conv metrics (Table S3), of which seven represent the AU-i component. The MIC value for total aff. AU-i (≥ 6.5 activity counts per second) was identical for both AU_conv and specified AU metrics. The discriminative performance of MIC for AU_conv metrics was slightly lower (range AUC 0.62–0.80), with accuracies ranging from 62 to 82%.

Table 5 Distribution of change in clinical measures and AU metrics across periods

Table 6 Distribution of change in clinical measures and diagnostic performance of MIC estimates

Table 7 Distribution of change in AU metrics and diagnostic performance of MIC estimates

Discussion

We investigated construct validity and responsiveness for clinical upper limb measures and sensor-based AU metrics in the acute, subacute, and chronic phases after stroke. These measures cover the distinct ICF domains of upper limb body functions, activity capacity, perceived AU performance, and real-life AU performance. Notably, we differentiated sensor-based activity types and specified AU performance by excluding gait activities to prevent its influence on correlations with clinical upper limb measures. Regarding construct validity, our results showed moderate to strong correlations between function, capacity measures and AU metrics, as well as between capacity measures and perceived performance in all recovery phases.

Criterion responsiveness was highest across measures within the first month poststroke. Compared to clinical upper limb measures, correlations between changes in AU metrics and perceived performance were considerably lower. Changes in intensity and duration metrics were most consistently correlated to capacity measures in period D10-D28.

MIC values were estimated for eight clinical measures and nine AU metrics within the first three months poststroke, especially in D10-D28. Beyond this period, MIC values could only be calculated for the FMA-UE and ARAT, total aff. AU-i and bilateral AU-d. Overall, the discriminative power of the MICs showed acceptable to good accuracies.

Construct validity

We investigated construct validity for the clinical measures FMA-UE, ARAT, BBT, and MAL, most frequently used in stroke rehabilitation research [77] and recommended for research and clinical stroke rehabilitation [6, 7, 51]. Within capacity measures, our results presented very strong relationships in the acute, early subacute, and chronic phases after stroke, suggesting high construct validity remained throughout the first year after stroke. Previous studies reported similar associations at single time points in the subacute [25, 55] and chronic phases [27, 34]. Although each measurement instrument assesses upper limb function or capacity differently, their underlying constructs share broad similarities, reinforcing that upper limb function is a prerequisite to carrying out activities [78, 79]. Despite these strong interrelations, the FMA-UE and ARAT should not be used interchangeably since they encompass distinct diagnostic and prognostic values [50, 80, 81]. Between capacity and perceived performance, we found predominantly strong correlations across all time points (range r_s 0.66–0.90), indicating that participants mostly recalled using their abilities during daily life. Other studies reported lower correlation coefficients between patient-perceived AU and capacity measures in the subacute (range r_s 0.43–0.52) [25] and chronic phases after stroke (range r_s 0.37–0.62) [27]. High variability of associations with perceived AU may relate to differences in individual and cultural contexts [82].

Real-world AU metrics from wearable sensors are well-suited auxiliary measures to complement patient-reported AU performance. Interestingly, correlations between perceived performance (MAL) and real-life AU performance of the affected upper limb (AU-i, AU-d) were weaker than those between perceived performance and capacity measures. This discrepancy might be due to some MAL items requiring a specific motor capacity, such as dexterity and force, but involving only minor movement amplitudes. For instance, the MAL-14 items holding a book (item 1), handwriting (item 7), stabilising (item 8), and buttoning clothes (item 14) could fall below the applied functional movement thresholds. However, discrepancies between the MAL and AU metrics were also observed in AU_conv metrics with low thresholds (Figure S2).

Discrepancies between outcome domains might be inherent to their different measurement constructs. Designated as the match-mismatch paradox, the discrepancies between capacity and perceived/real-life performance of the upper limb have been investigated extensively [11, 83,84,85].

The disparity between capacity and perceived performance or between perceived performance and AU metrics could be due to underestimating or overestimating performance by the individual. However, accurate judgment on over- or underestimation requires a valid criterion (ground truth), considering the context-dependent nature of real-world performance. Whether the current convention on the AU metrics’ calculation adequately reflects the actual AU within individual contexts remains unclear. By investigating criterion validity in a small validation sample of stroke survivors, we previously showed thresholds correctly classified functional/nonfunctional movement in 80% of cases after excluding whole-body movements [66]. This optimised thresholding increased specificity by 15% compared to AU_conv methods. However, transferability between the validation study and our longitudinal data remains uncertain, given the differences between the sample size and environmental settings (intensive care, acute hospitals, and rehabilitation clinics). In the acute hospitalisation phase (D3 and D10), we observed a less pronounced relation between AU-i metrics and upper limb capacity measures (range rs 0.44–0.60), possibly due to environmental constraints in acute care settings. Factors other than hemiparesis, such as infusion lines and cables for vital parameter measurement, can limit movement amplitude and speed.

Correlations between upper limb capacity (FMA-UE, ARAT) and total affected AU-d metrics ranging from r_s 0.66 to 0.92 across time points were considerably higher than previously reported (range r_s 0.44 to 0.66) for the subacute [86] and chronic phase [87]. Previously, Geed et al. (2023) similarly removed gait and non-functional upper limb movements and reported comparable magnitudes in the correlation between the use duration ratio and the ARAT (r = 0.82) for the chronic phase after stroke [88]. Leuenberger et al. (2017) reported significant differences by correlating total aff. AU-i with the BBT when including gait (r = 0.69) versus excluding gait (r = 0.93), which was not seen in our data (D90 r_s 0.84 vs. 0.85, respectively). Therefore, we wrongly hypothesised that specifying AU performance through additional criteria would positively influence construct validity for capacity measures. Overall, we found strong relationships with capacity for both specified AU and AU_conv data sets. Although some AU-d metrics showed higher correlations with clinical measures, differences in correlations are mostly minor between data sets (Figure S3). Nevertheless, specifying AU performance is crucial to improving the criterion validity [66], which is more important than the relationship to different measurement constructs (construct validity). Importantly, specifying AU metrics influences the magnitude and dispersion of data, which needs to be considered when comparing results between different methods.

Construct validity is often evaluated between unilateral capacity and the AU symmetry between the affected and nonaffected upper limbs [88,89,90]. Interestingly, our data shows that symmetry metrics are consistently more strongly associated with unilateral capacity than with unilaterally affected AU. This might be because daily activities are less frequently carried out with the affected arm, whereas bilateral activity makes up for the largest proportion of AU patterns [91, 92]. Compensatory AU strategies were represented by inverse associations in our data between impairment and unilateral nonaff. AU-d in the late subacute and chronic phase, which has also been reported in previous investigations [92, 93].

Clinical implications for construct validity

A large proportion of variance (> 50%) is explained by capacity and the duration of real-life AU (r_s >0.7) across recovery phases. However, a remaining proportion remains unexplained between these different measurement constructs, which justifies a holistic assessment of upper limb functioning. Optimally, each measurement construct should be assessed individually to draw conclusions regarding transfer of functional abilities into real-life activities. Lower correlations between capacity and AU performance reveal discrepancies that require further investigation on individual or subgroup levels. Counter-intuitively, lower correlations suggest added value through the addition of unique information. For example, the MAL reflects limitations in perceived performance for specific daily tasks, while AU performance indirectly reflects limitations. In this regard, metrics quantifying AU patterns can reveal compensatory behaviour and add information not captured by one-sided capacity measures. This differentiation of AU patterns should be considered when implementing AU performance monitoring in clinical stroke rehabilitation.

Responsiveness

Using perceived change as a criterion, we observed strong relationships between upper limb capacity measures in the period D10-D28. These relationships were moderate within the first days poststroke and at later periods. Previously, a linear relationship between the GRPC scores, the FMA-UE [94], and the ARAT [40] was only reported for the first few weeks poststroke. Our findings extend this evidence by detailing the change relationships across multiple upper limb measures for poststroke phases.

Responsiveness can be negatively influenced by floor and ceiling effects [95], which are particularly reported in the FMA-UE and ARAT [96,97,98]. At D3, we observed floor effects for the FMA-UE (17%), but no ceiling effects were identified. For the ARAT, floor effects decreased from 36% at D10 to 17% at D90, while ceiling effects increased from 12% at D10 to 30% at D90. Individuals with high scores who perceived meaningful change in daily activities might have influenced the magnitudes of MIC values. By not excluding these incidents, the reported MIC values remain applicable to moderate to mild upper limb impairments.

Responsiveness was moderate for the MAL and low for AU metrics at periods D10-D28, whereas at D28-D90, only changes in total aff. and bilateral AU-i showed a weak relationship with perceived changes (GRPC scores). Although we found linear relationships between perceived change and observed changes in sensor-based AU, these correlations were highly time-dependent. The weaker associations in the early weeks after stroke illustrate that changes in upper limb capacity translate to AU performance to some extent. Inpatient rehabilitation, which applied to 80% of participants in this period, might have influenced these associations through physical exercise and behavioural education. Future research should explore potential modulators affecting the responsiveness of sensor-based AU performance. Our results indicate that the responsiveness of clinical measures and AU metrics is highly period-specific poststroke.

Minimally important change

We present eight MIC values for clinical upper limb outcome measures and nine MIC values for sensor-based AU metrics, applying to conventional stroke rehabilitation within the first three months after stroke. The outcome measures’ criteria responsiveness is a prerequisite for estimating MIC cut-off values that accurately distinguish between important and nonimportant change.

Clinical upper limb measures

Considering criterion responsiveness, MICs for the FMA-UE were estimated for all three periods investigated. These MIC values ranged between 4 and 7 points (i.e., 6.1% and 10.1% of the scale, respectively) within the first three months after stroke, illustrating that relevant change varies over time. By gradually expanding the duration between measurement time points, our study design aligned with the motor recovery curve, in which most improvements occur in the first weeks after stroke, tapering off in the following weeks. This recovery trajectory was found to be similar in upper limb capacity [80, 99] and AU performance [100]. Despite period D10-D28 spanning half the duration of period D28-D90, change proportions remained relatively constant for the FMA-UE, ARAT and total affected AU-d (Table 5).

We did not evaluate criterion responsiveness for the period D90-D365 to prevent GRPC ratings from inflating responses when recalling the long-ago health status. Recall bias and response shift are two distinct phenomena that are known to cause over- or estimation of stroke survivors’ responses [101, 102]. Therefore, our evidence for D90-D365 is limited to construct responsiveness (Fig. 4) and distribution-based properties (Table 5), showing a positive change in capacity outcome, whereas AU performance declined.

Criterion-based MIC values should be interpreted probabilistically using their classification performance, sample characteristics and period durations. In our sample of mild-to-moderate motor impairment in D10-D28, an improvement of at least 4 points in the FMA-UE correctly indicated a meaningful change in 75% of individuals. Factors such as the period duration, baseline severity, intrinsic patient’s expectation of recovery, intervention type/intensity, and type of criterion measure can influence MIC values and should be considered for interpretation [10, 103,104,105]. For instance, Lundquist et al. (2017) reported an accurate MIC of ≥ 4 points in the FMA-UE with comparable accuracy concerning a 3-week conventional rehabilitation in patients with mild-to-moderate impairment [106]. In line with our sample, individuals with cognitive or mental impairment were included in the latter study. In contrast, considerably higher MICs were reported in studies with different sample characteristics and study designs: A MIC ≥ 9 points was estimated within four weeks of intensive repetitive task training [41] and a MIC within eight weeks of conventional rehabilitation (MIC ≥ 13 points) [94]. These higher MICs were estimated for samples excluding cognitive impairments but featuring severe motor impairment that exhibited larger change rates. The interpretation of MIC values depends on whether their magnitude ranges above or below MDC values. The MIC ≥ 7 points exceeds the MDC of 5.2 points (7,8%) determined in a small sample. Although the FMA-UE shows excellent interrater reliability [52], robust evidence on MDC values is needed to interpret MIC values.

We provide criterion-based MICs for the ARAT, BBT, and MAL for conventional rehabilitation settings. These cut-off values show high accuracy for the BBT, ARAT, and MAL within the period D10-D28, ranging from 81 to 87%. In our sample, an increase of ≥ 5 points in the ARAT score benchmarks an important change in the first weeks poststroke. This relatively small MIC exceeds the MDC estimated at 2.3 points in the early subacute recovery phase [56].

Sensor-based AU-metrics

We provide MIC values for sensor-based AU metrics for clinical stroke rehabilitation. Seven MIC values applying to the first month poststroke cover intensity, duration, and symmetry of AU. In this period, a change in total aff. AU-d of ≥ 32 min correctly classified meaningful change in 59% but also correctly classified nonimportant change below this cut-off in 74% of our sample (see Table 7). Noteworthy, total aff. AU-d includes both unilateral and bilateral functional movements. Accordingly, change rates and the MIC for unilateral aff. AU-d were lower.

The negative MIC value for unilateral affected AU-i must be treated cautiously. This negative value is due to balanced weights on sensitivity and specificity, reflecting incidents where negative change values aligned with a perceived meaningful change. The MIC estimate for unilateral aff. AU-i_conv also showed a low magnitude (≥ 1 activity count) and lower discriminative performance (67% accuracy). In the D28-D90 period, low or negative change rates in AU performance could be linked to most of our cohort being discharged from the rehabilitation clinic (69% of cases). Rehabilitation discharge potentially perturbs behaviour and increases the risk for learned non-use, as movement therapy and self-reflection of movement behaviour are reduced.

We estimated MICs based on recommended minimal correlation coefficients of ≥ 0.3 between perceived and objective change [72]. However, the change values between the groups perceiving and not perceiving change in a certain period were variable and partly overlapped.

The weak relationship between perceived change and change in AU illustrates that methods evaluating meaningful change in real-life performance need to be refined. We revealed discrepancies between change magnitude in capacity, AU-d and important change from a patient’s perspective across recovery phases (Fig. 5). Notably, these discrepancies appeared more prominent in AU performance. Criterion responsiveness in the period D10-D28 indicates that the variance in GRPC scores is explained by variances in AU change by small percentages (10-27% corresponding to squared r_s). This might be related to the GRPC scale’s limited ability to reflect functional change. Despite the GRPC scale’s validity in the domains of pain, quality of life, and disability [107], it has been criticised for lacking face validity [108] and longitudinal instability [109] when used in functional outcomes. Future research should determine whether GRPC’s face validity improves by specifying the scale for essential functional tasks. Associating change with meaningful daily tasks like drinking or eating could improve face validity and lead to robust MIC estimates.

Clinical implications for responsiveness and MICs

High construct validity does not necessarily translate to good responsiveness. Relationships of change apply differently to the distinct measurement constructs of upper limb capacity, perceived performance, and real-life performance. Our MIC values can be used to interpret the meaningfulness of change in the respective recovery phase. Clinicians should ensure that the patient’s characteristics and recovery stage match the sample characteristics when selecting MICs from literature or our work. An important change in an FMA-UE score does not necessarily translate to an important change in AU performance (Fig. 5), so functional status and limitations should be assessed individually. Discrepancies between the magnitude of observed change and its perceived relevance are frequent [110] and should be further determined at the individual and subgroup levels.

Sensor-based AU metrics can provide valuable feedback to patients and therapists [111, 112]. for effectively tailoring behavioural techniques to individual needs [113]. Particularly, the MICs for AU symmetry demonstrate balanced classification rates, which serve as a valuable benchmark for evaluation and goal-setting in stroke rehabilitation.

Limitations

The following limitations should be considered when interpreting our findings.

First, sample sizes varied across time points, particularly at D28 and D365. Missing data could have influenced correlation coefficients at these time points. We evaluated differences in distributions of patients’ characteristics by complete or partially missing data. Amongst characteristics, we only found differences in the proportions of the dominant affected side at D28 and D365, which could influence AU behaviour.

Secondly, AU metrics derived from wrist sensors reflect central tendencies of end-effector accelerations but are not sensitive to more distal activities such as hand dexterity. Fine motor tasks might evoke only low acceleration magnitudes falling below functional thresholds. Wearable devices that enable quantification of hand dexterity in clinical and home environments are needed to specify upper limb performance further. In addition, a general limitation of sensor-based metrics is that it remains uncertain if movements (even above thresholds) were actually purposeful or completed successfully. Criterion-based validation of movement classifiers in larger stroke samples and various environments is needed to enhance the transferability of accurately detecting purposeful movements in real-world environments.

Thirdly, we did not evaluate the proportional effects of applied activity classification methods on AU metrics. Aiming to provide reference data on construct validity and responsiveness, we used data sets including functional movements excluding gait and a data set including conventional thresholds for activity and including gait. Future research should evaluate the partial effect of functional movement, and gait exclusion versus non-specified data.

Fourthly, we omitted p-values to avoid misinterpretation due to sample size dependency, excluding a direct measure of uncertainty. Although confidence intervals could offer this, they are difficult to represent clearly in large correlation matrices.

Lastly, the absence of a test-retest scenario in stable patient conditions in our study design underscores the necessity for future research. Standardised measurement errors could not be estimated within our sample, making it crucial for future studies to determine minimal detectable change values. This is particularly important when interpreting MIC values, i.e., whether MICs are below or above measurement error.

Conclusions

This study lays important groundwork for understanding the interrelations between clinical measures and AU performance after stroke. Clinical measures of motor capacity retain strong relationships across recovery phases. Patient-reported and sensor-based AU performance cover distinct measurement constructs, providing unique information for researchers and clinicians. Low associations between clinical measures and AU metrics early after a stroke could be related to the acute hospital setting. At later time points, strength in correlations increases for AU intensity, duration, and symmetry. Quantified AU patterns, such as unilateral and bilateral movements, can reveal behavioural adaptations, such as compensational strategies. The established measurement properties of a comprehensive set of clinical measures and AU metrics enable the development and testing of further hypotheses on recovery in individuals and groups of patients.

Criterion responsiveness was especially good within the first weeks after a stroke. Changes in clinical scores are more strongly related to perceived changes than changes in sensor-based AU. Estimated MIC values apply to the clinical rehabilitation pathway but should be used considering their probabilistic properties. We provide the first MIC values on both specified AU metrics, which retain only functional and non-gait movements, and conventional AU metrics. Estimated MICs for AU symmetry overlap for specified and non-specified AU classifications and can, therefore, be suggested as a benchmark to evaluate change in AU performance. These measurement properties for sensor-based AU are an essential step toward regular activity monitoring within the stroke care continuum.