We briefly review the history of measuring perceptions of health and quality of life, followed by an examination of conceptual issues related to terminology that have led to potentially conflicting ontologies. Then, we discuss challenges posed by the lack of consensus on common meaning and the proliferation of measures. Next, we suggest a solution grounded in an ontology adopted by the National Institutes of Health (NIH) funded Patient-Reported Outcomes Measurement Information System (PROMIS) project. We conclude by discussing issues associated with mapping the PROMIS domain framework onto other familiar ontologies and recommend a way forward for PROMIS to provide a sustainable ontological structure to enable coherent common measurement.

Background

To guide measure development, National Institutes of Health-supported Patient reported Outcomes Measurement Information System (PROMIS) investigators developed a hierarchical domain framework. The framework specifies health domains at multiple levels. The initial PROMIS domain framework specified that physical function and symptoms such as Pain and Fatigue indicate Physical Health (PH); Depression, Anxiety, and Anger indicate Mental Health (MH); and Social Role Performance and Social Satisfaction indicate Social Health (SH). We used confirmatory factor analyses to evaluate the fit of the hypothesized framework to data collected from a large sample.

Methods

We used data (n=14,098) from PROMIS's wave 1 field test and estimated domain scores using the PROMIS item response theory parameters. We then used confirmatory factor analyses to test whether the domains corresponded to the PROMIS domain framework as expected.

Results

A model corresponding to the domain framework did not provide ideal fit [root mean square error of approximation (RMSEA)=0.13; comparative fit index (CFI)=0.92; Tucker Lewis Index (TLI)=0.88; standardized root mean square residual (SRMR)=0.09]. On the basis of modification indices and exploratory factor analyses, we allowed Fatigue to load on both PH and MH. This model fit the data acceptably (RMSEA=0.08; CFI=0.97; TLI=0.96; SRMR=0.03).

Discussion

Our findings generally support the PROMIS domain framework. Allowing Fatigue to load on both PH and MH improved fit considerably.

Purpose

Individual change on a patient-reported outcome (PRO) measure can be assessed by statistical significance and meaningfulness to patients. We explored the relationship between these two criteria by varying the confidence levels of the coefficient of repeatability (CR) on the Patient-Reported Outcomes Measurement Information System (R) Physical Function (PF) 10a (PF10a) measure.

Methods

In a sample of 1129 adult cancer patients, we estimated individual-change thresholds on the PF10a from baseline to 6 weeks later with the CR at 50%, 68%, and 95% confidence. We also assessed agreement with group- and individual-level thresholds from anchor-based methods [mean change and receiver operating characteristic (ROC) curve] using a PF-specific patient global impression of change (PGIC).

Results

CRs at 50%, 68%, and 95% confidence were 3, 4, and 7 raw score points, respectively. The ROC- and mean-change-based thresholds for deterioration were -4 and -6; for improvement they were both 2. Kappas for agreement between anchor-based thresholds and CRs for deterioration ranged between κ = 0.65 and 1.00, while for improvement, they ranged between 0.35 and 0.83. Agreement between the PGIC and all CRs always fell below "good" (κ < 0.40) for deterioration (0.30-0.33) and were lower for improvement (0.16-0.28).

Conclusions

In comparison to the CR at 95% confidence, CRs at 50% and 68% confidence (considered likely change indexes) have the advantage of maximizing the proportion of patients appropriately classified as changed according to statistical significance and meaningfulness.

Background

Despite the increasing use of panel surveys, little is known about the differences in data quality across panels.

Objective

The aim of this study was to characterize panel survey companies and their respondents based on (1) the timeliness of response by panelists, (2) the reliability of the demographic information they self-report, and (3) the generalizability of the characteristics of panelists to the US general population. A secondary objective was to highlight several issues to consider when selecting a panel vendor.

Methods

We recruited a sample of US adults from 7 panel vendors using identical quotas and online surveys. All vendors met prespecified inclusion criteria. Panels were compared on the basis of how long the respondents took to complete the survey from time of initial invitation. To validate respondent identity, this study examined the proportion of consented respondents who failed to meet the technical criteria, failed to complete the screener questions, and provided discordant responses. Finally, characteristics of the respondents were compared to US census data and to the characteristics of other panels.

Results

Across the 7 panel vendors, 2% to 9% of panelists responded within 2 days of invitation; however, approximately 20% of the respondents failed the screener, largely because of the discordance between self-reported birth date and the birth date in panel entry data. Although geographic characteristics largely agreed with US Census estimates, each sample underrepresented adults who did not graduate from high school and/or had annual incomes less than US $15,000. Except for 1 vendor, panel vendor samples overlapped one another by approximately 20% (ie, 1 in 5 respondents participated through 2 or more panel vendors).

Conclusions

The results of this head-to-head comparison provide potential benchmarks in panel quality. The issues to consider when selecting panel vendors include responsiveness, failure to maintain sociodemographic diversity and validated data, and potential overlap between panels.

Patient reported outcomes provide the patient's perspective on the effectiveness of treatment. The draft Food and Drug Administration guidance on patient reported outcomes for labeling and promotional claims raises a number of method and measurement issues that require further clarification, including methods of determining responsiveness and minimal important differences. For clinical trials, instruments need to be based on a clear conceptual framework, have evidence supporting content validity and acceptable psychometric qualities. The measures must also have evidence documenting responsiveness and interpretation guidelines (i.e., minimal important difference) to be most useful as effectiveness endpoints in clinical trials. The recommended approach is to estimate the minimal important difference based on several anchor-based methods, with relevant clinical or patient-based indicators, and to examine various distribution-based estimates (i.e., effect size, standardized response mean, standard error of measurement) as supportive information, and then to triangulate on a single value or small range of values for the MID. Confidence in a specific MID value evolves over time and is confirmed by additional research evidence, including clinical trial experience. The MID may vary by population and context, and no one MID will be valid for all study applications involving a PRO instrument. Responsiveness and MID must be demonstrated and documented for the particular study population, and these measurement characteristics are needed for PRO labeling and promotional claims.

Background

The United States has a culturally and demographically diverse populace, and the aim of this study was to examine differences in health preferences by sex, age, ethnicity, and race.

Methods

We assessed preferences for health outcomes defined by the PROMIS-29 survey in a sample of the US population. On the basis of the survey's 540 paired-comparisons trading off lifespan and 7 domains of health-related quality of life (HRQoL), we compared the choices between men and women, adults age 18-54 years and 55 years and older, Hispanics and non-Hispanics, and non-Hispanic blacks and whites. For each subgroup, we estimated the value of 122 HRQoL outcomes on a quality-adjusted life year scale and tested for subgroup differences.

Results

Compared with men, women preferred reduced lifespan over losses in HRQoL, particularly for depression. Compared with the younger adults, older adults preferred reduced lifespan over the symptoms of depression, anxiety, and fatigue. Compared with non-Hispanic whites, Hispanics preferred reduced lifespan over depression and sleep disturbance, but held similar values on losses in physical functioning. Among non-Hispanics, blacks preferred reduced lifespan over losses in ability to climb stairs and to fall asleep compared with whites, but held similar values on mental health outcomes.

Conclusions

With the growing emphasis on patient-centeredness and culturally sensitive treatment, it is important to recognize the diversity in values placed on potential losses in HRQoL, particularly mental health outcomes. Demographic differences in preferences may influence comparative or cost effectiveness of treatments as perceived by one or another subgroup.

Background

Preference-based health-related quality of life (HR-QOL) scores are useful as outcome measures in clinical studies, for monitoring the health of populations, and for estimating quality-adjusted life-years.

Methods

This was a secondary analysis of data collected in an internet survey as part of the Patient-Reported Outcomes Measurement Information System (PROMIS(®)) project. To estimate Health Utilities Index Mark 3 (HUI-3) preference scores, we used the ten PROMIS(®) global health items, the PROMIS-29 V2.0 single pain intensity item and seven multi-item scales (physical functioning, fatigue, pain interference, depressive symptoms, anxiety, ability to participate in social roles and activities, sleep disturbance), and the PROMIS-29 V2.0 items. Linear regression analyses were used to identify significant predictors, followed by simple linear equating to avoid regression to the mean.

Results

The regression models explained 48 % (global health items), 61 % (PROMIS-29 V2.0 scales), and 64 % (PROMIS-29 V2.0 items) of the variance in the HUI-3 preference score. Linear equated scores were similar to observed scores, although differences tended to be larger for older study participants.

Conclusions

HUI-3 preference scores can be estimated from the PROMIS(®) global health items or PROMIS-29 V2.0. The estimated HUI-3 scores from the PROMIS(®) health measures can be used for economic applications and as a measure of overall HR-QOL in research.

Purpose

The PROMIS-29 v2.0 profile assesses pain intensity using a single 0-10 numeric rating item and seven health domains (physical function, fatigue, pain interference, depressive symptoms, anxiety, ability to participate in social roles and activities, and sleep disturbance) using four items per domain. This paper describes the development of physical and mental health summary scores for the PROMIS-29 v2.0.

Method

We conducted factor analyses of PROMIS-29 scales on data collected from two internet panels (n = 3000 and 2000).

Results

Confirmatory factor analyses provided support for a physical health factor defined by physical function, pain (interference and intensity), and ability to participate in social roles and activities, and a mental health factor defined primarily by emotional distress (anxiety and depressive symptoms). Reliabilities for these two summary scores were 0.98 (physical health) and 0.97 (mental health). Correlations of the PROMIS-29 v2.0 physical and mental health summary scores with chronic conditions and other health-related quality of life measures were consistent with a priori hypotheses.

Conclusions

This study develops and provides preliminary evidence supporting the reliability and validity of PROMIS-29 v2.0 physical and mental health summary scores that can be used in future studies to assess impacts of health care interventions and track changes in health over time. Further evaluation of these and alternative summary measures is recommended.

The use of global health items permits an efficient way of gathering general perceptions of health. These items provide useful summary information about health and are predictive of health care utilization and subsequent mortality. Analyses of 10 self-reported global health items obtained from an internet survey as part of the Patient-Reported Outcome Measurement Information System (PROMIS) project. We derived summary scores from the global health items. We estimated the associations of the summary scores with the EQ-5D index score and the PROMIS physical function, pain, fatigue, emotional distress, and social health domain scores. Exploratory and confirmatory factor analyses supported a two-factor model. Global physical health (GPH; 4 items on overall physical health, physical function, pain, and fatigue) and global mental health (GMH; 4 items on quality of life, mental health, satisfaction with social activities, and emotional problems) scales were created. The scales had internal consistency reliability coefficients of 0.81 and 0.86, respectively. GPH correlated more strongly with the EQ-5D than did GMH (r = 0.76 vs. 0.59). GPH correlated most strongly with pain impact (r = −0.75) whereas GMH correlated most strongly with depressive symptoms (r = −0.71). Two dimensions representing physical and mental health underlie the global health items in PROMIS. These global health scales can be used to efficiently summarize physical and mental health in patient-reported outcome studies.