In 1986 the International Journal of Epidemiology published "Identifiability, Exchangeability and Epidemiological Confounding". We review the article from the perspective of a quarter century after it was first drafted and relate it to subsequent developments on confounding, ignorability, and collapsibility.

Personalized medicine asks if a new treatment will help a particular patient, rather than if it improves the average response in a population. Without a causal model to distinguish these questions, interpretational mistakes arise. These mistakes are seen in an article by Demidenko [2016] that recommends the "D-value," which is the probability that a randomly chosen person from the new-treatment group has a higher value for the outcome than a randomly chosen person from the control-treatment group. The abstract states "The D-value has a clear interpretation as the proportion of patients who get worse after the treatment" with similar assertions appearing later. We show these statements are incorrect because they require assumptions about the potential outcomes which are neither testable in randomized experiments nor plausible in general. The D-value will not equal the proportion of patients who get worse after treatment if (as expected) those outcomes are correlated. Independence of potential outcomes is unrealistic and eliminates any personalized treatment effects; with dependence, the D-value can even imply treatment is better than control even though most patients are harmed by the treatment. Thus, D-values are misleading for personalized medicine. To prevent misunderstandings, we advise incorporating causal models into basic statistics education.

Objective

To illustrate an approach to compare CD4 cell count and HIV-RNA monitoring strategies in HIV-positive individuals on antiretroviral therapy (ART).

Design

Prospective studies of HIV-positive individuals in Europe and the USA in the HIV-CAUSAL Collaboration and The Center for AIDS Research Network of Integrated Clinical Systems.

Methods

Antiretroviral-naive individuals who initiated ART and became virologically suppressed within 12 months were followed from the date of suppression. We compared 3 CD4 cell count and HIV-RNA monitoring strategies: once every (1) 3 ± 1 months, (2) 6 ± 1 months, and (3) 9-12 ± 1 months. We used inverse-probability weighted models to compare these strategies with respect to clinical, immunologic, and virologic outcomes.

Results

In 39,029 eligible individuals, there were 265 deaths and 690 AIDS-defining illnesses or deaths. Compared with the 3-month strategy, the mortality hazard ratios (95% CIs) were 0.86 (0.42 to 1.78) for the 6 months and 0.82 (0.46 to 1.47) for the 9-12 month strategy. The respective 18-month risk ratios (95% CIs) of virologic failure (RNA >200) were 0.74 (0.46 to 1.19) and 2.35 (1.56 to 3.54) and 18-month mean CD4 differences (95% CIs) were -5.3 (-18.6 to 7.9) and -31.7 (-52.0 to -11.3). The estimates for the 2-year risk of AIDS-defining illness or death were similar across strategies.

Conclusions

Our findings suggest that monitoring frequency of virologically suppressed individuals can be decreased from every 3 months to every 6, 9, or 12 months with respect to clinical outcomes. Because effects of different monitoring strategies could take years to materialize, longer follow-up is needed to fully evaluate this question.

Decisions about when to start or switch a therapy often depend on the frequency with which individuals are monitored or tested. For example, the optimal time to switch antiretroviral therapy depends on the frequency with which HIV-positive individuals have HIV RNA measured. This paper describes an approach to use observational data for the comparison of joint monitoring and treatment strategies and applies the method to a clinically relevant question in HIV research: when can monitoring frequency be decreased and when should individuals switch from a first-line treatment regimen to a new regimen? We outline the target trial that would compare the dynamic strategies of interest and then describe how to emulate it using data from HIV-positive individuals included in the HIV-CAUSAL Collaboration and the Centers for AIDS Research Network of Integrated Clinical Systems. When, as in our example, few individuals follow the dynamic strategies of interest over long periods of follow-up, we describe how to leverage an additional assumption: no direct effect of monitoring on the outcome of interest. We compare our results with and without the "no direct effect" assumption. We found little differences on survival and AIDS-free survival between strategies where monitoring frequency was decreased at a CD4 threshold of 350 cells/μl compared with 500 cells/μl and where treatment was switched at an HIV-RNA threshold of 1000 copies/ml compared with 200 copies/ml. The "no direct effect" assumption resulted in efficiency improvements for the risk difference estimates ranging from an 7- to 53-fold increase in the effective sample size.

Background

Clinical guidelines vary with respect to the optimal monitoring frequency of HIV-positive individuals. We compared dynamic monitoring strategies based on time-varying CD4 cell counts in virologically suppressed HIV-positive individuals.

Methods

In this observational study, we used data from prospective studies of HIV-positive individuals in Europe (France, Greece, the Netherlands, Spain, Switzerland, and the UK) and North and South America (Brazil, Canada, and the USA) in The HIV-CAUSAL Collaboration and The Centers for AIDS Research Network of Integrated Clinical Systems. We compared three monitoring strategies that differ in the threshold used to measure CD4 cell count and HIV RNA viral load every 3-6 months (when below the threshold) or every 9-12 months (when above the threshold). The strategies were defined by the threshold CD4 counts of 200 cells per μL, 350 cells per μL, and 500 cells per μL. Using inverse probability weighting to adjust for baseline and time-varying confounders, we estimated hazard ratios (HRs) of death and of AIDS-defining illness or death, risk ratios of virological failure, and mean differences in CD4 cell count.

Findings

47 635 individuals initiated an antiretroviral therapy regimen between Jan 1, 2000, and Jan 9, 2015, and met the eligibility criteria for inclusion in our study. During follow-up, CD4 cell count was measured on average every 4·0 months and viral load every 3·8 months. 464 individuals died (107 in threshold 200 strategy, 157 in threshold 350, and 200 in threshold 500) and 1091 had AIDS-defining illnesses or died (267 in threshold 200 strategy, 365 in threshold 350, and 459 in threshold 500). Compared with threshold 500, the mortality HR was 1·05 (95% CI 0·86-1·29) for threshold 200 and 1·02 (0·91·1·14) for threshold 350. Corresponding estimates for death or AIDS-defining illness were 1·08 (0·95-1·22) for threshold 200 and 1·03 (0·96-1·12) for threshold 350. Compared with threshold 500, the 24 month risk ratios of virological failure (viral load more than 200 copies per mL) were 2·01 (1·17-3·43) for threshold 200 and 1·24 (0·89-1·73) for threshold 350, and 24 month mean CD4 cell count differences were 0·4 (-25·5 to 26·3) cells per μL for threshold 200 and -3·5 (-16·0 to 8·9) cells per μL for threshold 350.

Interpretation

Decreasing monitoring to annually when CD4 count is higher than 200 cells per μL compared with higher than 500 cells per μL does not worsen the short-term clinical and immunological outcomes of virally suppressed HIV-positive individuals. However, more frequent virological monitoring might be necessary to reduce the risk of virological failure. Further follow-up studies are needed to establish the long-term safety of these strategies.

Funding

National Institutes of Health.

Background

When a clinical treatment fails or shows suboptimal results, the question of when to switch to another treatment arises. Treatment switching strategies are often dynamic because the time of switching depends on the evolution of an individual's time-varying covariates. Dynamic strategies can be directly compared in randomized trials. For example, HIV-infected individuals receiving antiretroviral therapy could be randomized to switching therapy within 90 days of HIV-1 RNA crossing above a threshold of either 400 copies/ml (tight-control strategy) or 1000 copies/ml (loose-control strategy).

Methods

We review an approach to emulate a randomized trial of dynamic switching strategies using observational data from the Antiretroviral Therapy Cohort Collaboration, the Centers for AIDS Research Network of Integrated Clinical Systems and the HIV-CAUSAL Collaboration. We estimated the comparative effect of tight-control vs. loose-control strategies on death and AIDS or death via inverse-probability weighting.

Results

Of 43 803 individuals who initiated an eligible antiretroviral therapy regimen in 2002 or later, 2001 met the baseline inclusion criteria for the mortality analysis and 1641 for the AIDS or death analysis. There were 21 deaths and 33 AIDS or death events in the tight-control group, and 28 deaths and 41 AIDS or death events in the loose-control group. Compared with tight control, the adjusted hazard ratios (95% confidence interval) for loose control were 1.10 (0.73, 1.66) for death, and 1.04 (0.86, 1.27) for AIDS or death.

Conclusions

Although our effective sample sizes were small and our estimates imprecise, the described methodological approach can serve as an example for future analyses.

In Africa, antiretroviral therapy (ART) is delivered with limited laboratory monitoring, often none. In 2003-2004, investigators in the Development of Antiretroviral Therapy in Africa (DART) Trial randomized persons initiating ART in Uganda and Zimbabwe to either laboratory and clinical monitoring (LCM) or clinically driven monitoring (CDM). CD4 cell counts were measured every 12 weeks in both groups but were only returned to treating clinicians for management in the LCM group. Follow-up continued through 2008. In observational analyses, dynamic marginal structural models on pooled randomized groups were used to estimate survival under different monitoring-frequency and clinical/immunological switching strategies. Assumptions included no direct effect of randomized group on mortality or confounders and no unmeasured confounders which influenced treatment switch and mortality or treatment switch and time-dependent covariates. After 48 weeks of first-line ART, 2,946 individuals contributed 11,351 person-years of follow-up, 625 switches, and 179 deaths. The estimated survival probability after a further 240 weeks for post-48-week switch at the first CD4 cell count less than 100 cells/mm(3) or non-Candida World Health Organization stage 4 event (with CD4 count <250) was 0.96 (95% confidence interval (CI): 0.94, 0.97) with 12-weekly CD4 testing, 0.96 (95% CI: 0.95, 0.97) with 24-weekly CD4 testing, 0.95 (95% CI: 0.93, 0.96) with a single CD4 test at 48 weeks (baseline), and 0.92 (95% CI: 0.91, 0.94) with no CD4 testing. Comparing randomized groups by 48-week CD4 count, the mortality risk associated with CDM versus LCM was greater in persons with CD4 counts of <100 (hazard ratio = 2.4, 95% CI: 1.3, 4.3) than in those with CD4 counts of ≥100 (hazard ratio = 1.1, 95% CI: 0.8, 1.7; interaction P = 0.04). These findings support a benefit from identifying patients immunologically failing first-line ART at 48 weeks.