Nancy A. Perrin, PhD

 Examine association of comprehensiveness of colorectal cancer (CRC) screening discussion by primary care physicians (PCPs) with completion of CRC screening.

Observational study in Kaiser Permanente Northwest, a group-model health maintenance organization.

 A total of 883 participants overdue for CRC screening received an automated telephone call (ATC) between April and June 2009 encouraging CRC screening. Between January and March 2010, participants completed a survey on PCPs’ discussion of CRC screening and patient beliefs regarding screening. Primary outcome measure: receipt of CRC screening (assessed by electronic medical record [EMR], 9 months after ATC). Primary independent variable: comprehensiveness of CRC screening discussion by PCPs (7-item scale). Secondary independent variables: perceived benefits of screening (4-item scale assessing respondents’ agreement with benefits of timely screening) and primary care utilization (EMR; 9 months after ATC). The independent association of variables with CRC screening was assessed with logistic regression.

 Average scores for comprehensiveness of CRC discussion and perceived benefits were 0.4 (range 0-1) and 4.0 (range 1-5), respectively. A total of 28.2% (n = 249) completed screening, 84% of whom had survey assessments after their screening date. Of screeners, 95.2% completed the fecal immunochemical test. More comprehensive discussion of CRC screening was associated with increased screening (odds ratio [OR] = 1.51, 95% confidence interval [CI] = 1.03-2.21). Higher perceived benefits (OR = 1.46, 95% CI = 1.13-1.90) and 1 or more PCP visits (OR = 5.82, 95% CI = 3.87-8.74) were also associated with increased screening.

 More comprehensive discussion of CRC screening was independently associated with increased CRC screening. Primary care utilization was even more strongly associated with CRC screening, irrespective of discussion of CRC screening.

 More comprehensive discussion of CRC screening by PCPs was associated with higher screening rates among adults aged 51 to 80 years who were overdue for screening.

 Higher perceived benefits of CRC screening by patients was also associated with increased CRC screening.

 1 or more PCP visits was strongly associated with increased CRC screening, irrespective of discussion of CRC screening.

Colorectal cancer (CRC) is the third-most common cancer and the second-leading cause of cancer-related death in the United States.

 Recent estimates suggest that CRC will be newly diagnosed in about 112,340 people annually, resulting in over 52,180 deaths per year.

Total direct costs associated with treating CRC are $17 billion annually, with 9.8 million work days lost annually to hospitalization for colon cancer.

The early detection of high-risk lesions, or CRC itself, through appropriate screening is associated with decreased incidence of and mortality from CRC.

 The US Preventive Services Task Force recommends that men and women of average risk begin screening for CRC starting at age 50 years,

 and specialty care guidelines provide recommendations for higher-risk groups.

 Currently, recommended CRC screening modalities for average-risk patients include annual fecal occult blood testing (FOBT), colonoscopy every 10 years, or flexible sigmoidoscopy every 5 years, with or without interval FOBT.

 In addition, as a result of improved test performance and usability, in 2008 multiple professional societies endorsed the use of 4 types of fecal immunochemical tests (FITs) for CRC screening to replace FOBT.

Despite the proven benefits of CRC screening, screening rates remain suboptimal. Although more than half of adults 50 years and older in the United States have ever received a CRC screening test, less than half receive screening tests at recommended intervals.

and specific health-system characteristics are associated with receipt of CRC screening (having usual sources of care,

 and assistance with scheduling of colonoscopy),

 less is known about how providerpatient interactions and comprehensiveness of CRC discussion by primary care physicians (PCPs) influence CRC screening. Previous studies indicate that physician recommendation for screening

 may increase CRC screening. However, less is known about how the comprehensiveness of discussion by PCPs about testing options, screening intervals, and potential test complications may influence actual CRC screening. Study of the association of the comprehensiveness of CRC discussion by PCPs with receipt of CRC screening can inform the development of quality-improvement interventions.

The primary objective of this study was to determine the independent association of comprehensiveness of CRC screening discussion by PCPs, as reported by patients, with receipt of CRC screening.

The protocol for this study was approved by the institutional review board within the study health maintenance organization (HMO).

Study Site, Survey, and Electronic Data Sources

The study was conducted at Kaiser Permanente Northwest (KPNW), a not-for-profit HMO with about 485,000 members in southwest Washington and the Portland, Oregon, metro area. The members’ demographic characteristics (age, gender, race/ethnicity) are similar to those of the area population.

KPNW regional electronic databases provided data on patient membership, demographics, primary care assignment, clinical data (including weight and height, laboratory results, and other healthcare utilization, including CRC screening), and clinician data. These data capture over 95% of all medical care and pharmacy services members receive, and data are linked through each member’s health record number.

We included HMO members aged 50 to 80 years who were at average risk for developing CRC, were overdue for CRC screening, and received an automated telephone call as part of a CRC screening-outreach program between March and June 2009.

 The specifics of the automated call program, including the definition of “average risk,” have been described previously.

 Essentially, the average-risk population includes those who are overdue for CRC screening and do not meet any of the following

criteria: (1) active colon cancer/risk factors that would indicate need for non-routine screening, (2) ever diagnosed with adenomatous polyps or human immunodeficiency virus/acquired immunodeficiency syndrome, (3) referred for colonoscopy or sigmoidoscopy in previous 3 months, or (4) receipt of plavix or warfarin medications in previous 4 months that may increase risk of false-positive FIT. The brief automated telephone calls (ATCs) included information about the benefits of CRC screening, encouraged FOBT as a relatively simple and lowrisk method of screening, and allowed patients to request an FOBT by pushing a number on their telephone. The date of the telephone call was defined as the intervention index date.

Of the population receiving automated telephone calls, a random sample of 2000 adults were mailed a survey in January 2010. We chose to begin survey outreach efforts in January 2010 in order to have the most time for those efforts, while still ensuring that sufficient time elapsed (6-9 months) for patients who received ATCs between March and June 2009 to complete screening.

The process flow of the survey administration process is described in the 

 We sent initial surveys to study respondents on January 15, 2010. Next, we sent a reminder postcard to non-responders 2 weeks after the initial mailing. Finally, we mailed the survey a second time to non-responders about 3 weeks after the reminder postcard. Survey enrollment was completed by March 31, 2010. Of the whole population, 1816 (90.8%) were contacted. A total of 184 participants were not contacted due to incorrect addresses. Of those contacted, 883 completed the survey, for a response rate of 48.6% (883/1816). Compared with non-respondents, respondents were more likely to be white (respondents = 95.3% vs non-respondents = 87.5%, 

<.0001), female (respondents = 58.3% vs non-respondents = 48.4%, 

<.0001), and taking more medications (respondents [mean ± standard deviation (SD)] = [3.1 ± 3.8]). Responders and non-responders did not differ in age or length of health plan membership.

This survey was designed to better understand the barriers and facilitators that patients encountered in their efforts to complete CRC screening. We focused on 2 patient-reported domains: (1) patients’ perceptions of clinician-patient counseling, such as comprehensiveness of CRC screening discussion by the PCP, and (2) patient beliefs; namely, perceived benefits of CRC screening. Comprehensiveness of CRC discussion was the primary independent variable, while primary care utilization and perceived benefits of CRC screening were the secondary independent variables.

Comprehensiveness of CRC Screening Discussion, Based on Patient Self-Report

Comprehensiveness of CRC screening discussion.

 These items were based on previous work from Braddock and colleagues

 that assessed levels of informed decision making occurring during patient-provider communications regarding CRC screening. Informed decision making, a means for facilitating patient participation in decision making, includes (1) providing relevant information about the clinical situation, alternatives, and risks and benefits; (2) assessing the patient’s understanding; and (3) giving the patient a clear opportunity to voice a preference.

 Seven yes/no items measured the content of patients’ discussions with their PCPs. We asked patients whether their PCPs did each of the following at any time in the 2 years prior to survey: (1) explained benefits of CRC screening, (2) discussed how often each screening test should be done, (3) provided information on how to do various tests, (4) explained the accuracy of each screening test, (5) explained potential test complications from screening tests, (6) asked if the patient understood or had any questions about CRC screening, and (7) asked if the patient had the information he or she wanted about CRC screening. These 7 items had high internal consistency (Cronbach’s alpha = 0.92). To calculate an average score, each of the 7 items was summed together (0.0 = no, 1.0 = yes) and divided by 7. A score of 1 indicates that patients answered “yes” to all 7 questions, while a score of 0 indicates a “no” answer to all 7 questions. Only the responses of patients who answered all 7 questions were included in the variable describing this scale.

Perceived Benefits of CRC Screening, Based on Patient Self-Report

Four items measured perceived benefits of CRC screening, scored on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Similar to the scale for comprehensiveness of CRC discussion, the perceived-benefits scale also had good internal consistency (Cronbach’s alpha = 0.80). To calculate an average score, each of the 4 items was summed together (1 = strongly disagree, 5 = strongly agree) and divided by 4. A score of 5 indicates that the respondent answered “strongly agree” to all 4 questions, and a score of 1 indicates an answer of “strongly disagree” to all 4 questions. Only the responses of patients who answered all 4 questions were included in the variable describing this scale. These questions were adapted from Myers and colleagues28-30 and measured general agreement with the following statements: (1) CRC screening is necessary even if no family history exists, (2) getting screening for CRC can protect health, (3) when colon polyps are found and removed, CRC can be prevented, and (4) when CRC is found early, it can be cured. The rationale for selecting perceived benefits is that beliefs about colon cancer have been identified in the literature as an important characteristic associated with CRC screening.

All patients were due for screening at the time of the ATC; the primary outcome measure was whether the patient completed CRC screening in the 9 months following the ATC. CRC screening included any of the following: 1 sample FIT, receipt of colonoscopy, and receipt of flexible sigmoidoscopy. The distribution of first completed CRC screening test for this study population was FIT (95.2%), colonoscopy (2.8%), and sigmoidoscopy (2.0%).

We identified patient age in relation to the intervention index date, and we identified gender from member eligibility files. We also determined length of KPNW membership. Individual racial categories were obtained from electronic databases for 267 (30.4%) participants, and missing data were imputed using the census tract block corresponding to each subject’s mailing address. Patient race was categorized as white or non-white (African American, Pacific Islander, Asian, and Native American). In addition, as a measure intended to reflect disease burden,

we determined the mean number (± SD) of unique generic drugs dispensed within the 6 months prior to index date for each participant. We also determined whether survey respondents had any primary care visits (internal medicine, family practice; >1 visits vs none) in the 9 months after the intervention index date.

We used logistic regression to model CRC screening during the 9 months following index date (receipt of ATC). Final variables were entered simultaneously and included age (continuous, by 10 years), gender (female vs male), number of medications (continuous), comprehensiveness of CRC discussion scale (continuous), perceived benefits scale (continuous), and having any primary care visit during the follow-up period. With the exception of age and gender, all variables with significant bivariate associations with CRC screening completion were included in the final multivariable model. Although not significant in bivariate analysis, age and gender were retained in final logistic models to adjust for demographics, given that age and gender have been associated with CRC screening in previous research. We lacked complete data on some of the central variables (ie, PCP discussion scale, perceived benefits scale), and we imputed missing data by multiple imputation (MI). Multiple imputation is considered a scientifically rigorous approach to handle missing data.

The MI and MIANALYZE procedures in SAS used 5 replications. All analyses were conducted using SAS version 9.2 (SAS Institute, Cary, North Carolina).

 presents the population characteristics of patients who completed the patient survey. The mean age was 59 years, nearly 60% were female, 95% were white, and the average number of years with KPNW was 14. The average number of medications taken at baseline was 3.1. The average score on the comprehensiveness of CRC discussion scale was less than 0.5 (mean = 0.4, SD = 0.4, possible range 0 to 1), indicating that patients, on average, answered “yes” to fewer than 4 of the 7 screening discussion questions. In addition, the average score on the perceived benefits scale was high (mean = 4.0, SD = 0.7; possible range 1 to 5), suggesting that most believed in the benefits of CRC screening. Most (60%) reported 1 or more primary care visits in the 9 months after the intervention index date.

presents unadjusted and adjusted logistic regression results. Patients’ perceptions of the comprehensiveness of CRC screening discussions and perceived benefits had independent significant (

 <.05) associations with completion of CRC screening, after simultaneous adjustment. A higher score on the comprehensiveness of CRC screening discussion scale was associated with greater odds of completing CRC screening in both unadjusted (OR, 1.84; 95% CI, 1.20-2.81, 

 = .005) and adjusted (OR, 1.51; 95% CI, 1.03-2.21, 

 = .035) analyses. A higher score on the perceived benefits scale was associated with greater odds of completing CRC screening in both unadjusted (OR, 1.70; 95% CI, 1.32-2.18, 

<.0001) and adjusted (OR, 1.46; 95% CI, 1.13-1.90,

Of patient-demographic and clinical-characteristic covariates, 1 or more primary care visits in the follow-up period (9 months after ATC) was associated with greater odds of completing CRC screening in both unadjusted (OR, 6.16; 95% CI, 4.16-9.14, 

 <.0001) and adjusted (OR, 5.82; 95% CI, 3.87- 8.74, 

 <.0001) analyses. Having more medications (OR, 1.05; 95% CI, 1.01-1.09, 

= .012) was associated with greater odds of completing CRC screening in the unadjusted analysis; however, this finding did not remain significant in multivariable analysis.

Articles

More comprehensive discussion of colorectal screening by primary care physicians was associated with higher screening rates among adults aged 51 to 80 years who were overdue for screening.

More Comprehensive Discussion of CRC Screening Associated With Higher Screening

 To determine whether adding selfreported health and functional status data to a diagnostic risk-score model explains additional variance in predicting inpatient admissions and costs.

 We used data from a Health Status Questionnaire (HSQ), completed by 6407 Kaiser Permanente Northwest Medicare patients between December 2006 and October 2008. We used answers from 3 items on the HSQ: (1) General Self-rated Health score, (2) needing help with 1 or more activities of daily living, and (3) having a bothersome health condition. We calculated a DxCG relative risk score from utilization information in the year prior to the survey, using electronic medical records. We compared: (1) DxCG as the sole independent variable and (2) DxCG plus the 3 items as independent variables. We estimated area under the curve (AUC) for each model. Any inpatient admission (yes/no) and being in the top 10% of costs (in the year after survey) were the dependent variables for the first and second logistic regression models, respectively.

The 3 items explained an additional 2.8% and 4.0% of variance for inpatient admissions and top 10% of costs, respectively, in addition to the variance explained by the DxCG score alone. For DxCG alone, the AUC was 0.686 (95% confidence interval [CI] 0.663-0.710) and 0.741 (95% CI 0.719- 0.764), respectively, for inpatient admissions and top 10% of costs and improved to 0.709 (95% CI 0.687-0.730) and 0.770 (95% CI 0.749-0.790) when the 3 self-reported items were added.

 Using self-reported health information improved the predictive power of a DxCG model to forecast inpatient admissions and patient cost-tier.

	In a Medicare population, self-reported information about being in poorer health was associated with higher inpatient admissions and being in the top tier for costs. Items associated with these outcomes were: (1) lower score on the General Self-rated Health score item, (2) answering yes to “do you need help with 1 or more activities of daily living?” and, (3) answering yes to “do you have a bothersome health condition?” These items:

		Explained an additional 2.8% and 4.0% of admission and cost variance, respectively.

		Were independently predictive of future inpatient admissions and being in the top 10% cost group.

	Predictive healthcare models are valuable tools for identifying individuals at risk for adverse outcomes, including high healthcare utilization and mortality. Predictive models are typically based on diagnostic history from administrative data or sometimes on patient-reported outcomes, but rarely both. The purpose of our analyses was to determine whether combining these 2 types of data in an integrated predictive model improved upon the predictive power compared with models based on administrative data alone.

	Claims-based predictive models are the most common because of the ubiquitous availability of claims data. There are a variety of commercially available claims-based predictive models, and their validity is well documented.

 The Society of Actuaries has conducted periodic comparisons of the predictive power of these models for following-year costs.

 In these models, health diagnoses and prescriptions, in addition to prior expenses, are the variables with most predictive accuracy. Laboratory results have also been shown to add predictive power.

	Predictive models based on patient-reported outcomes are less common because such measures are not routinely collected and reported. However, patient-reported outcomes have been shown to have strong predictive value for forecasting costs, utilization, and mortality. In particular, the predictive value of a single General Self-rated Health (GSRH) question is well documented. DeSalvo (2005) concluded in a meta-analysis that people with “poor” self-rated health had double the mortality risk of people with “excellent” self-rated health, even after adjustment for key factors such as functional status, depression, and comorbidity.

 GSRH has also been shown to be responsive to change. In a study of older adults, self-reported health declined steeply prior to adverse events, such as death or stroke.

 A study of repeated administrations of 2 GSRH questions showed good reliability, reproducibility, and discriminate scale performance.

 Although the single-question GSRH measure is seldom used in clinical practice settings, it is widely used in medical and economic research.

 In an investigation of the causal relationship between self-reported health and subsequent outcomes, Jylha suggests that aspects of self-reported health may reflect important physiological dysregulations, such as increases in tissue inflammation.

	Similarly, DeSalvo (2009) notes that respondents provide composite assessments of their physical and mental health and social status when answering questions about their overall health, which helps explain the strong performance of GSRH in predicting future risk.

 The literature includes several examples of patientreported outcomes used in predictive models. DeSalvo (2009) compared predictive ability of single- and multi-item assessments of self-reported health to a comorbidity index and diagnostic history in predicting next year’s costs. They found that the single-item self-reported health question predicted costs as well as the multi-item assessment and the comorbidity index, but not as well as the diagnostic history model.

 Using data from a survey of Medicare members, Brody et al developed highly sensitive indices of frailty and advanced illness.

 The models use survey items about self-reported health and function, such as poor general health and assistance needed for medications, bathing, or dressing, and were shown to be superior to clinical judgment in predicting frailty and advanced illness.

	There are, however, few examples of predictive models that combine diagnostic history and patient-reported outcomes. Fleishman et al found that measures of health status improved prediction of subsequent medical expenditures in a model that included demographics, chronic conditions, and previous expenditures in a nationally representative sample.

 Fleishman and Cohen later found that measures of self-rated health and function improved prediction of high expenditures beyond more detailed information on medical conditions for a nationally representative sample.

 These results are consistent with previous studies based on nonrepresentative samples. In a study of Veterans Affairs (VA) beneficiaries, self-reported physical and mental health improved mortality prediction when combined with a diagnostic history model.

Similarly in a previous study of VA beneficiaries, a multidomain health assessment added predictive power to a diagnostic history model.

	In this study, we examined whether inclusion of answers to a few brief self-reported items improved the prediction of future inpatient admissions and medical care costs compared with using an administrative risk score alone to predict the same outcomes. This study is unique in that it is the first of which we are aware to conduct this analysis in a Medicare population. Further, the Medicare population represents a unique population, given the large proportion with chronic health conditions, which contributes to increased inpatient admissions and costs—as the US population ages. We selected the well-validated, single-item general health question, a question about activities of daily living,

 and a question indicating whether or not the person currently has a physical condition that bothers him or her. These 3 items were selected because they are questions that could be quickly administered during an office visit, over the phone, by mail, or the Internet and are not tailored to a specific population (eg, those with chronic conditions) and capture information not readily attainable through administrative data.

	Study subjects were members of Kaiser Permanente Northwest (KPNW) and were aged 65 years and older. To be included in the study, members needed to complete a Health Status Questionnaire (HSQ) between December 2006 and October 2008, and have 1 year of membership eligibility prior to and after the completion of the HSQ. This time frame allowed us to compute the DxCG scores. When the member had completed more than 1 HSQ in the time frame of interest, 1 HSQ was selected at random for inclusion in the study.

	During the period of study, the HSQ was mailed to new Medicare members aged 65 years and older. This population included those newly enrolled to KPNW and current members who recently became Medicare eligible. In addition, it was mailed annually to a Medicare expanded-care population. The general response rate for the HSQ is 76%.

	The HSQ is a 45-item survey that includes measures of physical function, activities of daily living, health conditions, self-care deficits, and general health. For this study we used data from 3 items. The self-reported general health measure is scored in a range of 1 (excellent) to 4 (poor) by answering the question “compared to other persons your age, would you say your health is…?” The second item asked “because of a disability or health problem, do you need or receive help from another person for any of the following everyday activities…?” The activities listed include preparing meals, shopping for groceries, doing routine household chores, managing money, doing laundry, taking medications, getting to places not within walking distance, and using the telephone. This item was scored 0 if no items were checked and if 1 or more items were checked. The third item was “Is there any physical condition, illness, or health problem that bothers you now?” and was scored 0 for “no” and 1 for “yes.”

 is based on an individual’s total predicted cost in the next year relative to the population mean and is used to assign individuals to a Diagnostic Cost Group (DCG). DxCG Risk Solutions software

 is used to calculate the DxCG relative risk score. DCG cost estimates are calculated using 1 year of inpatient and outpatient 

International Classification of Diseases, Ninth Revision, Clinical Modification 

) codes to classify patients into groups with similar cost patterns. The diagnoses are classified into mutually exclusive DCGs, based on predicting future inpatient and outpatient costs. The software algorithm for DCG classifications was developed using 3 different populations: a privately insured population, a Medicaid population, and a Medicare population. Diagnostic Cost Groups for this study were based on the Medicare-derived DCG grouping. The range of Medicare DxCG scores ranged from 0.04 (minimum) to 15.59 (maximum). Individuals with a score of 1.0 have the relative risk equivalent to the national Medicare reference population. This reference population includes any of the following (excluding those with end-stage renal disease): (1) individuals 65 years or older, (2) disabled (under age 65 years) Medicare eligible, or (3) Medicare/Medicaid dual eligible. Previous research has used the DxCG relative risk score to predict future costs and mortality estimates.

	In the year following the date of the HSQ, we used electronic medical records to determine if the subject had any inpatient hospital admissions. Actual health plan costs for each person for this same period were extracted from administrative data and presented in 2008 dollars.

	Prior to logistic regression analysis, we tested for collinearity among the self-reported health items and DxCG risk score by calculating a Pearson product-moment correlation coefficient. We used 2 sets of logistic regressions, with 2 models in each set: DxCG as the sole independent variable and DxCG plus the 3 self-reported items as the independent variables. We calculated sensitivity, specificity, receiver operating characteristic (ROC) curves, and the area under the curve (AUC) for each model. Any inpatient admission (yes/no) was the dependent variable for the first set of regression models and being in the top 10% of costs (based on the observed study population) was the dependent variable for the second set of regression models. Prior to conducting the main analyses, those with and without complete data were compared using t tests and χ

	A total of 7545 HSQs were returned between December 2006 and October 2008 from 7496 unique individuals. Of those, 6407 people met the inclusion criteria of 1 year of eligibility prior to and after the HSQ date. Of those, 5815 (90.8%) had complete data on the 3 self-report items of interest and the DxCG. Those with complete data were younger (mean = 68.55 vs 70.71, 

 <.001), more likely to be male (44.9% vs 39.6%, 

 = .016), more likely to have attended college (67.0% vs 59.7%, 

 <.001), more likely to be married (69.0% vs 59.5%, 

 <.001), and to be white (93.1% vs 88.0%, 

 <.001). There were no significant differences between those with and without complete data on the likelihood of being in the top 10% of costs (

 = .680) or of having an inpatient admission (

 the average age was 68.55 (standard deviation [SD] = 6.34) with a range of 65 to 97. The majority of the sample was female (55.3%), married (69.0%), white (93.1%), and had attended some college (67.0%). Only 1.0% of the sample reported their race as black or African American; 1.9% as Asian or Pacific Islander; 0.3% as Aleut, Eskimo, or American Indian; and 1.3% as other. When asked to rate their health compared with other persons of their age, 27.8% rated it as excellent, 55.4% as good, 14.9% as fair, and 2.0% as poor. Only 10.9% needed help with 1 or more instrumental daily activities and 40.5% had a physical condition, illness, or health problem that bothered them at the time of taking the survey. The mean DxCG relative risk score was 0.68 (SD = 0.75) with a range of 0.19 to 9.91. Slightly more than 10% (10.8%) had 1 or more inpatient stays during the year following the survey.

Comparison of DxCG-only Model and DxCG—plus–3–self-report–items Model

	We used logistic regression to test whether addition of 3 self-report items significantly improved the DxCG model’s accuracy in predicting inpatient admissions and being in the top 10% of costs. The correlations among the self-report items and DxCG ranged from 0.26 to 0.45 (results not shown), which did not indicate a high degree of multicollinearity. For predicting inpatient admissions, the 3 self-report items made a significant contribution to the model compared with using DxCG alone (χ

 <.001), accounting for an additional 2.8% of the variance in having 1 or more inpatient admissions. 

 summarizes results of the model. Higher DxCG scores, poorer self-reported health, needing help with at least 1 everyday activity, and having a physical condition that was bothersome at the time of completing the survey were all associated with having an inpatient admission. For each additional point on the DxCG, the likelihood of an inpatient admission increased by 63%. A change of 1 point on self-reported health (eg, from good to fair) increased the likelihood of an inpatient admission by 30%. Needing help with at least 1 instrumental daily activity and having a physical condition that bothered the respondent at the time of the survey increased the likelihood of an inpatient admission by 52% and 53%, respectively. 

 presents the ROC curves for the model that used DxCG alone, and for the model that used DxCG plus the 3 self-report variables. The AUC was 0.686 (

 <.001, 95% confidence interval [CI] 0.663-0.710) for the model with DxCG alone, and improved to 0.709 (

 <.001, 95% CI 0.687-0.730) when the 3 self-report items were added. 

 provides results for sensitivity, specificity, positive predictive value, and negative predictive value for both models. Sensitivity was 0.467 for the DxCG-only model and improved to 0.529 when the 3 self-report items were added. Specificity was 0.807 for the DxCGonly model and 0.741 when the 3 self-report items were added to the model.

	Similarly, analysis for the models’ ability to predict being in the top 10% of costs showed that the 3 self-report items made a significant contribution to the model compared with the model that used DxCG alone (χ

 <.001). The 3 self-report items accounted for an additional 4.0% of the variance explained in being in the top 10% cost group (Table 3). Table 2 summarizes the results of the full logistic model with DxCG and the 3 self-report items. Higher DxCG scores, poorer self-reported health, needing help with at least 1 everyday activity, and having a physical condition that bothered the participant at the time of completing the survey were all associated with having an inpatient admission. For each additional point on the DxCG, the likelihood of being in the top 10% of costs increased by 104%. A change of 1 point on the self-reported health score (eg, good to fair) increased the likelihood of being in the top 10% of costs by 30%. Needing help with at least 1 instrumental daily activity and having a bothersome physical condition increased the likelihood of being in the top 10% of costs by 78% and 82%, respectively. 

presents the ROC curves for the model using DxCG alone and for the model with DxCG plus the 3 self-report variables. The AUC was 0.741 (

 <.001, 95% CI 0.719-0.764) for the model that used DxCG alone and improved to 0.770 (

 <.001, 95% CI 0.749-0.790) when the 3 self-report items were added to the model. Sensitivity was 0.522 for the DxCG-only model and improved to 0.611 when the 3 self-report items were added to the model. Specificity was 0.821 for the DxCG-only model and 0.771 when the 3 self-report items were added to the model (see Table 3).

	This study found that self-reported items add predictive power for future healthcare costs and inpatient admissions beyond the DxCG relative risk score. Specifically, poorer self-reported health, needing help with at least 1 activity of daily living and having a bothersome physical condition added predictive power for higher costs and inpatient admissions in the 12 months after survey administration. This study is one of only a few studies to examine the additive effect of self-reported health items to predict subsequent costs and inpatient admissions over and above the DxCG relative risk measure for a Medicare population.

	Results from this study confirm findings from other studies. Several studies have compared self-reported health items with administrative data—based risk measures to predict future costs, but did not examine the additive effect of self-reported items over administrative data–based risk measures. Using data from the Medical Expenditure Panel Survey, DeSalvo et al found that a 1-item GSRH score was predictive of future healthcare costs, but less robust than models that use administrative data–based risk measures.

 Similarly, using administrative and survey data from a VA population, Maciejewski et al found the SF-36 Physical Component Survey and Mental Component Survey were predictive of future healthcare costs, but again less robust than administrative data—based risk measures.

 In addition, similar to our findings, Fleishman et al found that measures of health status improved prediction of subsequent medical expenditures beyond demographics, chronic conditions, and previous expenditures.

	We also found that the DxCG risk score and observed self-report health items were not highly correlated with each other. Similar to our findings, Wang et al found that DCG (similar to DxCG) and SF-36 physical and mental health scores were both predictive of future healthcare costs, but not highly correlated.

 The implication of these results is that DxCG and the individual self-report health items are measuring different underlying constructs and do not overlap greatly.

	Findings from our study have implications for care delivery and quality improvement initiatives. Asking questions about self-rated health status, activities of daily living, and bothersome physical conditions is easy to do and can be administered in a variety of care delivery settings. The strong predictive ability of these self-report questions are worthy of integration into enhanced care management and patient outreach efforts. This information is especially important when diagnostic history is not available, or is too expensive to collect.

	This study is not without limitations. First, the DxCG relative risk measure was the only administrative data—based risk measure analyzed in this study. This limitation is addressed in part in that previous research found the DxCG relative risk measure has greater predictive ability than other administrative data–based measures.

 Second, the study population was enrolled in an integrated health maintenance organization health delivery system, so study findings cannot be generalized to non—managed care settings. Third, the study population included few racial/ethnic minorities, again limiting the generalizability of the study findings. Fourth, we were not able to compare the characteristics of those who did not return an HSQ with those who did return the HSQ. Fifth, patient-level cost data include information where both utilization and matching general ledger information are available. Therefore, costs for services with no, or inadequate, available utilization information are not included, so costs may be understated for some patients. Last, the DxCG predictive model did not include laboratory and biometric data, as well as information on receipt of evidence-based care based on clinical condition, which are important factors that can improve predictive modeling.

	Future research is needed to evaluate the use of self-reported health items to enhance care management activities. Specifically, randomized studies are needed to compare enhanced delivery models using self-reported health items to usual care. Within this approach, more formal cost-effectiveness analysis is needed to identify potential reductions in hospital admissions through rigorous case finding and enhanced care management approaches. The primary objective would be to determine the extent to which the addition of self-reported health information enhances the ability of healthcare organizations to better manage overall costs and utilization over time.

 From Center for Health Research (NAP, DMM, AB, ES), Kaiser Permanente Northwest, Portland OR; Department of Care and Service Quality (MS), Kaiser Permanente, Oakland, CA; Decision Support Services (EMD), Kaiser Permanente, Oakland, CA.

Among a Medicare population, use of 3 self-reported health items improves predicted inpatient admissions and healthcare costs when used with risk-prediction model.

Self-Reported Health and Functional Status Information Improves Prediction of Inpatient Admissions and Costs

To evaluate the effect of a patient panel-support tool (PST) on care delivery for diabetes mellitus (DM) and cardiovascular disease (CVD). 

Retrospective longitudinal cohort study among primary care providers (PCPs), with 2005 as the preintervention, 2006 as the implementation, and 2007 as the postintervention period.

We estimated the intervention effect using electronic medical record data and hierarchical linear models. The intervention was a PST displaying "care gaps" and recommendations for glycosylated hemoglobin, low-density lipoprotein cholesterol, and blood pressure screening and control; retinopathy, nephropathy, and foot screening; aspirin, statin, and angiotensinconverting enzyme inhibitor or beta-blocker use; and influenza and pneumococcal vaccination. Participants were qualifying PCPs and health maintenance organizations; patients. Patients had DM or CVD and 12 months of membership (n =30,273 DM; 26,414 CVD). Main measures were mean percentages of care recommendations that were met by PCPs per patient per month (the care score).

 From 2005 to 2007, the mean care score (95% confidence interval) increased for both DM and CVD, from 63.5 (62.7, 64.3) to 70.6 (69.8, 71.4) and from 67.9 (67.2, 68.7) to 72.6 (71.9, 73.3), respectively. After adjustments, DM and CVD patients had improvements in the care score of 7.6 and 5.1, respectively, in 2007 compared with 2005 (

Delivery of care recommendations for DM and CVD improved after implementation of a PST. More research is necessary to optimize results and determine whether patient outcomes improve.

	The effect of a patient panel-support tool (PST) on care delivery for diabetes mellitus (DM)and cardiovascular disease (CVD) was evaluated in a retrospective longitudinal cohort study among primary care providers.

		The mean percentage of care recommendations met (care score) increased for DM and CVD from 63.5 to 70.6 and from 67.9 to 72.6, respectively.

		Future work is needed to maximize the effects of PSTs on care recommendations met and to determine whether improvements in recommendations met translate to improved patient outcomes.

	Electronic medical records (EMRs) hold great promise for changing care management by improving implementation of care guidelines and providing data and decision support to prompt and facilitate action.

 Electronic tools utilizing EMR data such as alerts, reminders, and other embedded guideline advice have proved effective at improving clinical practice.

 However, many tools address a single group or a small group of care needs, and depend on a defined patient encounter (visit or phone call) to be useful to practitioners. The need for more global tools to harness data and technology stored in EMRs is particularly evident when treating patients with chronic diseases. For example, patients with diabetes mellitus (DM) or cardiovascular disease (CVD) receive only about 50% of the recommended care for which they are eligible.

 These pervasive care gaps in screening, monitoring, preventive medication use, and risk factor control7-11 result in unnecessary morbidity, mortality, and healthcare costs.

	We evaluated the effect of a new tool, the panel-support tool (PST),

 on delivery of care recommendations by primary care providers (PCPs) for patients with DM and/or CVD. The PST uses EMR data to graphically display “care gaps” for each patient based on current evidence and provides summary information to assist practitioners and care teams in evaluating these gaps and ordering needed services. Such tools have been referred to as “quality dashboards”

and are becoming integrated into available EMRs

 or are in the development phase in homegrown systems.

 However, despite their potential to be of enormous use to managed care organizations, published reports evaluating their success are limited.

	The study was conducted in 2008 at Kaiser Permanente Northwest, a nonprofit group-model health maintenance organizations (HMO) operating in southern Washington and northern Oregon, with 15 medical clinics and 485,000 members. Members are demographically similar to the area population.

 Electronic databases provided information on clinician characteristics and patient membership, demographics, diagnoses, medications dispensed, over-the-counter medication documented by clinicians, blood pressure (BP), laboratory results, and procedures from inpatient and outpatient settings. These data captured more than 95% of all medical care and pharmacy services received by members.

 A fully electronic medical record (Kaiser Permanente HealthConnect, Epic System Inc) has been in place since 1996. A 99.5% specific diabetes registry

 has existed since 1989; its sensitivity is estimated at 99% compared with expert chart review. A CVD registry derived from diagnoses and procedures (including acute myocardial infarction, ischemic heart disease or angina, coronary artery bypass procedures, percutaneous transcoronary angioplasty, stroke, and peripheral vascular disease) also has been in place since 1996. The study site’s clinical guidelines for diabetes and CVD care are consistent with the American Diabetes Association (ADA) guidelines

 and the American Heart Association’s guidelines.

 During the study period (and as implemented in the PST), the ADA recommended aspirin for primary prevention in DM in those older than 40 years. More recent evidence led the ADA to recommend aspirin only in the presence of at least 1 additional CVD risk factor.

Other than the implementation of the PST, care processes for patients with DM and CVD remained stable during the study period.

	The protocol for this study was approved by the institutional review board within the study HMO.

) has been previously described in detail.

 It is Internet-based and graphically displays care gaps for each patient in a PCP’s panel, based on current evidenceof recommended care.

 Sorting functions allow care teams to select patients for follow-up based on several parameters. The PST provides data on gaps in screening,  medication use, monitoring, risk factor control, and immunizations, coded by color and number according to urgency. Gaps in medication use and risk factor control were emphasized through use of higher care gap scores for these services. For preventive needs, green indicates up-to-date; yellow, due soon; and red, overdue. For cardiovascular risk factor control, green indicates good control (eg BP, <135/85 mm Hg; glycosylated hemoglobin [A1C] <7%; low-density lipoprotein cholesterol [LDL-C] <100 mg/dL); yellow indicates modest elevations; and red indicates poor control. Prompts recommend conducting a screening test, considering starting/restarting a preventive medication, adjusting a medication dose, or adding an additional control medication, as indicated. Sets of clinical orders addressing each care gap facilitate clinician follow-up. Graphical displays allow care teams to compare their performance with that of other teams. Primary care providers and teams were trained in use of the PST and were expected by the HMO to use it to maximize team-member contributions during the implementation year (2006). Use of the PST by PCP teams gradually increased to 50% of teams by August 2006, 65% by December 2006, and more than 95% by February 2007. Although the PST was designed for use during clinic visits and for patient outreach, during the study period the training and implementation protocols were for use of the tool during visits.

	This retrospective longitudinal cohort study evaluated the effectiveness of the PST at improving rates at which PCPs met care recommendations for patients with DM or CVD. Our unit of analysis was the PCP. Separate analyses were conducted for DM and CVD, and patients could be in one or both analyses.

	We identified all 204 adult PCPs (in internal medicine or family practice) who had patient panels during all 3 time periods: the pre-PST period (2005), the PST implementation period (2006), and the post-PST period (2007). To achieve stability in our estimates, we limited analyses to the 167 PCPs who had at least 20 eligible DM patients during each study month and the 143 PCPs who had at least 20 eligible CVD patients each study month.

	The study period was divided into 36 months, and data for each PCP were extracted monthly based on eligible patients in that month. Patients were eligible if they were at least 18 years of age and known to have DM or CVD. To enable the collection of covariates and outcomes, we also required that patients be continuous members during and 11 months before the study month.

	The primary outcome was the mean percentage of care recommendations met by PCPs per member per month (for their eligible patient panel each month) or the “care score,” with all included care recommendations weighted equally. Therefore, the care score could range from 0 (none of patients’ care recommendations were met) to 100 (100% of patients’ care recommendations were met). The composite outcome mirrored that used by others in DM and CVD

and in analyses that aimed to measure improvement over multiple clinical care needs.

 The care recommendations were consistent with national guidelines

and mirrored the decision support constructed in the PST (

). We included  all screening, preventive medication use, risk factor control, and immunizations relevant to the care of DM or CVD that also were addressed by the PST: A1C, LDLC, BP screening and control (unmeasured values were considered to be uncontrolled); retinopathy, nephropathy, and foot screening; aspirin, HMG-CoA reductase inhibitor (statin), angiotensin-converting enzyme (ACE) inhibitor or beta-adrenergic blocker (betablocker) use; and influenza and pneumococcal vaccinations. Care recommendations not specific to this population were excluded (eg, cancer screening). To aggregate data at the PCP level for the care score, we determined the total number of services for which each member of the PCP’s patient panel was eligible and the percentage of those services that were met each month. Results for each individual care score component (percent met) also were calculated.

	We included a series of covariates (PCP and patient panel characteristics and measures of healthcare utilization) that could impact the likelihood of care quality improvement resulting from the PST. The PCP demographics were determined at baseline (January 2005) and included age, sex, number of years employed at study site, adult primary care specialty (family practice or internal medicine), and degree type (medical doctor or allied health [nurse practitioner, physician’s assistant]). At the beginning of each study year, we determined the following PCP patient panel characteristics: size, percentage of patients with condition of interest (DM or CVD), mean age of patients, percentage of patients who were nonwhite, and percentage of the DM or CVD population with an annual family income of less than $40,000. Individual racial categories were obtained from electronic  databases for 23,453 (78.1%) patients with DM and 23,647 (90.1%) patients with CVD, and missing race information was geocoded (estimated from neighborhood measures of socioeconomic status using the census-tract block corresponding to each subject’s mailing address).

 Family income also was obtained from geocoded data.

	For healthcare utilization variables that were likely to change more frequently and that provided opportunities for meeting care needs, at the beginning of each study month we determined the following additional PCP patient panel characteristics, using eligible patients of interest (DM or CVD) that month: mean number of outpatient specialty visits likely to impact outcomes (ie, endocrinology, cardiology, pulmonology, nephrology), mean number of outpatient visits to the assigned PCP, mean number of outpatient visits to other PCPs (per patient), percentage of patients with a hospitalization, and percentage of patients in selected care management programs likely to impact outcomes (specifically, nurse and pharmacist cardiovascular risk factor management, lipid or diabetes programs) in the prior year.

	We used 3-level hierarchical linear models to determine whether the care score changed across the 3 time periods (pre-PST year, PST implementation year, and post-PST year), controlling for relevant factors. The first level of the model inn cluded variables that were measured monthly (time in months and utilization measures); the second level included  variables that were measured annually (panel size and patient demographics and indicators of the PST period [PST implementation vs pre-PST, post-PST vs pre-PST]); and the third level included characteristics of the PCP. To achieve parsimonious models, we used a stepped process to determine variables to retain in the models. First, we developed the list of variables that could impact the likelihood of care quality improvement resulting from the PST, as noted above. Then at the first level of the model, we retained those utilization covariates (mean number of PCP visits, mean number of specialty care visits, and percentage of patients in selected care management programs) that made a significant contribution to the care score. Next, we added the second level to the model, retaining the panel characteristics (for DM/CVD patients: number per PCP, the proportion of DM/CVD patients in the PCP’s panel, mean age, and the proportion that were nonwhite) that were significant. In the last step, the third level of the model was added, retaining characteristics of the PCP (PCP age, years in the plan, and degree) that were significant. This stepped approach allowed us to include theoretically relevant variables as covariates without overfitting the model. Time (in months) by period (pre-PST, PST implementation, post-PST) interaction was included in the model to test whether the rate of change from January to December of each year differed for the 3 periods. We expected to see very little monthly change in the pre-PST period; a greater monthly change during the PST implementation  period, as more clinics and PCPs were trained; and a leveling off of the monthly change in the post-PST period. A base rate for time by period interaction was included to test  whether differences between periods (in monthly rates of change) varied according to PCPs’ starting care score. All analyses were conducted with SAS 9.1 (SAS Institute Inc., Cary, NC) or HLM 6.0 for Windows (Raudenbush SW, Bryk AS, Congdon R. Lincolnwood, IL: Scientific Software International, Inc; 2004).

	A mean of 20,705 (SD = 1369; range 18,401-22,574) patients per month across all PCPs and a mean of 123 (SD = 55; range 22-285) patients per month per PCP qualified for the DM analysis. A mean of 17,840 (SD = 1464; range = 15,168-19,473) patients per month across all PCPs and a mean of 124 (SD = 61; range 23-302) patients per month per PCP qualified for the CVD analysis. A total of 30,473 unique DM patients and 26,414 CVD patients were included in the monthly calculations across the 3 time periods (8543 were in both analyses).

	Characteristics of PCPs and their patients are presented in 

. The PCPs in both analyses were similar in mean age and years of practice. The PCP panel patient characteristics were fairly stable through the study period. The mean total panel size was approximately 1500 patients, and PCP panels were about 9% DM and 8.7% CVD patients. The mean age of DM patients was 61 years, compared with 70 years for CVD patients. The DM patients were about 12% nonwhite, compared with 7% of CVD patients, and nearly 20% of patients in both analyses had a family income of less than $40,000 per year. Both patient groups had similar numbers of mean annual visits to primary care clinics, but CVD patients used selected specialty visits, hospitalizations, and care management programs at nearly double the rate of DM patients.

 shows the unadjusted care scores and the individual components of the composite measure across the study years. The care scores across the study years (95% confidence interval) increased for DM patients from 63.5 (62.7, 64.3) pre-PST in 2005 to 65.7 (64.9, 66.6) during implementation in 2006, and to 70.6 (69.8, 71.4) post-PST in 2007. Corresponding increases for CVD patients were from 67.9 (67.2, 68.7) to 69.4 (68.7, 70.2), and then to 72.6 (71.9, 73.3). Improvements in the composite measure were largely due to improvements in rates of nephropathy screening, foot exams, all medications used for secondary prevention (clinician documentation of aspirin use and dispensing of statins, beta-blockers, and ACE inhibitors/angiotensin II receptor blockers), LDL-C control, and pneumococcal vaccine use. Blood pressure control showed small improvements, and A1C control declined.

 presents modeling results of the care score (in DM and CVD analyses), adjusting for selected healthcare utilization, PCP, and patient panel size and demographic characteristics. For DM, the main effect for time period shows that, controlling for other variables, during the PST implementation year PCPs had an absolute improvement in the mean care score of 2.3 (coefficient or b = 2.3, 

 <.001) compared with the pre-PST year; in the post-PST year, the care score improved 7.6 (b = 7.6, 

 <.001). For CVD, PCPs had an absolute improvement in the care score of 1.6 during the implementation year (b = 1.6, 

<.001) and of 5.1 in the post-PST year (b = 5.1, 

 <.001). The difference in care score predicted by the models with covariates is very similar to the observed unadjusted difference in care scores between the periods (DM: improvement of 2.2 in the implementation year and 7.1 post-PST; CVD: improvement of 1.5 in the implementation year and 3.2 post-PST). Time-by-period interactions reveal that during PST implementation, the care score began to increase at a rate of 1.7 per month in DM (b = 1.7, 

<.001) and 2.1 per month in CVD (b = 2.1, 

 <.001) compared with the prior year. These findings, which are graphically displayed in 

, confirm our hypothesized pattern of results: flat performance pre-PST, rapid improvement during implementation, and a slower continued rate of improvement postimplementation. A separate sensitivity analysis including only CVD patients without DM found the same pattern (data not shown).

	Several other factors significantly affected the care scores for both patient cohorts. The PCPs with higher baseline care scores had smaller increases in the rate of monthly change from pre-PST to PST implementation period for both DM (b = -0.02, 

 <.001). Having more selected DM- and CVD-relevant outpatient specialty visits, being enrolled in care management programs, and higher number of visits to the patient’s assigned PCP per month were associated with higher care scores (

 <.001 for all). The care score was 1.9 higher in DM patients and 1.4 higher in CVD patients for each additional specialty visit, and it was 2.6 higher in DM patients and 1.5 higher in CVD patients for each additional visit to the patient’s PCP. Also, for each additional year of mean panel age, the care score was 0.41 higher in DM (

A panel-support tool in a managed care setting improved the percentage of care recommendations met for patients with diabetes mellitus or cardiovascular disease.

Effect of a Patient Panel-Support Tool on Care Delivery

 To test the efficiency and cost-effectiveness of interventions aimed at enhancing laboratory monitoring of medication.

 Patients of a not-for-profit, group-model HMO were randomized to 1 of 4 interventions: an electronic medical record reminder to the clinician, an automated voice message to patients, pharmacy-led outreach, or usual care. Patients were followed for 25 days to determine completion of all recommended baseline laboratory monitoring tests. We measured the rate of laboratory test completion and the cost-effectiveness of each intervention. Direct medical care costs to the HMO (repeated testing, extra visits, and intervention costs) were determined using trial data and a mix of other data sources.

The average cost of patient contact was $5.45 in the pharmacy-led intervention, $7.00 in the electronic reminder intervention, and $4.64 in the automated voice message reminder intervention. The electronic medical record intervention was more costly and less effective than other methods. The automated voice message intervention had an incremental cost-effectiveness ratio (ICER) of $47 per additional completed case, and the pharmacy intervention had an ICER of $64 per additional completed case.

 Using the data available to compare strategies to enhance baseline monitoring, direct clinician messaging was not an efficient use of resources. Depending on a decision maker’s willingness to pay, automated voice messaging and pharmacy-led efforts can be efficient choices to prompt therapeutic baseline monitoring, but direct clinician messaging is probably a less efficient use of resources.

Patients of a not-for-profit, group-model HMO were randomized to 1 of 4 interventions: an electronic medical record reminder to the clinician, an automated voice message to patients, pharmacy-led outreach, or usual care.

The electronic medical record intervention was more costly and less effective than other

Depending on a decision maker’s willingness to pay, pharmacy-led outreach or automated

	voice messages to patients can be cost-effective methods to enhance laboratory monitoring of

	medications at initiation of therapy compared with usual care.

Recent studies indicate that laboratory monitoring of medications at initiation of therapy is below the level recommended by guidelines, with as many as 39% of patients not receiving recommended testing.

 Lack of monitoring is a concern because of potential adverse events (eg, hyperkalemia associated with inhibitors of angiotensin) and because of failure to achieve therapeutic benefit due to inadequate blood levels of medication. Additionally, failure to establish baseline levels makes it difficult to determine a patient’s trends in laboratory values.

These concerns regarding patient safety and clinical effectiveness have led researchers to test methods to enhance laboratory-based medication monitoring. Several types of interventions (including pharmacyled efforts, electronic reminders to clinicians, and automated telephone call reminders to patients) to improve laboratory monitoring of medications at therapy initiation have been effective in randomized trials.

 Although all these interventions improve monitoring, the most efficient intervention methods are not clear, and no economic analyses have been done to inform policy makers in this area. Some efforts may be particularly resource intensive, but could be worth the added expenditure when the potential adverse outcome is severe. Without careful analysis of the balance between costs and benefits, one cannot determine which (if any) interventions ought to be funded by healthcare payers. The efficiency of alternative approaches to therapeutic monitoring is of growing importance to healthcare providers as this monitoring is now a focus of quality measurement.

Although some interventions may enhance medication safety, an electronic medical record reminder to providers may not be an efficient use of resources.


Nancy A. Perrin, PhD

Articles

More Comprehensive Discussion of CRC Screening Associated With Higher Screening

Self-Reported Health and Functional Status Information Improves Prediction of Inpatient Admissions and Costs

Effect of a Patient Panel-Support Tool on Care Delivery

Improving Laboratory Monitoring of Medications: An Economic Analysis Alongside a Clinical Trial

Selection Bias Between 2 Medicare Capitated Benefit Programs