Roger T. Anderson, PhD

 To describe patient and healthcare system correlates of receipt of recommended care in North Carolina (NC) as indicated by receipt of adjuvant radiation therapy (RT) after breast-conserving surgery (BCS).

 Subjects were 344 women diagnosed as having primary breast cancer in 1998 and 1999, who were classified as being alive at least 12 months after treatment with BCS. Medicaid claims were used to supplement central cancer registry (CCR) data about adjuvant RT, and hospital medical record verification was performed when no RT was documented. Health system characteristics (size and volume) were obtained from existing databases.

 Of 344 NC women enrolled in Medicaid and treated with BCS, one third did not receive RT. The following patient and health system characteristics were associated with lack of receipt of adjuvant RT after BCS: older age (=65 years), residing in a lowâ€“population density county, receiving BCS at a smaller hospital, and living in a county classified as a whole-county specialist scarcity area.

 Some low-income women do not access RT following BCS, placing them at risk for worse outcomes than those associated with standard mastectomy. We identify geographic isolation and scarcity of healthcare specialists as possible leverage points for interventions.

Omission of adjuvant radiation therapy (RT) after breast-conserving surgery leads to higher recurrence and mortality rates. One third of Medicaid-insured women with low socioeconomic status did not receive adjuvant RT.

We propose that geographic isolation and scarcity of healthcare specialists are possible leverage points for interventions in this regard.

Breast cancer is one of the most frequently occurring diseases among women

 and is expected to account for almost 49,000 deaths in the United States in 2007.

 Decades of clinical trial research have established treatments that minimize risk of recurrence,

 extend survival, and improve quality of life. To achieve maximum potential benefit of treatment, the National Cancer Institute

 and the National Comprehensive Cancer Network

 endorse that all women diagnosed as having breast cancer have access to care that is timely and is based on current professional guidelines. For breast cancer, the National Institutes of Health 1990 Consensus Conference

 recommended breast-conserving surgery (BCS) plus radiation therapy (RT ) as a safe and effective alternative to mastectomy for most women with stage I and stage II breast cancer based on evidence from the National Surgical Adjuvant Breast Project protocol

 and other prospective randomized trials. These studies proved that, among women with nodepositive or node-negative breast cancer, BC S followed by adjuvant RT resulted in the same breast cancer&#8211;related survival rates as those associated with mastectomy. More recently, evidence shows that receipt of RT after BCS is associated with improved survival.

 Using data from the National Cancer Institute&#8217;s Surveillance, Epidemiology, and End Results (SEER) registry, Vinh-Hung and colleagues

 found a 34% increase in mortality among patients treated with BC S without RT , and Clark and colleagues

 in a 2005 meta-analysis of 15 trials reported, the mortality effect remained, albeit smaller (8.9%) than that observed in registry data. The effect of RT on survival among low-income women is especially important to consider because they are at greater risk of having late-stage tumors at the time of diagnosis and may be especially vulnerable to an increased risk of local recurrence with subsequently larger tumors.

The choice between BCS plus RT and mastectomy for stage I and stage II tumors depends on individual circumstances, personal preference, and provider recommendation. However, potential barriers to adjuvant RT with BCS faced by the patient may include an appreciable time burden (normally 5 days per week for 6 weeks), which may be logistically difficult and costly if the patient lives a long distance from a treatment facility.

 Radiation therapy may also cause undesirable effects, including swelling, fatigue, pain, skin pigmentation, and fibrosis of the breast, and a patient treated with BCS might reweigh her preference to receive this required treatment. There is a substantial body of evidence indicating that some women treated with BCS do not receive RT .

 reported that only 60% of Pennsylvania Medicaid-insured patients with breast cancer undergoing BC S received RT in 1990, although National Institutes of Health

 recommendations for RT and BC S were only emerging at this time. Roetzheim et al

 found that only 67.9% of patients with breast cancer enrolled in Florida&#8217;s Medicaid program received RT after BC S. In these studies, the reasons for omitting RT after BC S were hypothesized to include physician- and patient-led considerations such as patient refusal

 and barriers to access such as distance to the nearest RT facility.

 Healthcare system features may be important in assisting patients in weighing the &#8220;best&#8221; surgical approach based on patient preference and counseling about decision making. Because RT is not a surgical service, it requires patient referral and follow-up processes across providers and perhaps institutions. The latter functions may be challenging within fragmented healthcare systems.

Previous work demonstrated that approximately one third of Medicaid-enrolled patients with breast cancer in North Carolina (NC) who underwent BCS did not have RT listed in Medicaid claims or in consolidated central cancer registry (CCR) data.

 In the present study, we examine geographic and healthcare facility correlates of receipt of adjuvant RT among all Medicaid-enrolled women treated with BCS in 1998 and 1999 to identify potential areas for quality-of-care improvement.

Data for this study originated from NC Medicaid claims files and from 1998 and 1999 NC CCR data which were linked by patient identifiers (name, date of birth, and Social Security number) to produce an analytic file. To perform this dataset linkage, a probabilistic match routine was run against the Medicaid eligibility file based on 2 matching strings (1 for each round of matching). The first string consisted of a Social Security number and the first 3 characters of the first name (3 characters of the first name to prevent mismatching a wife&#8217;s cancer record with hear husband&#8217;s Medicaid record). Nonmatches were passed to a second string consisting of the first 5 characters of the last name, first 3 characters of the first name, date of birth, and county of residence. During the process of the linkage, we assessed nonmatched cases that were closely linked on a case-by-case basis, but the overall success of the linkage was not assessed.

Institutional review board and state agency approval was obtained to conduct this study and included all CCR and Medicaid data along with stringent data protection and patient privacy protection procedures. Subjects eligible for inclusion in this study were all female cases listed in the CCR with a diagnosis of nonmetastatic primary breast cancer (ie, in situ, local, or regional) in 1998 or 1999, who were classified as being alive at least 12 months after treatment with BCS. The latter was determined by the absence of a death record notification within the Social Security Administration&#8217;s Death Master File (DMF) in a search conducted in 2000. Eligibility also required evidence of enrollment in NC Medicaid at the time of diagnosis listed in the CCR and during the period of 1 to 3 months before the exact date of diagnosis. The sample size in the study years allowed a between-group difference of 10% (25% vs 35%) at &#945; = 0.05 in the proportions with RT omitted following BCS. The time frame was selected to allow survival outcomes to be observed.

Health system characteristics were determined from CCR facility codes for the location of surgery. Hospital size and patient volume of the surgery facility were determined using a 2005 listing from the American Hospital Directory

 that pertained to all patient stays. We classified NC counties as areas of specialist physician shortage in 2003, based on Centers for Medicare & Medicaid Services lists

 as those having the lowest specialty care ratios of Medicare beneficiaries to active physicians in a rural census county. Specialist shortage area designations exclude primary care physicians, including general practice, family practice, internal medicine, and obstetrics/gynecology.

To qualify for NC Medicaid and for full Medicaid coverage for the treatment of breast cancer, an adult woman generally must have an annual income of less than 180% below the federal poverty level and must have 1 of the following characteristics: she must be 65 years or older, blind or disabled according to Social Security standards, pregnant, or the parent or caretaker relative of a child who is younger than 19 years. A woman aged 18 to 64 years with breast cancer who is not eligible for Medicaid but who has no creditable medical insurance coverage may be eligible under the breast and cervical cancer Medicaid program. To qualify for this program, she must be enrolled and screened for breast or cervical cancer through the Breast and Cervical Cancer Control Program that is administered by the Division of Public Health of the NC Department of Health and Human Services. Full NC Medicaid coverage includes full payment of all medically prescribed cancer treatment at rates negotiated by the Centers for Medicare & Medicaid Services. For persons enrolled in Medicaid with dual Medicare insurance (eg, for those legally blind or disabled or those &#8805;65 years), NC Medicaid pays the deductible and the coinsurance.

The study claims database included all paid NC Medicaid claims and crossover claims that originated under fee-for-service plans. For the period of study, almost all NC Medicaid was under fee for service, with only 1 small managed care organization in Mecklenberg County that covered approximately 10,000 lives (men, women, and children). During 1998 and 1999, paid Medicare claims were &#8220;crossed over&#8221; to the NC Medicaid claims processing contractor by tape and were listed in the study database regardless of the source of reimbursement. We used the date of cancer diagnosis and the NC Medicaid eligibility file to determine continuous eligibility; for our sample, we selected all patients who were enrolled at least 1 month before the date of diagnosis and who had 12 months of continuous enrollment in NC Medicaid. This procedure ensured that the cancer care of study subjects could be observed in our dataset. 

To estimate the accuracy of the archived consolidated cancer registry RT data, we performed hospital record audits when there was a lack of agreement between cancer registry and Medicaid claims data for RT among 288 patients with BC S who were continuously enrolled in Medicaid for ~12 months following the date of diagnosis. The latter enrollment requirement ensured that Medicaid claims would be present during the treatment period pertinent to the review of registry data. The audit process included accessing updated consolidated cancer registry files for new treatment field entries and field notes to capture additional details not included in standard data releases. Patients with multiple source documents (cases reported by >1 facility) were placed on reconcile hold for side-by-side review of cases, including codes and text. For records in which updated information was not present, or if the discrepancy between data sources persisted after reviewing updated information, staff from the hospital registry office were contacted to provide supplemental information. This included reviewing the original file and following up with physician notes and freestanding RT facilities to clarify if a procedure or treatment was given. Only notes documented in the registry or medical records were reported back to the registry staff. Edits and recodes resulting from this review were used to recalculate percentage agreement per cancer care service category. Based on the results of this quality review, which showed high specificity and sensitivity of the study&#8217;s adjuvant RT data (results presented herein), the entire sample of 344 patients listed as having BC S was included in this study, using Medicaid claims for RT to substitute for registry data that showed that RT was not planned or administered or that the treatment status was unknown.

Of 288 patients with BCS having continuous Medicaid eligibility, 33 (11.5%) showed lack of agreement between Medicaid claims and the CCR about having any receipt of adjuvant RT within 1 year following diagnosis. Review of hospital registrar information demonstrated that 94% of these discrepant cases had no registry RT information when Medicaid RT claims were present; 2 cases had no RT claims when the CCR indicated that RT was given. Based on data obtained from rereview and updated fields as the gold standard, the sensitivity of the original CCR RT data was 84%, and the specificity was 100%. Medicaid claims RT information had a sensitivity of 95% and a specificity of 93%. The combined CCR&#8211;Medicaid claims dataset had a sensitivity of 97% and a specificity of 98% when information indicating adjuvant RT was unilaterally accepted from either source. In a logistic model, predictors (

 <.05) of the odds of having matching RT information in the 1-year period between data sources were the following: having cancer treatment provided by more than 1 medical facility, being treated at a facility with a hospital registrar, and having fewer days until RT after surgery (data not shown). Based on these results, for all 344 cases we took the approach of using CCR data as the primary source of RT data and used Medicaid  data to classify the RT status of CCR patients for whom RT was missing, not given, or unknown.

Characteristics of the study population are given in 

. The mean age was 62.4 years (age range, 23-102 years), with 46.2% being 65 years or older; 49.1% were listed as being of nonwhite race/ethnicity, with 47.0% listed as being of black race/ethnicity. Most subjects (68.3%) had localized disease; approximately 11.0% and 20.6% had in situ and regional disease, respectively. None of the patients were listed as having unstaged disease. More than half (61.3%) had a tumor size of less than 2 cm, and 19.5% had 1 or more lymph nodes removed. 

The results of the present study indicate that, after adjusting for all model covariates, women managed with BCS are at higher risk of not receiving adjuvant RT if they are 65 years or older, were surgically treated at small hospitals, or reside in group homes or assisted-living facilities. This builds on previous work

 and on the larger literature about patterns of breast cancer care among underserved populations

 by revealing that, like barriers to diagnostic follow-up of mammography,

 barriers to the receipt of adjuvant RT following BCS are multilevel, including individual (age), residential location (rural), and healthcare system (hospital size and patient volume) inputs. This is important because it is well established that omission of RT increases the risk for tumor recurrences following BC S (1990 National Institutes of Health 1990 Consensus Conference

). Although randomized clinical trials have not found a benefit of increased survival with RT, recent population and cohort studies

 show poorer longer-term survival for patients without RT. We hypothesize that the pattern of results for RT use found in this study reflects the larger issue of quality of care (eg, through care coordination processes among healthcare organizations) and a varying availability of resources focused on seamless comprehensive cancer care. The latter is a process that leads with patient education and assistance in decision making about the choice of surgery through the facilitation of a patient&#8217;s navigation of postsurgical care and secondary preventive care. Smaller hospitals with lower cancer patient volumes typically have fewer resources dedicated specifically to the care coordination of patients with cancer than larger comprehensive cancer facilities, where an array of services across the continuum of cancer care can be integrated within a single information system or community region. However, future studies are needed to directly test this hypothesis and to suggest interventions to improve the rate of access to required treatment among low-income populations. For example, Bickell and colleagues

 in a physician recall survey study of 119 women surgically treated for breast cancer 4 years earlier who did not receive standard adjuvant therapies found that adjuvant treatment was underused when surgical care was delivered at a municipal hospital and was associated with a pattern of referral to a clinic rather than to a specifically named oncologist. In addition, system failures were more common among Medicaid-enrolled patients than among Medicare-enrolled patients. Schrag et al

 reported that enrollment in Medicaid is associated with greater fragmentation of hospital care among chronically ill New York City residents than enrollment in other insured groups. This suggests that Medicaid recipients are vulnerable to disjointed care or disruptions in care. Transportation, also a likely barrier in Medicaid populations, has been shown to influence the receipt of RT through association with the distance that a patient must travel to receive care.

 Small hospitals in nonmetropolitan counties likely have fewer resources on site and may require their patients undergoing BCS to travel to other facilities for RT.

The association of age as a barrier to adjuvant RT following BCS reflects the larger controversy about optimal management of breast cancer in older women and is consistent with the reported underuse of RT in SEER data.

 However, there is evidence that older women, despite having fewer survival years remaining, may benefit from RT alone

 by a reduction in risk for local disease recurrence. Finally, the finding that women listed in the Medicaid enrollment files as living in a group home or assisted-living facility were at risk for omission of RT after adjusting for all model covariates is intriguing and merits further study. Bradley et al

 studied nursing home patients insured under Michigan Medicaid and found underuse of cancer services in this population. Patients in assisted-living facilities are likely frail and may be deemed likely to succumb to competing causes of death. However, these patients are not autonomous, and some may be subjected to lack of follow-up care through deficiencies in care coordination that may be present in some facilities. 

Certain limitations of this study deserve to be noted. We did not have a data source independent of hospital registrar data in this study (ie, independent reabstraction of medical record data); therefore, the extent of false-positive listings in the CCR or in claims could not be assessed. However, Cress et al

 examined the correspondence of registry data to physician medical records and found 2% disagreement with registry information. Similarly, Malin et al

 found that the medical record is generally positive when the registry says that no treatment was delivered. Another limitation is that our results pertain to cases of primary breast cancer identified in the NC CCR in 1998 and 1999 and might not be generalizable to systems in other states or to time frames that vary from ours in the degree of data completeness. The NC registry system has received &#8220;gold&#8221; and &#8220;silver&#8221; ratings on the quality of its 1998 and 1999 data by the North American Association of Central Cancer Registries based on case ascertainment and has received silver ratings based on case completeness. North Carolina Medicaid policy required crossing over of all Medicare claims to NC Medicaid for determination of coinsurance payment of deductibles and of noncovered charges associated with a treatment or procedure. States that do not require this level of detail will likely lack claims for the dually insured; therefore, Medicare data will be necessary to fill this gap. Finally, we did not obtain data from other insurance plans; therefore, we cannot conclude that Medicaid-enrolled patients treated with BC S are at a greater disadvantage than non&#8211;Medicaid-enrolled patients in accessing adjuvant RT . There is other evidence to suggest that Medicaid-enrolled patients are less likely to receive guideline-concordant therapy. In a study of SEER data, Harlan et al

 found that among patients with cancer diagnosed from 1995 through 1999, non-Hispanic blacks with Medicaid coverage had significantly lower rates of guideline-concordant therapy compared with non-Hispanic blacks having Medicare coverage only, private insurance, or no insurance.

In conclusion, we find that factors related to geographic isolation and to scarcity of healthcare specialists are related to omission of adjuvant RT after BCS in this Medicaid population characterized by low-income status. Because RT is a key component of local definitive therapy for breast cancer, its omission may adversely affect survival outcomes. As such, this work provides potential target interventions to improve care among Medicaid-enrolled and low-income populations to help us work toward eliminating disparities in cancer outcomes.

 Department of Public Health Services, Pennsylvania State University (RT A, FC), Hershey; Duke University Medical Center (GGK), Durham, State Center for Health Statistics (JTW) and Division of Public Health (CD), North Carolina Department of Health and Human Services, Raleigh, and Health Sciences Center, Wake Forest University (EAL, FMT), Winston-Salem; and School of Public Health, Ohio State University (RB), Columbus.

 This study was supported by grant RSGT-04-005-01 from the American Cancer Society.

 The authors (RT A, GGK, FC, JTW, CD, EAL, FMT, RB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. 

 Concept and design (RT A, GGK, JTW, EAL, FMT, RB); acquisition of data (RT A, GGK, JTW, CD, EAL); analysis and interpretation of data (GGK, FC, JTW, RB ); drafting of the manuscript (RT A, GGK, RB); critical revision of the manuscript for important intellectual content (RT A, GGK, EAL, FMT, RB); statistical analysis (FC, RB); obtaining funding (RT A); administrative, technical, or logistic support (CD, FMT); supervision (RT A) and quality control (CD).

 Gretchen G. Kimmick, MD, MS, Duke University Medical Center, Ste 3800 Duke S, Box 3204, Durham, NC 27710. E-mail: 

1. Jemal A, Murray T, Samuals A, Ghafoor A, Ward E, Thun MJ.

 Treatment of early-stage breast cancer. 

Treatment of early-stage breast cancer: June 18-21, 1990. 8th ed. Bethesda, MD: National Institutes of Health; 1990.

7. Fisher ER, Anderson S, Redmond C, Fisher B. 

Pathologic findings from the National Surgical Adjuvant Breast Project protocol B-06: 10-year pathologic and clinical prognostic discriminants. 

Postoperative radiotherapy for breast cancer: growing evidence for an impact on survival. 

11. Vinh-Hung V, Voordeckers M, Van de Steene J, Soete G, Lamote J, Storme G.

 Omission of radiotherapy after breast-conserving surgery: survival impact and time trends. 

13. Kimmick G, Camacho F, Foley KL, Levine EA, Balkrishnan R, Anderson R.

 Racial differences in patterns of care among Medicaid-enrolled breast cancer patients. 

15. Athas WF, Adams-Cameron M, Hunt WC, Amir-Fazli A, Key CR.

 Travel distance to radiation therapy and receipt of radiotherapy following breast-conserving surgery. 

17. Malin JL, Kwan L, Adams J, Laouri M, Gantz PA.

 Accuracy of cancer registry data when treatment is in the ambulatory setting: implications for quality measurement. 

 Cancer treatment policies and their effects on survival: female breast, 1986-1994. Key Sites Study 6. Leeds: Northern and Yorkshire Cancer Registry and Information Service. 2001. 

http://www.nycris.org.uk/reports/Nycris_publications_1995-2007.pdf

21. Bickell NA, Wang JJ, Oluwole S, et al.

 Missed opportunities: racial disparities in adjuvant breast cancer treatment. 

23. Chan A, Pintilie M, Vallis K, Girourd C, Goss P.

 Breast cancer in women <35 years: review of 1002 cases from a single institution. 

25. Schrag D, Xu F, Hanger M, Elkin E, Bickell NA, Bach PB.

 Fragmentation of care for frequently hospitalized urban residents. 

www.cms.hhs.gov/HPSAPSAPhysicianBonuses/downloads/specialtyPSA

 counties.zip. Accessed September 22, 2008.

Series IV: Cancer Case Ascertainment. Procedure Guidelines for Cancer Registries

. Springfield, IL: North American Association of Central Cancer Registries; February 2002.

Rural-Urban Continuum Codes for Metro and Nonmetro Counties, 1993. 

Beltsville, MD: Agriculture and Rural Economy Division, Economic Research Service, US Dept of Agriculture; 1994. Staff report 9425 (September).

33. Hershman D, McBride R, Jacobson J, et al.

 Racial disparities in treatment and survival among women with early-stage breast cancer. 

35. Kaplan CP, Crane LA, Stewart S, Juarez-Reyes M.

 Factors affecting follow-up among low-income women with breast abnormalities. 

37. Ballard-Barbush R, Potosky AL, Harlan LC, Nayfield SG, Kessler LG.

 Factors associated with surgical and radiation therapy for early stage breast cancer in older women. 

 Ten-year results of the treatment of early-stage breast carcinoma in elderly women using breast-conserving surgery and definitive breast irradiation. 

 Absence of cancer diagnosis and treatment in elderly Medicaid-insured nursing home residents. 

43. Harlan LC, Greene AL, Clegg LX, Mooney M, Stevens JL, Brown ML.

 Insurance status and the use of guideline therapy in the treatment of selected cancers. 

Articles

Omission of radiation therapy after breast-conserving surgery leads to poor outcomes. Geographic isolation and scarcity of healthcare specialists correlate with low adjuvant radiation therapy use.

Health System Correlates of Receipt of Radiation Therapy After Breast-Conserving Surgery: A Study of Low-Income Medicaid-Enrolled Women

 To determine whether classification tree techniques used on survey data collected at enrollment from older adults in a Medicare HMO could predict the likelihood of an individual being in a low, medium, high, or very high cost group in the subsequent year. 

 Data from comprehensive health risk assessment (HRA) screening of 11 744 new enrollees (age &#8805;65 years) continuously enrolled in a Medicare HMO for at least 24 months between 1997 and 2002 were combined with complete healthcare service utilization data for each enrollee in the postenrollment period to create cost groups. An original clinically devised algorithm and a Classification and Regression Tree (CART) that used the HRA data were compared with respect to their ability to correctly place an individual within the cost groups over 12 months of enrollment. 

 The variables that best classified enrollees into 12-month cost groups included quality of life, age, and preenrollment health services. Classification was best for CART: a sensitivity of 39.8% and a specificity of 83% was achieved for very high cost enrollees. Compared with the clinical algorithm, CART also utilized only one third as many predictor variables to define risk categories. 

 Brief self-report survey data collected at enrollment can be modeled to provide moderate sensitivity in predicting very high costs in the postenrollment year. The ease of applicability of software to create empirically derived profiles of high-cost enrollees precludes the need to rely on clinically devised algorithms to identify enrollees at risk for very high costs. 

Providing timely healthcare to an aging population is of increasing concern for health policymakers in the United States. Persons who survive to age 65 years today have a life expectancy of nearly 18 more years. A large number will require care for chronic diseases such as arthritis, hypertension, diabetes, and chronic obstructive pulmonary disease. These highly prevalent conditions among noninstitutionalized older adults (age &#8805;65 years) account for a large portion of healthcare expenditures for this age group, and contribute to an overall higher rate of consumption of healthcare resources among this age group than among any other.

In 1996, the average older adult had nearly 12 annual physician visits and 6.5 days of annual hospital stay, and consumed 36% of all hospital stays and 49% of all hospital days.

 At the same time, managed care enrollment of older adults, which grew nearly 4-fold in the decade from 1989 to 1999, has experienced a reversing trend, with more than 2 million enrollees involuntarily disenrolled from Medicare managed care plans that withdrew from the market because of financial constraints.

 This trend brings into renewed focus the need to cost-effectively provide care by identifying high-risk older adults and providing interventions to eliminate potentially avoidable medical outcomes and associated resource utilization. 

Because the health and well-being of older adults depend on both access to and timing of components and outcomes associated with care,

care management efforts increasingly rely on health risk assessments to characterize enrollees' healthcare needs and the likelihood of adverse medical outcomes (eg, hospitalization, emergency care). Traditionally, "risk" has been defined in the context of becoming a high-cost user of medical care, or the propensity of enrollees to consume healthcare resources in a particular time period (eg, on an annual basis), based on total payments or billed costs.

Individuals deemed to be at "high risk" may be referred to appropriate disease risk management programs.

Although it is congressionally mandated that all the Medicare health maintenance organizations (HMOs) in the country conduct risk screening, there is little guidance on which strategy of identifying risk is most effective and beneficial to enrollees. A practical challenge is implementing a valid and reliable risk assessment process early in the enrollee's health plan tenure, when it can be most useful. Whereas claims-based strategies and clinician assessment require sufficient time in-plan to accrue patient information,

 self-report information supports early risk classification as it enables data collection near time of enrollment and allows elements such as inclusion of disability, self-rated health, and behavioral health variables known to predict service utilization

 that are not routinely recorded in medical information systems. Potential disadvantages of self-report information are possible low sensitivity of self-reported diagnosed and treated conditions,

and imprecise collection of information on disease severity. 

Given the potential value of self-report screening data, a critical challenge is constructing an algorithm from the output of statistical models that maximizes sensitivity and specificity of the cost group classification. Most classification studies in the literature do not involve cost groups, but have relied on basic regression techniques to identify predictors of service utilization. 

Although these procedures are sound and valuable tools for deriving a "best set of predictors" of the outcome and percent variation explained, the model parameters that relate to probability do not easily lend themselves to risk categorization because they provide no information on the accuracy of prediction of cost groups in terms of sensitivity and specificity, and because they may not address complex interactions among variables. As a result there is considerable uncertainty over the validity and precision of converting regression weighted scores into decision rules to classify new members into actionable risk-propensity groups. 

This study sought to develop and test a risk classification tool for use at the time of enrollment based on its ability to correctly classify individuals into cost groups. Subjects were older adults enrolled in a Medicare managed care plan in the southeastern region of the United States. 

A prospective cohort was drawn from 11 744 newly enrolled members in a Medicare HMO who completed the comprehensive health risk assessment battery on entry into the plan and for whom complete healthcare service utilization data were available for at least 2 years postenrollment. Claims data were use to follow this cohort annually for 2 years postenrollment. The health plan was a Medicare managed care plan in the southeastern region of United States, and was the sole provider of medical care to enrollees ("lock-in" risk benefit plan). For the purposes of this analysis, we included only subjects who had not died, moved away, or switched to a fee-for-service plan during the study follow-up period. 

Each enrollee was sent a mailed questionnaire that asked questions about demographics, clinical conditions, health service utilization in the year before joining the plan, lifestyle and health behaviors, depression, disability, and health-related quality of life. Since its inception in 1996, the response rate for this questionnaire has been more than 80%. Information from the survey data was then used to derive risk profiles of patients based on a clinician-derived algorithm, which was then provided to case managers for disease state management of high-risk patients. 

Demographic characteristics and health status of this population are shown in 

. Subjects were more likely to be female (58.7%) and had an average age of 75.1 years on enrollment. About 75% rated their general health as good to excellent, while 6% reported their health worsening in the last year (pre-enrollment year). 

The comprehensive risk assessment tool included self-reported information on conditions known ever to have been diagnosed (heart problems, stroke, chest pain, lung problems, asthma, arthritis, diabetes, poor circulation, hypertension, cancer, bladder problems, stomach/bowel problems, other); lifestyle risk (eg, smoking status, physical activity, alcohol), as adapted from the Centers for Disease Control and Prevention Behavioral Risk Factor Surveillance System; general health (the Medical Outcomes Survey 12-Item Short-Form Health Survey [SF-12])

; functional status, assessed as impairments in instrumental activities of daily living (IADLs) and activities of daily living (ADLs) as determined by the Duke Older Americans Resources and Services (OARS) screening method

; depression (Center for Epidemiological Studies Depression scale [CES-D], short form)

; number of falls in the last year; current medication status (number and conditions treated); and use of healthcare services during the last year. This battery of measures represents key aspects of health considered to indicate need or use of health services. 

The responses to the patient questionnaire formed the basis for the demographic and health status variables in the study and are shown in 

. The demographic variables were age, sex, and living alone (as opposed to with others, including caregivers, spouse, relatives, etc). The health status variables were indicators for sedentary status (physical activity and walking for at least 30 minutes a week), falls during the previous year, smoking status, alcohol consumption, presence of depression (as indicated by a score of .06 on the CES-D short form), number of difficulties in ADLs and IADLs, number of conditions not treated before enrollment, number of comorbid conditions, perception of health status, and the physical and mental subscore components of the SF-12 questionnaire. Healthcare utilization was determined from the number of annual hospitalizations, emergency department visits, and healthcare payments in the 2-year postenrollment period; this information was obtained from enrollee claims data collected by the HMO. Total healthcare payments were obtained from administrative claims records and consisted of all recorded healthcare service utilization and prescription payments. We utilized claims paid by the HMO to calculate the costs associated with each enrollee. 

The mean payment per member per month [pmpm], unadjusted for period inflation, was $296. The cost levels were created using the following split of the distribution of total healthcare costs for 2 years postenrollment: 10% (very high), 20% (high), 30% (medium), and 40% (low). These cutoffs were chosen to produce estimates of cost risk most relevant to Medicare HMO risk-related decision making, based on consultation with Medicare HMO policymakers as well as previously conducted studies.

 Enrollees were classified as low cost if they had less than $75 pmpm postenrollment total payments, medium cost if they had payments ranging from $75 to $270 pmpm, high cost if they had payments ranging from $271 to $792 pmpm, and very high cost if they had payments equal to or more than $793 pmpm. 

We focused on 3 methods to classify patients into the different cost levels specified above. The first method involved applying an original cost estimation algorithm proposed by clinicians and experts in the field (see 

 for a description). The second approach involved using Salford Systems' Classification and Regression Tree (CART) software.

 CART analysis is a nonparametric technique ideally suited to identify patterns or "groups" in complex data. It has been used in aging research,

 when disability group status was of primary interest,

 and by medical research to develop treatment decision rules for clinicians

; thus, it has the potential for a more general application to managed care risk models. For comparison purposes, the third method involved assigning cost levels at random based on the distribution of cost levels. 

Candidate CARTs were first built, evaluated, and compared with each other using a set of data consisting of two thirds of the available sample (the training set); the best tree was then further evaluated for its actual performance using the remaining data (referred to in the literature as the test set).

 With this approach, we could select the best-performing tree without bias and judge its true performance with new data. 

CART and other tree-structured classifiers attempt to recursively partition observations into groups to maximize the classification rate.

The CART decision tree identifies groups in the optimal partition produced by the algorithm by asking a series of hierarchical, binary, Yes/No questions, and is easy to depict, as can be seen in 

. In medical diagnosis situations, binary trees produced by CART tend to be smaller and have slightly lower error rates than multiway trees.

The detailed methodology involved in developing the most appropriate CART can be found on the developers' Web page.

The CART algorithms used for this study is available from the authors upon request. 

How unknown values are treated may affect any classification algorithm used for HRA data, because sample observations often contain missing values. The largest proportion of missing values in our data was 13% for the SF-12 Physical Component Subscore. Missing values may be easier to manage with decision trees than they are with other classification methods.

 The tree-building algorithm in the Salford System CART software uses a method of "surrogate" variables, in which observations with missing predictor variables used in the tree are not dropped from the analysis; instead, the missing predictor is replaced by a proxy variable best approximating its performance.

 present data on the characteristics of the sample. Table 2 and Table 3 break down the descriptive statistics in the population by the categories of risk based on the original clinical algorithm and the CART, respectively. Table 4 lists potential predictors considered in the various cost-based risk models. The original risk estimation algorithm and the CART decision tree are shown in Figures 1 and 2, respectively. 

Results from the comparisons of the 3 classification approaches are shown in 

. The overall classification rates show the proportion of enrollees who were correctly classified in their subsequent 12-month cost group, based on the number of enrollees actually classified in each group. There was a significant difference between the classification rate of the CART and that of the original algorithm (McNemar test 

< .0001, with the CART improving overall classification by 7.1 percentage points), as well as a significant difference between the classification rates of the CART and the random classifier (

< .0001, with the CART improving overall classification by 6.8 percentage points). However, there was no significant difference between the rates of the original (clinician-based) algorithm and the random classifier (

 shows the sensitivity, specificity, and positive predictive values of each classifier. The probability of correct classification among high-cost patients is highest for the CART (39.8%). For enrollees assigned to the very high cost group, the CART also had the highest positive predictive value (20.4%), although this varied by only 1.2 percentage points from the original, clinician-based algorithm (19.2%). The CART had lower sensitivity than both the original algorithm and the random classifier for classifying enrollees in the middle range (high and medium) cost groups (Table 3) and was slightly more sensitive than the random classifier for enrollees in the high-cost group (21.9% vs 19.0%). In contrast, the original clinician-based algorithm showed distinctly lower sensitivity for the very high risk cost group and the low-cost group compared with the CART. 

To increase the sensitivity of the CART for the middlecost groups, we instructed the tree to modify the classification of "very high" to "high" risk if there was evidence of self-reported physical activity or of nonsmoking status. The modifications are described in Figure 2, and the performance of this classifier also is described in Tables 5 and 6. In general, we observed that the performance of this modified classifier was indistinguishable from that of the original tree, except that the sensitivity of the model for the medium-cost level increased by 3.7 percentage points, at the expense of a decrease of 6.5 percentage points at the very high cost level. 

In the literature, various methodologic approaches to risk assessment have been proposed.

Medical and claims-based information is sometimes viewed as providing more details regarding medical need, but these data are rarely available at the time of plan enrollment. To satisfy the need to identify subsets of newly enrolled patients who may require timely and focused care to avert negative health events, and to ensure that a new enrollee's needs do not go unrecognized in the transition from one plan to another, self-reported survey information may be a valuable option. It is well known that survey data have value in predicting cost outcomes.

We focused on total payments for new enrollees of a Medicare managed care plan and included a comprehensive assessment of comorbidity and lifestyle risk factors to select our models. More importantly, we compared classification methods on their ability to correctly classify enrollees into subsequent 12-month cost groups. We know of no other reports in the literature that investigated this essential property of classification. We used CART models as an established method for group classification and sought to extend their utility in predicting specific health services utilization groups relevant to clinical operations.

The significant variables predicting high-cost risk differed between the original clinical algorithm and the CART. The CART predictors were a parsimonious set of 4 predictor variables (age, self-reported physician visits, hospitalization in the preenrollment year, and the SF-12 Physical Component Subscore), in contrast to the 8 complex categories (with 12 predictor variables) used to define high risk in the clinical algorithm. 

Most regression-based studies of health services utilization in the literature have ended up selecting variable sets based on significant correlations with healthcare costs and categorical outcomes associated with morbidity and high-cost risk (eg, hospitalizations).

Although this work is valuable in identifying the nature of exposures and conditions that lead to utilization, the questions of how well these selected variables can classify individuals correctly into cost groups and whether risk scores produced from regression weights apply across the continuum of cost or utilization were not addressed. Advantages of CART for assessing risk include its ability to identify risk groups that lend themselves to decision rules, its ability to handle potentially complex interactions among predictor variables, the avoidance of model errors due to highly skewed data, and its relative simplicity, enabling interpretation by clinicians or nonstatisticians.

Our CART classification models were contrasted with a model of chance and with expert opinion to enable relative comparisons to be made across defined healthcare cost groups. The results were complex. The data-derived model (based on CART) appeared to be superior in sensitivity for extremes of very high cost and low cost, both of which are most important to Medicare HMO decision makers and care planners. For high-utilization risk classification, we obtained superior results using only 5 predictors compared with a more subjective methodology based on clinician judgment. Unlike regression models that yield parameter estimates, the decision tree results, once tabulated and depicted, are simple to interpret and use. It is noteworthy that empirically derived CART models can be obtained for care management decision rules by applying readily available software to enrollee risk survey data. The software, while technical in origin, is easy to use and may be operated by those trained to perform regression analyses.

There are some important limitations in the study. First, we tested a limited set of classification approaches that we selected variously based on their use in related fields: a straightforward decision-tree format of the results and a more traditional classification approach based on clinical judgment and impression. There may be other classification models superior to those tested here, such as propensity scores

and discriminant analytic models. We are not aware of an appreciable literature on classification of risk for healthcare services utilization, and our study was designed to test a leading potential approach. Another limitation is the potential for idiosyncratic policy and practice issues in the study HMO that could have influenced use of services&#150;either blunting or biasing our risk estimates. Our data came from 1 HMO, and thus are not sufficiently diverse to represent variations in policy and referral encountered elsewhere. Our population of enrollees may not have been typical of other HMO populations. However, we are not aware of unique or local policy or administrative issues that could have biased our results. 

The sample size for our study also was limited, but other risk assessment studies used similar numbers of patients to illustrate the effectiveness of various risk assessment approaches. 

The results of our illustrative study, however, have to be validated with other populations before a definitive assessment of the general applicability of classification approaches like CART can be made. The self-report of some of the risk screen variables also could introduce an element of respondent bias. For example, we found that nearly three quarters of our sample reported their health to be good or excellent. This unusually positive response may be in part because these are ambulatory older adults, and the risk screens were completed at the time of enrollment. Thus, there may have been a positive response bias in subjects who feared losing coverage if they reported their health status otherwise. In support of our sample, we found the prevalence of comorbidities and the disability status of our population to be highly similar to national estimates for older adults reported in the National Health Interview survey.

We also examined the mean annualized payments of enrollees who did not return a survey and found them to be highly similar to those of enrollees who did return a survey. Certain limitations exist because our sample size of approximately 12 000 enrollees could be considered small, especially when dealing with relatively rare events that have very high costs. Finally, our study examined total healthcare utilization and payments, as opposed to examining disease-specific utilization and payments. This was done to avoid problems inherent in attributing all billings for healthcare service utilization and payments to a specific condition due to miscoding of the diagnosis. These limitations do not, in our view, undermine the validity of the potential benefits of our study of alternative risk classification methodologies such as CART for comprehensive risk classification of older adults. 

We found that 5 self-reported health status variables from newly enrolled Medicare managed care enrollees can be used to classify cost groups for use in disease management and risk reduction efforts of HMOs. Clarification and regression tree analysis can be easily applied to health risk assessment data, thus providing HMO decision makers with more sensitive estimates of cost risk than those based on clinically derived risk algorithms. This method offers an important early opportunity to gather health information on enrollees to aid in risk management, needs surveillance, and interventions to improve care. 

From the Section on Social Sciences and Health Policy, Department of Public Health Sciences, Wake Forest University Health Sciences, Winston-Salem, NC (RTA, FC); and the Department of Management and Policy Sciences, University of Texas School of Public Health, Houston, Tex (RB). 

This study was funded by an intramural grant from the Wake Forest University, Winston-Salem, NC. 

Presented as a poster at the Academy for Health Services Research and Health Policy Annual Meeting, Washington, DC, June 23-25, 2002. Address correspondence to: Roger T. Anderson, PhD, Section on Social Sciences and Health Policy, Department of Public Health Sciences, Wake Forest University Health Sciences, 2000 West First Street, Piedmont Plaza II, 2nd Floor, Winston-Salem, NC 27104. E-mail: 

 Health status of Medicare enrollees in HMOs and fee-for-service in 1994. 

2. Garfinkel SA, Riley GF, Iannacchione VG. 

. Washington, DC: Mathematica Policy Research Inc; January 2002. Available at: http://www.mathematica-mpr.com/PDFs/redirect_PubsDB. asp?strSite=fastfacts7.pdf. Accessed October 2, 2003.

4. American Association of Retired Persons.

Trends and Indicators in the Changing Health Care

. Marketplace: A Chartbook. Menlo Park, Calif: The Henry J. Kaiser Family Foundation; 2002.

Roger T. Anderson, PhD

Articles

Health System Correlates of Receipt of Radiation Therapy After Breast-Conserving Surgery: A Study of Low-Income Medicaid-Enrolled Women

Risk Classification of Medicare HMO Enrollee Cost Levels Using a Decision-Tree Approach

Effect of Prescription Benefit Changes on Medical Care Utilization in a Medicare HMO Population

Predictive Power of a Risk-Assessment Questionnaire Across Different Disease States: Results in an Elderly Managed Care Enrolled Population