The American Journal of Managed Care March 2011
Fracture Risk Tool Validation in an Integrated Healthcare Delivery System
Objective: To evaluate the utility of the Fracture Risk Calculator (FRC, Foundation for Osteoporosis Research and Education) for predicting 10-year hip fracture risk within a “real world” population.
Study Design: Retrospective cohort study.
Methods: We identified female members of Kaiser Permanente Northern California aged ≥50 years with bone mineral density (BMD) measured during 1997-2003. Hospitalization for hip fracture was ascertained up to 10 years following the BMD date, and 10-year observed hip fracture probabilities were calculated. Baseline data for fracture risk calculation were extracted from health plan databases, including age, race/ethnicity, smoking, body mass index, prior fracture, rheumatoid arthritis, glucocorticoid use, disorders associated with bone loss, and femoral neck BMD. Predicted 10-year FRC hip fracture probabilities were compared with observed 10-year hip fracture probabilities.
Results: Among 94,489 women (mean age 62.8 ± 8.6 years, average femoral neck Z-score 0.1), the median duration of follow-up was 6.6 years, during which 1579 (1.7%) hip fractures occurred. Using the FRC, 23% met or exceeded the National Osteoporosis Foundation’s 3% hip fracture threshold. The FRC somewhat underestimated observed hip fracture probabilities; across 10-year risk categories <1%, 1% to 2.9%, and 3% to 4.9%, ratios of observed to median predicted probabilities ranged from 1.3 to 1.4.
Conclusions: The FRC tool can be applied to assess fracture risk in large populations using data from administrative databases. Despite some underestimation, this relatively simple tool may assist targeting of at-risk populations for more complete fracture risk assessment.
(Am J Manag Care. 2011;17(3):188-194)
The utility of the Fracture Risk Calculator (FRC) for predicting 10-year hip fracture risk within a “real world” population was evaluated.
- The FRC provided rapid assessment of population fracture risk with some underestimation.
- Bone mineral density as an input parameter appeared to have little overall effect on the tool’s discrimination.
- While the FRC underestimated observed 10-year hip fracture probabilities by 30% to 40%, the ability to rapidly assess fracture risk using population data may be useful for osteoporosis programs in the initial identification of high-risk patients.
Several Web-based fracture risk tools are available,5-8 but these are designed for entry of individual patient-level information and currently do not function in an easily accessible, large-scale, batch mode where data on thousands of individuals can be instantly uploaded for fracture risk estimation. The most widely used fracture risk tool is FRAX, developed by the World Health Organization Collaborating Centre for Metabolic Bone Disease, University of Sheffield, UK.6,9 This tool first solicits information on a number of key, independent clinical risk factors and then calculates 10-year probabilities of hip and of any 1 of 4 major osteoporotic fractures. While some of these risk variables are easily obtained from administrative data sets (eg, age, sex), others may be obtained with some effort (eg, race/ethnicity, height and weight, prior fracture, high-dose glucocorticoid exposure, history of rheumatoid arthritis, other conditions known to contribute to osteoporosis, current smoking). Certain variables (eg, family history of osteoporosis, heavy alcohol use) may not be obtained without direct patient questioning. In addition, although many patients have undergone bone mineral density (BMD) testing, numeric results may not be easily accessible; fortunately, fracture risk tools can produce results with or without BMD. Indeed, fracture risk tools can provide accurate probability estimates with less than complete input data.10 Especially for the purpose of population categorization and targeting osteoporosis outreach programs, results based on a limited data input tool would be considered adequate.
In this study, we modified an existing Web-based fracture risk tool (the Fracture Risk Calculator [FRC])7 to provide batch outputs using input values obtained from administrative data in a large population of women undergoing BMD testing. We evaluated the performance of the FRC batch tool against observed 10-year hip fracture rates. The FRC tool was chosen because it was highly accessible, efficient, and transparent. Furthermore, the FRC batch estimates were provided at no cost, and results were instantly available via a Web interface. We compared observed 10-year fracture rate estimates with average predicted rates in more than 90,000 female members over age 50 years who received a BMD test in Kaiser Permanente Northern California (KPNC) between 1997 and 2003. Our primary goals were to determine what proportion met or exceeded the National Osteoporosis Foundation (NOF) 3% 10-year hip fracture risk threshold4,11 and to assess whether the tool overestimated or underestimated the true risk of hip fracture.
Kaiser Permanente Northern California is a large integrated healthcare delivery system serving more than 3 million members in Northern California; approximately 0.5 million are women over the age of 50 years. Since 1991, bone densitometry has been available using Hologic dual-energy x-ray absorptiometry (DXA) scanners (Waltham, MA). By 1995, KPNC had a fully integrated, systemwide patient data collection system that included ambulatory visit diagnoses, radiologic records, and prescription drugs that complemented long-standing hospitalization databases. Therefore, we selected as our study cohort all women aged 50 to 85 years who underwent a hip bone density scan on a Hologic scanner (models QDR 2000, 4500, or Delphi), selecting the first scan during 1997-2003. We excluded those who did not have at least 1 year of continuous (<90-day gap) membership both prior to and following the DXA scan date, those for whom DXA data were not electronically accessible, and those with missing race/ethnicity. We also excluded women who had filled a prescription for a bisphosphonate in the year prior to the DXA test. The study was approved by the Institutional Review Board of KPNC.
Fracture Risk Calculator
The fracture model estimate began with population-based 10-year fracture probability for age, sex, and race/ethnicity. Next, specific patient characteristics were compared with those of the base population, and relative risks were applied to factors that differed between the individual patient and the base population. In very simplistic terms, the product of base rate times the risk differences yielded the predicted absolute 10-year risk. The current base US 10-year fracture risks for men and for women used by the FRC are those calculated from the 2006 US National Inpatient Survey by Ettinger et al.12 The FRC model’s relative risks for various clinical risk factors are shown in Table 1. A detailed manual of operations for the Web-based FRC batch tool is available through the Foundation for Research and Education (www.fore.org).
Data Input Variables
Age, race/ethnicity, and body mass index (BMI) were determined at the index BMD scan date. Those with missing BMI were assigned a null value of 25 kg/m2, which was the median value in our cohort. Using ambulatory care, hospitalization, and pharmacy databases, we obtained each patient’s data from the 1 year prior to the DXA measurement to secure the following exposures and diagnoses: glucocorticoid use ≥1825 mg of cumulative prednisone dose equivalent in the prior year (average 5 mg/day), rheumatoid arthritis diagnoses, and secondary causes of bone loss (diabetes mellitus with insulin use, malabsorption syndrome, chronic liver disease, and osteogenesis imperfecta). We determined prior history of fracture after age 45 years (up to 10 years prior to the DXA date) based on hospitalization and outpatient diagnoses of fracture (International Classification of Diseases,Ninth Revision [ICD-9] codes 800-829), excluding open fractures, fractures related to severe trauma (ICD-9 E-codes 800-845), and fractures of the fingers, toes, facial bones, and skull, since these are not generally considered to be osteoporotic. Information on alcohol consumption and parental history of hip fracture was not available, and smoking status was not uniformly available. We assumed that all missing input values were null.
Output From Fracture Tools
Under the auspices of institutional agreements for protection of proprietary interests, batch data without patient identifiers were securely uploaded to the Foundation for Osteoporosis Research and Education FRC Web site. Ten-year fracture probabilities were returned, calculated both with and without BMD inputs.
Follow-up and Observed Hip Fracture Outcome
Individual patient data were examined from the index BMD scan date until the earliest of the following: the fourth prescription for a bisphosphonate, likely indicating 1 year of exposure because typical prescriptions provide a 3-month supply (N = 19,440, 20.6%); a principal diagnosis of hip fracture (ICD-9 code 820.0X, 820.2X, and 820.8X, excluding open fractures and those associated with major trauma, ICD-9 E-codes 800- 845); death; disenrollment (>90-day gap in membership); or when 10 years had elapsed from the index date.
Differences between women with and without subsequent hip fracture were compared using the X2 test for categorical variables and the Student t test for continuous variables. Incident hip fracture rates and 95% confidence intervals (CIs) were calculated per 1000 person-years up to 10 years followup, the censoring date, or December 2009 (whichever occurred first). We used Kaplan Meier product-limit estimates to calculate the 10-year hip fracture probabilities from observed events over time and compared these with the median predicted 10- year hip fracture risk for the subcategories <1%, 1% to 2.9%, and 3% to 4.9%, as well as 5% to 6.9%, 7% to 9.9%, and >10%. We also compared the proportions of women in each decade of age meeting or exceeding the NOF’s 3% cost-effectiveness hip fracture risk threshold4,11 both with and without inclusion of BMD as an input parameter. We used the area under the receiver operating characteristic (ROC) curve (C statistic) to compare sensitivity and specificity of the FRC results, with and without BMD.13 The C statistic ranges from 0.5 (the predictions are no better than chance) to 1.0 (a perfect predictive model). Statistical analyses were performed using SAS version 9.1 (SAS Institute, Cary, NC) and Stata version 9.2 (Stata- Corp, College Station, TX).
Hologic BMD data were available for 94,489 women between the ages of 50 and 85 years, after excluding those not meeting health plan membership criteria (n = 19,178), those with missing race/ethnicity (n = 318), those with missing BMD data (n = 257), and those who received bisphosphonate drugs in the year prior to the index BMD date(n = 2730).
Table 2 shows the characteristics of the cohort at baseline. The mean femoral neck Z-score was 0.14, suggesting that the study population overall was comparable to the Hologic reference range (National Health and Nutrition Examination Survey III).14 The median duration of follow-up was 6.6 years (interquartile range 3.6-8.3 years), during which 1579 hip fractures were observed. Those who subsequently suffered a hip fracture were older, more likely to be white, had lower BMI, had other risk factors for osteoporosis, and were less likely to be users of hormone therapy. The mean age at hip fracture was 78 years and hip fracture incidence rates increased markedly with age (Table 3). The incidence of hip fracture in nonwhites (Asians, Blacks, and Hispanics) was approximately half the incidence in whites.