Risk Adjusting Community-acquired Pneumonia Hospital Outcomes Using Automated Databases

Using laboratory and administrative data, large managed care organizations can assign severity of illness scores to patients with pneumonia for risk adjustment and reporting.
Published Online: March 01, 2008
Gabriel J. Escobar, MD; Bruce H. Fireman, MA; Ted E. Palen, MD, PhD, MSPH; Marla N. Gardner, BA; Janelle Y. Lee, DrPH; Mark P. Clark, MD; and Patricia Kipnis, PhD

Objective: To describe the development and assessment of the Abbreviated Fine Severity Score (AFSS), a simplified version of the Pneumonia Severity Index (PSI) suitable for providing risk-adjusted reports to clinicians caring for patients hospitalized with communityacquired pneumonia.

Study Design: Retrospective cohort study.

Methods: We defined the AFSS based on data available in administrative and laboratory databases. We downloaded and linked these hospitalization and laboratory data from 2 cohorts (11,030 patients and 6147 patients) hospitalized with community-acquired pneumonia in all Kaiser Permanente Medical Care Program hospitals in northern California. We then assessed the relationship between the AFSS and mortality, length of stay, intensive care unit admission, and the use of assisted ventilation. Using logistic regression analysis, we assessed the performance of the AFSS and determined the area under the receiver operating characteristic curve (c statistic). Using a combination of manual and electronic medical record review, we compared the AFSS with the full PSI in 2 subsets of patients in northern California and Denver, Colorado, whose medical records were manually reviewed.

Results: The AFSS compares favorably with the PSI with respect to predicting mortality. It has good discrimination with respect to inhospital (c = 0.74) and 30-day (c = 0.75) mortality. It also correlates strongly with the PSI (r = 0.87 and r = 0.93 in the 2 medical record review subsets).

Conclusions: The AFSS can be used to provide clinically relevant risk-adjusted outcomes reports to clinicians in an integrated healthcare delivery system. It is possible to apply risk-adjustment methods from research settings to operational ones.

(Am J Manag Care. 2008;14(3):158-166)

Risk adjustment using physiologic data has been limited to intensive care unit admissions or to research studies.

It is possible for integrated healthcare delivery systems to conduct risk adjustment for hospitalized patients with community-acquired pneumonia.

The Abbreviated Fine Severity Score (AFSS) has good discrimination and has the advantage of incorporating laboratory results from automated databases. It compares well with the Pneumonia Severity Index, which requires manual medical record review.

The use of the AFSS is an intermediate strategy because the use of more complex severity scores will be possible once fully automated medical records are available. 

Risk adjustment using physiologic data has been limited to intensive care unit admissions or research studies.

More than 4 million cases of community-acquired pneumonia (CAP) occur in the United States each year, with 1.3 million patients hospitalized.1-3 CAP remains the leading infectious cause of mortality in the United States.4 Its incidence among older persons is 18.3 cases per 10005 and accounts for almost 7% of all US patient hospital costs.6,7 CAP is also the most common cause of severe sepsis.8

Given its importance, managed care organizations seeking to improve quality of care must find ways of addressing practice variation in the management of CAP. The presence and persistence of practice variation have undermined the credibility of hospitals and physicians with purchasers and with the public.9-12 A major challenge facing managed care organizations seeking to reduce practice variation is how to respond to clinicians’ concerns regarding differences in patient illness severity. The most commonly used administrative data sources for risk adjustment are hospital discharge abstracts, which are based on International Classification of Diseases (ICD) codes (usually grouped into diagnosis-related groups13). These readily available sources have 2 major disadvantages. First, they use information that is unavailable at the time of clinical decision making (eg, an ICD code that indicates that a patient experienced assisted ventilation). Second, they do not contain information about a patient’s physiologic state. Research investigations address these limitations by incorporating data acquired through manual medical record review such as vital signs and laboratory test results.13,14 However, because of the high costs of acquiring such data, it is difficult for managed care organizations to incorporate them into routine reports provided to clinicians.

A few recent studies have shown that it is possible to assign abbreviated forms of severity scores using laboratory data from automated databases15,16 or from manual data collection in combination with electronic data.17 Render et al16,18 developed a method for risk adjusting adult intensive care unit (ICU) outcomes that combines laboratory data with administrative data, while Graham and Cook17 combined manually acquired diagnostic and electronically captured laboratory data. More recently, Pine et al14 highlighted the value of supplementing condition-specific riskadjustment models with laboratory data. Although it is clear that incorporation of vital signs, radiologic findings, and other findings of the physical examination in severity scores is desirable, many hospitals do not yet have ready access to these data in electronic format. Consequently, it seems reasonable to use these abbreviated scores during this transitional phase in medicine.

In this article, we describe the development and assessment of such a transitional tool, an abbreviated version of an existing severity of illness score, the Pneumonia Severity Index (PSI).19,20 Our Abbreviated Fine Severity Score (AFSS) combines available laboratory data with administrative data and is used for routine reporting to clinicians at 18 hospitals in an integrated healthcare delivery system, the Kaiser Permanente Medical Care Program (KPMCP). We chose this score as our starting point because of the clinical importance of CAP and because the PSI is an integral part of the KPMCP CAP clinical practice guideline.21


Study Setting
We developed the AFSS using hospitalization data from 16 Northern California KPMCP hospitals between January 1, 2000, and March 30, 2002 (cohort 1). All of these facilities use the same comprehensive information systems linked by a common medical record number. By the time that ongoing operational reporting using the AFSS became routine, an additional 2 hospitals were in operation. Therefore, cohort 2 consisted of CAP admissions between July 1, 2004, and June 30, 2005, at 18 KPMCP hospitals in northern California.

Limited resources allowed review of medical records for only the following 3 patient subsets: (1) We compared the electronically assigned AFSS with the PSI based on manual medical record review in the Colorado region of the KPMCP, which has 398,706 members, including admissions to Exempla St Joseph Hospital (ESJH) in Denver, Colorado (which is the primary contract hospital for KPMCP patients in Denver), and the Kaiser Permanente outpatient clinics from which patients were referred between November 1, 2004, and April 22, 2005. (2) We performed a similar medical record review using randomly selected hospitalizations from cohort 1 at the KMPCP Oakland Medical Center. (3) We reviewed a randomly selected group of patients from cohort 1 to assess the accuracy of the diagnosis of pneumonia.

We obtained approval from the institutional review boards for the protection of human subjects in the northern California and Colorado regions of the KPMCP. The approval included a waiver of individual informed consent.

Identification of Patients With CAP
Methods used to identify KPMCP patients and to link their hospitalization records to each other (in the case of transported patients) and to their laboratory test results have been described.22-27 Our initial step was to identify all nonobstetric, nonpsychiatric patients at least 18 years of age hospitalized between January 1, 2000, and March 31, 2002, with at least 1 stay in a KPMCP hospital. From this cohort, we first identified patients with pneumonia and then selected patients who met the following eligibility criteria19,20: (1) A principal discharge diagnosis of pneumonia, defined according to the list of ICD codes by Fine et al,19,20 was required. (2) In patients with multiple CAP hospitalizations, only the first was used. (3) Patients with prior hospitalizations within 7 days were excluded. (4) Patients with selected conditions (eg, prior organ transplantation or infection with human immunodeficiency virus) were excluded. (5) Among patients with a  hospitalization involving more than 1 hospital, patients with a first hospital stay outside the KPMCP were excluded. (6) Patients who were not KPMCP members at the time of CAP hospitalization were excluded. For identification of patients at ESJH, we used the same criteria. Membership in the Colorado KPMCP was required instead of membership in the Northern California KPMCP.

Abbreviated Fine Severity Score
The PSI uses 19 predictors (eg, arterial pH and altered mental status) and has a maximum value of 285 plus the patient’s age in years (Table 1).20 Of these 19 predictors, the following 7 are not readily available in hospital discharge abstracts or in laboratory databases and could not be included: whether a patient was a nursing home resident, 5 physical examination components (eg, respiratory rate), and the presence of pleural effusion. The remaining 12 predictors, which constitute the AFSS, are available in KPMCP databases. Based on the PSI scoring scheme, these 12 items permit a maximum number of points equaling 180 plus the patient’s age in years.

We scanned the KPMCP and ESJH hospitalization and laboratory databases and downloaded all relevant test results obtained on a patient during the 24-hour period preceding hospital admission and linked these to the electronic discharge abstracts. In cases in which more than 1 test of a given type was obtained in the time frame, we selected the test result that would give the highest point assignment. If an individual test result was not obtained for a patient (eg, arterial pH), it was imputed as normal, and 0 points were assigned.


We ascertained inhospital mortality, assisted ventilation, ICU admission, and total hospital length of stay (LOS) from KPMCP and ESJH databases in northern California and Denver. To ascertain 30-day mortality, we linked KPMCP records to California or Colorado death certificates and to publicly available Medicare files using previously described methods.28

We ascertained the use of assisted ventilation based on ICD procedure codes 96.7 (other continuous mechanical ventilation), 96.70 (continuous mechanical ventilation of unspecified duration), 96.71 (continuous mechanical ventilation <96 hours), or 96.72 (continuous mechanical ventilation ≥96 hours). To establish total LOS for patients transferred between hospitals, we linked records involving multiple hospital stays, so LOS is defined as the exact time in days and hours between the first hospital admission in a linked hospital stay and the final discharge to home or a skilled nursing facility or death.

Manual Medical Record Abstraction
Because the Colorado KPMCP had an operational outpatient electronic medical record during the study period, we could manually audit all of the Denver patients’ outpatient notes before admission, admission histories and physical examination findings, and dictated discharge summaries. These latter 2 items are retrievable through the electronic medical record.

To compare the AFSS and the PSI in Northern California KPMCP patients, we randomly selected 200 CAP hospitalizations that began at the KPMCP Oakland hospital between January 1, 2000, and March 30, 2002. Of these, 100 were randomly selected hospitalizations in which the patient died or was admitted to the ICU, and 100 were randomly selected hospitalizations in which the patient survived and did not experience ICU admission.

PDF is available on the last page.
Diabetes Compendium
Oncology Compendium