Exclusions to Analysis

The in-hospital mortality analysis includes all inpatient records categorized into one of the 15 selected DRGs with the following exceptions:

- patients who left against medical advice or discontinued care
- patients who were transferred to another short-term acute care hospital
- patients with an invalid (or missing) ASG score
- patients with an invalid (or missing) age

For the actual number and percent of statewide cases excluded from analysis, refer to Table 2. For the actual number and percent of regional cases excluded from analysis, refer to the Table of Contents for the appropriate table.

Construction of Reference Database

A Pennsylvania statewide comparative database was computed for the 1998 Pennsylvania acute care hospital inpatient data. The reference database for the measure of in-hospital mortality is indexed by DRG, ASG score, cancer status, and age category. ASG score, cancer status and age category were used as risk adjustment factors in the statistical analysis for in-hospital mortality. Indirect standardized was adopted as the risk-adjustment technique. In order to best support the statistical methods that were utilized, it was decided that the patient count in each of the final ASG/cancer/age categories should be twenty or more. When the number of patients in an ASG/cancer/age category did not meet this minimum threshold, combination of categories was warranted. (There is a maximum of 45 different combinations of ASG/cancer/age categories. Because of the high volume DRGs that were selected for this HPR, it was not typically necessary to combine categories in order to achieve a minimum number of 20 patients per combination of ASG/cancer/age category.)

The algorithm used to combine categories was determined under the premise that ASG is regarded as the best indicator of patient risk, followed by cancer status, then age category. (Note that age in years, as an independent predictor of mortality, was already evaluated and retained where statistically significant in the Atlas severity score developed by CIC-MediQual).

When an age category had a small patient count, it was combined with an adjacent age category. Age categories are defined as:

- age 64 and under
- age 65 through age 79
- age 80 and over

Patients were next risk-adjusted with respect to cancer status. When small patient counts were encountered, the adjustment algorithm combined patients with a history of cancer with those patients who do not have cancer diagnosis codes present. Cancer categories were combined only when age category collapsing would not improve small patient counts. The three applicable categories are:

- no cancer (that is, no cancer diagnosis codes present)
- malignant neoplasm or cancer in situ (ICD.9.CM diagnosis codes 140.0 – 208.9 inclusive or 230.0 – 234.9 inclusive)
- history of cancer (ICD.9.CM diagnosis codes V10.00 – V10.90 inclusive)

When the patient count for an ASG level was small, and all acceptable collapsing of cancer status categories were performed, collapsing of ASG levels was necessary. When combining ASG levels, counts for scores 0 and 1 may have been combined; and counts for scores 3 and 4 may have been combined. An ASG score of 2 was considered an independent category. The following table displays the conversion of probabilities to admission severity categories:

Admission Severity Group Probability of Death0 (no risk of clinical instability) 0.000 – 0.001

1 (minimum risk of clinical instability) 0.002 – 0.011

2 (moderate risk of clinical instability) 0.012 – 0.057

3 (severe risk of clinical instability) 0.058 – 0.499

4 (maximum risk of clinical instability) 0.500 – 1.000

Calculation of the Expected Mortality Rate

Using the Pennsylvania comparative database, the statewide mortality rate was calculated for the final ASG/cancer/age category combinations within each of the fifteen DRGs. These mortality rates were computed for each ASG/cancer/age category by dividing the total number of deaths in that category by the total number of patients in that ASG/cancer/age category. This mortality rate, computed across all hospitals, provides a standard comparison for individual hospitals.

Actual In-hospital Mortality Compared With Expected In-hospital Mortality

The number of deaths for each hospital within each DRG during 1998 is the actual or observed in-hospital mortality. The number of deaths "expected" for each hospital within each DRG is calculated using the statewide expected mortality rates for each of ASG/cancer/age category combinations for a particular DRG.

Note, that since there are 5 ASG categories, 3 cancer categories, and 3 age categories, there are potentially 45 different combinations of these categories for every DRG. In the notation presented for the calculation of the Expected Number of Deaths, the variable, i, can range from 1 to 45. (For a particular DRG, the maximum of the variable, i, is given by the number of final combinations of ASG/cancer/age categories.) The i^{th} combination is a generic term used to signify each of the final combinations of ASG/cancer/age categories.

The expected number of mortalities for each DRG within each hospital is calculated as follows:

Expected Number of Deaths = (p

_{i}x n_{i})where, for each of the final combinations of ASG/cancer/age within the DR

p

_{i }= the statewide mortality rate for the i^{th}combinationn

_{i }= the number of cases for the hospital of the i^{th}combinationThe estimated probability of death, p, for each DRG within each hospital is calculated as follows:

p = (Expected Number of Deaths)/N

where, N

_{ }= the total number of cases for a particular DRG within a particular hospital(n

_{i}= N)

The following example illustrates the calculation of the expected mortality for DRG 127 (Heart Failure and Shock) at Hospital "A".

DRG 127 (Heart Failure and Shock)

Hospital "A"

Statewide Comparative Database

Expected Mortality at Hospital "A"

Admission Severity Group

Cancer Status

Age Category

Number of Patients Treated

Mortality Rate

Product: Number x

Mortality Rate

0

No Cancer

Under Age 65

2

0.019608

0.0392

1

No Cancer

Under Age 65

10

0.004388

0.0439

Age 65-79

5

0.004261

0.0213

Age 80 & Over

1

0.017094

0.0171

2

No Cancer

Under Age 65

20

0.009498

0.1900

Age 65-79

80

0.013526

1.0821

Age 80 & Over

70

0.020612

1.4429

Malignant Neoplasm or Cancer In Situ

Under Age 65

1

0.055901

0.0559

Age 65-79

5

0.039179

0.1959

Age 80 & Above

3

0.030471

0.0914

History of Cancer

Age 80 & Above

5

0.009050

0.0452

3

No Cancer

Under Age 65

10

0.069703

0.6970

Age 65-79

50

0.075641

3.7821

Age 80 & Over

150

0.086699

13.0049

Malignant Neoplasm or Cancer In Situ

Under Age 65

1

0.135135

0.1351

Age 65-79

7

0.110887

0.7762

Age 80 & Above

12

0.090056

1.0807

History of Cancer

Under Age 65

2

0.000000

0.0000

Age 65-79

4

0.032338

0.1294

Age 80 & Above

6

0.064350

0.3861

4

No Cancer

Under Age 65

2

0.266667

0.5333

Age 65-79

5

0.343096

1.7155

Age 80 & Over

7

0.406417

2.8449

History of Cancer

All Age Categories Combined

5

0.250000

1.2500

Total

463

29.5601

Summary

Hospital "A" had a total of 463 patients treated for DRG 127 (Heart Failure and Shock). Of those 463 patients, the expected number of deaths is 29.5601. (Note: For display purposes the calculations for the expected mortality within each combination of ASG/cancer/age category have been rounded to four decimal places.)

Binomial Test

Though a hospital's observed mortality may be comparable to the state norm, random variation plays a factor in these comparisons. Statistical evaluation can determine when the difference between the observed and the expected value is too large to be attributed solely to chance. Statistical evaluation of in-hospital mortality was performed for each DRG using the binomial test. The binomial test is appropriate for situations where the possible outcome is dichotomous; in this case, death or survival for each patient. The binomial test is based upon the following assumptions:

- the probability of death for each patient within a DRG is the expected mortality rate provided by the reference database. This probability of death is assumed to be a constant number from one patient to the next.
- the death or survival of one patient has no impact on the death or survival of any other patient. In other words, patients are independent entities.

Inferential Error

A type of inferential error that can be made in statistics is called Type I error or false positive. The probability of committing a Type I error is equal to the level of significance established by the researcher. For this analysis, the level of significance has been set to 0.05. In the context of the Hospital Performance Report, a Type I error occurs when the difference between the observed in-hospital mortality and the expected in-hospital mortality is declared statistically significant, when in fact, the difference is due to chance. That is, for a particular DRG, the hospital is declared to be statistically higher or lower than expected, when in reality the hospital's level of performance is comparable to the state norm. Since the level of significance has been set to 0.05, there is a 5% (or 1 in 20) chance of committing this type of error.

P-value Calculation

Calculating the p-value for the binomial test is defined by a formula which sums discrete probabilities based upon the binomial distribution. The binomial formula is written as:

P(X=a) = [(N!)/(a!(N-a)!)] p

^{a}(1-p)^{N-a}where,

X is the binomial random variable (X is a discrete random variable and 0

<X<N)a is the actual number of mortalities for a particular hospital's DRG

N is the number patients for a particular hospital's DRG

p is the estimated probability of patient death for a particular hospital's DRG

For each DRG within each hospital, p is calculated using the expected mortality for each DRG within each hospital. For further information regarding this computation, refer to the section "Actual In-hospital Mortality Compared With Expected In-hospital Mortality."

Statistical Rating

A statistical rating is assigned to each hospital if the difference between what was observed and what was expected in a particular DRG is statistically significant. The p-value, calculated in terms of a "two-tailed" test is compared to the level of significance.

- If the calculated p-value is greater than 0.05, then the conclusion is made that the difference between what was expected and what was observed is not statistically significantly different. It cannot be concluded that the in-hospital mortality for that particular DRG is different from the state norm.

- If the calculated p-value is less than or equal to 0.05,
then the conclusion is made that the difference between what was expected
and what was observed is statistically significant.
- If the observed in-hospital mortality is less than the
statewide in-hospital mortality, the hospital is assigned the symbol
labeled "Mortality significantly less than Expected" for a
particular DRG.

- If the observed in-hospital mortality is higher than the statewide in-hospital mortality, the hospital is assigned the symbol labeled "Mortality significantly greater than Expected" for a particular DRG.

- If the observed in-hospital mortality is less than the
statewide in-hospital mortality, the hospital is assigned the symbol
labeled "Mortality significantly less than Expected" for a
particular DRG.