FAQ | Measures | Future | NC Hospitals
Frequently Asked Questions (click a question below to view the answer)
What is the Quality Dashboard?
Since January 2007, the North Carolina Center for Hospital Quality and Patient Safety (NC Quality Center) has been posting on this site the quality measures that derive from the data submitted by hospitals to the Quality Improvement Organization (QIO) Clinical Data Warehouse (part of the Centers for Medicare & Medicaid Services (CMS)). The NC Quality Center has been reporting these measures in the reports section in categories by condition (heart attack, heart failure, pneumonia, and surgical care).
With the Dashboard, you can now click on your hospital of choice and see the scores on up to 12 measures for that one hospital. Furthermore, the dashboard is a one-page report that includes color-coded cells within a table. Each color denotes which quartile the hospital's score lies within the distribution of scores for all hospitals.
The NC Quality Center is thankful to receive quarterly data from The Carolinas Center for Medical Excellence that allows for 1.) the posting of scores with less lag than the scores on Hospital Compare, and 2) shorter 6-month timeframes vs the 12-month timeframes on www.hospitalcompare.hhs.gov referred to as HospitalCompare
In addition to four optimal care scores, the dashboard includes two HCAHPS measures ("patient perception of care survey") and six outcome measures. The source of these eight measures are files that are downloadable from HospitalCompare. The HCAHPS scores are survey results for a 12-month period and are updated quarterly on HospitalCompare. The outcome measures show risk-adjusted rates for a 36-month period and are updated annually on HospitalCompare.
All the measures on the dashboard are measures that are publicly reported on CMS' HospitalCompare. To see a table of the measures, their data source, and the measure's timeframe that is being reported on this website, please review the files in the Downloads section.
More information about the optimal care scores and the HCAHPS measures can be found below in the FAQ section.
How are the 30-day Mortality and Readmission Rates Developed? (Outcome Measures)
The outcome measures are calculated by the Centers for Medicare and Medicaid Services using a complex, statistical procedure. "Outcome of Care Measures" show what happened after patients with certain conditions received hospital care. The death rates focus on whether patients died within 30 days of their hospitalization. The rates of readmission focus on whether patients were hospitalized again within 30 days. Death rates and rates of readmission show whether a hospital is doing its best to prevent complications, teach patients at discharge, and ensure patients make a smooth transition to their home or another setting such as a nursing home.
The hospital death rates and rates of readmission are calculated using Medicare enrollment and claims records and are based on people with Medicare who are 65 and older. The death rates and rates of readmission are "risk-adjusted", meaning that the calculations take into account how sick patients were when they went in for their initial hospitalization. When the rates are risk-adjusted, it helps make comparisons fair and meaningful.
More information on these measures can be found at HospitalCompare.
How are the Dashboard Quartiles developed?
Quartiles represent the four percentile ranges of a population's distribution. The population that was used for each measure included each NC hospital that had complete data and 10 or more patients for a given measure and time period. Statistical procedures using SAS version 9.1 (SAS Institute, Cary, N.C.) were performed on each of the 12 measures to determine the quartile cut-points and to assign quartiles to each hospitals' scores.
Please refer to the graph below as an example of quartiles. The graph shows the distribution of the pneumonia optimal care score for 101 hospitals. The quartile cut-points or thresholds were a score of 80.9% for the 25th percentile, 87.2% for the 50th percentile (also called the median), and 91.5% for the 75th percentile. A hospital with a score of 91.5% or higher would fall in to the top quartile. Confusingly enough, the top quartile is often referred to as the TOP 25%.
There may be very little difference between the score of a hospital in quartile two versus quartile three. For example, using the graph below, a hospital with a score of 87.1% would fall in to the 2nd quartile while a hospital with a score of 87.2% would fall in to the 3rd quartile. Please note that numbers reported on the dashboard are rounded.
Why do some hospitals have "no data" for a measure in the dashboard?
Beginning in July 2007, hospitals that treat Medicare patients and that are subject to inpatient prospective payment system (IPPS) provisions ("subsection (d) hospitals") must collect and submit data on these 12 measures to CMS in order to receive their full IPPS annual payment update (APU) each fiscal year. IPPS hospitals that fail to report the required quality measures, which include the HCAHPS survey, may receive an APU that is reduced by 2.0 percentage points.
Therefore, almost all non-federal North Carolina hospitals will have a dashboard on this website with at least one measure except for hospitals that primarily serve children, psychiatric, or rehabilitation patients and non-IPPS hospitals, such as Critical Access Hospitals (CAH). To be designated as a CAH, a hospital must be located in a rural area, provide 24-hour emergency services; have an average length-of-stay for its patients of 96 hours or less; be located more than 35 miles (or more than 15 miles in areas with mountainous terrain) from the nearest hospital or be designated by its State as a "necessary provider". Hospitals may have no more than 25 beds.
The submission of the process measure data and HCAHPS is voluntary for a CAH hospital. Often a smaller hospital does participate in the data collection of a measure, yet their data can not be reported because their patient volume was too low for that measure.
Why are you reporting measurement scores from 2007 and 2008?
Hospital quality measurement has been evolving over the years to be more transparent. However, with the increase in transparency, comes the need for standardization of meaningful measures that can be used for accurate comparability of hospital performance. For both the hospital and the reporting organization, the process of collecting the data, validating the data, and preparing the data for public reporting is long and complicated, resulting in measurement score timeframes with 6 to 12-month lag between the time health care service took place (i.e., admission, surgery, survey, etc) and when the measure is reported.
For the mortality and readmission rates, the Hospital Compare website displays hospital-specific confidence intervals (a statistical range around a hospital's average or rate) for the purpose of determining if a hospital's rate is significantly better or worse than the national norm. During testing, CMS reported rates for the most recent two years of data available (2007-2008), however, the patient volume was not adequate for many hospitals. Large volume is needed to create tight enough confidence intervals that result in finding "outliers" (those significantly different than the nation norm.) Therefore, CMS has decided to use the latest three years of claims data to create the outcome measures.
Currently the outcome measures reported on both HospitalCompare and this site include data on discharges that took place between 7/1/2005 - 6/30/2008. The mortality and readmission rate data will be refreshed on Hospital Compare on June 24, 2010. The rates will reflect outcomes for hospital discharges that took place between 7/1/2006 - 6/30/2009.
What is HCAHPS?
The Hospital Consumer Assessment of Healthcare Providers & Systems - HCAHPS (pronounced H-caps) is a national, standardized survey of patient's perception of their hospital experience. HCAHPs is a requirement of Medicare's CMS Quality Initiative for the majority of acute care hospitals in the nation.
The survey includes 27 questions that are used to report 10 measures (six summary, two individual and two global measures) that reflect the patient's perception of the care that they have received in the hospital.
The Patient Safety Group has developed a HCAHPS Benchmark Ranking Map Tool at their web site.
Overview of HCAHPS questions
Summary measures:
- Communication with nurses (Questions 1, 2, and 3)
- Communication with doctors (Questions 5, 6, and 7)
- Responsiveness of hospital staff (Questions 4 and 11)
- Pain management (Questions 13 and 14)
- Communication about medicines (Questions 16 and 17)
- Discharge information (Questions 19 and 20)
- Cleanliness of hospital environment (Question 8)
- Quietness of hospital environment (Question 9)
- Overall rating of hospital (Question 21)
- Willingness to recommend hospital (Question 22)
Questions 10, 12, 15 and 18 are screener questions that direct the respondent to answer specific questions in a section. Questions 23, 24 and 27 are used with items from hospital administrative records to adjust for differences in patient mix across hospitals. Questions 25 and 26 are related to race and ethnicity and will be used for work related to health care disparities. They will also provide information for the Agency for Healthcare Research and Quality's congressionally mandated reports on health care disparities.
How is the HCAHPS survey administered?
The HCAHPS survey data is collected 48 hours to six weeks after discharge from adult patients. Participating hospitals may either use an approved survey vendor or collect their own survey data if approved by Centers for Medicare and Medicaid Services (CMS) to do so. Hospitals must survey patients each month of the year. Hospitals may use a random sample of patients, but must submit at least 300 completed surveys per year. If a hospital does not meet this requirement they must survey all patients.
Are the HCAHPS survey results adjusted?
The HCAHPS survey results are adjusted for the mode of survey administration (mail, telephone and/or interactive voice response) and patient characteristics (health status, education, service line, age, ER admission, response percentile, service by linear age interactions, and primary language other than English). For more information about the HCAHPS survey visit www.hcahpsonline.org.
What are the goals of the HCAHPS survey?
The HCAHPS is designed to meet three goals:
- To produce information that allows objective and meaningful comparisons of hospitals on topics that are important to consumers;
- Public reporting of the survey results creates new incentives for hospitals to improve their quality of care; and
- Public reporting increases public accountability by providing information on the quality of hospital care provided in return for the payment the hospital receives for that care.
What is the source of the process measure data?
The Carolinas Center for Medical Excellence provides to the NC Quality Center quarterly hospital level data from the CMS/QIO data warehouse. The data include numerators and denominators from the most recent 3 months for each clinical process measure per hospital for all hospitals that have signed a consent form to share their data with the NC Quality Center.
The source for HCAHPS data is www.HospitalCompare.hhs.gov.
The hospital scores for HCAHPS and the individual quality measures displayed on this site (excluding the optimal care score) have been publicly reported by U.S. Dept. of Health and Human Services, through their online resource called Hospital Compare.
A list of when each measure became part of the Centers for Medicare and Medicaid (CMS) reporting program can be found here: MeasComp2010.pdf
Another website - WhyNotTheBest.org - shows the latest performance data on the process-of-care and hospital patient experience measures that are publicly reported by the Centers for Medicare and Medicaid Services. Users of the site can conduct side-by-side comparisons of 4,500 hospitals nationwide, track performance over time against numerous benchmarks, and download tools to improve health care quality.
What is an "Optimal Care" score?
The optimal care measure is a condition-level summary score that uses the "all or none" methodology to determine if a patient received all of the recommended treatment for which they were eligible. For each condition on this site (heart attack, heart failure, pneumonia, surgical care), performance is determined at the patient level and then summarized per hospital.
A hospital optimal care percentage score is calculated with this formula (per condition):
| Total # patients receiving all care measures | ||
| % = | / | |
| Total # patients qualifying for any care |
||
The steps to the methodology are:
Per discharged patient, determine if he/she met the eligibility to be categorized as a patient with a primarily diagnosis of heart attack, heart failure, pneumonia, or admitted for surgery.
Determine which measures per condition a patient has been deemed eligible for from a patient's medical chart.
Did the patient receive the recommended care for all eligible measures? If yes, patient receives a 1. If no, patient receives a 0.
Per hospital, sum the numbers from step 3 (above.) This is the sum of the numerators.
Per hospital, sum the total number of patients that had at least one eligible measure. This is the sum of the denominators and is a count of unique patients.
EXAMPLE: Let's say a patient having cardiac surgery was eligible to receive an antibiotic before incision to reduce the risk of infection and also to receive medication to prevent venous thromboembolism (i.e., blood clots). He was previously taking a beta blocker making him eligible for all 8 surgical care process measures. If the patient's timing to receive one of these treatments did not comply with the treatment guidelines then that patient received only 7 of the 8 elements. Thus, the score for that patient would be 0%. (Not a rate of 7/8 = 87.5%.) The numerator is either a 1 or a 0 and each patient has a denominator of 1. A hospital's rate per condition = the sum of the patient numerators divided by the total number of patients.
| Optimal Care: Surgical Care Example | |||
| # of Measures Eligible | Received All Eligible (1=yes) |
Patient Count | |
| Patient #1 | 8 | 1 | 1 |
| Patient #2 | 6 | 0 | 1 |
| Patient #3 | 7 | 0 | 1 |
| Patient #4 | 8 | 1 | 1 |
| Patient #5 | 5 | 1 | 1 |
| Patient #6 | 7 | 1 | 1 |
| SCIP-8 O.C. Totals | 4 | 6 | |
| Calculated SCIP-8 Optimal Care Score: 4/6 = .666 or 67% | |||
The SCIP8 optimal care measure includes eight surgical care measures and has replaced the 5-measure SIP Optimal Care score in the regional comparison graphs and the hospital-specific trend graphs.
What are Hospital "Highlights?"
The website has a new section to highlight hospitals that we label as "High Reliable" as measured by the condition-specific optimal care score across time. For example, if a hospital's heart failure optimal care score is at or above 90% across the four 6-month time periods on this site then they meet the High Reliable criteria for heart failure care. This means that the hospital's compliance to the evidence-based recommended treatments for their patients are consistent and exceptional.
Criteria for High Reliable Hospital
Heart Attack: Optimal care score is 95% or higher for four 6-month time periods. Heart Failure: Optimal care score is 90% or higher for four 6-month time periods. Pneumonia: Optimal care score is 90% or higher for four 6-month time periods. Surgical Care: Optimal care score is 90% or higher for four 6-month time periods.
Another distinction in the new highlight section is a hospital deemed "Most Improved" per condition. These are the 12 hospitals that have seen the most improvement in their optimal care scores over time. More specifically, these are the top 3 hospitals per condition with the largest absolute value increase in optimal care score from 6-month baseline to the last 6-month time period without any substantial dips (10 percentage pts or more) in performance over 18 months.
Determining Most Improved Hospitals
Four time periods for the latest release were used:
- Jan 1, 2008 - June 30, 2008 (baseline)
- Jul 1, 2008 - Dec 31, 2008
- Jan 1, 2009 - Jun 30, 2009
- Jul 1, 2009 - Dec 31, 2009 (last 6 months available)
Steps taken to determine top 3 Most Improved hospitals per condition:
- Exclude hospitals with < 10 patients during the 3rd and 4th time periods.
- Determine the absolute difference in the O.C. score between baseline and last 6 months.
- Exclude hospitals where time 4 score <= (time 3 score - 10 percentage points). (Only small dip allowed.)
- Exclude hospitals where time 3 score <= (time 2 score - 10 percentage points). (Only small dip allowed.)
- Exclude hospitals where time 2 score <= (time 1 score - 10 percentage points). (Only small dip allowed.)
- Per condition, list rank the remaining hospitals by descending absolute difference from baseline (calculated in step 2) and select the top 3 hospitals to be highlighted.
What is a PCI-capable hospital?
A PCI-capable hospital is a hospital that performs percutaneous coronary interventions (i.e., angioplasty, stenting) for patients suffering from a heart attack. To be grouped with PCI-capable hospitals on this website, the hospital performed these interventions on 10 or more patients during the last 6-month time period. This subgroup of NC hospitals is used to develop a separate mean score and 90th percentile score for the heart attack optimal care measure for only those hospitals that were performing PCI.
The subgroup was created so that hospitals with patients eligible for the measure titled "PCI Within 90 Minutes of Arrival" could be compared to each other in one graph. In addition, in the report tables, a hospital is highlighted in green if they meet or exceed the 90th percentile optimal care score of all hospitals (100% score) OR they meet or exceed the 90th percentile optimal care score of PCI hospitals (99%).
Why are some hospitals missing from the regional bar graph or trend graph?
This site graphically displays hospital performance on four optimal care measures for heart attack, heart failure, pneumonia, and surgical care.
There are two reasons why a hospital may not have an optimal care score for graphing. The first reason is patient volume that is too low (< 10 patients) to adequately compare or trend a hospital's performance. For example, during the six-month time period, if a hospital cared for less than 10 patients that had a heart attack, they do not have enough volume to be shown on this site's heart attack graphs. The second reason that a hospital does not have an optimal care score is incomplete data for a time-period. Both of these "Too Few" reasons are indicated with a "TF" in the report tables' column for the optimal care score.
A list of hospitals, per condition, that are excluded from this site's bar and trend graphs can be found in this downloadable spreadsheet: g0_dec09.xls
The report tables on this site include measurement information on 110 non-federal, general acute-care hospitals in North Carolina. Three percent of NC's acute care hospitals have yet to join the NC Hospital Quality Performance Report site. Lastly, hospitals that primarily serve children, psychiatric, or rehabilitation patients are not included in this site.
How are the individual measures developed?
The information posted on this site comes from the quality data submitted by hospitals to the Quality Improvement Organization (QIO) Clinical Data Warehouse (part of the Centers for Medicare & Medicaid Services (CMS)) for all inpatient discharges. Except where noted, all rates are derived from data reflecting two quarters (six months) of experience, which is updated on a rolling basis.
Each of the measures tells you the proportion of cases where a hospital provided the recommended process of care. Only patients meeting the inclusion criteria for a measure are included in the calculation of the rate for a measure. A rate of 88% means that the hospital provided the recommended process of care 88% of the time. For example, the rates for aspirin at discharge for individuals who have had an acute myocardial infarction -- a heart attack -- tell you the percentage of patients who received an aspirin when they are discharged from the hospital. The ultimate goal for each measure is 100%. Hospitals with effective quality improvement programs are continually working toward this goal.
From CMS' QualityNet site, specification manuals are available that provide the detailed information on the specifications and the methodology used to calculate each measure's score. These manuals represent the result of efforts by the CMS and the Joint Commission to achieve identity among common national hospital performance measures and to share a single set of common documentation.
What are the average and top 10% scores?
On the top of each condition table there are two rows that display state-level scores. The first line says "Average for NC Hospitals". This is the mean score for the N.C. hospitals that currently participate in the NC Hospital Quality Performance Report. This average is unweighted by patient volume. For example, the score for the hospital with 20 patients has as much "weight" in calculating the average as the hospital with 200 patients.
The second line says "Top 10% of NC Hospitals". The values shown are the 90th percentile score for all the hospitals participating in the NC Hospital Quality Performance Report.
The optimal care state average and the 90th percentile were developed using only the scores from hospitals with 10 or more patients per condition.
What is being done to improve performance?
Hospital-level and statewide performance on the process measures displayed on this site is monitored by the NC Center for Hospital Quality and Patient Safety. To improve surgical care improvement, which includes reducing the risk of post-surgical infection, the Quality Center and the Carolinas Center for Medical Excellence are leading the N.C. Surgical Care Improvement Project collaborative with the help of statewide partners. The goal is to reduce surgical complications and to improve performance on surgical care processes of care.
How can consumers and patients use this information?
This information helps patients, family, and friends compare the quality and safety of care in North Carolina hospitals. The quality measures reported on this site are well accepted nationally as steps that should be taken in the care of the conditions. Scientific evidence shows that high quality care leads to fewer repeat hospitalizations, hospital-acquired infections and medical errors, thereby reducing costs.
This site is a resource to inform consumers about hospital quality care and to help North Carolinians make good decisions about health care. Consumers should view this information as a starting point for educating themselves about hospital quality, for talking to their doctors about choosing a hospital for medical care, and for asking questions while receiving care in the hospital. Consumers may want to review the quality measure information when considering a hospital visit for a scheduled procedure.
In this report, each quality measure includes only patients whose history and condition indicate the treatment is appropriate. Talk to your doctor if you have questions about your treatment.
How can healthcare providers use this data?
This information is used by the medical community to heighten their awareness of the opportunity that exists to improve the care that they currently deliver. The public reporting of this data also provides an incentive for hospitals to continue to improve their use of recommended best treatments.
How can I get more information about this site?
Additional questions or comments about this site should be directed to info@nchospitalquality.org.
Who is involved in this project?
The North Carolina Center for Hospital Quality and Patient Safety ("NC Quality Center") initiated the NC Performance Reporting Committee to determine which clinical measures should be included on this site. The Committee is a multi-disciplinary team consisting of physicians, nurses, and executives representing hospitals, health systems, insurance, industry, the Carolinas Center for Medical Excellence (CCME), NC Medical Society, and NC Department of Health and Human Services. The principles set by the group were to include measures that were actionable, standardized, well-defined, available, and would not add burden to hospital data collection efforts. In September 2009, the Workgroup's recommendation to include HCAHPS to this site were approved by the NC Quality Center's Board and endorsed by the NCHA Policy Development Committee.