FAQ | Measures | Future | NC Hospitals
Frequently Asked Questions
What is Value-Based Purchasing and how are the Scores Developed?
The Affordable Care Act (ACA) of 2010 mandates that the Centers for Medicare and Medicaid Services (CMS) implement an inpatient hospital Value-Based Purchasing (VBP) program, a pay-for-performance program that will link Medicare payment to the quality performance of acute care hospitals paid under the inpatient Prospective Payment System (IPPS).
Hospitals' quality performance, as evaluated by CMS under the final rule, will affect Medicare inpatient fee-for-service payments to hospitals beginning October 1, 2012 (federal fiscal year (FFY) 2013). CMS has also set forth rules for future program years and will continue to do so in future rulemaking.
As required by the ACA, a pool of funds to be redistributed to hospitals based on quality performance under the VBP program will be created by reducing Medicare IPPS payments for all participating hospitals - a 1.0% reduction in FFY 2013, increasing by 0.25% each year until the reduction reaches 2.0% for FFY 2017 and thereafter.
CMS will base the FFY 2013 VBP Program on quality measures in two domains: clinical process of care and patient experience of care as measured by HCAHP survey questions. All of the measures in these two domains are currently reported under the Hospital Inpatient Quality Reporting (IQR) Program, as required by the ACA.
Under the VBP program, VBP scores will be calculated for each hospital based on its performance on the selected quality measures during two specific time periods. These time periods are defined by CMS as a "baseline period" and a "performance period."
• Baseline Period: Quality data collected by hospitals during the 9-month period, July 1, 2009 through March 31, 2010, will be used for the baseline period. This data will serve as the baseline for determining hospital quality improvement and establishing the VBP national performance standards.
• Performance Period: As proposed, quality data collected by hospitals during the 9-month period, July 1, 2011 through March 31, 2012, will be used for the performance period. This data will be used to determine hospitals' achievement scores by comparing hospital performance to the national performance standards derived from the baseline period data.
CMS has established national benchmarks and thresholds for each VBP Program quality measure. The benchmarks represent the highest achievement levels on quality measures; the thresholds represent the minimum achievement levels. Hospitals' performance on individual quality measures will be compared to these national performance standards to calculate VBP "achievement" and "improvement" scores.
• National Benchmarks: CMS will set the national benchmark for each process measure and each HCAHPS measure at the average performance score for the top 10% of all hospitals during the baseline period.
• National Thresholds: CMS will set the national threshold for each process measure and each HCAHPS measure at the median performance score (50th percentile) for all hospitals during the baseline period.
A hospital's overall VBP score will determine its payments from the VBP incentive pool. CMS will calculate an overall VBP score for each hospital by combining the process of care domain score and the patient experience of care domain score. CMS is required by the ACA to assign weights to each domain. CMS will apply a weight of 70% to the clinical process of care domain and a weight of 30% to the patient experience of care domain.
For the FFY 2013 Hospital VBP Program, the VBP payment incentive pool will be funded by reducing Medicare IPPS payments by 1.0%. Per the ACA, the VBP program must be budget neutral; all pool dollars must be released back to hospitals.
CMS is adopting the use of a "linear exchange function" to calculate each hospital's payment adjustment under the VBP program. Using the adopted scoring methodology, this payout function distributes the VBP pool dollars based on the overall VBP scores of all hospitals participating in the program. Not all hospitals will earn back everything they contribute to the pool and some hospitals will earn back more than they contribute to the pool.
How is the overall value-based purchasing score developed??
For each measure, a formula is used to determine whether performance or improvement was greater, and only the larger of the two is included in the component score. The two component scores are then weighted to create one overall score. Download a presentation about VBP and the scoring methodology. A more detailed description of the program is available at www.gpo.gov/fdsys/pkg/FR-2011-05-06/pdf/2011-10568.pdf.
What is the Quality Dashboard?
Since January 2007, the North Carolina Center for Hospital Quality and Patient Safety (NC Quality Center) has been posting on this site the quality measures that derive from the data submitted by hospitals to the Quality Improvement Organization (QIO) Clinical Data Warehouse (part of the Centers for Medicare & Medicaid Services (CMS)). The NC Quality Center has been reporting these measures in the reports section in categories by condition (heart attack, heart failure, pneumonia, and surgical care).
With the Dashboard, you can now click on your hospital of choice and see the scores on up to 12 measures for that one hospital. Furthermore, the dashboard is a one-page report that includes color-coded cells within a table. Each color denotes which quartile the hospital's score lies within the distribution of scores for all hospitals.
The NC Quality Center is thankful to receive quarterly data from The Carolinas Center for Medical Excellence that allows for 1.) the posting of scores with less lag than the scores on Hospital Compare, and 2) shorter 6-month timeframes vs the 12-month timeframes on www.hospitalcompare.hhs.gov referred to as HospitalCompare
In addition to four optimal care scores, the dashboard includes two HCAHPS measures ("patient perception of care survey") and six outcome measures. The source of these eight measures are files that are downloadable from HospitalCompare. The HCAHPS scores are survey results for a 12-month period and are updated quarterly on HospitalCompare. The outcome measures show risk-adjusted rates for a 36-month period and are updated annually on HospitalCompare.
All the measures on the dashboard are measures that are publicly reported on CMS' HospitalCompare. To see a table of the measures, their data source, and the measure's timeframe that is being reported on this website, please review the files in the Downloads section.
More information about the optimal care scores and the HCAHPS measures can be found below in the FAQ section.
How are the 30-day Mortality and Readmission Rates Developed? (Outcome Measures)
The outcome measures are calculated by the Centers for Medicare and Medicaid Services using a complex, statistical procedure. "Outcome of Care Measures" show what happened after patients with certain conditions received hospital care. The death rates focus on whether patients died within 30 days of their hospitalization. The rates of readmission focus on whether patients were hospitalized again within 30 days. Death rates and rates of readmission show whether a hospital is doing its best to prevent complications, teach patients at discharge, and ensure patients make a smooth transition to their home or another setting such as a nursing home.
The hospital death rates and rates of readmission are calculated using Medicare enrollment and claims records and are based on people with Medicare who are 65 and older. The death rates and rates of readmission are "risk-adjusted", meaning that the calculations take into account how sick patients were when they went in for their initial hospitalization. When the rates are risk-adjusted, it helps make comparisons fair and meaningful.
More information on these measures can be found at HospitalCompare.
How are the Dashboard Quartiles developed?
Quartiles represent the four percentile ranges of a population's distribution. The population that was used for each measure included each NC hospital that had complete data and 10 or more patients for a given measure and time period. Statistical procedures using SAS version 9.1 (SAS Institute, Cary, N.C.) were performed on each of the 12 measures to determine the quartile cut-points and to assign quartiles to each hospitals' scores.
Please refer to the graph below as an example of quartiles. The graph shows the distribution of the pneumonia optimal care score for 101 hospitals. The quartile cut-points or thresholds were a score of 80.9% for the 25th percentile, 87.2% for the 50th percentile (also called the median), and 91.5% for the 75th percentile. A hospital with a score of 91.5% or higher would fall in to the top quartile. Confusingly enough, the top quartile is often referred to as the TOP 25%.
There may be very little difference between the score of a hospital in quartile two versus quartile three. For example, using the graph below, a hospital with a score of 87.1% would fall in to the 2nd quartile while a hospital with a score of 87.2% would fall in to the 3rd quartile. Please note that numbers reported on the dashboard are rounded.
Why do some hospitals have "no data" for a measure in the dashboard?
Beginning in July 2007, hospitals that treat Medicare patients and that are subject to inpatient prospective payment system (IPPS) provisions ("subsection (d) hospitals") must collect and submit data on these 12 measures to CMS in order to receive their full IPPS annual payment update (APU) each fiscal year. IPPS hospitals that fail to report the required quality measures, which include the HCAHPS survey, may receive an APU that is reduced by 2.0 percentage points.
Therefore, almost all non-federal North Carolina hospitals will have a dashboard on this website with at least one measure except for hospitals that primarily serve children, psychiatric, or rehabilitation patients and non-IPPS hospitals, such as Critical Access Hospitals (CAH). To be designated as a CAH, a hospital must be located in a rural area, provide 24-hour emergency services; have an average length-of-stay for its patients of 96 hours or less; be located more than 35 miles (or more than 15 miles in areas with mountainous terrain) from the nearest hospital or be designated by its State as a "necessary provider". Hospitals may have no more than 25 beds.
The submission of the process measure data and HCAHPS is voluntary for a CAH hospital. Often a smaller hospital does participate in the data collection of a measure, yet their data can not be reported because their patient volume was too low for that measure.
What does it mean to be a PSO member?
Hospitals can voluntarily join a Patient Safety Organization (PSO) and report their serious adverse events, near misses and unsafe condition for analysis and learning. This data is aggregated, analyzed and shared among PSO members. As a result, risk reduction strategies are identified and shared with members to promote safer, reliable processes, thus reducing the possibility of harm to patients. This approach promotes rapid collective learning about underlying causes of risks and harms in health care processes.
PSOs also foster a culture of safety by encouraging widespread internal reporting of these events so everyone can learn from one another why these events occur and what can be done to improve health care quality. Participation in a PSO reflects an organization's commitment to its patients, staff and community that patient safety is a priority.
For information on joining the NC Quality Center PSO please visit the NC Quality Center site. For information about other PSOs see the AHRQ site.
Why are you reporting measurement scores from 2008 and 2009?
Hospital quality measurement has been evolving over the years to be more transparent. However, with the increase in transparency, comes the need for standardization of meaningful measures that can be used for accurate comparability of hospital performance. For both the hospital and the reporting organization, the process of collecting the data, validating the data, and preparing the data for public reporting is long and complicated, resulting in measurement score timeframes with 6 to 12-month lag between the time health care service took place (i.e., admission, surgery, survey, etc) and when the measure is reported.
For the mortality and readmission rates, the Hospital Compare website displays hospital-specific confidence intervals (a statistical range around a hospital's average or rate) for the purpose of determining if a hospital's rate is significantly better or worse than the national norm. During testing, CMS reported rates for two years of data available (2007-2008), however, the patient volume was not adequate for many hospitals. Large volume is needed to create tight enough confidence intervals that result in finding "outliers" (those significantly different than the nation norm.) Therefore, CMS has decided to use the latest three years of claims data to create the outcome measures.
Currently the latest outcome measures reported on both HospitalCompare and this site include data on discharges that took place between 7/1/2008 - 6/30/2011.
What is HCAHPS?
The Hospital Consumer Assessment of Healthcare Providers & Systems - HCAHPS (pronounced H-caps) is a national, standardized survey of patient's perception of their hospital experience. HCAHPs is a requirement of Medicare's CMS Quality Initiative for the majority of acute care hospitals in the nation.
The survey includes 27 questions that are used to report 10 measures (six summary, two individual and two global measures) that reflect the patient's perception of the care that they have received in the hospital.
The Patient Safety Group has developed a HCAHPS Benchmark Ranking Map Tool at their web site.
Overview of HCAHPS questions
Summary measures:
- Communication with nurses (Questions 1, 2, and 3)
- Communication with doctors (Questions 5, 6, and 7)
- Responsiveness of hospital staff (Questions 4 and 11)
- Pain management (Questions 13 and 14)
- Communication about medicines (Questions 16 and 17)
- Discharge information (Questions 19 and 20)
Individual measures:
- Cleanliness of hospital environment (Question 8)
- Quietness of hospital environment (Question 9)
Global measures:
- Overall rating of hospital (Question 21)
- Willingness to recommend hospital (Question 22)
Questions 10, 12, 15 and 18 are screener questions that direct the respondent to answer specific questions in a section. Questions 23, 24 and 27 are used with items from hospital administrative records to adjust for differences in patient mix across hospitals. Questions 25 and 26 are related to race and ethnicity and will be used for work related to health care disparities. They will also provide information for the Agency for Healthcare Research and Quality's congressionally mandated reports on health care disparities.
How is the HCAHPS survey administered?
The HCAHPS survey data is collected 48 hours to six weeks after discharge from adult patients. Participating hospitals may either use an approved survey vendor or collect their own survey data if approved by Centers for Medicare and Medicaid Services (CMS) to do so. Hospitals must survey patients each month of the year. Hospitals may use a random sample of patients, but must submit at least 300 completed surveys per year. If a hospital does not meet this requirement they must survey all patients.
Are the HCAHPS survey results adjusted?
The HCAHPS survey results are adjusted for the mode of survey administration (mail, telephone and/or interactive voice response) and patient characteristics (health status, education, service line, age, ER admission, response percentile, service by linear age interactions, and primary language other than English). For more information about the HCAHPS survey visit www.hcahpsonline.org.
What are the goals of the HCAHPS survey?
The HCAHPS is designed to meet three goals:
- To produce information that allows objective and meaningful comparisons of hospitals on topics that are important to consumers;
- Public reporting of the survey results creates new incentives for hospitals to improve their quality of care; and
- Public reporting increases public accountability by providing information on the quality of hospital care provided in return for the payment the hospital receives for that care.
What is the source of the process measure data?
The Carolinas Center for Medical Excellence provides to the NC Quality Center quarterly hospital level data from the CMS/QIO data warehouse. The data include numerators and denominators from the most recent 3 months for each clinical process measure per hospital for all hospitals that have signed a consent form to share their data with the NC Quality Center.
The source for HCAHPS data is www.HospitalCompare.hhs.gov.
The hospital scores for HCAHPS and the individual quality measures displayed on this site (excluding the optimal care score) have been publicly reported by U.S. Dept. of Health and Human Services, through their online resource called Hospital Compare.
A list of when each measure became part of the Centers for Medicare and Medicaid (CMS) reporting program can be found here: MeasComp2012.pdf
Another website - WhyNotTheBest.org - shows the latest performance data on the process-of-care and hospital patient experience measures that are publicly reported by the Centers for Medicare and Medicaid Services. Users of the site can conduct side-by-side comparisons of 4,500 hospitals nationwide, track performance over time against numerous benchmarks, and download tools to improve health care quality.
What is an "Optimal Care" score?
The optimal care measure is a condition-level summary score that uses the "all or none" methodology to determine if a patient received all of the recommended treatment for which they were eligible. For each condition on this site (heart attack, heart failure, pneumonia, surgical care), performance is determined at the patient level and then summarized per hospital.
A hospital optimal care percentage score is calculated with this formula (per condition):
| Total # patients receiving all care measures | ||
| % = | / | |
| Total # patients qualifying for any care |
||
The steps to the methodology are:
Per discharged patient, determine if he/she met the eligibility to be categorized as a patient with a primarily diagnosis of heart attack, heart failure, pneumonia, or admitted for surgery.
Determine which measures per condition a patient has been deemed eligible for from a patient's medical chart.
Did the patient receive the recommended care for all eligible measures? If yes, patient receives a 1. If no, patient receives a 0.
Per hospital, sum the numbers from step 3 (above.) This is the sum of the numerators.
Per hospital, sum the total number of patients that had at least one eligible measure. This is the sum of the denominators and is a count of unique patients.
EXAMPLE: Let's say a patient having cardiac surgery was eligible to receive an antibiotic before incision to reduce the risk of infection and also to receive medication to prevent venous thromboembolism (i.e., blood clots). He was previously taking a beta blocker making him eligible for all 8 surgical care process measures. If the patient's timing to receive one of these treatments did not comply with the treatment guidelines then that patient received only 7 of the 8 elements. Thus, the score for that patient would be 0%. (Not a rate of 7/8 = 87.5%.) The numerator is either a 1 or a 0 and each patient has a denominator of 1. A hospital's rate per condition = the sum of the patient numerators divided by the total number of patients.
| Optimal Care: Surgical Care Example | |||
| # of Measures Eligible | Received All Eligible (1=yes) |
Patient Count | |
| Patient #1 | 8 | 1 | 1 |
| Patient #2 | 6 | 0 | 1 |
| Patient #3 | 7 | 0 | 1 |
| Patient #4 | 8 | 1 | 1 |
| Patient #5 | 5 | 1 | 1 |
| Patient #6 | 7 | 1 | 1 |
| SCIP-8 O.C. Totals | 4 | 6 | |
| Calculated SCIP-8 Optimal Care Score: 4/6 = .666 or 67% | |||
What are Hospital "Highlights?"
The website has a new section to highlight hospitals that we label as "High Reliable" as measured by the condition-specific optimal care score across time. For example, if a hospital's heart failure optimal care score is at or above 90% across the four 6-month time periods on this site then they meet the High Reliable criteria for heart failure care. This means that the hospital's compliance to the evidence-based recommended treatments for their patients are consistent and exceptional.
Criteria for High Reliable Hospital
Heart Attack: Optimal care score is 95% or higher for four 6-month time periods. Heart Failure: Optimal care score is 95% or higher for four 6-month time periods. Pneumonia: Optimal care score is 95% or higher for four 6-month time periods. Surgical Care: Optimal care score is 95% or higher for four 6-month time periods.
Another distinction in the new highlight section is a hospital deemed "Most Improved" per condition. These are the 12 hospitals that have seen the most improvement in their optimal care scores over time. More specifically, these are the top 3 hospitals per condition with the largest absolute value increase in optimal care score from 6-month baseline to the last 6-month time period without any substantial dips (10 percentage pts or more) in performance over 18 months.
Determining Most Improved Hospitals
Four time periods for the latest release were used:
- Apr 1, 2010 - Sep 30, 2010 (baseline)
- Oct 1, 2010 - Mar 31, 2011
- Apr 1, 2011 - Sep 30, 2011
- Oct 1, 2011 - Mar 31, 2012 (last 6 months available)
Steps taken to determine top 3 Most Improved hospitals per condition:
- Exclude hospitals with < 10 patients during the 4th time periods.
- Determine the absolute difference in the O.C. score between baseline and last 6 months.
- Exclude hospitals where time 4 score <= (time 3 score - 10 percentage points). (Only small dip allowed.)
- Exclude hospitals where time 3 score <= (time 2 score - 10 percentage points). (Only small dip allowed.)
- Exclude hospitals where time 2 score <= (time 1 score - 10 percentage points). (Only small dip allowed.)
- Per condition, list rank the remaining hospitals by descending absolute difference from baseline (calculated in step 2) and select the top 3 hospitals to be highlighted.
What is a PCI-capable hospital?
A PCI-capable hospital is a hospital that performs percutaneous coronary interventions (i.e., angioplasty, stenting) for patients suffering from a heart attack. To be grouped with PCI-capable hospitals on this website, the hospital performed these interventions on 10 or more patients during the last 6-month time period. This subgroup of NC hospitals is used to develop a separate mean score and 90th percentile score for the heart attack optimal care measure for only those hospitals that were performing PCI.
The subgroup was created so that hospitals with patients eligible for the measure titled "PCI Within 90 Minutes of Arrival" could be compared to each other in one graph. In addition, in the report tables, a hospital is highlighted in green if they meet or exceed the 90th percentile optimal care score of all hospitals (100% score) OR they meet or exceed the 90th percentile optimal care score of PCI hospitals (99%).
Why are some hospitals missing from the regional bar graph or trend graph?
This site graphically displays hospital performance on four optimal care measures for heart attack, heart failure, pneumonia, and surgical care.
There are two reasons why a hospital may not have an optimal care score for graphing. The first reason is patient volume that is too low (< 10 patients) to adequately compare or trend a hospital's performance. For example, during the six-month time period, if a hospital cared for less than 10 patients that had a heart attack, they do not have enough volume to be shown on this site's heart attack graphs. The second reason that a hospital does not have an optimal care score is incomplete data for a time-period. Both of these "Too Few" reasons are indicated with a "TF" in the report tables' column for the optimal care score.
The report tables on this site include measurement information on 110 non-federal, general acute-care hospitals in North Carolina. Three percent of NC's acute care hospitals have yet to join the NC Hospital Quality Performance Report site. Lastly, hospitals that primarily serve children, psychiatric, or rehabilitation patients are not included in this site.
How are the individual measures developed?
The information posted on this site comes from the quality data submitted by hospitals to the Quality Improvement Organization (QIO) Clinical Data Warehouse (part of the Centers for Medicare & Medicaid Services (CMS)) for all inpatient discharges. Except where noted, all rates are derived from data reflecting two quarters (six months) of experience, which is updated on a rolling basis.
Each of the measures tells you the proportion of cases where a hospital provided the recommended process of care. Only patients meeting the inclusion criteria for a measure are included in the calculation of the rate for a measure. A rate of 88% means that the hospital provided the recommended process of care 88% of the time. For example, the rates for aspirin at discharge for individuals who have had an acute myocardial infarction -- a heart attack -- tell you the percentage of patients who received an aspirin when they are discharged from the hospital. The ultimate goal for each measure is 100%. Hospitals with effective quality improvement programs are continually working toward this goal.
From CMS' QualityNet site, specification manuals are available that provide the detailed information on the specifications and the methodology used to calculate each measure's score. These manuals represent the result of efforts by the CMS and the Joint Commission to achieve identity among common national hospital performance measures and to share a single set of common documentation.
What are the average and top 10% scores?
On the top of each condition table there are two rows that display state-level scores. The first line says "Average for NC Hospitals". This is the mean score for the N.C. hospitals that currently participate in the NC Hospital Quality Performance Report. This average is unweighted by patient volume. For example, the score for the hospital with 20 patients has as much "weight" in calculating the average as the hospital with 200 patients.
The second line says "Top 10% of NC Hospitals". The values shown are the 90th percentile score for all the hospitals participating in the NC Hospital Quality Performance Report.
The optimal care state average and the 90th percentile were developed using only the scores from hospitals with 10 or more patients per condition.
What is being done to improve performance?
Hospital-level and statewide performance on the process measures displayed on this site is monitored by the NC Center for Hospital Quality and Patient Safety. Quality improvement initiatives are performed in hospitals to continuously improve evidence-based processes of care and to improve the health outcomes of all patients. To see a list of focused collaboratives led by the NC Quality Center: www.ncqualitycenter.org/collaboratives.lasso
Nationally, leaders of major hospitals, employers, physicians, nurses, and patient advocates along with state and federal governments are working in a shared effort called "Partnership for Patients" to make hospital care safer, more reliable, and less costly.
The North Carolina-Virginia Hospital Engagement Network (NoCVA HEN) works to meet the goals of the Partnership for Patients, a national initiative which aims to reduce patient harm by 40% and reduce readmissions by 20%. Hospitals are identifying strategies to reduce harmful patient events, share solutions, and have access to learning opportunities. For more information: www.ncqualitycenter.org/nocva/index.lasso
How can consumers and patients use this information?
This information helps patients, family, and friends compare the quality and safety of care in North Carolina hospitals. The quality measures reported on this site are well accepted nationally as steps that should be taken in the care of the conditions. Scientific evidence shows that high quality care leads to fewer repeat hospitalizations, hospital-acquired infections and medical errors, thereby reducing costs.
This site is a resource to inform consumers about hospital quality care and to help North Carolinians make good decisions about health care. Consumers should view this information as a starting point for educating themselves about hospital quality, for talking to their doctors about choosing a hospital for medical care, and for asking questions while receiving care in the hospital. Consumers may want to review the quality measure information when considering a hospital visit for a scheduled procedure.
In this report, each quality measure includes only patients whose history and condition indicate the treatment is appropriate. Talk to your doctor if you have questions about your treatment.
How can healthcare providers use this data?
This information is used by the medical community to heighten their awareness of the opportunity that exists to improve the care that they currently deliver. The public reporting of this data also provides an incentive for hospitals to continue to improve their use of recommended best treatments.
How can I get more information about this site?
Additional questions or comments about this site should be directed to eroedder@ncha.org.
Who is involved in this project?
The North Carolina Center for Hospital Quality and Patient Safety ("NC Quality Center") initiated the NC Performance Reporting Committee to determine which clinical measures should be included on this site. The Committee is a multi-disciplinary team consisting of physicians, nurses, and executives representing hospitals, health systems, insurance, industry, the Carolinas Center for Medical Excellence (CCME), NC Medical Society, and NC Department of Health and Human Services. The principles set by the group were to include measures that were actionable, standardized, well-defined, available, and would not add burden to hospital data collection efforts. In September 2009, the Workgroup's recommendation to include HCAHPS to this site were approved by the NC Quality Center's Board and endorsed by the NCHA Policy Development Committee.