Download as PDF

ACO Survey on CAHPS Vendors

July 2016 

The National Association of ACOs (NAACOS) is pleased to release the results of a survey of Accountable Care Organizations’ (ACOs) rating of vendors for the Consumer Assessment of Healthcare Providers and Systems (CAHPS) process. The Centers for Medicare & Medicaid Services (CMS) requires ACOs participating in the Medicare Shared Savings Program (MSSP), the Pioneer ACO, and Next Generation ACO models to collect patient experience feedback annually from their beneficiaries. Measuring the experience of patients is not only important for ACOs to understand how they can improve the care process for their patients, but also for CMS to hear from Medicare beneficiaries across the country on how care is being delivered.

The survey instrument used to measure patient experience is the CAHPS for ACOs Survey and includes two versions, ACO-9 or ACO-12, provided as an option for ACOs. Both the ACO-9 and ACO-12 versions of the survey include seven required measures that are used to determine an ACO’s quality score and one measure from the clinician and group version of the survey (CG-CAHPS), along with either one (ACO-9) or four (ACO-12) additional measures that were developed from qualitative research. The selection and process of the random sample of beneficiaries is conducted by CMS, and the survey is commonly implemented from November to February each year.

Each summer, CMS provides a list of approved vendors for ACOs to select and then contract with to conduct the CAHPS for ACOs survey. The role of a vendor can range from simply following CMS’s survey requirements and timelines to providing feedback throughout the survey process on areas for improvement. Selecting a high quality vendor can make the difference of not only seeing the survey results but also knowing how to improve. This is especially important considering seven of the survey measures are reflected on the ACO’s quality score, which points to the core of what ACOs are all about – providing better patient care for patients.

Methods

In June 2016, NAACOS emailed all of the MSSP ACOs, including NAACOS members and non-members, with information about the ACO Survey on CAHPS Vendors. The online survey was open for three weeks (June 14 to July 1, 2016). There were 134 ACOs that completed the survey, representing 125 unique ACOs and a 29 percent response rate. The survey questions were developed with extensive stakeholder feedback at both the beginning and end of the drafting process to ensure the questions were meaningful to ACOs.

About the CAHPS Vendors

The CAHPS vendors included in this survey are those who were approved by CMS for the 2015 ACO CAHPS cycle and can be found in Table 1 below. To be approved by CMS, vendors must meet minimum business requirements and attend two webinar training sessions in order to administer the CAHPS for ACOs surveys. As seen in Table 1, out of the 18 vendors listed, around half (54 percent) of the ACOs completed the survey on behalf of Press Ganey. Additionally, there were three vendors that did not have any ACOs complete a survey on their behalf.

Table 1: List of CAHPS vendors included in the survey, the percentage of respondents by vendor, and if the vendor is on the CMS conditionally approved list for 2016.

Survey Findings: Contract Differences

When asking ACOs what process they used to select a CAHPS vendor, the results found a variety of responses. As seen in Figure 1 below, two methods consisted of the majority used, which were selecting a vendor that was already being used elsewhere in the system (38 percent) and going through a traditional Request for Proposal (RFP) process (34 percent). The minority of methods included selecting a vendor through word of mouth (10 percent), researching and contacting several options from the list (6 percent), randomly selecting from the list (6 percent), and past experience (6 percent).

Figure 1: Surveyed ACOs response to the question: How did your ACO select your CAHPS vendor?

In order to get a better understanding of contracting and vendor variation, ACOs were asked a number of contract related questions, including:

• Did the ACO contract with the vendor for other projects?
  • 25 percent said yes; 71 percent said no; 4 percent were unsure.
    • Of those who said yes, some of the “other project” contracting examples included CAHPS surveys for clinics and hospitals (CG-CAHPS and H-CAHPS).
    • Did the vendor submit the survey results to CMS by the required deadline?
      • 97 percent said yes; 0 percent said no; 3 percent were unsure.
      • Did the ACO employ any penalties for falling short of contractual expectations?
        • 0 percent said yes; 92 percent said no; 8 percent were unsure.

Additionally, ACOs were asked if the vendor provided proactive approaches based on the data received during the CAHPS survey process to support the ACO to improve the experience for their patients. As seen in Figure 2, about a quarter (26 percent) of the ACOs received support, almost half (48 percent) of the ACOs did not receive any support, and 26 percent were unsure about the support provided. Of those vendors that did provide support, some vendors provided one summary report after the survey was completed, where others provided monthly or even weekly reports throughout the process. Other types of support included recommending specific improvement strategies, providing presentations to quality committees, and one vendor even supplied a “predictive modeling tool to proactively remediate deficiencies.”

Figure 2: Surveyed ACOs response to the question: Did the vendor provide proactive approaches based on the data received during the survey to support the ACO?

When asked if the ACO contracted with their vendor for clinician-level results, the respondents were roughly split into thirds, with 35 percent saying yes, 35 percent saying no, and 30 percent were unsure, as seen in Figure 3 below. Of the 35 percent who responded yes, 17 percent indicated that the reports were available without any additional charge but 18 percent said they contracted for the clinician-level reports but had to pay an extra charge for the service.

Figure 3: Surveyed ACOs response to the question: Did the ACO also contract to obtain clinician-level results?

Survey Findings: Rating the Vendor

When asked how the ACOs rate their vendor’s ability to meet project milestones, supply data reports and communicate, the results are similar with 60 to 63 percent of vendors meeting expectations across each question. In Figure 4, results indicated that 32 percent of vendors exceeded expectations to meet project milestones, 5 percent were below expectations, and 63 percent met expectations. In Figure 5, survey results found that only 24 percent of vendors exceeded expectations in supplying data reports, 13 percent were below expectations, and 63 percent met expectations. In Figure 6 below, results found that 29 percent exceeded expectations on communicating during the survey process, 11 percent were below expectations, and 63 percent met expectations.

Figures 4 and 5: Surveyed ACOs response to the question: How did ACOs rate their vendor’s ability to meet project milestones in a timely manner? And surveyed ACOs response to the question: How did ACOs rate their vendor’s ability to supply data reports?

Figure 6: Surveyed ACOs response to the question: How did ACOs rate their vendor’s communication during the survey process?

In asking ACOs to provide an overall performance ranking of their vendor, an expected bell curve result occurred, with 61 percent of the vendors achieving performance standards. As seen in Figure 7, on the next page, among those who exceeded performance, 26 percent exceeded performance standards and 6 percent significantly exceeded performance. Of those who were below standard, 6 percent barely achieved performance standards and only 1 percent were significantly below performance standards.

Figure 7: Surveyed ACOs response to the question: How does the ACO rate the overall performance of the vendor?

As seen in Figure 8 below, in asking the ACOs if they plan to contract with the same ACO again, 74 percent said they planned on renewing with the same vendor, 3 percent said no, and 23 percent said other. When asked if the ACO would recommend their vendor to another ACO, 84 percent said yes, 5 percent said no, and 11 percent said other (see Figure 9 below). This indicates that many of the ACOs are satisfied with their current vendors, but about 16 to 26 percent of ACOs are potentially looking for a new vendor.

 Figures 8 and 9: Surveyed ACOs response to the questions: Does the ACO intend to rehire the vendor? And does the ACO recommend the vendor to another ACO?

 Conclusion

As seen in the report’s findings, there is a variety of type of contracts, the services provided by vendors, and the experiences ACOs are having with their vendors. A common theme, however, was that ACOs rated at least 60 percent (Figures 4-9) of the vendors to have similar, satisfactory findings. Only a minority of vendors stood out as good or bad.

In looking at the biggest area for improvement, only 26 percent of vendors provided proactive feedback (Figure 2) to ACOs. Throughout the survey process, vendors can begin to see where patients feel they may be experiencing poor care. If vendors provide this data-guided feedback, ACOs can better target their limited resources to improving care for their patients.

Let us know! NAACOS would like to hear how this information may be helpful to ACOs in developing RFPs and evaluating survey vendors.

Acknowledgement and Contact Information

The survey and report was prepared by NAACOS with special assistance provided by Karen Furbush, Creatif Strategies.

For more information on the ACO Survey on CAHPS Vendors, contact Teresa Litton, MPH at [email protected]

Appendix: Survey Questions