Relationship between accreditation scores and the public disclosure of accreditation reports: a cross sectional study
http://www.100md.com
《健康质量安全杂志》
1 Health Sciences Division, Minister’s Secretariat, Ministry of Health, Labour and Welfare, 1-2-2 Kasumigaseki, Chiyoda-ku, Tokyo, Japan
2 Hospital Evaluation Services, Japan Council for Quality Health Care, Tokyo, Japan
ABSTRACT
Objective: To examine the association between accreditation scores and the disclosure of accreditation reports.
Design: A cross sectional study.
Setting: Hospitals participating in an accreditation programme in Japan.
Participants: 547 of the 817 hospitals accredited by the Japan Council for Quality Health Care (JCQHC) by January 2003.
Main outcome measures: Data on participation in public disclosure of accreditation reports through the JCQHC website were obtained from the JCQHC database. Comments on the disclosure were obtained using a questionnaire based survey.
Results: A total of 508 (93%) of the participating hospitals disclosed their accreditation reports on the JCQHC website. Public hospitals were significantly more committed to public disclosure than private hospitals, and larger hospitals were significantly more likely to participate in public disclosure than smaller hospitals. Accreditation scores were positively related to the public disclosure of hospital accreditation reports. Scores for patient focused care and efforts to meet community needs were significantly higher in actively disclosing hospitals than in non-disclosing hospitals. Among the large hospitals, scores for safety management were significantly higher in hospitals advocating disclosure than in non-disclosing hospitals.
Conclusions: There was a positive correlation between accreditation scores and public disclosure. Our results suggest that the public disclosure of accreditation reports should be encouraged to improve public accountability and the quality of care. Future studies should investigate the interaction between public disclosure, processes and outcomes.
Keywords: accreditation; public disclosure
The growing emphasis on quality of care data has created pressure on healthcare organisations to publicly disclose information.1,2 The information can be classified into two groups—performance and organisational assessment.
With regard to the disclosure of performance, the US has been at the forefront of releasing healthcare performance statistics.3 The main aims of disclosure are to facilitate informed choice in consumers, promote improvements in quality, ensure professional accountability, and encourage the control of costs.4–6 Evidence suggests that the availability of this information has changed the behaviour of providers,7,8,9,10 although it has had a limited impact on decision making in consumers.4,7,11,12 The National Voluntary Hospital Reporting Initiative has been disclosing performance measurement data on the web site of the Centers for Medicare & Medicaid Services since 2003.13 The UK is also moving towards the publication of more information, not to facilitate market competition but rather to improve public accountability.11 At present most UK statistics are of hospital performance and comparisons of mortality rates.14 Despite the global demand for publicly available information on quality of care, the best method of disclosing these performance data remains controversial.
Regarding disclosure of organisational assessment, the accreditation of healthcare organisations has traditionally been recognized as a symbol of quality assurance that reflects the commitment of an organisation to continuous improvement.15 Accreditation information was originally intended for use in internal quality assurance and organisation based efforts to improve the care, so it has generally been kept confidential.7,16 The American accreditation body, the Joint Commission on Accreditation of Healthcare Organisations (JCAHO), has recently begun to provide public performance reports and a comparative summary of accreditation scores on its website,17 and has made accreditation data public since July 2004.18 In contrast, the Healthcare Commission in the UK has produced a "compulsory" system of public reports of healthcare organisations.
The most distinctive characteristic of the Japanese healthcare system is to assure free access to health care under universal health insurance (box 1). The Japanese people have a right to receive medical care at any healthcare organization and hospitals cannot refuse any patients. In addition, more than 80% of hospitals are owned by non-public, non-profit organisations. Under such a healthcare system accreditation is a relatively recent phenomenon in Japan, but the Japan Council for Quality Health Care (JCQHC) has fully disclosed accreditation reports at the request of each hospital on the internet since September 2002. Are the accreditation scores related to disclosure of accreditation reports of hospitals To answer this question, we carried out a questionnaire based survey on the disclosure of accreditation reports in accredited hospitals in Japan and compared the accreditation scores.
Box 1 Healthcare system in Japan
Universal health insurance system
Free access to any hospitals
Primary fee for services, partly diagnostic procedure combination (similar to diagnostic related groups)
75 hospitals per million residents
Ownership: public (19%), private (81%)
Background
Unlike the USA, the healthcare system in Japan is not driven by market forces; in fact, similar to the situation in the UK, it is the Japanese government and consumers who have encouraged hospitals to increase public accountability. One of the main targets of recent regulatory reforms has been transparency in healthcare information.
Besides the government inspection, the JCQHC—which was founded in 1995 as a counterpart of the JCAHO—has developed standards and criteria for accreditation and began carrying out on-site assessments in 1997.19 Hospital accreditation is voluntary and requires an application fee from hospitals. There were 9122 hospitals in Japan in 2003. By October 2003, 1049 hospitals had been accredited by the JCQHC (817 hospitals by January 2003). This figure represented only 11.5% by October 2003 (9.0% by January 2003) of all hospitals in Japan. However, the number of applications for accreditation is rapidly increasing since the third party accreditation bodies were linked with certain reimbursement requirements in 2002.
Hospitals receive scores for each item in all areas with comments from the JCQHC in the standard accreditation process. The data disclosed by the JCQHC to the public include summary comments and accreditation scores for all the items assessed. So far the JCQHC has only disclosed information at the request of the accredited hospital. In addition, hospitals have been permitted to disclose their accreditation status and reports themselves. However, the JCQHC does not allow hospitals to disclose only selected parts of their accreditation reports; this policy is intended to give consumers access not only to favorable aspects of the report but also to information about services that require improvement. There are therefore two forms of disclosure: (1) self-disclosure to the public directly by hospitals and (2) disclosure by the JCQHC with agreement from the hospital concerned.
METHODS
Study group
In February 2003 a questionnaire survey was sent to all hospitals that had received JCQHC accreditation by January 2003. Of the 817 accredited hospitals, 547 (67%) responded to the survey. The group that responded comprised 219 (40.0%) small/medium sized hospitals with 200 or fewer beds, 260 (47.5%) large hospitals with more than 200 beds, and 68 (12.4%) other types of hospitals including psychiatric hospitals and long term care institutions. There were no significant differences between hospitals that did and did not respond in their size and location (urban/rural). The response rate of public hospitals (74.4%) was significantly higher than that of private hospitals (63.4%, 2 = 9.3, df = 1, p<0.01).
Questionnaire
The questionnaire was designed to gather data about hospital characteristics along with comments about the JCQHC’s disclosure of accreditation reports. Hospital characteristics included number of beds, ownership (public/private), and location (urban/rural) of the hospital. Public hospitals were defined as those run by national, prefectural (state), and local authorities. Urban areas were defined as cities with a population of more than 500 000.
Comments about the disclosure of accreditation reports were categorized into five general subject areas: (1) impact of disclosure on the public, (2) advantages to the hospital, (3) risks to the hospital, (4) JCQHC disclosure, and (5) hospital self-disclosure of information—that is, voluntary disclosure by the hospital by, for example, a pamphlet or a notice on a billboard in the hospital. Responders were asked to complete the 13 item questionnaire by rating each item as "agree" or "disagree". The questionnaire was sent to the Chief Executive Officer (CEO) of the hospitals for completion and all questionnaires were returned to us by the CEO of each hospital. The administrative staff responsible for managing the hospital completed the questionnaire in 504 (92.1%) of the 547 hospitals participating in the study.
Accreditation score
The public disclosure status and accreditation scores were obtained from the JCQHC database. The numbers of items assessed differed according to the type of hospital: a total of 233 for small/medium sized hospitals and 408 for large hospitals. The accreditation scores were combined with the responses to the questionnaire survey. The accreditation surveyors rated each item from 1 (poor) to 5 (excellent). Items were categorized into the following six areas: (1) mission and policy, (2) efforts to meet community needs, (3) medical/healthcare, (4) nursing care systems, (5) patient satisfaction and safety, and (6) administration. "Efforts to meet community needs" was divided into subcategories such as recognition of roles and functions in the community, collaboration with other healthcare organisations. An example is shown in table 1. The total scores for each area were calculated as a representative indicator. Internal consistency was determined by calculating Cronbach’s coefficient alpha20 which gave a mean value for each score of 0.75 indicating acceptable internal consistency (95% confidence interval 0.72 to 0.78).
We also calculated the scores for two specific areas—patient focused care and safety management—across all of the six categories described above. The former is assessed by the JCAHO in the USA and the latter is one of the most important topics in health care. In order to define these categories, the authors independently identified items among the JCQHC accreditation standards that were similar to those assessed by the JCAHO, then discussed inconsistencies between the list of items produced and reached a general consensus. As a result, the patient focused care category included 14 items for small/medium sized hospitals and 20 items for large hospitals, while the safety management category included 10 and 19 items, respectively. An example for safety management is "Management of incidents on safety is appropriate". The mean Cronbach’s coefficient alpha of each was 0.68, which indicated moderately acceptable internal consistency (95% confidence interval 0.64 to 0.72).
Analysis of data
The questionnaire data from all 547 respondents were used as a single data set for the initial analysis in which the 2 test was used to compare public disclosure with both hospital characteristics and comments about public disclosure. In the second analysis we combined the questionnaire survey data and accreditation scores from the JCQHC database. However, because standards and criteria vary depending on hospital size and are regularly revised every few years, it was difficult to analyse the scores as one data set. We therefore analysed the hospitals by size using only the latest data for the accreditation scores which were available for 159 of the 219 small/medium sized hospitals and 159 of the 260 large hospitals. The hospitals that disclosed information were divided into three groups according to the level of self-disclosure, and two groups according to the attitude towards disclosure. One way analysis of variance was performed with post hoc Bonferroni multiple paired comparisons. All tests were two tailed and analyses were performed using SPSS version 11.0 (SPSS Inc, Chicago, IL, USA).
RESULTS
A total of 508 (92.9%) of 547 responding hospitals allowed the JCQHC to disclose their accreditation reports. Table 2 summarizes the public disclosure data and hospital characteristics. Public hospitals were significantly more likely to agree to the disclosure of their accreditation reports than private hospitals, those in rural areas agreed to public disclosure significantly more often than those in urban areas, and larger hospitals were more committed to public disclosure than smaller hospitals.
Comments on the public disclosure of accreditation reports and JCQHC disclosure status are shown in table 3. More than 60% of the hospitals among the 547 respondents perceived public disclosure as good for consumers and hospitals. Considering the advantages to the hospital, 409 (80.5%) of the hospitals who disclosed their accreditation reports to the JCQHC agreed that "disclosure provides incentives for improving the quality of care because consumers in the community read accreditation reports". Hospitals in agreement with public disclosure were also significantly more likely than hospitals not in agreement with public disclosure to report that "disclosure increases the credibility of hospitals with consumers in the community". There were no significant differences in the perceived risks to the hospital associated with disclosure between the hospitals in favour and those not in favour of disclosure, although a total of 304 (55.6%) of all 547 respondents were concerned because "people are sensitive to unfavourable scores".
Among the 508 hospitals who agreed to disclosure by the JCQHC, 410 (80.7%) reported that their public disclosure was helpful. A total of 489 of the 547 respondents (89.4%) disclosed their accreditation reports themselves: 366 disclosed only their accreditation status and 123 disclosed more than this. Significantly more of the hospitals who agreed to disclosure by the JCQHC also released information than those who were not in favour of disclosure by the JCQHC.
Table 4 shows accreditation scores and disclosure of accreditation reports in small/medium sized hospitals. Hospitals with an active commitment to public disclosure had higher scores for the accreditation measures of "efforts to meet community needs" and "medical/healthcare" than those not in favour of disclosure. In addition, among the small/medium sized hospitals, scores for the patient focused care process were significantly higher in hospitals in agreement with active disclosure than in non-disclosing hospitals.
Among large hospitals there was no significant relationship between the disclosure of accreditation reports and accreditation scores (table 5). However, the accreditation scores differed according to the attitudes of hospitals toward the public disclosure. Accreditation scores in advocates of disclosure were significantly higher for measures of "efforts to meet community needs", "medical/healthcare", "administration", as well as the total scores.
The scores for patient focused care differed significantly according to the extent of the disclosure of accreditation reports, although multiple paired comparisons did not reach significant levels. In addition, among the large hospitals, scores for safety management were significantly higher in advocates of disclosure than in hospitals not in favour of disclosure.
DISCUSSION
Public disclosure was related to higher accreditation scores. Hospitals with disclosure of their accreditation reports had higher scores for efforts to meet the community needs and patient focused care. Large hospitals with disclosed data also had higher scores for safety management. Although deficiencies in care were also disclosed, most JCQHC accredited hospitals were highly positive about disclosure, believing that releasing information would improve their credibility with the community and provide incentives for internal quality improvement. However, there were concerns about some of the potential consequences of disclosure, including the misinterpretation of accreditation reports and overreaction to negative data.
Our survey was sent to all accredited hospitals in Japan and achieved a response rate of almost 70%, which was a representative sample. This is the first study to examine the association between public disclosure and accreditation scores. It is worth noting that there was a positive relationship between public disclosure and higher accreditation scores even though the majority of hospitals disclosed information and the power to make comparison is limited. Our approach allowed us to analyse both the subjective views of hospitals about disclosure and the objective accreditation scores.
Previous research has suggested that accreditation status might not be a true guarantor of high quality.14 However, our analysis of accreditation scores in this study showed that publicly disclosed hospitals had higher scores for efforts to meet community needs and for patient focused care, as well as higher total scores. This finding is consistent with previous studies in other countries which found that publicly reported hospitals with lowest performance scores were most negative about the validity of the data.21 Regardless of differences in healthcare systems, we can speculate that health care is moving towards public disclosure to assure accountability for quality of care.
Donabedian22 argued that neither the measurement of processes nor that of outcomes is inherently superior, and associations between the two can be important in the assessment of quality. We focused only on process data in this study. It is therefore important that future research examines the implications of public disclosure on outcome data.
Our results suggest that hospitals that are aware of the advantages of public disclosure actively disclose accreditation information. The advantages of public disclosure to hospitals should be used to encourage hospitals to disclose accreditation information in order to improve public accountability and the quality of care. Consumers have a right to know about the care that they receive and hospitals have a duty to provide this information.23 Hospitals should therefore proactively disclose information on their quality of care. However, in order to avoid unintended behaviour,24 it is essential to ensure the appropriate use of information.
Key messages
There is a positive correlation between public disclosure and accreditation scores.
Publicly disclosed hospitals had higher scores for efforts to meet community needs, patient focused care, and safety management, as well as higher total scores.
Most accredited hospitals were highly positive about disclosure.
Concerns remain about the possible consequences of disclosure including the misinterpretation of accreditation reports and overreaction to negative data.
Two limitations should be taken into account when considering our results. Firstly, the total number of accredited hospitals was relatively small as a result of the short history of accreditation in Japan, so our results are not representative of all hospitals. Also, the hospitals that have applied for JCQHC accreditation might be the leading hospitals providing a relatively high quality of care, so our sample might have been biased accordingly. Secondly, the measurement scale used for accreditation was not standardised even though several investigators measured each item during on-site visits and there was adequate internal consistency within the total scores. As the accreditation standards differed according to hospital size and were periodically updated, we divided the hospitals into two groups for the purpose of our analysis and used only the latest version of the standards.
The activities of the JCQHC, including public disclosure, have been covered in the media. Some magazines ranked accredited hospitals using the public disclosure data, and television and newspapers featured special programs on quality of care and publicized the activities of the JCQHC which resulted in dissemination of the third party evaluation. The impact of the public disclosure of accreditation reports on stakeholders over time is not known; the interaction between public disclosure, processes, and outcomes needs to be studied.
FOOTNOTES
This study was funded by the Ministry of Health, Labour, and Welfare, Japan.
The opinions expressed in this article are those of the authors and do not represent the official views of the Ministry of Health, Labour and Welfare.
REFERENCES
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638–42.
Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State cardiac surgery mortality reports. Med Care 1998;36:945–54.
Epstein AM. Rolling down the runway: the challenges ahead for quality report cards. JAMA 1998;279:1691–6.
Marshall MN, Shekelle PG, Leatherman S, et al. The public release of performance data: what do we expect to gain A review of the evidence. JAMA 2000;283:1866–74.
McCormick D, Himmelstein DU, Woolhandler S, et al. Relationship between low quality-of-care scores and HMO’s subsequent public disclosure of quality-of-care services. JAMA 2002;288:1484–90.
Localio AR, Hamory BH, Fisher AC, et al. The public release of hospital and physician mortality data in Pennsylvania. A case study. Med Care 1997;35:272–86.
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638–42.
Hannan EL, Kilburn H Jr, Racz M, et al. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA 1994;271:761–6.
Longo DR, Land G, Schramm W, et al. Consumer reports in health care: do they make a difference in patient care JAMA 1997;278:1579–84.
Schneider EC, Epstein AM. Influence of cardiac surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335:251–6.
Davies HT, Marshall MN. Public disclosure of performance data: does the public get what the public wants Lancet 1999;353:1639–40.
Hibbard JH, Slovic P, Peters E, et al. Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res 2002;37:291–313.
Centers for Medicare & Medicaid Services. National Voluntary Hospital Reporting Initiative. http://www.cms.hhs.gov/quality/hospital/NVHRIFactSheet.pdf (accessed 14 August 2004).
Marshall MN, Hiscock J, Sibbald B. Attitudes to the public release of comparative information on the quality of general practice care: qualitative study. BMJ 2002;325:1278.
Dean Beaulieu N, Epstein AM. National Committee on Quality Assurance health-plan accreditation: predictors, correlates of performance, and market impact. Med Care 2002;40:325–37.
Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989;320:53–6.
Joint Commission on Accreditation of Healthcare Organisations. 2003 Comprehensive accreditation manual for hospitals: the official handbook. Oakbrook Terrace: Joint Commission on Accreditation of Healthcare Organisations, 2003.
Joint Commission on Accreditation of Healthcare Organizations. Quality check. http://www.jcaho.org/quality+check/index.htm (accessed 14 August 2004).
Ito H, Iwasaki S, Nakano Y, et al. Direction of quality improvement activities of health care organizations in Japan. Int J Qual Health Care 1988;10:361–3.
Cronbach LJ. Coefficent alpha and the internal structure of tests. Psychometrika 1951;16:297–334.
Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts Health Aff (Millwood) 2003;22:84–94.
Donabedian A . The quality of care: How can it be assessed JAMA 1988;260:1743–8.
Berwick DM. Public performance reports and the will for change. JAMA 2002;288:1523–4.
Jacobson B . Hospital mortality league tables. BMJ 2003;326:777–8.(H Ito and H Sugawara)
2 Hospital Evaluation Services, Japan Council for Quality Health Care, Tokyo, Japan
ABSTRACT
Objective: To examine the association between accreditation scores and the disclosure of accreditation reports.
Design: A cross sectional study.
Setting: Hospitals participating in an accreditation programme in Japan.
Participants: 547 of the 817 hospitals accredited by the Japan Council for Quality Health Care (JCQHC) by January 2003.
Main outcome measures: Data on participation in public disclosure of accreditation reports through the JCQHC website were obtained from the JCQHC database. Comments on the disclosure were obtained using a questionnaire based survey.
Results: A total of 508 (93%) of the participating hospitals disclosed their accreditation reports on the JCQHC website. Public hospitals were significantly more committed to public disclosure than private hospitals, and larger hospitals were significantly more likely to participate in public disclosure than smaller hospitals. Accreditation scores were positively related to the public disclosure of hospital accreditation reports. Scores for patient focused care and efforts to meet community needs were significantly higher in actively disclosing hospitals than in non-disclosing hospitals. Among the large hospitals, scores for safety management were significantly higher in hospitals advocating disclosure than in non-disclosing hospitals.
Conclusions: There was a positive correlation between accreditation scores and public disclosure. Our results suggest that the public disclosure of accreditation reports should be encouraged to improve public accountability and the quality of care. Future studies should investigate the interaction between public disclosure, processes and outcomes.
Keywords: accreditation; public disclosure
The growing emphasis on quality of care data has created pressure on healthcare organisations to publicly disclose information.1,2 The information can be classified into two groups—performance and organisational assessment.
With regard to the disclosure of performance, the US has been at the forefront of releasing healthcare performance statistics.3 The main aims of disclosure are to facilitate informed choice in consumers, promote improvements in quality, ensure professional accountability, and encourage the control of costs.4–6 Evidence suggests that the availability of this information has changed the behaviour of providers,7,8,9,10 although it has had a limited impact on decision making in consumers.4,7,11,12 The National Voluntary Hospital Reporting Initiative has been disclosing performance measurement data on the web site of the Centers for Medicare & Medicaid Services since 2003.13 The UK is also moving towards the publication of more information, not to facilitate market competition but rather to improve public accountability.11 At present most UK statistics are of hospital performance and comparisons of mortality rates.14 Despite the global demand for publicly available information on quality of care, the best method of disclosing these performance data remains controversial.
Regarding disclosure of organisational assessment, the accreditation of healthcare organisations has traditionally been recognized as a symbol of quality assurance that reflects the commitment of an organisation to continuous improvement.15 Accreditation information was originally intended for use in internal quality assurance and organisation based efforts to improve the care, so it has generally been kept confidential.7,16 The American accreditation body, the Joint Commission on Accreditation of Healthcare Organisations (JCAHO), has recently begun to provide public performance reports and a comparative summary of accreditation scores on its website,17 and has made accreditation data public since July 2004.18 In contrast, the Healthcare Commission in the UK has produced a "compulsory" system of public reports of healthcare organisations.
The most distinctive characteristic of the Japanese healthcare system is to assure free access to health care under universal health insurance (box 1). The Japanese people have a right to receive medical care at any healthcare organization and hospitals cannot refuse any patients. In addition, more than 80% of hospitals are owned by non-public, non-profit organisations. Under such a healthcare system accreditation is a relatively recent phenomenon in Japan, but the Japan Council for Quality Health Care (JCQHC) has fully disclosed accreditation reports at the request of each hospital on the internet since September 2002. Are the accreditation scores related to disclosure of accreditation reports of hospitals To answer this question, we carried out a questionnaire based survey on the disclosure of accreditation reports in accredited hospitals in Japan and compared the accreditation scores.
Box 1 Healthcare system in Japan
Universal health insurance system
Free access to any hospitals
Primary fee for services, partly diagnostic procedure combination (similar to diagnostic related groups)
75 hospitals per million residents
Ownership: public (19%), private (81%)
Background
Unlike the USA, the healthcare system in Japan is not driven by market forces; in fact, similar to the situation in the UK, it is the Japanese government and consumers who have encouraged hospitals to increase public accountability. One of the main targets of recent regulatory reforms has been transparency in healthcare information.
Besides the government inspection, the JCQHC—which was founded in 1995 as a counterpart of the JCAHO—has developed standards and criteria for accreditation and began carrying out on-site assessments in 1997.19 Hospital accreditation is voluntary and requires an application fee from hospitals. There were 9122 hospitals in Japan in 2003. By October 2003, 1049 hospitals had been accredited by the JCQHC (817 hospitals by January 2003). This figure represented only 11.5% by October 2003 (9.0% by January 2003) of all hospitals in Japan. However, the number of applications for accreditation is rapidly increasing since the third party accreditation bodies were linked with certain reimbursement requirements in 2002.
Hospitals receive scores for each item in all areas with comments from the JCQHC in the standard accreditation process. The data disclosed by the JCQHC to the public include summary comments and accreditation scores for all the items assessed. So far the JCQHC has only disclosed information at the request of the accredited hospital. In addition, hospitals have been permitted to disclose their accreditation status and reports themselves. However, the JCQHC does not allow hospitals to disclose only selected parts of their accreditation reports; this policy is intended to give consumers access not only to favorable aspects of the report but also to information about services that require improvement. There are therefore two forms of disclosure: (1) self-disclosure to the public directly by hospitals and (2) disclosure by the JCQHC with agreement from the hospital concerned.
METHODS
Study group
In February 2003 a questionnaire survey was sent to all hospitals that had received JCQHC accreditation by January 2003. Of the 817 accredited hospitals, 547 (67%) responded to the survey. The group that responded comprised 219 (40.0%) small/medium sized hospitals with 200 or fewer beds, 260 (47.5%) large hospitals with more than 200 beds, and 68 (12.4%) other types of hospitals including psychiatric hospitals and long term care institutions. There were no significant differences between hospitals that did and did not respond in their size and location (urban/rural). The response rate of public hospitals (74.4%) was significantly higher than that of private hospitals (63.4%, 2 = 9.3, df = 1, p<0.01).
Questionnaire
The questionnaire was designed to gather data about hospital characteristics along with comments about the JCQHC’s disclosure of accreditation reports. Hospital characteristics included number of beds, ownership (public/private), and location (urban/rural) of the hospital. Public hospitals were defined as those run by national, prefectural (state), and local authorities. Urban areas were defined as cities with a population of more than 500 000.
Comments about the disclosure of accreditation reports were categorized into five general subject areas: (1) impact of disclosure on the public, (2) advantages to the hospital, (3) risks to the hospital, (4) JCQHC disclosure, and (5) hospital self-disclosure of information—that is, voluntary disclosure by the hospital by, for example, a pamphlet or a notice on a billboard in the hospital. Responders were asked to complete the 13 item questionnaire by rating each item as "agree" or "disagree". The questionnaire was sent to the Chief Executive Officer (CEO) of the hospitals for completion and all questionnaires were returned to us by the CEO of each hospital. The administrative staff responsible for managing the hospital completed the questionnaire in 504 (92.1%) of the 547 hospitals participating in the study.
Accreditation score
The public disclosure status and accreditation scores were obtained from the JCQHC database. The numbers of items assessed differed according to the type of hospital: a total of 233 for small/medium sized hospitals and 408 for large hospitals. The accreditation scores were combined with the responses to the questionnaire survey. The accreditation surveyors rated each item from 1 (poor) to 5 (excellent). Items were categorized into the following six areas: (1) mission and policy, (2) efforts to meet community needs, (3) medical/healthcare, (4) nursing care systems, (5) patient satisfaction and safety, and (6) administration. "Efforts to meet community needs" was divided into subcategories such as recognition of roles and functions in the community, collaboration with other healthcare organisations. An example is shown in table 1. The total scores for each area were calculated as a representative indicator. Internal consistency was determined by calculating Cronbach’s coefficient alpha20 which gave a mean value for each score of 0.75 indicating acceptable internal consistency (95% confidence interval 0.72 to 0.78).
We also calculated the scores for two specific areas—patient focused care and safety management—across all of the six categories described above. The former is assessed by the JCAHO in the USA and the latter is one of the most important topics in health care. In order to define these categories, the authors independently identified items among the JCQHC accreditation standards that were similar to those assessed by the JCAHO, then discussed inconsistencies between the list of items produced and reached a general consensus. As a result, the patient focused care category included 14 items for small/medium sized hospitals and 20 items for large hospitals, while the safety management category included 10 and 19 items, respectively. An example for safety management is "Management of incidents on safety is appropriate". The mean Cronbach’s coefficient alpha of each was 0.68, which indicated moderately acceptable internal consistency (95% confidence interval 0.64 to 0.72).
Analysis of data
The questionnaire data from all 547 respondents were used as a single data set for the initial analysis in which the 2 test was used to compare public disclosure with both hospital characteristics and comments about public disclosure. In the second analysis we combined the questionnaire survey data and accreditation scores from the JCQHC database. However, because standards and criteria vary depending on hospital size and are regularly revised every few years, it was difficult to analyse the scores as one data set. We therefore analysed the hospitals by size using only the latest data for the accreditation scores which were available for 159 of the 219 small/medium sized hospitals and 159 of the 260 large hospitals. The hospitals that disclosed information were divided into three groups according to the level of self-disclosure, and two groups according to the attitude towards disclosure. One way analysis of variance was performed with post hoc Bonferroni multiple paired comparisons. All tests were two tailed and analyses were performed using SPSS version 11.0 (SPSS Inc, Chicago, IL, USA).
RESULTS
A total of 508 (92.9%) of 547 responding hospitals allowed the JCQHC to disclose their accreditation reports. Table 2 summarizes the public disclosure data and hospital characteristics. Public hospitals were significantly more likely to agree to the disclosure of their accreditation reports than private hospitals, those in rural areas agreed to public disclosure significantly more often than those in urban areas, and larger hospitals were more committed to public disclosure than smaller hospitals.
Comments on the public disclosure of accreditation reports and JCQHC disclosure status are shown in table 3. More than 60% of the hospitals among the 547 respondents perceived public disclosure as good for consumers and hospitals. Considering the advantages to the hospital, 409 (80.5%) of the hospitals who disclosed their accreditation reports to the JCQHC agreed that "disclosure provides incentives for improving the quality of care because consumers in the community read accreditation reports". Hospitals in agreement with public disclosure were also significantly more likely than hospitals not in agreement with public disclosure to report that "disclosure increases the credibility of hospitals with consumers in the community". There were no significant differences in the perceived risks to the hospital associated with disclosure between the hospitals in favour and those not in favour of disclosure, although a total of 304 (55.6%) of all 547 respondents were concerned because "people are sensitive to unfavourable scores".
Among the 508 hospitals who agreed to disclosure by the JCQHC, 410 (80.7%) reported that their public disclosure was helpful. A total of 489 of the 547 respondents (89.4%) disclosed their accreditation reports themselves: 366 disclosed only their accreditation status and 123 disclosed more than this. Significantly more of the hospitals who agreed to disclosure by the JCQHC also released information than those who were not in favour of disclosure by the JCQHC.
Table 4 shows accreditation scores and disclosure of accreditation reports in small/medium sized hospitals. Hospitals with an active commitment to public disclosure had higher scores for the accreditation measures of "efforts to meet community needs" and "medical/healthcare" than those not in favour of disclosure. In addition, among the small/medium sized hospitals, scores for the patient focused care process were significantly higher in hospitals in agreement with active disclosure than in non-disclosing hospitals.
Among large hospitals there was no significant relationship between the disclosure of accreditation reports and accreditation scores (table 5). However, the accreditation scores differed according to the attitudes of hospitals toward the public disclosure. Accreditation scores in advocates of disclosure were significantly higher for measures of "efforts to meet community needs", "medical/healthcare", "administration", as well as the total scores.
The scores for patient focused care differed significantly according to the extent of the disclosure of accreditation reports, although multiple paired comparisons did not reach significant levels. In addition, among the large hospitals, scores for safety management were significantly higher in advocates of disclosure than in hospitals not in favour of disclosure.
DISCUSSION
Public disclosure was related to higher accreditation scores. Hospitals with disclosure of their accreditation reports had higher scores for efforts to meet the community needs and patient focused care. Large hospitals with disclosed data also had higher scores for safety management. Although deficiencies in care were also disclosed, most JCQHC accredited hospitals were highly positive about disclosure, believing that releasing information would improve their credibility with the community and provide incentives for internal quality improvement. However, there were concerns about some of the potential consequences of disclosure, including the misinterpretation of accreditation reports and overreaction to negative data.
Our survey was sent to all accredited hospitals in Japan and achieved a response rate of almost 70%, which was a representative sample. This is the first study to examine the association between public disclosure and accreditation scores. It is worth noting that there was a positive relationship between public disclosure and higher accreditation scores even though the majority of hospitals disclosed information and the power to make comparison is limited. Our approach allowed us to analyse both the subjective views of hospitals about disclosure and the objective accreditation scores.
Previous research has suggested that accreditation status might not be a true guarantor of high quality.14 However, our analysis of accreditation scores in this study showed that publicly disclosed hospitals had higher scores for efforts to meet community needs and for patient focused care, as well as higher total scores. This finding is consistent with previous studies in other countries which found that publicly reported hospitals with lowest performance scores were most negative about the validity of the data.21 Regardless of differences in healthcare systems, we can speculate that health care is moving towards public disclosure to assure accountability for quality of care.
Donabedian22 argued that neither the measurement of processes nor that of outcomes is inherently superior, and associations between the two can be important in the assessment of quality. We focused only on process data in this study. It is therefore important that future research examines the implications of public disclosure on outcome data.
Our results suggest that hospitals that are aware of the advantages of public disclosure actively disclose accreditation information. The advantages of public disclosure to hospitals should be used to encourage hospitals to disclose accreditation information in order to improve public accountability and the quality of care. Consumers have a right to know about the care that they receive and hospitals have a duty to provide this information.23 Hospitals should therefore proactively disclose information on their quality of care. However, in order to avoid unintended behaviour,24 it is essential to ensure the appropriate use of information.
Key messages
There is a positive correlation between public disclosure and accreditation scores.
Publicly disclosed hospitals had higher scores for efforts to meet community needs, patient focused care, and safety management, as well as higher total scores.
Most accredited hospitals were highly positive about disclosure.
Concerns remain about the possible consequences of disclosure including the misinterpretation of accreditation reports and overreaction to negative data.
Two limitations should be taken into account when considering our results. Firstly, the total number of accredited hospitals was relatively small as a result of the short history of accreditation in Japan, so our results are not representative of all hospitals. Also, the hospitals that have applied for JCQHC accreditation might be the leading hospitals providing a relatively high quality of care, so our sample might have been biased accordingly. Secondly, the measurement scale used for accreditation was not standardised even though several investigators measured each item during on-site visits and there was adequate internal consistency within the total scores. As the accreditation standards differed according to hospital size and were periodically updated, we divided the hospitals into two groups for the purpose of our analysis and used only the latest version of the standards.
The activities of the JCQHC, including public disclosure, have been covered in the media. Some magazines ranked accredited hospitals using the public disclosure data, and television and newspapers featured special programs on quality of care and publicized the activities of the JCQHC which resulted in dissemination of the third party evaluation. The impact of the public disclosure of accreditation reports on stakeholders over time is not known; the interaction between public disclosure, processes, and outcomes needs to be studied.
FOOTNOTES
This study was funded by the Ministry of Health, Labour, and Welfare, Japan.
The opinions expressed in this article are those of the authors and do not represent the official views of the Ministry of Health, Labour and Welfare.
REFERENCES
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638–42.
Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State cardiac surgery mortality reports. Med Care 1998;36:945–54.
Epstein AM. Rolling down the runway: the challenges ahead for quality report cards. JAMA 1998;279:1691–6.
Marshall MN, Shekelle PG, Leatherman S, et al. The public release of performance data: what do we expect to gain A review of the evidence. JAMA 2000;283:1866–74.
McCormick D, Himmelstein DU, Woolhandler S, et al. Relationship between low quality-of-care scores and HMO’s subsequent public disclosure of quality-of-care services. JAMA 2002;288:1484–90.
Localio AR, Hamory BH, Fisher AC, et al. The public release of hospital and physician mortality data in Pennsylvania. A case study. Med Care 1997;35:272–86.
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279:1638–42.
Hannan EL, Kilburn H Jr, Racz M, et al. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA 1994;271:761–6.
Longo DR, Land G, Schramm W, et al. Consumer reports in health care: do they make a difference in patient care JAMA 1997;278:1579–84.
Schneider EC, Epstein AM. Influence of cardiac surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335:251–6.
Davies HT, Marshall MN. Public disclosure of performance data: does the public get what the public wants Lancet 1999;353:1639–40.
Hibbard JH, Slovic P, Peters E, et al. Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res 2002;37:291–313.
Centers for Medicare & Medicaid Services. National Voluntary Hospital Reporting Initiative. http://www.cms.hhs.gov/quality/hospital/NVHRIFactSheet.pdf (accessed 14 August 2004).
Marshall MN, Hiscock J, Sibbald B. Attitudes to the public release of comparative information on the quality of general practice care: qualitative study. BMJ 2002;325:1278.
Dean Beaulieu N, Epstein AM. National Committee on Quality Assurance health-plan accreditation: predictors, correlates of performance, and market impact. Med Care 2002;40:325–37.
Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989;320:53–6.
Joint Commission on Accreditation of Healthcare Organisations. 2003 Comprehensive accreditation manual for hospitals: the official handbook. Oakbrook Terrace: Joint Commission on Accreditation of Healthcare Organisations, 2003.
Joint Commission on Accreditation of Healthcare Organizations. Quality check. http://www.jcaho.org/quality+check/index.htm (accessed 14 August 2004).
Ito H, Iwasaki S, Nakano Y, et al. Direction of quality improvement activities of health care organizations in Japan. Int J Qual Health Care 1988;10:361–3.
Cronbach LJ. Coefficent alpha and the internal structure of tests. Psychometrika 1951;16:297–334.
Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts Health Aff (Millwood) 2003;22:84–94.
Donabedian A . The quality of care: How can it be assessed JAMA 1988;260:1743–8.
Berwick DM. Public performance reports and the will for change. JAMA 2002;288:1523–4.
Jacobson B . Hospital mortality league tables. BMJ 2003;326:777–8.(H Ito and H Sugawara)