Monitoring global health: time for new solutions
http://www.100md.com
《英国医生杂志》
1 Harvard Initiative for Global Health, Harvard University, 104 Mount Auburn Street, Cambridge, MA 02138, USA, 2 School of Population Health, University of Queensland, Herston Road, Herston, Queensland 4006, Australia, 3 Office of Permanent Secretary, Ministry of Public Health, Tiwanond Road, Nonthaburi 11000, Thailand
Correspondence to: C J L Murray (christopher_murray@harvard.edu)
Improved global health monitoring requires new technologies and methods, strengthened national capacity, norms and standards, and gold standard global reporting. The World Health Organization's many functions limit its capacity for global reporting, and a new global health monitoring organisation is needed to provide independent gold standard health information to the world
Introduction
For several critical measures of health, delivery of interventions, and financial and human resources including many millennium development goal indicators, the current measurement technology is inadequate. Although validated methods have been developed to measure child mortality through household surveys in settings where vital registration is incomplete or non-existent,2-4 no adequate method exists to measure adult mortality in such settings.5 Although antibody tests for HIV mean that the prevalence of infection in the population can be ascertained from sample surveys,6 no affordable and feasible methods are currently available to assess tuberculosis in a community. Advances in immunology, proteomics, genomics, metabolomics, survey science, and statistical methods may result in new technologies or methods coming on line in the next decade that will dramatically improve our ability to monitor population health.7
Strengthening capacity
A third building block for effective health information is the establishment of global norms and standards on key indicators for different health programmes and for health systems overall, the best measurement methods for these indicators given current technology and analytical methods, and standardised definitions and classification systems. As illustrated by the International Classification of Diseases and Injuries over the past 50 years, WHO can play a powerful part in all this.12 For overall health statistics and many disease specific or risk factor specific areas, WHO can and should remain the leading institution. Nevertheless, WHO has shifted away from work on global norms and standards to emphasise country implementation of disease programmes,13 which will create a void that other institutions may need to fill.
Global reporting
WHO very often finds itself in the multiple roles of global advocate, provider of technical assistance to countries, monitor of progress towards targets, and evaluator of what works and what does not. The commitment and dedication of WHO's staff working in the technical departments is unquestionable. The problem is that staff representatives inevitably feel the tension between advocacy, monitoring, and evaluation. In other arenas, such as business, it would be unthinkable to ask a company to audit themselves. Two examples can serve to illustrate the consequences of this problem.
In the early 1990s, the global burden of disease study was initiated in part because the sum of deaths claimed by different WHO programmes exceeded the total number of deaths in the world several times.16 The intense pressure on technical programmes to keep their figures as large as possible was evident. Crudely, size of problem often translated into dollars from donor agencies. The global burden of disease study, published in 1996, set new benchmarks for internal consistency, comparability, and comprehensiveness of epidemiological information.17
Since 1998, WHO has adopted and implemented the study's approach to producing coherent epidemiological information. Given the importance of figures for advocacy, technical programmes would exert intense competitive pressure on the epidemiology and burden of disease team that was charged with bringing together WHO's annual assessments. These pressures were withstood because of the strong commitment of the senior management to valid, reliable, and comparable epidemiological information. This process, although imperfect, meant that internally consistent, comparable, and comprehensive information on incidence, prevalence, mortality, and disabling sequelae by age and sex for 14 epidemiological subregions of the world were published each year. Over the past year, as the focus of WHO has shifted to technical assistance for countries, the epidemiology and burden of disease team has been reduced from 22 to two staff members, and it seems unlikely that the effort to produce coherent comparable epidemiological data by major causes of death and ill health will continue.
WHO's tuberculosis programme provides another example of the tensions that inevitably exist in an organisation simultaneously developing advocacy material, providing countries with technical assistance, monitoring progress towards the global targets, and evaluating its own directly observed treatment, short course (DOTS) strategy. Direct measurements of tuberculosis in populations come from four potential sources: registered tuberculosis deaths, notified new cases of tuberculosis, purified protein derivative standard (PPD) skin testing of BCG scar negative children, and sputum prevalence surveys. For most low income countries, the only source is case notifications. WHO has estimated true incident cases from case notifications.18 19 Such estimates are useful for planning purposes but should not be used for monitoring progress or evaluating the DOTS strategy. In many countries, case notifications are being used to calculate the numerator of the case detection rate and to estimate the denominator. Trends in case detection (one of two key indicators for tuberculosis programmes) are derived exclusively from changes in the assumptions.
The figure shows, by using isoquants, all possible combinations of true incidence and case detection rates that are consistent with the number of notified, smear positive cases of tuberculosis in Mozambique for each year from 1996 to 2002. The choice of a unique combination of true incidence and case detection rates from each year's isoquant is arbitrary. WHO has a set of assumptions on which it bases the case detection rates for each year. As estimates for a given year are arbitrary, estimates of trends in case detection rates over time have no empirical basis—the isoquants in the figure are consistent with increasing or decreasing trends in case detection rates, and no information is available on the true trend. It should also be noted that WHO's assumed case detection rates and true incidence for the same year differ in the published global tuberculosis reports and on its website.20 Serial guessing is not a sound basis for monitoring progress towards a global target of 70% case detection. Because the WHO programme is also the global advocate for tuberculosis control, it cannot and does not bring attention to the fact that essentially no empirical basis exists to assess the trend in case detection in regions where tuberculosis is most prevalent, including sub-Saharan Africa.
Relation between case detection rate and true incidence of tuberculosis, Mozambique, 1996-2000. Isoquants represent all possible combinations of true incidence and case detection rates that are consistent with the number of smear positive cases for each year
National politics and impossible figures
To sustain increased investments in global health, gold standard information is essential. In the long run, such information will require better measurement methods and technologies, capacity strengthening in developing countries, and global norms and standards. To fulfil our need for gold standard information in the short term and to fuel government commitment to better health information in the long term, the institutional problems of global reporting by WHO and other UN agencies need to be solved. Although our analysis has stressed the structural issues of WHO, the problems of mixing advocacy, technical assistance, monitoring and evaluation roles, and maintaining independence from country pressure apply to other agencies and organisations as well.
Experience with the SARS outbreak shows that courageous leadership can withstand political pressure
Credit: GREG BAKER/AP PHOTOS
Solutions such as creating an independent arm of the WHO that reports directly to the executive board, a strategy adopted by the World Bank and the International Monetary Fund, are possible but pose nearly insurmountable political challenges. Initiatives that are specific to a disease or intervention to undertake independent monitoring to a gold standard may be helpful but will miss out on the many opportunities for improved global reporting that are common across programmes. We believe that the only viable solution will be to create a new, independent, health monitoring organisation.
The objective of this body would be to report regularly to the world on what is spent on health, what health services are delivered, and the impact of these efforts on population health. This organisation would be small as its main role would be to collate, analyse, and disseminate the best available evidence. Much of this work would be in close partnership with various actors such as the WHO technical programmes, the Global Alliance for Vaccines and Immunizations, and the Global Fund for AIDS, Tuberculosis and Malaria. To be effective, the organisation would need to be as sheltered as possible from the needs of advocacy on the one hand and country political interference on the other.
Those familiar with the complex governance issues that new entities such as the global alliance or the global fund have faced will recognise that solving the governance and financing issues for this organisation will not be easy, but it can be done. Success of such an organisation would depend on several key factors. Firstly, all representatives from the key stakeholders in global reporting would need to have a voice in the governance of this effort. Key stakeholders would include national governments, multilateral institutions (WHO, UNICEF, UNAIDS, UN Development Programme, World Bank, European Union, and others), bilateral donor agencies, a range of non-governmental organisations, and the research community.
Secondly, to be effective such a health monitoring organisation would have to be committed to the principles of validity, reliability, comparability of figures, an explicit data audit trail, and open consultation. As health information reaches a wider audience and touches on issues salient to everybody's life, scrutiny of this information will intensify. The criticisms of the World Health Report 2000 and the subsequent recommendations of the scientific peer review group27-29 highlight the importance of total transparency in the process of measurement.
Summary points
Improved global health monitoring requires new technologies and methods, strengthened national capacity, norms and standards, and gold standard global reporting
WHO's ability to undertake independent global reporting is limited by its simultaneous roles as global advocate, technical adviser to countries, monitor, and evaluator—and by intense political pressure
A new global health monitoring organisation is needed to provide independent gold standard health information to the world
Validity and reliability are familiar concepts in health measurement. Comparability means that results should be reported in a manner that allows meaningful comparisons to be made between countries and over time. Committing to an explicit data audit trail is costly but essential. This means that every step in the development of a figure should be made publicly available, including the primary data, details on the methods used to analyse the primary data (including corrections for known biases), and adequate documentation of all steps in the analysis. The extraordinary commitment of the Human Genome Project to put all primary data in the public domain with effectively no time lag should be held up as a model to follow. Finally, open consultation means that both governments and the scientific community at large should be able to comment and critique figures that are published. Fostering healthy debate on measurements will lead to better data collection and analysis.
Thirdly, an independent monitoring organisation, although it could be a relatively small undertaking of the order of $50m-70m (￡27m-38m;39m-55m) per year, would require stable core resources. Securing the right combination of governance and stable core resources is the main challenge for creating such an organisation. Without core resources, any organisation—regardless of governance structure—could be captured by its agencies that fund specific projects. Several financing models would be possible, ranging from endowment to assessed contributions from entities that would benefit from the dissemination of gold standard information to revenue generating services such as accreditation of figures.
In an era when the credibility of global health organisations is under attack, providing the public with credible, clear, and comparable health information will strengthen the commitment and resolution to scale up efforts on global health. At the Bangkok AIDS conference in July 2004, a journalist asked whether WHO figures on the delivery of antiretroviral therapy were "`Enron'-like."30 Although such comments are clearly unfair, doubts about what has been achieved and whether we are pursuing the right strategies to make progress could undermine the consensus that investing in global health is vital. It is in the interest of all of the global community to invest in solid independent monitoring and reporting.
Acknowledgements: We thank Julio Frenk, Sudhir Anand, Sue Goldie, and Anthony Rodgers for helpful comments. Jesse Abbot Klafter provided invaluable assistance on the analysis of tuberculosis coverage.
Contributors: CJLM, ADL and SW contributed the analysis and writing. All authors approved the final version.
Competing interests: None declared.
References
United Nations Development Programme. Human development report, 2003. The millennium development goals: a compact among nations to end human poverty. New York: Oxford University Press, 2003.
Boerma JT, Sommerfelt AE. Demographic and health surveys (DHS): contributions and limitations. World Health Stat Q 1993;46: 222-6.
Hill AG, Brass W, eds. The analysis of maternity histories. Liege, Belgium: Editions Derouaux-Ordina, 1992.
USAID, Macro International. Demographic and health surveys. www.measuredhs.com (accessed 15 Oct 2004).
Gakidou E, Hogan M, Lopez AD. Adult mortality: time for a reappraisal. Int J Epidemiol 2004;33: 710-7.
Boerma JT, Ghys PD, Walker N. Estimates of HIV-1 prevalence from national population-based surveys as a new gold standard. Lancet 2003;362: 1929-31.
Varmus H, Klausner R, Zerhouni E, Acharya T, Daar AS, Singer PA. Public health. Grand challenges in global health. Science 2003;302: 398-9.
Heymann DL, Aylward RB. Eradicating polio. N Engl J Med 2004;351: 1275-7.
INDEPTH Network. Population and health in developing countries: volume 1 population, health and survival at INDEPTH sites. Ottawa Canada: International Development Research Center, 2002. http://web.idrc.ca/es/ev-32411-201-1-DO_TOPIC.html (accessed 15 Oct 2004).
Cash RA, Narasimhan V. Impediments to global surveillance of infectious diseases: consequences of open reporting in a global economy. Bull World Health Org 2000;78: 1358-67.
Malison MD, Bull DL, Abamonte PA, Johnson SN, Lyons CC, Wilson GA. Management for international public health. Leadership Public Health 2000;5: 24-8.
World Health Organization. International statistical classification of diseases and related health problems. 10th rev. Geneva: WHO, 1992.
Lee JW. Global health improvement and WHO: shaping the future. Lancet 2003;362: 2083-8.
World Health Organization. The world health report. Geneva: WHO. www.who.int/whr/ (accessed 15 Oct 2004).
Lopez AD, Ahmad O, Guillot M, Inoue M, Fergusson B, Salomon J, et al. World mortality in 2000: life tables for 191 countries. Geneva: World Health Organization, 2002.
Murray CJL, Lopez AD. Evidence-based health policy—lessons from the global burden of disease study. Science 1996;274: 740-3.
Murray CJL, Lopez AD, eds. The global burden of disease: a comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Cambridge: Harvard University Press, 1996.
Corbett EL, Watt CJ, Walker N, Maher D, Williams BG, Raviglione MC, Dye C. The growing burden of tuberculosis. Global trends and interactions with the HIV epidemic. Arch Intern Med 2003;163: 1009-21.
Raviglione MC. The TB epidemic from 1992 to 2002. Tuberculosis 2003;83: 4-14.
World Health Organization. Global tuberculosis control—surveillance, planning, financing. Geneva: WHO. www.who.int/tb/ (accessed 15 Oct 2004).
Murray CJL, Shengelia B, Gupta N, Moussavi S, Tandon A, Thieren M. Validity of reported vaccination coverage in 45 countries. Lancet 2003;362: 1022-7.
Global Alliance for Vaccines and Immunization. Progress and challenges 2004. Geneva: GAVI, 2004.
The LATH Consortium. Immunization data quality audit evaluation report: final report from Deloitte Touche Tohmatsu Emerging Markets, Euro Health Group, and Liverpool Associates in Tropical Health. Geneva: World Health Organization, 2001.
Heymann DL, Rodier GR, WHO Operational Support Team to the Global Outbreak Alert and Response Network. Hot spots in a wired world: WHO surveillance of emerging and re-emerging infectious diseases. Lancet Infect Dis 2001;1: 345-53.
Enserink M. Emerging infectious diseases. A global fire brigade responds to disease outbreaks. Science 2004;303: 1605.
Gostin LO. International infectious disease law. Revision of the World Health Organization's international health regulations. JAMA 2004;291: 2623-7.
Almeida C, Braveman P, Gold MR, Szwarcwald CL, Ribeiro JM, Miglionico A, et al. Methodological concerns and recommendations on policy consequences of the World Health Report 2000. Lancet 2001;357: 1692-7.
Williams A. Science or marketing at WHO? A commentary on "World Health 2000." Health Econ 2001;10: 93-100.
Anand S, Ammar W, Evans T, Hasegawa T, Kissimova-Skarbek K, Langer A, et al. Report of the scientific peer review group on health systems performance assessment. In: Murray CJL, Evans DB, eds. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization, 2003: 839-915.
Naik G. WHO misses AIDS-treatment goal. Wall Street J 2004 July 12: B2.(Christopher J L Murray, d)
Correspondence to: C J L Murray (christopher_murray@harvard.edu)
Improved global health monitoring requires new technologies and methods, strengthened national capacity, norms and standards, and gold standard global reporting. The World Health Organization's many functions limit its capacity for global reporting, and a new global health monitoring organisation is needed to provide independent gold standard health information to the world
Introduction
For several critical measures of health, delivery of interventions, and financial and human resources including many millennium development goal indicators, the current measurement technology is inadequate. Although validated methods have been developed to measure child mortality through household surveys in settings where vital registration is incomplete or non-existent,2-4 no adequate method exists to measure adult mortality in such settings.5 Although antibody tests for HIV mean that the prevalence of infection in the population can be ascertained from sample surveys,6 no affordable and feasible methods are currently available to assess tuberculosis in a community. Advances in immunology, proteomics, genomics, metabolomics, survey science, and statistical methods may result in new technologies or methods coming on line in the next decade that will dramatically improve our ability to monitor population health.7
Strengthening capacity
A third building block for effective health information is the establishment of global norms and standards on key indicators for different health programmes and for health systems overall, the best measurement methods for these indicators given current technology and analytical methods, and standardised definitions and classification systems. As illustrated by the International Classification of Diseases and Injuries over the past 50 years, WHO can play a powerful part in all this.12 For overall health statistics and many disease specific or risk factor specific areas, WHO can and should remain the leading institution. Nevertheless, WHO has shifted away from work on global norms and standards to emphasise country implementation of disease programmes,13 which will create a void that other institutions may need to fill.
Global reporting
WHO very often finds itself in the multiple roles of global advocate, provider of technical assistance to countries, monitor of progress towards targets, and evaluator of what works and what does not. The commitment and dedication of WHO's staff working in the technical departments is unquestionable. The problem is that staff representatives inevitably feel the tension between advocacy, monitoring, and evaluation. In other arenas, such as business, it would be unthinkable to ask a company to audit themselves. Two examples can serve to illustrate the consequences of this problem.
In the early 1990s, the global burden of disease study was initiated in part because the sum of deaths claimed by different WHO programmes exceeded the total number of deaths in the world several times.16 The intense pressure on technical programmes to keep their figures as large as possible was evident. Crudely, size of problem often translated into dollars from donor agencies. The global burden of disease study, published in 1996, set new benchmarks for internal consistency, comparability, and comprehensiveness of epidemiological information.17
Since 1998, WHO has adopted and implemented the study's approach to producing coherent epidemiological information. Given the importance of figures for advocacy, technical programmes would exert intense competitive pressure on the epidemiology and burden of disease team that was charged with bringing together WHO's annual assessments. These pressures were withstood because of the strong commitment of the senior management to valid, reliable, and comparable epidemiological information. This process, although imperfect, meant that internally consistent, comparable, and comprehensive information on incidence, prevalence, mortality, and disabling sequelae by age and sex for 14 epidemiological subregions of the world were published each year. Over the past year, as the focus of WHO has shifted to technical assistance for countries, the epidemiology and burden of disease team has been reduced from 22 to two staff members, and it seems unlikely that the effort to produce coherent comparable epidemiological data by major causes of death and ill health will continue.
WHO's tuberculosis programme provides another example of the tensions that inevitably exist in an organisation simultaneously developing advocacy material, providing countries with technical assistance, monitoring progress towards the global targets, and evaluating its own directly observed treatment, short course (DOTS) strategy. Direct measurements of tuberculosis in populations come from four potential sources: registered tuberculosis deaths, notified new cases of tuberculosis, purified protein derivative standard (PPD) skin testing of BCG scar negative children, and sputum prevalence surveys. For most low income countries, the only source is case notifications. WHO has estimated true incident cases from case notifications.18 19 Such estimates are useful for planning purposes but should not be used for monitoring progress or evaluating the DOTS strategy. In many countries, case notifications are being used to calculate the numerator of the case detection rate and to estimate the denominator. Trends in case detection (one of two key indicators for tuberculosis programmes) are derived exclusively from changes in the assumptions.
The figure shows, by using isoquants, all possible combinations of true incidence and case detection rates that are consistent with the number of notified, smear positive cases of tuberculosis in Mozambique for each year from 1996 to 2002. The choice of a unique combination of true incidence and case detection rates from each year's isoquant is arbitrary. WHO has a set of assumptions on which it bases the case detection rates for each year. As estimates for a given year are arbitrary, estimates of trends in case detection rates over time have no empirical basis—the isoquants in the figure are consistent with increasing or decreasing trends in case detection rates, and no information is available on the true trend. It should also be noted that WHO's assumed case detection rates and true incidence for the same year differ in the published global tuberculosis reports and on its website.20 Serial guessing is not a sound basis for monitoring progress towards a global target of 70% case detection. Because the WHO programme is also the global advocate for tuberculosis control, it cannot and does not bring attention to the fact that essentially no empirical basis exists to assess the trend in case detection in regions where tuberculosis is most prevalent, including sub-Saharan Africa.
Relation between case detection rate and true incidence of tuberculosis, Mozambique, 1996-2000. Isoquants represent all possible combinations of true incidence and case detection rates that are consistent with the number of smear positive cases for each year
National politics and impossible figures
To sustain increased investments in global health, gold standard information is essential. In the long run, such information will require better measurement methods and technologies, capacity strengthening in developing countries, and global norms and standards. To fulfil our need for gold standard information in the short term and to fuel government commitment to better health information in the long term, the institutional problems of global reporting by WHO and other UN agencies need to be solved. Although our analysis has stressed the structural issues of WHO, the problems of mixing advocacy, technical assistance, monitoring and evaluation roles, and maintaining independence from country pressure apply to other agencies and organisations as well.
Experience with the SARS outbreak shows that courageous leadership can withstand political pressure
Credit: GREG BAKER/AP PHOTOS
Solutions such as creating an independent arm of the WHO that reports directly to the executive board, a strategy adopted by the World Bank and the International Monetary Fund, are possible but pose nearly insurmountable political challenges. Initiatives that are specific to a disease or intervention to undertake independent monitoring to a gold standard may be helpful but will miss out on the many opportunities for improved global reporting that are common across programmes. We believe that the only viable solution will be to create a new, independent, health monitoring organisation.
The objective of this body would be to report regularly to the world on what is spent on health, what health services are delivered, and the impact of these efforts on population health. This organisation would be small as its main role would be to collate, analyse, and disseminate the best available evidence. Much of this work would be in close partnership with various actors such as the WHO technical programmes, the Global Alliance for Vaccines and Immunizations, and the Global Fund for AIDS, Tuberculosis and Malaria. To be effective, the organisation would need to be as sheltered as possible from the needs of advocacy on the one hand and country political interference on the other.
Those familiar with the complex governance issues that new entities such as the global alliance or the global fund have faced will recognise that solving the governance and financing issues for this organisation will not be easy, but it can be done. Success of such an organisation would depend on several key factors. Firstly, all representatives from the key stakeholders in global reporting would need to have a voice in the governance of this effort. Key stakeholders would include national governments, multilateral institutions (WHO, UNICEF, UNAIDS, UN Development Programme, World Bank, European Union, and others), bilateral donor agencies, a range of non-governmental organisations, and the research community.
Secondly, to be effective such a health monitoring organisation would have to be committed to the principles of validity, reliability, comparability of figures, an explicit data audit trail, and open consultation. As health information reaches a wider audience and touches on issues salient to everybody's life, scrutiny of this information will intensify. The criticisms of the World Health Report 2000 and the subsequent recommendations of the scientific peer review group27-29 highlight the importance of total transparency in the process of measurement.
Summary points
Improved global health monitoring requires new technologies and methods, strengthened national capacity, norms and standards, and gold standard global reporting
WHO's ability to undertake independent global reporting is limited by its simultaneous roles as global advocate, technical adviser to countries, monitor, and evaluator—and by intense political pressure
A new global health monitoring organisation is needed to provide independent gold standard health information to the world
Validity and reliability are familiar concepts in health measurement. Comparability means that results should be reported in a manner that allows meaningful comparisons to be made between countries and over time. Committing to an explicit data audit trail is costly but essential. This means that every step in the development of a figure should be made publicly available, including the primary data, details on the methods used to analyse the primary data (including corrections for known biases), and adequate documentation of all steps in the analysis. The extraordinary commitment of the Human Genome Project to put all primary data in the public domain with effectively no time lag should be held up as a model to follow. Finally, open consultation means that both governments and the scientific community at large should be able to comment and critique figures that are published. Fostering healthy debate on measurements will lead to better data collection and analysis.
Thirdly, an independent monitoring organisation, although it could be a relatively small undertaking of the order of $50m-70m (￡27m-38m;39m-55m) per year, would require stable core resources. Securing the right combination of governance and stable core resources is the main challenge for creating such an organisation. Without core resources, any organisation—regardless of governance structure—could be captured by its agencies that fund specific projects. Several financing models would be possible, ranging from endowment to assessed contributions from entities that would benefit from the dissemination of gold standard information to revenue generating services such as accreditation of figures.
In an era when the credibility of global health organisations is under attack, providing the public with credible, clear, and comparable health information will strengthen the commitment and resolution to scale up efforts on global health. At the Bangkok AIDS conference in July 2004, a journalist asked whether WHO figures on the delivery of antiretroviral therapy were "`Enron'-like."30 Although such comments are clearly unfair, doubts about what has been achieved and whether we are pursuing the right strategies to make progress could undermine the consensus that investing in global health is vital. It is in the interest of all of the global community to invest in solid independent monitoring and reporting.
Acknowledgements: We thank Julio Frenk, Sudhir Anand, Sue Goldie, and Anthony Rodgers for helpful comments. Jesse Abbot Klafter provided invaluable assistance on the analysis of tuberculosis coverage.
Contributors: CJLM, ADL and SW contributed the analysis and writing. All authors approved the final version.
Competing interests: None declared.
References
United Nations Development Programme. Human development report, 2003. The millennium development goals: a compact among nations to end human poverty. New York: Oxford University Press, 2003.
Boerma JT, Sommerfelt AE. Demographic and health surveys (DHS): contributions and limitations. World Health Stat Q 1993;46: 222-6.
Hill AG, Brass W, eds. The analysis of maternity histories. Liege, Belgium: Editions Derouaux-Ordina, 1992.
USAID, Macro International. Demographic and health surveys. www.measuredhs.com (accessed 15 Oct 2004).
Gakidou E, Hogan M, Lopez AD. Adult mortality: time for a reappraisal. Int J Epidemiol 2004;33: 710-7.
Boerma JT, Ghys PD, Walker N. Estimates of HIV-1 prevalence from national population-based surveys as a new gold standard. Lancet 2003;362: 1929-31.
Varmus H, Klausner R, Zerhouni E, Acharya T, Daar AS, Singer PA. Public health. Grand challenges in global health. Science 2003;302: 398-9.
Heymann DL, Aylward RB. Eradicating polio. N Engl J Med 2004;351: 1275-7.
INDEPTH Network. Population and health in developing countries: volume 1 population, health and survival at INDEPTH sites. Ottawa Canada: International Development Research Center, 2002. http://web.idrc.ca/es/ev-32411-201-1-DO_TOPIC.html (accessed 15 Oct 2004).
Cash RA, Narasimhan V. Impediments to global surveillance of infectious diseases: consequences of open reporting in a global economy. Bull World Health Org 2000;78: 1358-67.
Malison MD, Bull DL, Abamonte PA, Johnson SN, Lyons CC, Wilson GA. Management for international public health. Leadership Public Health 2000;5: 24-8.
World Health Organization. International statistical classification of diseases and related health problems. 10th rev. Geneva: WHO, 1992.
Lee JW. Global health improvement and WHO: shaping the future. Lancet 2003;362: 2083-8.
World Health Organization. The world health report. Geneva: WHO. www.who.int/whr/ (accessed 15 Oct 2004).
Lopez AD, Ahmad O, Guillot M, Inoue M, Fergusson B, Salomon J, et al. World mortality in 2000: life tables for 191 countries. Geneva: World Health Organization, 2002.
Murray CJL, Lopez AD. Evidence-based health policy—lessons from the global burden of disease study. Science 1996;274: 740-3.
Murray CJL, Lopez AD, eds. The global burden of disease: a comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Cambridge: Harvard University Press, 1996.
Corbett EL, Watt CJ, Walker N, Maher D, Williams BG, Raviglione MC, Dye C. The growing burden of tuberculosis. Global trends and interactions with the HIV epidemic. Arch Intern Med 2003;163: 1009-21.
Raviglione MC. The TB epidemic from 1992 to 2002. Tuberculosis 2003;83: 4-14.
World Health Organization. Global tuberculosis control—surveillance, planning, financing. Geneva: WHO. www.who.int/tb/ (accessed 15 Oct 2004).
Murray CJL, Shengelia B, Gupta N, Moussavi S, Tandon A, Thieren M. Validity of reported vaccination coverage in 45 countries. Lancet 2003;362: 1022-7.
Global Alliance for Vaccines and Immunization. Progress and challenges 2004. Geneva: GAVI, 2004.
The LATH Consortium. Immunization data quality audit evaluation report: final report from Deloitte Touche Tohmatsu Emerging Markets, Euro Health Group, and Liverpool Associates in Tropical Health. Geneva: World Health Organization, 2001.
Heymann DL, Rodier GR, WHO Operational Support Team to the Global Outbreak Alert and Response Network. Hot spots in a wired world: WHO surveillance of emerging and re-emerging infectious diseases. Lancet Infect Dis 2001;1: 345-53.
Enserink M. Emerging infectious diseases. A global fire brigade responds to disease outbreaks. Science 2004;303: 1605.
Gostin LO. International infectious disease law. Revision of the World Health Organization's international health regulations. JAMA 2004;291: 2623-7.
Almeida C, Braveman P, Gold MR, Szwarcwald CL, Ribeiro JM, Miglionico A, et al. Methodological concerns and recommendations on policy consequences of the World Health Report 2000. Lancet 2001;357: 1692-7.
Williams A. Science or marketing at WHO? A commentary on "World Health 2000." Health Econ 2001;10: 93-100.
Anand S, Ammar W, Evans T, Hasegawa T, Kissimova-Skarbek K, Langer A, et al. Report of the scientific peer review group on health systems performance assessment. In: Murray CJL, Evans DB, eds. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization, 2003: 839-915.
Naik G. WHO misses AIDS-treatment goal. Wall Street J 2004 July 12: B2.(Christopher J L Murray, d)