Observational Studies of Drug Safety — Aprotinin and the Absence of Transparency
http://www.100md.com
《新英格兰医药杂志》
The full safety profile of a new drug is rarely known at the time of approval by the Food and Drug Administration (FDA). Most drug-development programs designed for treatments of symptomatic indications are underpowered to detect any increased risk of rare drug reactions or change in background event rates attributable to the drug. Large, post-marketing, randomized, controlled trials provide robust data on drug safety but may be subject to multiple sources of bias. Observational studies of a drug's effects in clinical practice can offer additional information on risks. The recent discussions of aprotinin (Trasylol, Bayer) by the Cardiovascular and Renal Drugs Advisory Committee of the FDA, which I chair, provide insight into the strengths and weaknesses of using observational data to assess drug safety and highlight the importance of using a transparent and open process when reviewing such data.
Post-marketing observational studies permit the evaluation of drug safety in a large number of patients in a real-world setting, where practice patterns, including off-label use, can be assessed. Such studies have limited ability to determine causation, but they can detect signals that may suggest a safety concern.
In an observational study, decisions to use specific drugs are made by local physicians, according to their perceptions of the risks and benefits for particular patients. Such treatment allocation results in an imbalance in demographic and risk factors between patients given the drug of interest and those given an alternative or no treatment. An imbalance in various clinical factors will directly influence safety outcomes when patients at higher risk receive the drug. Therefore, appropriate statistical methods must be used to adjust for the nonrandom assignment to treatment groups.1
One way to address imbalances between groups is to use a propensity score that incorporates confounders and other covariates into a model predicting the probability of assignment to a particular treatment. This score can be used for adjustment or for matching patients who have similar probabilities of receiving a treatment. Differences in outcomes between treated and untreated patients (or patients treated with a comparison drug) with equal propensity scores provide a less biased estimate of treatment effect.
Aprotinin was approved by the FDA in 1993 as a means of reducing perioperative blood loss and the need for blood transfusion in patients undergoing coronary-artery bypass grafting. Neither the clinical trial database that led to approval nor the numerous randomized, controlled trials conducted after approval identified an association between aprotinin and any short-term increase in the risk of death or nonfatal cardiovascular events or any serious renal toxic effects (except for a transient increase in the serum creatinine concentration). However, in early 2006, two observational studies were published that raised serious concerns about the drug's safety.2,3
(Figure)
The Aprotinin Molecule.
Source of the aprotinin molecule data (Protein Data Bank number 2PTC): Research Collaboratory for Structural Bioinformatics Protein Data Bank (www.pdb.org).
One study, by Mangano et al.,2 evaluated 4374 patients undergoing coronary-artery bypass surgery. End points were prospectively defined, and data on a large number of clinical variables were collected for each patient. The decision to use aprotinin, aminocaproic acid, or tranexamic acid to inhibit fibrinolysis or to withhold antifibrinolytic therapy was made by the treating physician. Since group assignment was not random, it is not surprising that patients assigned to the group receiving aprotinin were at much higher risk for adverse cardiovascular outcomes than were the other patients in the study, who were treated with alternative therapies. When a propensity score was used in a logistic-regression model to adjust for these baseline differences, aprotinin was associated with a nonsignificant increase in the risk of death (odds ratio, 1.59; 95% confidence interval , 0.76 to 3.34) and with significant increases in the risks of renal events (odds ratio, 2.34; 95% CI, 1.27 to 4.31), cardiovascular events (odds ratio, 1.42; 95% CI, 1.09 to 1.86), and cerebrovascular events (odds ratio, 2.15; 95% CI, 1.14 to 4.06). The authors concluded that "the observed association between aprotinin and serious end-organ damage indicates that continued use is not prudent."
The second study, by Karkouti et al.,3 compared the risks associated with aprotinin with those associated with tranexamic acid in 898 patients undergoing high-risk cardiac surgery. The authors used a propensity score to match patients who had been given the different treatments — an approach that permitted the identification of a population of patients who were subsequently fully matched according to a large number of covariates. Calculations from the study data, performed by the Colorado Prevention Center, indicated no significant difference between the two treatments in the overall risk of myocardial infarction (odds ratio, 1.20; 95% CI, 0.52 to 2.75), stroke (odds ratio, 1.15; 95% CI, 0.56 to 2.40), or death (odds ratio, 0.91; 95% CI, 0.56 to 1.46). However, among patients who had abnormal renal function at baseline, those given aprotinin had a significantly increased risk of postoperative renal dysfunction (odds ratio, 1.69; 95% CI, 1.07 to 2.69). Because this study included fewer patients than the study by Mangano et al., it had less power to detect a safety problem.
These observational studies did not permit a definitive conclusion about cardiovascular or renal risk, but they did raise concern. The FDA appropriately issued a public health advisory about the potential risks posed by aprotinin and urged physicians to monitor their patients carefully for renal, cardiac, and cerebral toxic effects.4 The agency also convened a meeting of the Cardiovascular and Renal Drugs Advisory Committee on September 21, 2006, to evaluate the evidence on the cardiovascular and renal toxic effects of aprotinin. Our committee reviewed the published studies and the global safety and efficacy database submitted by Bayer, which included 45 randomized, controlled trials involving 2249 patients who received aprotinin and 2164 who received placebo.
The Bayer safety analysis was confined to the unadjudicated adverse events reported within 7 days after the administration of aprotinin, which included 520 deaths or serious cardiovascular or renal events. The results did not reveal any increased risk of fatal or nonfatal cardiovascular events. According to the Colorado Prevention Center's calculations based on Bayer's data, the odds ratio for death was 1.14 (95% CI, 0.80 to 1.62); for myocardial infarction, 1.17 (95% CI, 0.93 to 1.49); for stroke, 0.71 (95% CI, 0.42 to 1.18); and for renal failure, 1.15 (95% CI, 0.74 to 1.78). Two studies in which myocardial infarctions were adjudicated by an independent review committee showed a nonsignificant increase in the risk of myocardial infarction, with a point estimate of the odds ratio as high as 2.24 (95% CI, 0.56 to 9.00).
How can the results of these trials and the study by Karkouti et al. appear to be so different from the findings of Mangano et al.? Part of the problem may derive from the inability of observational studies to identify and measure all relevant covariates that may influence the outcome; similar studies using a few different covariates can sometimes come to opposite conclusions.
FDA advisory committee meetings usually include an independent review of data — either those provided by the sponsor or those used by the authors of relevant published studies. This independent and transparent process becomes even more critical in the evaluation of observational studies, whose conclusions can be highly influenced by the statistical methods used and the inability to identify all confounding factors. Unfortunately, Mangano did not give the FDA or the committee full access to his data, which would have allowed the agency to perform an independent review and analysis to validate his group's findings. Although there are many legitimate concerns with regard to the sharing of data (for instance, confidentiality and the need for informed consent), the lack of independent review greatly limited the committee's ability to draw conclusions from the study. Having reviewed all the data available, the committee decided that there was insufficient evidence to require an additional warning on aprotinin's labeling and agreed that the clinical data supported an acceptable safety and efficacy profile for aprotinin.
Days after the committee meeting, the FDA was made aware of additional observational data from the sponsor that had not been presented at the meeting. Bayer evidently had commissioned an observational study involving 67,000 patients who were given aprotinin.5 According to the initial FDA review of data from that study, aprotinin may be associated with "increased risk for death, kidney failure, congestive heart failure and stroke." The failure of Bayer to disclose all its data on aprotinin seriously undermined the advisory committee process and hindered the safety review.
Although observational studies cannot be definitive (and so should evoke measured responses), they can provide important new safety information that can direct further scientific and regulatory actions — if their findings can be confirmed. Since further analysis of the new data provided by Bayer is ongoing at the FDA, conclusions about the overall safety of aprotinin cannot be drawn at this time. Still, the aprotinin story demonstrates that full disclosure and a transparent process are essential in evaluating the findings of all studies germane to drug safety and the public health.
Source Information
Dr. Hiatt is a professor of medicine at the University of Colorado School of Medicine and president of the Colorado Prevention Center — both in Denver — and is the current chair of the Cardiovascular and Renal Drugs Advisory Committee of the FDA. Opinions expressed in this article are those of the author and do not necessarily reflect the opinions of the FDA or the advisory committee.
References
Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-763.
Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354:353-365.
Karkouti K, Beattie WS, Dattilo KM, et al. A propensity score case-control comparison of aprotinin and tranexamic acid in high-transfusion-risk cardiac surgery. Transfusion 2006;46:327-338.
Food and Drug Administration. Public health advisory: aprotinin injection (marketed as Trasylol). Updated September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/cder/drug/advisory/aprotinin.htm.)
Idem. FDA statement regarding new Trasylol data. September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/bbs/topics/NEWS/2006/NEW01472.html.)(William R. Hiatt, M.D.)
Post-marketing observational studies permit the evaluation of drug safety in a large number of patients in a real-world setting, where practice patterns, including off-label use, can be assessed. Such studies have limited ability to determine causation, but they can detect signals that may suggest a safety concern.
In an observational study, decisions to use specific drugs are made by local physicians, according to their perceptions of the risks and benefits for particular patients. Such treatment allocation results in an imbalance in demographic and risk factors between patients given the drug of interest and those given an alternative or no treatment. An imbalance in various clinical factors will directly influence safety outcomes when patients at higher risk receive the drug. Therefore, appropriate statistical methods must be used to adjust for the nonrandom assignment to treatment groups.1
One way to address imbalances between groups is to use a propensity score that incorporates confounders and other covariates into a model predicting the probability of assignment to a particular treatment. This score can be used for adjustment or for matching patients who have similar probabilities of receiving a treatment. Differences in outcomes between treated and untreated patients (or patients treated with a comparison drug) with equal propensity scores provide a less biased estimate of treatment effect.
Aprotinin was approved by the FDA in 1993 as a means of reducing perioperative blood loss and the need for blood transfusion in patients undergoing coronary-artery bypass grafting. Neither the clinical trial database that led to approval nor the numerous randomized, controlled trials conducted after approval identified an association between aprotinin and any short-term increase in the risk of death or nonfatal cardiovascular events or any serious renal toxic effects (except for a transient increase in the serum creatinine concentration). However, in early 2006, two observational studies were published that raised serious concerns about the drug's safety.2,3
(Figure)
The Aprotinin Molecule.
Source of the aprotinin molecule data (Protein Data Bank number 2PTC): Research Collaboratory for Structural Bioinformatics Protein Data Bank (www.pdb.org).
One study, by Mangano et al.,2 evaluated 4374 patients undergoing coronary-artery bypass surgery. End points were prospectively defined, and data on a large number of clinical variables were collected for each patient. The decision to use aprotinin, aminocaproic acid, or tranexamic acid to inhibit fibrinolysis or to withhold antifibrinolytic therapy was made by the treating physician. Since group assignment was not random, it is not surprising that patients assigned to the group receiving aprotinin were at much higher risk for adverse cardiovascular outcomes than were the other patients in the study, who were treated with alternative therapies. When a propensity score was used in a logistic-regression model to adjust for these baseline differences, aprotinin was associated with a nonsignificant increase in the risk of death (odds ratio, 1.59; 95% confidence interval , 0.76 to 3.34) and with significant increases in the risks of renal events (odds ratio, 2.34; 95% CI, 1.27 to 4.31), cardiovascular events (odds ratio, 1.42; 95% CI, 1.09 to 1.86), and cerebrovascular events (odds ratio, 2.15; 95% CI, 1.14 to 4.06). The authors concluded that "the observed association between aprotinin and serious end-organ damage indicates that continued use is not prudent."
The second study, by Karkouti et al.,3 compared the risks associated with aprotinin with those associated with tranexamic acid in 898 patients undergoing high-risk cardiac surgery. The authors used a propensity score to match patients who had been given the different treatments — an approach that permitted the identification of a population of patients who were subsequently fully matched according to a large number of covariates. Calculations from the study data, performed by the Colorado Prevention Center, indicated no significant difference between the two treatments in the overall risk of myocardial infarction (odds ratio, 1.20; 95% CI, 0.52 to 2.75), stroke (odds ratio, 1.15; 95% CI, 0.56 to 2.40), or death (odds ratio, 0.91; 95% CI, 0.56 to 1.46). However, among patients who had abnormal renal function at baseline, those given aprotinin had a significantly increased risk of postoperative renal dysfunction (odds ratio, 1.69; 95% CI, 1.07 to 2.69). Because this study included fewer patients than the study by Mangano et al., it had less power to detect a safety problem.
These observational studies did not permit a definitive conclusion about cardiovascular or renal risk, but they did raise concern. The FDA appropriately issued a public health advisory about the potential risks posed by aprotinin and urged physicians to monitor their patients carefully for renal, cardiac, and cerebral toxic effects.4 The agency also convened a meeting of the Cardiovascular and Renal Drugs Advisory Committee on September 21, 2006, to evaluate the evidence on the cardiovascular and renal toxic effects of aprotinin. Our committee reviewed the published studies and the global safety and efficacy database submitted by Bayer, which included 45 randomized, controlled trials involving 2249 patients who received aprotinin and 2164 who received placebo.
The Bayer safety analysis was confined to the unadjudicated adverse events reported within 7 days after the administration of aprotinin, which included 520 deaths or serious cardiovascular or renal events. The results did not reveal any increased risk of fatal or nonfatal cardiovascular events. According to the Colorado Prevention Center's calculations based on Bayer's data, the odds ratio for death was 1.14 (95% CI, 0.80 to 1.62); for myocardial infarction, 1.17 (95% CI, 0.93 to 1.49); for stroke, 0.71 (95% CI, 0.42 to 1.18); and for renal failure, 1.15 (95% CI, 0.74 to 1.78). Two studies in which myocardial infarctions were adjudicated by an independent review committee showed a nonsignificant increase in the risk of myocardial infarction, with a point estimate of the odds ratio as high as 2.24 (95% CI, 0.56 to 9.00).
How can the results of these trials and the study by Karkouti et al. appear to be so different from the findings of Mangano et al.? Part of the problem may derive from the inability of observational studies to identify and measure all relevant covariates that may influence the outcome; similar studies using a few different covariates can sometimes come to opposite conclusions.
FDA advisory committee meetings usually include an independent review of data — either those provided by the sponsor or those used by the authors of relevant published studies. This independent and transparent process becomes even more critical in the evaluation of observational studies, whose conclusions can be highly influenced by the statistical methods used and the inability to identify all confounding factors. Unfortunately, Mangano did not give the FDA or the committee full access to his data, which would have allowed the agency to perform an independent review and analysis to validate his group's findings. Although there are many legitimate concerns with regard to the sharing of data (for instance, confidentiality and the need for informed consent), the lack of independent review greatly limited the committee's ability to draw conclusions from the study. Having reviewed all the data available, the committee decided that there was insufficient evidence to require an additional warning on aprotinin's labeling and agreed that the clinical data supported an acceptable safety and efficacy profile for aprotinin.
Days after the committee meeting, the FDA was made aware of additional observational data from the sponsor that had not been presented at the meeting. Bayer evidently had commissioned an observational study involving 67,000 patients who were given aprotinin.5 According to the initial FDA review of data from that study, aprotinin may be associated with "increased risk for death, kidney failure, congestive heart failure and stroke." The failure of Bayer to disclose all its data on aprotinin seriously undermined the advisory committee process and hindered the safety review.
Although observational studies cannot be definitive (and so should evoke measured responses), they can provide important new safety information that can direct further scientific and regulatory actions — if their findings can be confirmed. Since further analysis of the new data provided by Bayer is ongoing at the FDA, conclusions about the overall safety of aprotinin cannot be drawn at this time. Still, the aprotinin story demonstrates that full disclosure and a transparent process are essential in evaluating the findings of all studies germane to drug safety and the public health.
Source Information
Dr. Hiatt is a professor of medicine at the University of Colorado School of Medicine and president of the Colorado Prevention Center — both in Denver — and is the current chair of the Cardiovascular and Renal Drugs Advisory Committee of the FDA. Opinions expressed in this article are those of the author and do not necessarily reflect the opinions of the FDA or the advisory committee.
References
Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med 1997;127:757-763.
Mangano DT, Tudor IC, Dietzel C. The risk associated with aprotinin in cardiac surgery. N Engl J Med 2006;354:353-365.
Karkouti K, Beattie WS, Dattilo KM, et al. A propensity score case-control comparison of aprotinin and tranexamic acid in high-transfusion-risk cardiac surgery. Transfusion 2006;46:327-338.
Food and Drug Administration. Public health advisory: aprotinin injection (marketed as Trasylol). Updated September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/cder/drug/advisory/aprotinin.htm.)
Idem. FDA statement regarding new Trasylol data. September 29, 2006. (Accessed November 2, 2006, at http://www.fda.gov/bbs/topics/NEWS/2006/NEW01472.html.)(William R. Hiatt, M.D.)