Practical Application of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) in a Public Sector Forensic Se
http://www.100md.com
《美国精神病学与法律学报》
Dr. Pinals is Associate Professor of Psychiatry and Co-Director, and Drs. Tillbrook and Mumley are Assistant Professors of Psychiatry, Law and Psychiatry Program, Department of Psychiatry, University of Massachusetts Medical School, Worcester, MA.
Abstract
Several instruments have been developed to assess defendants’ competence to stand trial (CST). These instruments have become increasingly sophisticated mainstays of CST evaluation research. Less is known, however, about their clinical utility. Continuous quality-improvement (CQI) efforts within forensic practice are one means of helping ascertain the effectiveness and efficiency of the use of these measures, especially in public sector settings. In this article, the authors describe a CQI project examining the use of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) on an inpatient forensic evaluation service. Results of a small, local CQI project cannot be generalized to all settings. However, from this CQI experience, the authors found the MacCAT-CA to be advantageous in assessing CST in certain defendants. Given some limitations of its practical utility, including its use with defendants putting forth poor effort, those with significant cognitive impairments, and those from various cultural backgrounds, the authors did not find that it would be an efficient practice to utilize the instrument in all CST assessments within a busy, public sector, inpatient forensic evaluation service. Having the instrument available for use as an adjunct to a clinical interview in particular cases may offer the most advantage. In addition, it is a useful tool for educating trainees about complex CST domains. This case-based CQI project reveals some basic strengths and weaknesses of the instrument and offers direction for further exploration in a systematic research study.
The prevailing legal standard for competence to stand trial has not changed since 1960 when the United States Supreme Court stated:
The test will be whether [the defendant] has sufficient present ability to consult with his lawyer with a reasonable degree of rational understanding, and whether he has a rational as well as factual understanding of the proceedings against him [Ref. 1, p 402].
The Supreme Court has upheld the substantive criteria enumerated in Dusky v. United States for four and a half decades,2 and most states have adopted this language with only minor modifications, if any, in wording.3 Numerous states mandate that mental health experts consider and address the Dusky factors during a competence evaluation and when rendering an opinion about a defendant’s ability to proceed to trial. Schreiber,4 however, noted the difficulty experienced by mental health experts in applying psychological and behavioral observations of defendants to the legal criteria of Dusky.
Over the past 35 years, numerous mental health and legal scholars have set out to operationalize the Dusky language to guide mental health professionals who are asked to perform a large number of competence evaluations, especially those in public mental health settings. Earlier estimates place the number of competence-to-stand-trial evaluations performed each year in the United States at around 20,000,5 with more recent statistics citing 60,000,6 making such evaluations a significant focus of mental health inquiry in the criminal justice system. To assist experts in evaluating competence to stand trial, several instruments have been developed, each differing slightly in their purpose, scope, design, and the degree to which they have been examined empirically. These instruments have taken various forms, including checklists of items,7,8 a sentence-completion questionnaire,9 and subjectively rated interview guides that provide some structure for assessment of the Dusky criteria.10 Before the past decade, widely used instruments included the Competence to Stand Trial Assessment Instrument,11 Georgia Court Competency Test-Mississippi State Hospital revision,12 Interdisciplinary Fitness Interview (IFI),13 and Fitness Interview Test (FIT).14 The last two measures, IFI and FIT, were revised to reflect specific changes in the legal statutes in the jurisdictions where they were developed and commonly employed. The IFI-Revised15 reflects changes to Utah’s new competence code, which mandates that specific competence domains be addressed, and the FIT-Revised16 was modified in accordance with changes in Canadian law.
Each of these "first-generation" measures has been utilized and reviewed extensively. Although they have each provided a unique contribution to the assessment of competence to stand trial, they have been critiqued regarding their psychometric shortcomings, their neglect of sampling important domains relevant to Dusky (e.g., rational decision-making), and their limited practical utility (for more comprehensive reviews, see Refs. 3,10,17). Grisso18 called on researchers to revise existing measures (i.e., resulting in the IFI-R and FIT-R) or to develop new competence assessment instruments that address these shortcomings and incorporate more standardized administration and scoring. A "second generation" of research resulted in the development of two relatively new competence measures, the MacArthur Competence Assessment Tool–Criminal Adjudication (MacCAT-CA)19,20 and the Evaluation of Competency to Stand Trial–Revised (ECST-R).21 Although legal challenges to these instruments may still be seen, the use of competence assessment instruments is found in case law (see, for example, Refs. 22–24), suggesting the value of distinct evaluations for competence to stand trial.21,25 Furthermore, reviews have indicated that competence-to-stand-trial evaluations in research settings using these specific assessment instruments are highly reliable and valid, although individually these measures differ in their conceptual assumptions.3,5,21 What has not been explored, however, is their practical applicability in improving clinical practice.
Enhancing practice and improving the quality of services provided are goals recognized throughout medicine.26,27 The need to look at the quality, effectiveness, and efficiency of forensic evaluations conducted for the courts within public sector settings is in line with that goal. To that end, we developed a continuous quality-improvement (CQI) project to examine our competence assessment process, especially given the lack of information available about the clinical utility of second-generation competence assessment instruments. In designing this project, we decided to examine the MacCAT-CA in more detail, with a specific focus on delineating important considerations when choosing to use this measure, as well as identifying the potential contributions of the MacCAT-CA to the competence evaluation process in public sector settings. In this article, we report our findings from this CQI project and discuss the MacCAT-CA’s utility in forensic practice, including some practical considerations pertaining to its administration, scoring, and use in assisting evaluators in formulating opinions of competence to proceed.
MacArthur Competence Assessment Tool–Criminal Adjudication
The MacCAT-CA, developed over years in response to the need for an objectively scored assessment tool incorporating Dusky criteria, was studied extensively between 1996 and 199828 (for a complete review of relevant MacArthur studies, see Ref. 29). It comprises 22 items that are organized into three sections, including criteria covered by Dusky. The MacCAT-CA includes a standard administration, beginning with a vignette of a hypothetical offense involving an aggravated assault at a pool hall. This vignette is read by the examiner and sets the context for the first 16 items. For the remaining six items, defendants are asked to make comparative judgments about their own cases and to explain their reasoning. Defendants’ responses are rated on a three-point scale (0, 1, or 2) against explicit scoring criteria.
The first section, Understanding, assesses defendants’ ability to understand general information about the legal system and the process of adjudication. Most of these items consist of two parts, the first of which is an assessment of defendants’ current knowledge of the legal system. If their answers are less than satisfactory (i.e., a score less than 2), they are read a short narrative that educates them on the essential components. They are then asked to paraphrase this information. The purpose of the disclosure is to assess defendants’ capacity to learn, remember, and apply new information relevant to the legal process.
The second section, Reasoning, evaluates defendants’ ability to discern the legal relevance of information and their capacity to reason about specific choices that confront defendants during the course of a typical criminal proceeding. For the first five of these eight items, defendants are asked to distinguish between more and less legally relevant factual information. The last three items require defendants to reason through hypothetical legal scenarios (e.g., plea bargain) and reach rational decisions.
Appreciation is the last scale, and it assesses defendants’ ability to appreciate the meaning and consequences of their own legal circumstances. These six items depart from the hypothetical vignette and investigate defendants’ beliefs and perceptions about their roles as defendants. Compared with the previous 16 items, these are scored on the basis of whether the answers are plausible (i.e., grounded in reality or influenced by delusional beliefs).
Otto et al.28 reported evidence of the MacCAT-CA’s construct validity, with its scales correlating in an expected pattern with measures of psychopathology, cognitive ability, and clinical judgments. In addition, the findings demonstrate good predictive utility with acceptable hit rate ranges (i.e., specificity, sensitivity, positive, and negative predictive values, and false-negative rates). The results of that study supported the development of national norms that examiners can use as guideposts to compare a particular defendant’s performance to others in similar circumstances.
Compared with earlier measures, it has been suggested that the MacCAT-CA does a more thorough job of sampling relevant competence-related abilities, including areas of plea bargaining,3,5 but there have been concerns about its ability to assess a defendant’s capacity to consult with counsel.30,31 Furthermore, according to Dusky, competence-related abilities should apply to the particular defendant being evaluated and his or her legal case. Much of the MacCAT-CA, however, focuses on a hypothetical vignette and assesses understanding and reasoning related to the fictional case as opposed to the defendant’s own situation. The psychometric properties of the MacCAT-CA have been reported20 and critiqued,3,30 but its clinical utility (i.e., ease of use by forensic examiners and ability to elicit data from evaluees) has not, to our knowledge, been investigated. This is of particular interest, because the format of the MacCAT-CA differs notably from earlier competence assessment instruments.
Continuous Quality-Improvement Project Purpose and Design
Among the goals of the CQI project was the identification of ways to improve service delivery by understanding factors associated with the clinical usefulness of the MacCAT-CA (e.g., clinical or case-related variables that suggest that the administration of this instrument would be helpful in rendering clinical opinions about competence to stand trial). We also wondered if there were circumstances in which using the MacCAT-CA might not be helpful in formulating opinions about a defendant’s competence to proceed.
We submitted a summary of this CQI project to the Massachusetts Department of Mental Health Research Review Committee and the Institutional Review Board of the University of Massachusetts Medical School for review. Both bodies determined that the project was exempt from review.
Setting
The CQI project was performed in a 15-bed coeducational inpatient court evaluation unit located in a state psychiatric hospital in Massachusetts. In this facility, court-ordered evaluations of psycholegal issues such as competence to stand trial, criminal responsibility, and aid in sentencing are conducted by staff of the hospital’s forensic service, which consists of psychologists and psychiatrists with specialized training in forensic evaluation. The forensic service also functions as a major training site for forensic evaluators, including psychiatry and psychology forensic fellows. In Massachusetts, public sector forensic evaluations can only be performed by mental health professionals certified to do so via a state-wide program that establishes standards for assessment procedures and reports. All evaluators who participated in the current CQI project either had this certification or were forensic trainees supervised by a psychologist or psychiatrist with this credential. Several evaluators were involved in this project, three of whom are the authors of this article. Finally, before participating, all evaluators involved received training in administration of the MacCAT-CA.
CQI Project Implementation
Utilizing a model incorporating the quality-improvement rubric of "Plan, Do, Check, Act,"32,33 after the planning stage, evaluators attempted to administer the MacCAT-CA as part of consecutive competence-to-stand-trial evaluations done by the forensic service during a six-month period. Clinicians adhered to a standard assessment procedure consisting of the following steps: history-gathering and establishing rapport, administration of the MacCAT-CA, and a forensic interview focused on assessing abilities related to competence to stand trial. After administering and scoring the measure, examiners were asked to document explicitly their experiences with the MacCAT-CA. To facilitate this process, we developed a form for recording basic demographic information regarding the defendants, their charges, psychiatric diagnoses, and whether the instrument was completed. If the MacCAT-CA was not completed, we noted the reasons. We also reported our forensic opinions based on our forensic interview, separate from the MacCAT-CA results. Finally, we noted our observations related to the use of the MacCAT-CA for each defendant, based on global clinical impressions. Malingering was primarily assessed via comparison of data obtained from routine forensic interviews, treatment staff observations, and collateral sources of information, although psychological testing was used in some cases. At the end of the six-month period, we had attempted to administer the MacCAT-CA to a total of 38 defendants. We then conducted the "check" of the CQI initiative by discussing and reviewing the evaluations, paying special attention to evaluators’ observations about the MacCAT-CA.
Clinical Experiences
Of the MacCAT-CA evaluations administered, approximately 60 percent were able to be completed. In nearly 30 percent of the evaluations examined, clinical factors (including psychotic or mood symptoms, cognitive limitations, or attempts to malinger) contributed to a failure to complete the MacCAT-CA. For example, in some cases, the severe thought disorganization, irritability, or pressured speech of the defendant interfered with an examiner’s ability to present items without repeated interruption or to obtain coherent responses. In other cases, impairments in abstract reasoning compromised a defendant’s ability to comprehend the hypothetical scenario that serves as the stimulus for the items on the Understanding and Reasoning scales. Response styles also played a role in the failure to complete the MacCAT-CA, in that some defendants exhibited poor motivation to participate or actively attempted to exaggerate or feign deficits, as often evidenced by repeated "I don’t know" responses in the context of data indicating they had the requisite knowledge (e.g., information from discussions with nursing staff). Also, the MacCAT-CA was not completed in certain cases in which there were data to suggest that repeated "I don’t know" responses were a function of significant cognitive impairments. We also noted non-clinical reasons for failure to complete the MacCAT-CA in approximately 13 percent of the cases. These included time constraints inherent in the court-ordered evaluations and language barriers. We relied on general impressions in this CQI project, and therefore did not systematically examine factors that might predict failure to complete.
Our review revealed that the MacCAT-CA offered data helpful in assessing adjudicative competence in some cases but not in others. Following is a summary, based on our experiences, of the benefits and problems associated with using this measure, as well as some corresponding case examples. These case descriptions represent composites of actual defendants admitted to our court evaluation unit for assessment of competence to stand trial and were written with limited and at times slightly altered detail to protect confidentiality. The cases were also reviewed with regard to confidentiality and approved by the Institutional Review Board of the Department of Mental Health. Of note, as the cases below illustrate, the MacCAT-CA results were interpreted in light of all information obtained in the general forensic interview, and our agreement with and understanding of the MacCAT-CA results varied accordingly.
Regarding the potential benefits of using the MacCAT-CA, we found that, with some paranoid or guarded defendants, the measure served to elicit information that was difficult to obtain in the clinical interview alone. That is, the measure provided another mechanism for gathering data from defendants who were resistant, particularly in cases in which their lack of cooperation was due to suspiciousness about the evaluator. It may be that such paranoid defendants find questions posed as part of a standardized measure less threatening and intrusive than those asked as part of a clinical interview. Although defendants’ paranoid beliefs about their own cases may cause them to provide little information about their situations, they are sometimes more able to respond to questions about a hypothetical scenario. Thus, at least with respect to factual understanding and reasoning capacities, the MacCAT-CA may allow for assessment of defendants’ abilities independent of their symptoms (e.g., paranoid concerns about their own cases). Such data could be used to rule out other variables that may account for apparent competence-related deficits, such as cognitive limitations or malingering.
Case 1
Ms. A., a woman charged with motor vehicle-related offenses, presented at the time of evaluation with delusions that she was being persecuted by her former supervisors and coworkers. She reported believing that several employees of the company where she worked were monitoring her movements and attempting to sabotage a lawsuit she had initiated against the company. She also expressed concerns that the staff on the court evaluation unit might be involved in this conspiracy, and she presented as quite guarded during her meetings with the forensic evaluator. Ms. A. provided enough information for the evaluator to opine that she had an adequate factual understanding of the legal proceedings and a trusting relationship with her attorney. Because of Ms. A.’s paranoia, however, the evaluator had difficulty eliciting information about her rational understanding of her case and the legal process. During the clinical interview, for example, she expressed concerns that she would be treated unfairly by the legal system, but she was unwilling to elaborate. Thus, the evaluator had difficulty determining whether her concerns were based on realistic or delusional factors. During the administration of the MacCAT-CA, however, Ms. A. revealed some basis for her concerns. In response to an Appreciation scale item, she reported believing that she was more likely to be found guilty than other defendants similarly charged because of her former colleagues’ influence with the court. Such data, which suggested deficits in the defendant’s rational understanding, secondary to delusional thinking, were crucial in helping the evaluator to clarify her opinion about Ms. A.’s competence to proceed.
Another potential benefit of incorporating the MacCAT-CA into competence-to-stand-trial assessments is that it provides an additional source and method of obtaining relevant data. This may be especially useful in cases of defendants who engage in inconsistent reporting during the clinical interview, including those who attempt to exaggerate or malinger cognitive deficits. Our review revealed that evaluees who attempted to feign memory deficits or exaggerate genuine cognitive deficits in the clinical interview sometimes failed to maintain this response style during administration of the MacCAT-CA. One possible explanation for this occurrence is that the structured format of the measure may cause some individuals to become less aware of their self-presentation. In other words, although it is possible to malinger on the MacCAT-CA, we found that certain (often less sophisticated) defendants who feigned deficits became somewhat distracted by the structured MacCAT-CA tasks and consequently lost sight of their interest in presenting in a particular way.
Case 2
Mr. B., a man in his early 20s charged with a drug offense, presented at the time of evaluation with apparent memory deficits and other neuropsychological impairments. He had a history of childhood exposure to lead, and he appeared to have genuine deficits in verbal processing and abstract reasoning; most likely these were results of this early brain insult. He exhibited no sign of psychiatric illness on the court evaluation unit but reported a history of polysubstance dependence. During the clinical interview portion of the competence-to-stand-trial assessment, Mr. B. provided conflicting responses when questioned about his understanding of factual aspects of the court system and his own case. For example, he alternated between accurately naming his charge and saying he did not know what it was, as well as saying he did not know the meaning of the basic trial verdicts and later using the terms correctly in conversation. On the MacCAT-CA, Mr. B.’s performance on the Understanding scale was unimpaired, suggesting that he had the requisite factual knowledge. Thus, in this case, the MacCAT-CA yielded data helpful in clarifying the defendant’s actual level of knowledge about the legal system and explaining his inconsistent reporting (i.e., that it was a function of an apparent attempt to exaggerate cognitive deficits).
Although the MacCAT-CA may be a useful tool in evaluating the competence to stand trial of defendants with certain types of clinical presentations, our review suggested problems with its practical utility in other types of cases. Also, even when it was useful, it did not adequately detect areas of difficulty in consulting with counsel. In the case of Ms. A., for example, the defendant’s persecutory delusions probably would have impaired her ability to work rationally with her attorney, but the MacCAT-CA did not yield data directly relevant to this concern.
We found that MacCAT-CA scores sometimes underestimated defendants’ actual abilities, in part as a function of the items’ complexity. Many of the items require substantial attentional capacities (e.g., most notably, the Reasoning scale’s lengthier questions regarding decision-making about pleading guilty or going to trial) and abstract reasoning abilities (e.g., the Appreciation scale items requiring defendants to compare their situations to other similar defendants). Evaluees with even mild impairments in frontal lobe functions such as attention, mental control, and abstract reasoning may perform poorly on the MacCAT-CA as a function of these instrument characteristics, which may not equate to absolute deficits in competence-related abilities. Real-life interactions with defense counsel would allow for the use of simpler language, explanation of legally relevant information, and further exploration of the defendant’s ability to work with counsel. The MacCAT-CA, however, does not permit deviation from established administration procedures. For these reasons, our review suggests that evaluators should proceed cautiously in interpreting MacCAT-CA scores from defendants with even mild deficits in the aforementioned neuropsychiatric domains.
Case 3
Mr. C. was a man facing several theft charges at the time of his admission to the Court Evaluation Unit. He had a history of Undifferentiated Schizophrenia and presented on admission with auditory hallucinations, attentional deficits, thought disorganization, paranoid delusions, and flat affect. With psychopharmacologic treatment, many of these symptoms improved over the course of his hospitalization. As a result of ongoing impairments in attentional capacities and ability to maintain mental set, however, he had difficulty encoding and retaining information on the complex MacCAT-CA items. Thus, his scores reflected clinically significant impairment on the Reasoning and Appreciation scales. In contrast, during the clinical interview, Mr. C. was able to engage in rational decision-making about his case when information was presented in simple, digestible terms and repeated as needed. In addition, he was able to ask questions of the evaluator and his lawyer when he did not understand a concept. In light of these data, the forensic evaluator opined that Mr. C. was not demonstrating significant impairment in his adjudicative competence abilities, despite his poor performance on the MacCAT-CA.
Another problem noted in our review of cases is the MacCAT-CA’s potential lack of sensitivity in assessing defendants’ delusional beliefs about their legal situations. Our analysis suggested that the items purported to assess defendants’ rational understanding of their own legal circumstances (i.e., the Appreciation scale) may fail to detect encapsulated delusions in particular. This occurrence may be an artifact of the wording of these items, which asks defendants to compare their situations to those of other similar defendants.
Case 4
Ms. D. was a woman in her late 50s charged with a violent offense. She had a history of manic and depressive episodes, as well as a long-standing delusion that her former employer wanted to have sex with her and that he was controlling members of her family and community. During the clinical interview portion of the competence-to-stand-trial evaluation, Ms. D. reported believing that her charges were the result of her former employer’s influence with the police department, that her former employer had instructed police officers to falsify the police report, and that he might influence her attorney and the judge such that she would not receive a fair trial. The MacCAT-CA failed to detect this circumscribed delusion, however, as evidenced by Ms. D.’s response to the Appreciation scale item concerning expectations about fair treatment by the legal system. She reported believing that she was more likely than similar defendants to be treated fairly because she was "not a guilty person." She did not mention in her responses to the questions in the instrument her previously reported concerns about her former employer’s influence on her case. Based in part on her report of these concerns during the clinical interview, the forensic evaluator opined that Ms. D.’s psychiatric symptoms compromised her rational understanding of the proceedings and her ability to rationally assist counsel in her defense.
Discussion
Over the past three decades, several competence assessment instruments have been developed.3,5 In a public sector setting where clinicians have grown accustomed to conducting evaluations in a particular way, the use of new instruments can be daunting. By incorporating the MacCAT-CA routinely in forensic evaluations conducted in a public sector setting, we were able to gain first-hand experience regarding its practical utility. As expected, the instrument was found to have advantages and disadvantages for routine clinical practice.
In our experience, the MacCAT-CA offered certain advantages. Specifically, the instrument allowed for an objective, standardized assessment of factual understanding, appreciation and reasoning related to the legal process. Its measurement of these latter two factors has been identified as one of the benefits offered by this instrument over others.17,29 These components allow for a more thorough examination of decision-making, one of the areas of competence that can be particularly difficult to assess. Also, in using the MacCAT-CA, evaluators were able to systematically educate defendants regarding areas where factual understanding may have been limited. By the same token, the lengthy hypothetical case example within the MacCAT-CA allowed for an in vivo exploration as to how defendants might attend to a complex proceeding and endure fatigue.
In many cases, the MacCAT-CA proved useful in ascertaining the level of defendants’ function with regard to their competence to proceed compared with a cohort of other individuals. This can be helpful in strengthening the reliability and validity of the forensic opinion, and thereby reduce the individual error rate. Although scoring norms were not available for defendants with significant mental retardation, we found that the instrument’s use with certain individuals with cognitive impairments yielded more structured data to detect difficulties. In addition, the hypothetical scenario provided a less threatening mechanism to assess defendants who were particularly paranoid about their own case, as noted in the example of Ms. A. On a more pragmatic level, the MacCAT-CA provided a novel set of questions for evaluators to utilize in eliciting nuances of defendants’ abilities or deficits.
Although there are many strengths of the instrument, we identified some limitations of its use. The structured administration, by definition, limited the ability of the examiners to probe further into areas where the defendant showed some deficits. The tool also did not include detailed questions related to consulting with counsel, a required element of Dusky. The Appreciation scale questions ask defendants to consider their case relative to other defendants, but the instrument does not specifically probe for information related to defendants’ abilities to consult with their own counsel. Responses to other items in the instrument at times may lead to responses related to that issue, so that some data regarding defendants’ abilities to consult with counsel may be gathered or inferred, depending on what they reveal. Another disadvantage of the MacCAT-CA, as noted by others,3,17 is that the use of the hypothetical vignette may not adequately reveal defendants’ views regarding their own cases, which is also critical to the Dusky standard. In our experience, this meant that the evaluators had to return later to these areas of assessment instead of exploring them at the moment, a strategy that compromised efficiency.
Another problem involved the use of language within the hypothetical case. The names used in the vignette sometimes became confused for defendants (and evaluators). The hypothetical case materials included terms, such as simple and aggravated assault, that are not utilized in our jurisdiction. Although the test uses these words to help assess defendants’ learning and understanding of criminal intent, we found that defendants at times became distracted by the unfamiliar words and less focused on the underlying concepts. Furthermore, fatigue at times made defendants appear more impaired in reasoning and appreciation, simply because those more abstract questions came toward the end of the MacCAT-CA administration.
Although the assessment of defendants with certain symptom patterns seemed to benefit from the utilization of the MacCAT-CA, as noted earlier, this did not apply across the board. The impact of certain encapsulated delusions on defendants’ competence seemed to be difficult to detect with the instrument, as seen in the case of Ms. D. The instrument was helpful in assessing attempts to exaggerate cognitive deficits in some cases (e.g., Mr. B.) but not in others. In fact, the lack of the MacCAT-CA’s capacity to assess specifically for malingering was one of the reasons for the design of the more recently released competence-to-stand-trial instrument, the ECST-R.21 Frequent "I don’t know" responses to MacCAT-CA items were returned by certain defendants who genuinely did not know the information or were thought to be malingering or showing poor effort. If this response pattern was pervasive, the evaluators generally stopped the administration of the MacCAT-CA after just a few questions. Defendants, particularly those with high degrees of irritability or impatience, sometimes expressed frustration and annoyance at the hypothetical scenario because it was not relevant to their particular legal situation. Indeed, we noted a relatively high rate of failure to complete the instrument, due to both clinical and non-clinical factors.
Many of the defendants presenting on our service come from a variety of cultural and ethnic backgrounds, and it is not uncommon to have non–English-speaking evaluees. Because we had no reliable translations of the instrument and because the instrument was normed on English-speaking defendants, its use with persons from other backgrounds raised certain concerns. Specifically, we concluded that if the MacCAT-CA were used with defendants from backgrounds not represented in the norming samples, the results should be interpreted with caution, if they are to be used at all. Similarly, several defendants with mental retardation were unable to comprehend the hypothetical scenario presented in the MacCAT-CA. In addition, as noted in the case of Mr. C., defendants with neuropsychological impairments (particularly in frontal lobe functioning) had difficulty retaining and manipulating information presented in the more complex items of the MacCAT-CA. Persons from certain minority groups and those with mental retardation or some degree of cognitive impairments are not uncommon in forensic mental health systems.34,35 These observations suggest that use of the instrument would not be indicated for every defendant who is evaluated in a public sector forensic setting such as ours.
As part of our CQI plan to "Check" and then "Act," we discussed whether to incorporate the MacCAT-CA as a routine part of all of our competence-to-stand-trial evaluations. Overall, this CQI project left us with the "action" of maintaining the MacCAT-CA within our armamentarium of competence-assessment instruments. We felt that its use on a case-by-case basis would best enhance the work done on our service. On balance, however, in reviewing our impressions as a group, we felt that the limitations outweighed the benefits for its regular use. Given this decision, our discussions yielded other areas for consideration.
As with any psychological or clinical tool, evaluators were required to think about whether and how to assess the results and incorporate them into their written reports. Scoring of the MacCAT-CA at times proved difficult. There were cases in which the exact words of the defendants recorded in the test booklet, when interpreted literally, would have meant they did not respond correctly. However, in considering the context of these responses, including inferences from prior responses, evaluators sometimes concluded that defendants performed better on items than the actual scores revealed. Thus, at times evaluators felt that defendants’ capacities were underestimated by the instrument, as exemplified in the case of Mr. C. In such cases, clinical judgments, which relied on data beyond that gleaned in the interview, generally trumped MacCAT-CA results, when deemed appropriate. For example, in the case of Mr. B., the findings of the MacCAT-CA were considered valid, despite clinical data that suggested cognitive deficits. With Ms. D., however, in the face of clinical data regarding delusions that were seemingly undetected by the MacCAT-CA, the results of the instrument were not considered an accurate assessment of the defendant’s competence to stand trial. In a case such as this, some explanation in the report to the court as to the basis for the forensic opinion in light of conflicting data seemed useful.
There are several limitations to the findings reported herein that are important to note. Although our experiences regarding the use of the MacCAT-CA may provide guidance to clinicians using the instrument, the results of this CQI project cannot be generalized across all forensic settings. They are limited by the CQI methodology, which by its nature33 involved a small sample of convenience and generated descriptive findings based on individual cases and overall impressions, rather than rigorous empirical analysis. In addition, the evaluations took place in a jurisdiction with a unique training program for its evaluators, wherein the evaluators conduct somewhat systematic, albeit not structured or standardized, in-depth clinical interviews. Although our findings are limited, they reveal some basic strengths and weaknesses of the MacCAT-CA. Given our preliminary impressions, future researchers would do well to focus on systematic analyses of larger samples to help assess the practical use of this instrument in forensic settings. Such studies could assess the frequency with which some of the weaknesses we noted come into play, whether malingering true deficits is detected differently across subscales, and whether specific factors (including those identified here or others) contribute to successes or failures of instrument completion.
The use of the MacCAT-CA for competence-to-stand-trial cases that presented during a specified period of time allowed evaluators to gain familiarity with one of the newer competence assessment instruments. There was a general consensus among the group that the CQI initiative itself was an important learning experience that fostered interesting discussion regarding existing competence assessment methods across evaluators. Furthermore, it allowed the evaluators to gain an in-depth understanding of the MacCAT-CA and to make decisions about when the instrument would be a useful adjunct in future cases.
The project also offered some unexpected advantages. For example, because the forensic service is a busy training site for forensic psychiatry fellows, postdoctoral forensic psychology fellows, psychiatric residents, medical students, and psychology practicum students, we found that the use of the instrument provided assistance as a training tool. Trainees of all disciplines can read the test booklet and manual to understand better the aspects of assessing competence to stand trial and the development of research in this area. Although not utilized in this project, the instrument may also be helpful in its test–retest value when competence restoration or training is at issue.
Competence assessment measures continue to be developed. The MacCAT-CA is but one of these instruments, and it can be used as an important supplement to other sources of data in competence evaluations. Newer instruments, such as the ECST-R,21 may provide further resources for forensic evaluators in practice, and future research may offer comparisons of the usefulness of these instruments in clinical settings and their admissibility in courts. Clinical interviewing, however, continues to be critical in the assessment of competence to stand trial, as the findings from these instruments cannot provide a sole basis for a forensic competence opinion. Nevertheless, evaluators of all disciplines would do well to maintain an awareness of developments in this area and to understand the strengths and limitations of these measures.
Acknowledgments
The authors wish to acknowledge Thomas Grisso, PhD, for his vision and guidance in the development of this paper; Valerie Williams, MA, and Chuck Lidz, PhD, for helpful thoughts and suggestions; and William Warnken, PsyD, for assistance in the conceptualization and realization of the project.
References
Dusky v. United States, 362 U.S. 402 (1960)
Cooper v. Oklahoma, 517 U.S. 348 (1996)
Grisso T: Competence to stand trial, in Evaluating Competencies: Forensic Assessments and Instruments. Edited by Grisso T. New York: Kluwer Academic, 2003, pp 69–148
Schreiber J: Professional judgment in the assessment of competence to stand trial: report of an evaluation study. Int J Law Psychiatry 5:331–40, 1982
Melton GB, Petrila J, Poythress NG, et al: Psychological Evaluations for the Courts (2nd ed.). New York: Guilford Press, 1997
Bonnie R, Grisso T: Adjudicative competence and youthful offenders, in Youth on Trial: A Developmental Perspective on Juvenile Justice. Edited by Grisso T, Shwartz R. Chicago: University of Chicago Press, 2000, pp 73–104
Bukatman B, Foy J, DeGrazia E: What is competence to stand trial Am J Psychiatry 125:1343–7, 1971
Robey A: Criteria for competence to stand trial: a checklist for psychiatrists. Am J Psychiatry 122:616–23, 1965
Lipsitt P, Lelos D, McGarry A: Competency for trial: a screening instrument. Am J Psychiatry 128:105–9, 1971
Zapf PA, Viljoen JL: Issues and considerations regarding the use of assessment instruments in the evaluation of competence to stand trial. Behav Sci Law 21:351–67, 2003
Laboratory of Community Psychiatry, Harvard Medical School: Competency to Stand Trial and Mental Illness. DHEW Publication No. ADM77-103. Rockville, MD: Department of Health, Education and Welfare, 1973
Nicholson R, Briggs S, Robertson H: Instruments for assessing competence to stand trial: how do they work Prof Psychol 19:383–94, 1988
Golding S, Roesch R, Schreiber J: Assessment and conceptualization of competency to stand trial: preliminary data on the Interdisciplinary Fitness Interview. Law Hum Behav 8:321–34, 1984
Roesch R, Webster CD, Eaves D: The Fitness Interview Test: A Method for Examining Fitness to Stand Trial. Toronto, Ontario, Canada: University of Toronto, Research Report of the Centre of Criminology, 1984
Golding SL: Interdisciplinary Fitness Interview-Revised: A Training Manual. Salt Lake City, UT: State of Utah Division of Mental Health, 1993
Roesch R, Zapf PA, Eaves D, et al: Fitness Interview Test (Revised Edition). Burnaby, British Columbia, Canada: Mental Health, Law and Policy Institute, Simon Fraser University, 1998
Stafford KP: Assessment of competence to stand trial, in Handbook of Psychology: Forensic Psychology (vol. 11). Edited by Goldstein AM. New York: Wiley, 2002, pp 359–80
Grisso T: Five-year research update (1986–1990): evaluations for competence to stand trial. Behav Sci Law 10:353–69, 1992
Hoge SK, Bonnie RJ, Poythress N, et al: The MacArthur Competence Assessment Tool-Criminal Adjudication. Odessa, FL: Psychological Assessment Resources, 1999
Poythress NG, Nicholson R, Otto RK, et al: Professional Manual for the MacArthur Competence Assessment Tool-Criminal Adjudication. Odessa, FL: Psychological Assessment Resources, 1999
Rogers R, Tillbrook CE, Sewell KW: Evaluation of Competency to Stand Trial-Revised (ECST-R) and Professional Manual. Lutz, FL: Psychological Assessment Resources, 2004
State v. Champagne, 497 A.2d 1242 (N.H. 1985)
State v. Hahn, 707 P.2d 699 (Wash. Ct. App. 1985)
United States v. Duhon, 104 F.Supp.2d 663 (W.D. La. 2000)
Gloldstein AM, Burd M: Role of delusions in trial competency evaluations: case law and implications for forensic practice. Forensic Rep 3:361–86, 1990
Committee on Quality of Health Care in America, Institute of Medicine: Crossing the quality chasm: A new health system for the 21st century. Washington, DC, National Academy Press, 2000
Rakich JS: Quality, CQI, and reengineering in health services organizations. J Health Soc Policy 13:41–58, 2001
Otto RK, Poythress NG, Nicholson RA, et al: Psychometric properties of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA). Psychol Assess 10:435–43, 1998
Poythress NG, Bonnie RJ, Monahan J, et al: Adjudicative Competence: The MacArthur Studies. New York: Kluwer/Plenum, 2002
Rogers R, Grandjean N, Tillbrook C, et al: Recent interview-based measures of competence to stand trial: a critical review augmented with research data. Behav Sci Law 19:503–18, 2001
Tillbrook CE: Competency to proceed: a comparative appraisal of approaches to assessment. Unpublished doctoral dissertation. Tuscaloosa, AL: The University of Alabama, 2000
Plan, Do, Check, Act: PDCA. Available at http://www.isixsigma.com/me/pdca. Accessed December 19, 2004
Byers JF, Beaudin CL: The relationship between continuous quality improvement and research. J Healthcare Qual 24:4–8, 2002
Pinals DA, Packer IK, Fisher W, et al: Relationship between race and ethnicity and forensic clinical triage dispositions. Psychiatr Serv 55:873–8, 2004
Wall BW, Krupp BH, Guilmette T: Restoration of competence to stand trial: a training program for persons with mental retardation. J Am Acad Psychiatry Law 31:189–201, 2003(Debra A. Pinals, MD, Chad E. Tillbrook, )
Abstract
Several instruments have been developed to assess defendants’ competence to stand trial (CST). These instruments have become increasingly sophisticated mainstays of CST evaluation research. Less is known, however, about their clinical utility. Continuous quality-improvement (CQI) efforts within forensic practice are one means of helping ascertain the effectiveness and efficiency of the use of these measures, especially in public sector settings. In this article, the authors describe a CQI project examining the use of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) on an inpatient forensic evaluation service. Results of a small, local CQI project cannot be generalized to all settings. However, from this CQI experience, the authors found the MacCAT-CA to be advantageous in assessing CST in certain defendants. Given some limitations of its practical utility, including its use with defendants putting forth poor effort, those with significant cognitive impairments, and those from various cultural backgrounds, the authors did not find that it would be an efficient practice to utilize the instrument in all CST assessments within a busy, public sector, inpatient forensic evaluation service. Having the instrument available for use as an adjunct to a clinical interview in particular cases may offer the most advantage. In addition, it is a useful tool for educating trainees about complex CST domains. This case-based CQI project reveals some basic strengths and weaknesses of the instrument and offers direction for further exploration in a systematic research study.
The prevailing legal standard for competence to stand trial has not changed since 1960 when the United States Supreme Court stated:
The test will be whether [the defendant] has sufficient present ability to consult with his lawyer with a reasonable degree of rational understanding, and whether he has a rational as well as factual understanding of the proceedings against him [Ref. 1, p 402].
The Supreme Court has upheld the substantive criteria enumerated in Dusky v. United States for four and a half decades,2 and most states have adopted this language with only minor modifications, if any, in wording.3 Numerous states mandate that mental health experts consider and address the Dusky factors during a competence evaluation and when rendering an opinion about a defendant’s ability to proceed to trial. Schreiber,4 however, noted the difficulty experienced by mental health experts in applying psychological and behavioral observations of defendants to the legal criteria of Dusky.
Over the past 35 years, numerous mental health and legal scholars have set out to operationalize the Dusky language to guide mental health professionals who are asked to perform a large number of competence evaluations, especially those in public mental health settings. Earlier estimates place the number of competence-to-stand-trial evaluations performed each year in the United States at around 20,000,5 with more recent statistics citing 60,000,6 making such evaluations a significant focus of mental health inquiry in the criminal justice system. To assist experts in evaluating competence to stand trial, several instruments have been developed, each differing slightly in their purpose, scope, design, and the degree to which they have been examined empirically. These instruments have taken various forms, including checklists of items,7,8 a sentence-completion questionnaire,9 and subjectively rated interview guides that provide some structure for assessment of the Dusky criteria.10 Before the past decade, widely used instruments included the Competence to Stand Trial Assessment Instrument,11 Georgia Court Competency Test-Mississippi State Hospital revision,12 Interdisciplinary Fitness Interview (IFI),13 and Fitness Interview Test (FIT).14 The last two measures, IFI and FIT, were revised to reflect specific changes in the legal statutes in the jurisdictions where they were developed and commonly employed. The IFI-Revised15 reflects changes to Utah’s new competence code, which mandates that specific competence domains be addressed, and the FIT-Revised16 was modified in accordance with changes in Canadian law.
Each of these "first-generation" measures has been utilized and reviewed extensively. Although they have each provided a unique contribution to the assessment of competence to stand trial, they have been critiqued regarding their psychometric shortcomings, their neglect of sampling important domains relevant to Dusky (e.g., rational decision-making), and their limited practical utility (for more comprehensive reviews, see Refs. 3,10,17). Grisso18 called on researchers to revise existing measures (i.e., resulting in the IFI-R and FIT-R) or to develop new competence assessment instruments that address these shortcomings and incorporate more standardized administration and scoring. A "second generation" of research resulted in the development of two relatively new competence measures, the MacArthur Competence Assessment Tool–Criminal Adjudication (MacCAT-CA)19,20 and the Evaluation of Competency to Stand Trial–Revised (ECST-R).21 Although legal challenges to these instruments may still be seen, the use of competence assessment instruments is found in case law (see, for example, Refs. 22–24), suggesting the value of distinct evaluations for competence to stand trial.21,25 Furthermore, reviews have indicated that competence-to-stand-trial evaluations in research settings using these specific assessment instruments are highly reliable and valid, although individually these measures differ in their conceptual assumptions.3,5,21 What has not been explored, however, is their practical applicability in improving clinical practice.
Enhancing practice and improving the quality of services provided are goals recognized throughout medicine.26,27 The need to look at the quality, effectiveness, and efficiency of forensic evaluations conducted for the courts within public sector settings is in line with that goal. To that end, we developed a continuous quality-improvement (CQI) project to examine our competence assessment process, especially given the lack of information available about the clinical utility of second-generation competence assessment instruments. In designing this project, we decided to examine the MacCAT-CA in more detail, with a specific focus on delineating important considerations when choosing to use this measure, as well as identifying the potential contributions of the MacCAT-CA to the competence evaluation process in public sector settings. In this article, we report our findings from this CQI project and discuss the MacCAT-CA’s utility in forensic practice, including some practical considerations pertaining to its administration, scoring, and use in assisting evaluators in formulating opinions of competence to proceed.
MacArthur Competence Assessment Tool–Criminal Adjudication
The MacCAT-CA, developed over years in response to the need for an objectively scored assessment tool incorporating Dusky criteria, was studied extensively between 1996 and 199828 (for a complete review of relevant MacArthur studies, see Ref. 29). It comprises 22 items that are organized into three sections, including criteria covered by Dusky. The MacCAT-CA includes a standard administration, beginning with a vignette of a hypothetical offense involving an aggravated assault at a pool hall. This vignette is read by the examiner and sets the context for the first 16 items. For the remaining six items, defendants are asked to make comparative judgments about their own cases and to explain their reasoning. Defendants’ responses are rated on a three-point scale (0, 1, or 2) against explicit scoring criteria.
The first section, Understanding, assesses defendants’ ability to understand general information about the legal system and the process of adjudication. Most of these items consist of two parts, the first of which is an assessment of defendants’ current knowledge of the legal system. If their answers are less than satisfactory (i.e., a score less than 2), they are read a short narrative that educates them on the essential components. They are then asked to paraphrase this information. The purpose of the disclosure is to assess defendants’ capacity to learn, remember, and apply new information relevant to the legal process.
The second section, Reasoning, evaluates defendants’ ability to discern the legal relevance of information and their capacity to reason about specific choices that confront defendants during the course of a typical criminal proceeding. For the first five of these eight items, defendants are asked to distinguish between more and less legally relevant factual information. The last three items require defendants to reason through hypothetical legal scenarios (e.g., plea bargain) and reach rational decisions.
Appreciation is the last scale, and it assesses defendants’ ability to appreciate the meaning and consequences of their own legal circumstances. These six items depart from the hypothetical vignette and investigate defendants’ beliefs and perceptions about their roles as defendants. Compared with the previous 16 items, these are scored on the basis of whether the answers are plausible (i.e., grounded in reality or influenced by delusional beliefs).
Otto et al.28 reported evidence of the MacCAT-CA’s construct validity, with its scales correlating in an expected pattern with measures of psychopathology, cognitive ability, and clinical judgments. In addition, the findings demonstrate good predictive utility with acceptable hit rate ranges (i.e., specificity, sensitivity, positive, and negative predictive values, and false-negative rates). The results of that study supported the development of national norms that examiners can use as guideposts to compare a particular defendant’s performance to others in similar circumstances.
Compared with earlier measures, it has been suggested that the MacCAT-CA does a more thorough job of sampling relevant competence-related abilities, including areas of plea bargaining,3,5 but there have been concerns about its ability to assess a defendant’s capacity to consult with counsel.30,31 Furthermore, according to Dusky, competence-related abilities should apply to the particular defendant being evaluated and his or her legal case. Much of the MacCAT-CA, however, focuses on a hypothetical vignette and assesses understanding and reasoning related to the fictional case as opposed to the defendant’s own situation. The psychometric properties of the MacCAT-CA have been reported20 and critiqued,3,30 but its clinical utility (i.e., ease of use by forensic examiners and ability to elicit data from evaluees) has not, to our knowledge, been investigated. This is of particular interest, because the format of the MacCAT-CA differs notably from earlier competence assessment instruments.
Continuous Quality-Improvement Project Purpose and Design
Among the goals of the CQI project was the identification of ways to improve service delivery by understanding factors associated with the clinical usefulness of the MacCAT-CA (e.g., clinical or case-related variables that suggest that the administration of this instrument would be helpful in rendering clinical opinions about competence to stand trial). We also wondered if there were circumstances in which using the MacCAT-CA might not be helpful in formulating opinions about a defendant’s competence to proceed.
We submitted a summary of this CQI project to the Massachusetts Department of Mental Health Research Review Committee and the Institutional Review Board of the University of Massachusetts Medical School for review. Both bodies determined that the project was exempt from review.
Setting
The CQI project was performed in a 15-bed coeducational inpatient court evaluation unit located in a state psychiatric hospital in Massachusetts. In this facility, court-ordered evaluations of psycholegal issues such as competence to stand trial, criminal responsibility, and aid in sentencing are conducted by staff of the hospital’s forensic service, which consists of psychologists and psychiatrists with specialized training in forensic evaluation. The forensic service also functions as a major training site for forensic evaluators, including psychiatry and psychology forensic fellows. In Massachusetts, public sector forensic evaluations can only be performed by mental health professionals certified to do so via a state-wide program that establishes standards for assessment procedures and reports. All evaluators who participated in the current CQI project either had this certification or were forensic trainees supervised by a psychologist or psychiatrist with this credential. Several evaluators were involved in this project, three of whom are the authors of this article. Finally, before participating, all evaluators involved received training in administration of the MacCAT-CA.
CQI Project Implementation
Utilizing a model incorporating the quality-improvement rubric of "Plan, Do, Check, Act,"32,33 after the planning stage, evaluators attempted to administer the MacCAT-CA as part of consecutive competence-to-stand-trial evaluations done by the forensic service during a six-month period. Clinicians adhered to a standard assessment procedure consisting of the following steps: history-gathering and establishing rapport, administration of the MacCAT-CA, and a forensic interview focused on assessing abilities related to competence to stand trial. After administering and scoring the measure, examiners were asked to document explicitly their experiences with the MacCAT-CA. To facilitate this process, we developed a form for recording basic demographic information regarding the defendants, their charges, psychiatric diagnoses, and whether the instrument was completed. If the MacCAT-CA was not completed, we noted the reasons. We also reported our forensic opinions based on our forensic interview, separate from the MacCAT-CA results. Finally, we noted our observations related to the use of the MacCAT-CA for each defendant, based on global clinical impressions. Malingering was primarily assessed via comparison of data obtained from routine forensic interviews, treatment staff observations, and collateral sources of information, although psychological testing was used in some cases. At the end of the six-month period, we had attempted to administer the MacCAT-CA to a total of 38 defendants. We then conducted the "check" of the CQI initiative by discussing and reviewing the evaluations, paying special attention to evaluators’ observations about the MacCAT-CA.
Clinical Experiences
Of the MacCAT-CA evaluations administered, approximately 60 percent were able to be completed. In nearly 30 percent of the evaluations examined, clinical factors (including psychotic or mood symptoms, cognitive limitations, or attempts to malinger) contributed to a failure to complete the MacCAT-CA. For example, in some cases, the severe thought disorganization, irritability, or pressured speech of the defendant interfered with an examiner’s ability to present items without repeated interruption or to obtain coherent responses. In other cases, impairments in abstract reasoning compromised a defendant’s ability to comprehend the hypothetical scenario that serves as the stimulus for the items on the Understanding and Reasoning scales. Response styles also played a role in the failure to complete the MacCAT-CA, in that some defendants exhibited poor motivation to participate or actively attempted to exaggerate or feign deficits, as often evidenced by repeated "I don’t know" responses in the context of data indicating they had the requisite knowledge (e.g., information from discussions with nursing staff). Also, the MacCAT-CA was not completed in certain cases in which there were data to suggest that repeated "I don’t know" responses were a function of significant cognitive impairments. We also noted non-clinical reasons for failure to complete the MacCAT-CA in approximately 13 percent of the cases. These included time constraints inherent in the court-ordered evaluations and language barriers. We relied on general impressions in this CQI project, and therefore did not systematically examine factors that might predict failure to complete.
Our review revealed that the MacCAT-CA offered data helpful in assessing adjudicative competence in some cases but not in others. Following is a summary, based on our experiences, of the benefits and problems associated with using this measure, as well as some corresponding case examples. These case descriptions represent composites of actual defendants admitted to our court evaluation unit for assessment of competence to stand trial and were written with limited and at times slightly altered detail to protect confidentiality. The cases were also reviewed with regard to confidentiality and approved by the Institutional Review Board of the Department of Mental Health. Of note, as the cases below illustrate, the MacCAT-CA results were interpreted in light of all information obtained in the general forensic interview, and our agreement with and understanding of the MacCAT-CA results varied accordingly.
Regarding the potential benefits of using the MacCAT-CA, we found that, with some paranoid or guarded defendants, the measure served to elicit information that was difficult to obtain in the clinical interview alone. That is, the measure provided another mechanism for gathering data from defendants who were resistant, particularly in cases in which their lack of cooperation was due to suspiciousness about the evaluator. It may be that such paranoid defendants find questions posed as part of a standardized measure less threatening and intrusive than those asked as part of a clinical interview. Although defendants’ paranoid beliefs about their own cases may cause them to provide little information about their situations, they are sometimes more able to respond to questions about a hypothetical scenario. Thus, at least with respect to factual understanding and reasoning capacities, the MacCAT-CA may allow for assessment of defendants’ abilities independent of their symptoms (e.g., paranoid concerns about their own cases). Such data could be used to rule out other variables that may account for apparent competence-related deficits, such as cognitive limitations or malingering.
Case 1
Ms. A., a woman charged with motor vehicle-related offenses, presented at the time of evaluation with delusions that she was being persecuted by her former supervisors and coworkers. She reported believing that several employees of the company where she worked were monitoring her movements and attempting to sabotage a lawsuit she had initiated against the company. She also expressed concerns that the staff on the court evaluation unit might be involved in this conspiracy, and she presented as quite guarded during her meetings with the forensic evaluator. Ms. A. provided enough information for the evaluator to opine that she had an adequate factual understanding of the legal proceedings and a trusting relationship with her attorney. Because of Ms. A.’s paranoia, however, the evaluator had difficulty eliciting information about her rational understanding of her case and the legal process. During the clinical interview, for example, she expressed concerns that she would be treated unfairly by the legal system, but she was unwilling to elaborate. Thus, the evaluator had difficulty determining whether her concerns were based on realistic or delusional factors. During the administration of the MacCAT-CA, however, Ms. A. revealed some basis for her concerns. In response to an Appreciation scale item, she reported believing that she was more likely to be found guilty than other defendants similarly charged because of her former colleagues’ influence with the court. Such data, which suggested deficits in the defendant’s rational understanding, secondary to delusional thinking, were crucial in helping the evaluator to clarify her opinion about Ms. A.’s competence to proceed.
Another potential benefit of incorporating the MacCAT-CA into competence-to-stand-trial assessments is that it provides an additional source and method of obtaining relevant data. This may be especially useful in cases of defendants who engage in inconsistent reporting during the clinical interview, including those who attempt to exaggerate or malinger cognitive deficits. Our review revealed that evaluees who attempted to feign memory deficits or exaggerate genuine cognitive deficits in the clinical interview sometimes failed to maintain this response style during administration of the MacCAT-CA. One possible explanation for this occurrence is that the structured format of the measure may cause some individuals to become less aware of their self-presentation. In other words, although it is possible to malinger on the MacCAT-CA, we found that certain (often less sophisticated) defendants who feigned deficits became somewhat distracted by the structured MacCAT-CA tasks and consequently lost sight of their interest in presenting in a particular way.
Case 2
Mr. B., a man in his early 20s charged with a drug offense, presented at the time of evaluation with apparent memory deficits and other neuropsychological impairments. He had a history of childhood exposure to lead, and he appeared to have genuine deficits in verbal processing and abstract reasoning; most likely these were results of this early brain insult. He exhibited no sign of psychiatric illness on the court evaluation unit but reported a history of polysubstance dependence. During the clinical interview portion of the competence-to-stand-trial assessment, Mr. B. provided conflicting responses when questioned about his understanding of factual aspects of the court system and his own case. For example, he alternated between accurately naming his charge and saying he did not know what it was, as well as saying he did not know the meaning of the basic trial verdicts and later using the terms correctly in conversation. On the MacCAT-CA, Mr. B.’s performance on the Understanding scale was unimpaired, suggesting that he had the requisite factual knowledge. Thus, in this case, the MacCAT-CA yielded data helpful in clarifying the defendant’s actual level of knowledge about the legal system and explaining his inconsistent reporting (i.e., that it was a function of an apparent attempt to exaggerate cognitive deficits).
Although the MacCAT-CA may be a useful tool in evaluating the competence to stand trial of defendants with certain types of clinical presentations, our review suggested problems with its practical utility in other types of cases. Also, even when it was useful, it did not adequately detect areas of difficulty in consulting with counsel. In the case of Ms. A., for example, the defendant’s persecutory delusions probably would have impaired her ability to work rationally with her attorney, but the MacCAT-CA did not yield data directly relevant to this concern.
We found that MacCAT-CA scores sometimes underestimated defendants’ actual abilities, in part as a function of the items’ complexity. Many of the items require substantial attentional capacities (e.g., most notably, the Reasoning scale’s lengthier questions regarding decision-making about pleading guilty or going to trial) and abstract reasoning abilities (e.g., the Appreciation scale items requiring defendants to compare their situations to other similar defendants). Evaluees with even mild impairments in frontal lobe functions such as attention, mental control, and abstract reasoning may perform poorly on the MacCAT-CA as a function of these instrument characteristics, which may not equate to absolute deficits in competence-related abilities. Real-life interactions with defense counsel would allow for the use of simpler language, explanation of legally relevant information, and further exploration of the defendant’s ability to work with counsel. The MacCAT-CA, however, does not permit deviation from established administration procedures. For these reasons, our review suggests that evaluators should proceed cautiously in interpreting MacCAT-CA scores from defendants with even mild deficits in the aforementioned neuropsychiatric domains.
Case 3
Mr. C. was a man facing several theft charges at the time of his admission to the Court Evaluation Unit. He had a history of Undifferentiated Schizophrenia and presented on admission with auditory hallucinations, attentional deficits, thought disorganization, paranoid delusions, and flat affect. With psychopharmacologic treatment, many of these symptoms improved over the course of his hospitalization. As a result of ongoing impairments in attentional capacities and ability to maintain mental set, however, he had difficulty encoding and retaining information on the complex MacCAT-CA items. Thus, his scores reflected clinically significant impairment on the Reasoning and Appreciation scales. In contrast, during the clinical interview, Mr. C. was able to engage in rational decision-making about his case when information was presented in simple, digestible terms and repeated as needed. In addition, he was able to ask questions of the evaluator and his lawyer when he did not understand a concept. In light of these data, the forensic evaluator opined that Mr. C. was not demonstrating significant impairment in his adjudicative competence abilities, despite his poor performance on the MacCAT-CA.
Another problem noted in our review of cases is the MacCAT-CA’s potential lack of sensitivity in assessing defendants’ delusional beliefs about their legal situations. Our analysis suggested that the items purported to assess defendants’ rational understanding of their own legal circumstances (i.e., the Appreciation scale) may fail to detect encapsulated delusions in particular. This occurrence may be an artifact of the wording of these items, which asks defendants to compare their situations to those of other similar defendants.
Case 4
Ms. D. was a woman in her late 50s charged with a violent offense. She had a history of manic and depressive episodes, as well as a long-standing delusion that her former employer wanted to have sex with her and that he was controlling members of her family and community. During the clinical interview portion of the competence-to-stand-trial evaluation, Ms. D. reported believing that her charges were the result of her former employer’s influence with the police department, that her former employer had instructed police officers to falsify the police report, and that he might influence her attorney and the judge such that she would not receive a fair trial. The MacCAT-CA failed to detect this circumscribed delusion, however, as evidenced by Ms. D.’s response to the Appreciation scale item concerning expectations about fair treatment by the legal system. She reported believing that she was more likely than similar defendants to be treated fairly because she was "not a guilty person." She did not mention in her responses to the questions in the instrument her previously reported concerns about her former employer’s influence on her case. Based in part on her report of these concerns during the clinical interview, the forensic evaluator opined that Ms. D.’s psychiatric symptoms compromised her rational understanding of the proceedings and her ability to rationally assist counsel in her defense.
Discussion
Over the past three decades, several competence assessment instruments have been developed.3,5 In a public sector setting where clinicians have grown accustomed to conducting evaluations in a particular way, the use of new instruments can be daunting. By incorporating the MacCAT-CA routinely in forensic evaluations conducted in a public sector setting, we were able to gain first-hand experience regarding its practical utility. As expected, the instrument was found to have advantages and disadvantages for routine clinical practice.
In our experience, the MacCAT-CA offered certain advantages. Specifically, the instrument allowed for an objective, standardized assessment of factual understanding, appreciation and reasoning related to the legal process. Its measurement of these latter two factors has been identified as one of the benefits offered by this instrument over others.17,29 These components allow for a more thorough examination of decision-making, one of the areas of competence that can be particularly difficult to assess. Also, in using the MacCAT-CA, evaluators were able to systematically educate defendants regarding areas where factual understanding may have been limited. By the same token, the lengthy hypothetical case example within the MacCAT-CA allowed for an in vivo exploration as to how defendants might attend to a complex proceeding and endure fatigue.
In many cases, the MacCAT-CA proved useful in ascertaining the level of defendants’ function with regard to their competence to proceed compared with a cohort of other individuals. This can be helpful in strengthening the reliability and validity of the forensic opinion, and thereby reduce the individual error rate. Although scoring norms were not available for defendants with significant mental retardation, we found that the instrument’s use with certain individuals with cognitive impairments yielded more structured data to detect difficulties. In addition, the hypothetical scenario provided a less threatening mechanism to assess defendants who were particularly paranoid about their own case, as noted in the example of Ms. A. On a more pragmatic level, the MacCAT-CA provided a novel set of questions for evaluators to utilize in eliciting nuances of defendants’ abilities or deficits.
Although there are many strengths of the instrument, we identified some limitations of its use. The structured administration, by definition, limited the ability of the examiners to probe further into areas where the defendant showed some deficits. The tool also did not include detailed questions related to consulting with counsel, a required element of Dusky. The Appreciation scale questions ask defendants to consider their case relative to other defendants, but the instrument does not specifically probe for information related to defendants’ abilities to consult with their own counsel. Responses to other items in the instrument at times may lead to responses related to that issue, so that some data regarding defendants’ abilities to consult with counsel may be gathered or inferred, depending on what they reveal. Another disadvantage of the MacCAT-CA, as noted by others,3,17 is that the use of the hypothetical vignette may not adequately reveal defendants’ views regarding their own cases, which is also critical to the Dusky standard. In our experience, this meant that the evaluators had to return later to these areas of assessment instead of exploring them at the moment, a strategy that compromised efficiency.
Another problem involved the use of language within the hypothetical case. The names used in the vignette sometimes became confused for defendants (and evaluators). The hypothetical case materials included terms, such as simple and aggravated assault, that are not utilized in our jurisdiction. Although the test uses these words to help assess defendants’ learning and understanding of criminal intent, we found that defendants at times became distracted by the unfamiliar words and less focused on the underlying concepts. Furthermore, fatigue at times made defendants appear more impaired in reasoning and appreciation, simply because those more abstract questions came toward the end of the MacCAT-CA administration.
Although the assessment of defendants with certain symptom patterns seemed to benefit from the utilization of the MacCAT-CA, as noted earlier, this did not apply across the board. The impact of certain encapsulated delusions on defendants’ competence seemed to be difficult to detect with the instrument, as seen in the case of Ms. D. The instrument was helpful in assessing attempts to exaggerate cognitive deficits in some cases (e.g., Mr. B.) but not in others. In fact, the lack of the MacCAT-CA’s capacity to assess specifically for malingering was one of the reasons for the design of the more recently released competence-to-stand-trial instrument, the ECST-R.21 Frequent "I don’t know" responses to MacCAT-CA items were returned by certain defendants who genuinely did not know the information or were thought to be malingering or showing poor effort. If this response pattern was pervasive, the evaluators generally stopped the administration of the MacCAT-CA after just a few questions. Defendants, particularly those with high degrees of irritability or impatience, sometimes expressed frustration and annoyance at the hypothetical scenario because it was not relevant to their particular legal situation. Indeed, we noted a relatively high rate of failure to complete the instrument, due to both clinical and non-clinical factors.
Many of the defendants presenting on our service come from a variety of cultural and ethnic backgrounds, and it is not uncommon to have non–English-speaking evaluees. Because we had no reliable translations of the instrument and because the instrument was normed on English-speaking defendants, its use with persons from other backgrounds raised certain concerns. Specifically, we concluded that if the MacCAT-CA were used with defendants from backgrounds not represented in the norming samples, the results should be interpreted with caution, if they are to be used at all. Similarly, several defendants with mental retardation were unable to comprehend the hypothetical scenario presented in the MacCAT-CA. In addition, as noted in the case of Mr. C., defendants with neuropsychological impairments (particularly in frontal lobe functioning) had difficulty retaining and manipulating information presented in the more complex items of the MacCAT-CA. Persons from certain minority groups and those with mental retardation or some degree of cognitive impairments are not uncommon in forensic mental health systems.34,35 These observations suggest that use of the instrument would not be indicated for every defendant who is evaluated in a public sector forensic setting such as ours.
As part of our CQI plan to "Check" and then "Act," we discussed whether to incorporate the MacCAT-CA as a routine part of all of our competence-to-stand-trial evaluations. Overall, this CQI project left us with the "action" of maintaining the MacCAT-CA within our armamentarium of competence-assessment instruments. We felt that its use on a case-by-case basis would best enhance the work done on our service. On balance, however, in reviewing our impressions as a group, we felt that the limitations outweighed the benefits for its regular use. Given this decision, our discussions yielded other areas for consideration.
As with any psychological or clinical tool, evaluators were required to think about whether and how to assess the results and incorporate them into their written reports. Scoring of the MacCAT-CA at times proved difficult. There were cases in which the exact words of the defendants recorded in the test booklet, when interpreted literally, would have meant they did not respond correctly. However, in considering the context of these responses, including inferences from prior responses, evaluators sometimes concluded that defendants performed better on items than the actual scores revealed. Thus, at times evaluators felt that defendants’ capacities were underestimated by the instrument, as exemplified in the case of Mr. C. In such cases, clinical judgments, which relied on data beyond that gleaned in the interview, generally trumped MacCAT-CA results, when deemed appropriate. For example, in the case of Mr. B., the findings of the MacCAT-CA were considered valid, despite clinical data that suggested cognitive deficits. With Ms. D., however, in the face of clinical data regarding delusions that were seemingly undetected by the MacCAT-CA, the results of the instrument were not considered an accurate assessment of the defendant’s competence to stand trial. In a case such as this, some explanation in the report to the court as to the basis for the forensic opinion in light of conflicting data seemed useful.
There are several limitations to the findings reported herein that are important to note. Although our experiences regarding the use of the MacCAT-CA may provide guidance to clinicians using the instrument, the results of this CQI project cannot be generalized across all forensic settings. They are limited by the CQI methodology, which by its nature33 involved a small sample of convenience and generated descriptive findings based on individual cases and overall impressions, rather than rigorous empirical analysis. In addition, the evaluations took place in a jurisdiction with a unique training program for its evaluators, wherein the evaluators conduct somewhat systematic, albeit not structured or standardized, in-depth clinical interviews. Although our findings are limited, they reveal some basic strengths and weaknesses of the MacCAT-CA. Given our preliminary impressions, future researchers would do well to focus on systematic analyses of larger samples to help assess the practical use of this instrument in forensic settings. Such studies could assess the frequency with which some of the weaknesses we noted come into play, whether malingering true deficits is detected differently across subscales, and whether specific factors (including those identified here or others) contribute to successes or failures of instrument completion.
The use of the MacCAT-CA for competence-to-stand-trial cases that presented during a specified period of time allowed evaluators to gain familiarity with one of the newer competence assessment instruments. There was a general consensus among the group that the CQI initiative itself was an important learning experience that fostered interesting discussion regarding existing competence assessment methods across evaluators. Furthermore, it allowed the evaluators to gain an in-depth understanding of the MacCAT-CA and to make decisions about when the instrument would be a useful adjunct in future cases.
The project also offered some unexpected advantages. For example, because the forensic service is a busy training site for forensic psychiatry fellows, postdoctoral forensic psychology fellows, psychiatric residents, medical students, and psychology practicum students, we found that the use of the instrument provided assistance as a training tool. Trainees of all disciplines can read the test booklet and manual to understand better the aspects of assessing competence to stand trial and the development of research in this area. Although not utilized in this project, the instrument may also be helpful in its test–retest value when competence restoration or training is at issue.
Competence assessment measures continue to be developed. The MacCAT-CA is but one of these instruments, and it can be used as an important supplement to other sources of data in competence evaluations. Newer instruments, such as the ECST-R,21 may provide further resources for forensic evaluators in practice, and future research may offer comparisons of the usefulness of these instruments in clinical settings and their admissibility in courts. Clinical interviewing, however, continues to be critical in the assessment of competence to stand trial, as the findings from these instruments cannot provide a sole basis for a forensic competence opinion. Nevertheless, evaluators of all disciplines would do well to maintain an awareness of developments in this area and to understand the strengths and limitations of these measures.
Acknowledgments
The authors wish to acknowledge Thomas Grisso, PhD, for his vision and guidance in the development of this paper; Valerie Williams, MA, and Chuck Lidz, PhD, for helpful thoughts and suggestions; and William Warnken, PsyD, for assistance in the conceptualization and realization of the project.
References
Dusky v. United States, 362 U.S. 402 (1960)
Cooper v. Oklahoma, 517 U.S. 348 (1996)
Grisso T: Competence to stand trial, in Evaluating Competencies: Forensic Assessments and Instruments. Edited by Grisso T. New York: Kluwer Academic, 2003, pp 69–148
Schreiber J: Professional judgment in the assessment of competence to stand trial: report of an evaluation study. Int J Law Psychiatry 5:331–40, 1982
Melton GB, Petrila J, Poythress NG, et al: Psychological Evaluations for the Courts (2nd ed.). New York: Guilford Press, 1997
Bonnie R, Grisso T: Adjudicative competence and youthful offenders, in Youth on Trial: A Developmental Perspective on Juvenile Justice. Edited by Grisso T, Shwartz R. Chicago: University of Chicago Press, 2000, pp 73–104
Bukatman B, Foy J, DeGrazia E: What is competence to stand trial Am J Psychiatry 125:1343–7, 1971
Robey A: Criteria for competence to stand trial: a checklist for psychiatrists. Am J Psychiatry 122:616–23, 1965
Lipsitt P, Lelos D, McGarry A: Competency for trial: a screening instrument. Am J Psychiatry 128:105–9, 1971
Zapf PA, Viljoen JL: Issues and considerations regarding the use of assessment instruments in the evaluation of competence to stand trial. Behav Sci Law 21:351–67, 2003
Laboratory of Community Psychiatry, Harvard Medical School: Competency to Stand Trial and Mental Illness. DHEW Publication No. ADM77-103. Rockville, MD: Department of Health, Education and Welfare, 1973
Nicholson R, Briggs S, Robertson H: Instruments for assessing competence to stand trial: how do they work Prof Psychol 19:383–94, 1988
Golding S, Roesch R, Schreiber J: Assessment and conceptualization of competency to stand trial: preliminary data on the Interdisciplinary Fitness Interview. Law Hum Behav 8:321–34, 1984
Roesch R, Webster CD, Eaves D: The Fitness Interview Test: A Method for Examining Fitness to Stand Trial. Toronto, Ontario, Canada: University of Toronto, Research Report of the Centre of Criminology, 1984
Golding SL: Interdisciplinary Fitness Interview-Revised: A Training Manual. Salt Lake City, UT: State of Utah Division of Mental Health, 1993
Roesch R, Zapf PA, Eaves D, et al: Fitness Interview Test (Revised Edition). Burnaby, British Columbia, Canada: Mental Health, Law and Policy Institute, Simon Fraser University, 1998
Stafford KP: Assessment of competence to stand trial, in Handbook of Psychology: Forensic Psychology (vol. 11). Edited by Goldstein AM. New York: Wiley, 2002, pp 359–80
Grisso T: Five-year research update (1986–1990): evaluations for competence to stand trial. Behav Sci Law 10:353–69, 1992
Hoge SK, Bonnie RJ, Poythress N, et al: The MacArthur Competence Assessment Tool-Criminal Adjudication. Odessa, FL: Psychological Assessment Resources, 1999
Poythress NG, Nicholson R, Otto RK, et al: Professional Manual for the MacArthur Competence Assessment Tool-Criminal Adjudication. Odessa, FL: Psychological Assessment Resources, 1999
Rogers R, Tillbrook CE, Sewell KW: Evaluation of Competency to Stand Trial-Revised (ECST-R) and Professional Manual. Lutz, FL: Psychological Assessment Resources, 2004
State v. Champagne, 497 A.2d 1242 (N.H. 1985)
State v. Hahn, 707 P.2d 699 (Wash. Ct. App. 1985)
United States v. Duhon, 104 F.Supp.2d 663 (W.D. La. 2000)
Gloldstein AM, Burd M: Role of delusions in trial competency evaluations: case law and implications for forensic practice. Forensic Rep 3:361–86, 1990
Committee on Quality of Health Care in America, Institute of Medicine: Crossing the quality chasm: A new health system for the 21st century. Washington, DC, National Academy Press, 2000
Rakich JS: Quality, CQI, and reengineering in health services organizations. J Health Soc Policy 13:41–58, 2001
Otto RK, Poythress NG, Nicholson RA, et al: Psychometric properties of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA). Psychol Assess 10:435–43, 1998
Poythress NG, Bonnie RJ, Monahan J, et al: Adjudicative Competence: The MacArthur Studies. New York: Kluwer/Plenum, 2002
Rogers R, Grandjean N, Tillbrook C, et al: Recent interview-based measures of competence to stand trial: a critical review augmented with research data. Behav Sci Law 19:503–18, 2001
Tillbrook CE: Competency to proceed: a comparative appraisal of approaches to assessment. Unpublished doctoral dissertation. Tuscaloosa, AL: The University of Alabama, 2000
Plan, Do, Check, Act: PDCA. Available at http://www.isixsigma.com/me/pdca. Accessed December 19, 2004
Byers JF, Beaudin CL: The relationship between continuous quality improvement and research. J Healthcare Qual 24:4–8, 2002
Pinals DA, Packer IK, Fisher W, et al: Relationship between race and ethnicity and forensic clinical triage dispositions. Psychiatr Serv 55:873–8, 2004
Wall BW, Krupp BH, Guilmette T: Restoration of competence to stand trial: a training program for persons with mental retardation. J Am Acad Psychiatry Law 31:189–201, 2003(Debra A. Pinals, MD, Chad E. Tillbrook, )