Buscar en
Revista de Psiquiatría y Salud Mental (English Edition)
Toda la web
Inicio Revista de Psiquiatría y Salud Mental (English Edition) Assessment of individual clinical outcomes: Regarding an electroconvulsive thera...
Journal Information
Vol. 8. Issue 1.
Pages 11-16 (January - March 2015)
Vol. 8. Issue 1.
Pages 11-16 (January - March 2015)
Original article
DOI: 10.1016/j.rpsmen.2015.04.002
Full text access
Assessment of individual clinical outcomes: Regarding an electroconvulsive therapy
Evaluación de resultados clínicos individuales: a propósito de un caso tratado con terapia electroconvulsiva
Ioseba Iraurgia,
Corresponding author

Corresponding author.
, Susana Gorbeñaa, Miren-Itxaso Martínez-Cubillosb, Margarita Escribanoc, Pablo Gómez-de-Maintenanta
a Departamento de Personalidad, Evaluación y Tratamientos Psicológicos, Universidad de Deusto, Bilbao, Vizcaya, Spain
b Fundación Eragintza, Bilbao, Vizcaya, Spain
c Unidad de Hospitalización, Avances Médicos (AMSA), Bilbao, Vizcaya, Spain
Article information
Full Text
Download PDF
Tables (1)
Table 1. Clinical and statistical significant change in NEO-PI-R factors, DBI and HAM-D.

Evaluation of therapeutic results and of the efficacy and effectiveness of treatments is an area of interest both for clinicians and researchers. In general, randomized controlled trial designs have been used as the methodology of choice in which intergroup comparisons are made having a minimum of participants in each arm of treatment. However, these procedures are seldom used in daily clinical practice. Despite this fact, the evaluation of treatment results for a specific patient is important for the clinician in order to address if therapeutic goals have been accomplished both in terms of statistical significance and clinical meaningfulness. The methodology based on Reliable Change Index (RCI – Jacobson and Truax)1 provides an estimate these two criteria. The goal of this article is to propose a procedure to apply the methodology with a single case study of a woman diagnosed with major depression and treated with Electroconvulsive Therapy (ECT).

Clinical assessment
Reliable change index
Case study

La valoración de resultados terapéuticos y la evaluación de la eficacia y efectividad de los tratamientos es un área interés para clínicos e investigadores. Usualmente se ha utilizado la metodología basada en diseños controlados aleatorios y la comparación intergrupos con un mínimo de participantes incluidos en cada brazo de tratamiento. No obstante, estos procedimientos raramente pueden ser aplicados en la práctica clínica habitual, pero sí es importante para el clínico evaluar si los resultados del tratamiento propuesto para una persona específica cumplen con los objetivos terapéuticos, tanto en su significación estadística como por su relevancia clínica. La metodología basada en el índice de cambio fiable (Jacobson y Truax)1 permite estimar esta doble consideración y el objetivo del artículo es proponer el procedimiento de aplicación a través del estudio de un caso único diagnosticado de depresión mayor tratado con Terapia Electro Convulsiva (TEC).

Palabras clave:
Evaluación clínica
Estudio de caso
Full Text

Evaluation is at the heart of the clinical process. It is a key element for diagnosis, treatment planning and evaluation of treatment effects. Obtaining the expected results implies both success of the intervention, and therapeutic discharge, while failure derives in a decision making process regarding change or removal of prescribed treatment or the assumption of new therapeutic recommendations. In any case, information regarding clinical outcomes of an intervention is of utmost importance for health providers.2

Specifically, evaluation of therapeutic efficacy based on scientific evidence relies on a methodology that uses comparisons of groups that are exposed to different interventions; one of which is the object of evaluation, the other with a proven therapeutic value. The basic design is known as random clinical trial, a procedure that defines a step-by-step protocol in order to avoid the effect of bias and preserves casualty of intervention on the therapeutic effect. Group variability control and statistic use allows for decision making regarding the cause (chance or intervention) of the observed changes.3

However, in day-to-day clinical practice, health providers rarely count on big groups to make comparisons and thus, more often than not, their decisions are based on specific cases. The habitual procedure is to consider change shown by the individual, analyzing variation of the state or trait of interest, normally assessed through biological, symptomatic or orectic indexes. For instance, in mental health, therapy has as a goal to change states (i.e. depressive signs and symptoms) or traits, a personality disposition (i.e. neuroticism). In order to do so, states or traits are assessed with standardized instruments that allow for the scoring of the case in relation to a normative group. Therefore, if a person is evaluated with the same instrument before and after treatment, the difference in scores can offer a quantitative index of change than can be incremental, decremental or equal. Besides, scores can be analyzed with reference to a normative sample, allowing for the ranking of the scores of the person in a functional or dysfunctional range. Scales usually provide information about cutoff points to facilitate clinical decisions.

However, compared with the methodology used for scientific decision making regarding efficacy of treatment modalities, decision making in individual cases lacks tools that could inform if the observed change is both clinically meaningful and statistically significant. We need to know if the change has a sufficient magnitude so as to be considered an important clinical outcome and rule out chance.

The goal of this paper, based on a clinical case treated with Electro Convulsive Therapy (ECT), is to offer an answer using the Reliable Change Index,1,4 a formula that can help in clinical and statistically significant decision making in individual cases.

Clinical case

The patient is a 45-year-old married woman, mother of two children and teacher on a sick leave of work. She was admitted in the Psychiatric Unit of a general hospital, being the fifth hospitalization. She was referred by a psychiatric due to severe depressive symptoms for the past several weeks in the context of a Bipolar Disorder.

The patient showed depressive symptoms, especially inability to enjoy daily activities, more exacerbated in the morning, severe asthenia, psychomotor slow-down, tensional headaches, attention and concentration impairments, and nondelirious thoughts of inadequacy, abandonment, disability and guilt.

Her daily functioning is characterized by being self-demanding, presenting herself as a very respectful, polite and well-mannered person, even rigid in response to a firm sense of duty. Her values showed a strong commitment to her responsibilities around her role as a good mother, daughter, wife and worker. She shows obsessive traits (order and control) and a notable denial of hostility. She refers a very organized daily routine that holds her together.

Her medical record indicates hypothyroidism treated with hormones. With reference to psychiatric antecedents, she relates several affective problems since youth, initially of a depressive sing that would remit with Benzodiazepine, even spontaneously. Three years ago she presented a manic episode with psychotic symptoms, congruent and incongruent (paranoid) with her mood. She refers no psychiatric family history.

The patient states that her first depressive episode goes back 6 years when her father died. It was treated with antidepressants, a medication she continues to use today. Initially she responds to IRSA antidepressants showing a quick and effective response with complete recovery. Since then, at least 5 depressive episodes have occurred with a poorer response to treatment, slower recovery without reaching a true euthymic state and maintaining subclinical depressive symptoms that interfere with functioning and quality of life. Medication was changed several times prescribing dual drugs (Venlafaxina) and later a combination of antidepressants with maximal dose (Venlafaxina and Mirtazapina) and incorporating other mood-stabilizing drugs (lithium salt or valproic acid). The intensity and functional disability caused by these symptoms required inpatient treatment. She also attends cognitive psychotherapy on a weekly basis showing good adherence in the past five years.

After several pharmacological interventions without therapeutic results (the latest Clorimipramine 225mg/d, suspended due to no response and severe secondary effects) Electro Convulsive Therapy is recommended. Twelve bilateral sessions are programmed, three per week on alternate days. A very favorable response was observed. During interviews after the third session she is euthymic, with a significant improvement in all symptoms. Finally, given the positive response, ten sessions were administered and maintenance sessions where recommended at discharge following Martínez-Amoros, Cardoner, Galvez and Urretavizcaya's suggestions.5

When the patient was admitted for ECT treatment, she participated in a clinical interview to assess frequency and intensity of depressive symptoms and personality traits. Three standardized instruments were used in the Spanish adapted version: Hamilton Depression Scale,6 Beck Depression Inventory,7 and NEO-PI-R8 a widely used personality test that measure six traits: Neuroticism, Extraversion, Openness to experience, Affability, Conscientiousness. The same protocol and instruments were used at the end of the second week. Table 1 shows her scores in the three scales at pretest and posttest.

Table 1.

Clinical and statistical significant change in NEO-PI-R factors, DBI and HAM-D.

  Pretest  Posttest  Clinical cutoff point  Clinical change  Change difference  RCI  Statistically significant change*  Interpretation 
Neuroticism  63  58  35/65  No  −5  −0.913  No  No change (NC) 
Extraversion  37  37  35/65  No  0.000  No  NC 
Openness  37  37  35/65  No  0.000  No  NC 
Agreeableness  58  57  35/65  No  −1  −0.183  No  NC 
Conscientiousness  53  47  35/65  No  −6  −1.096  No  NC 
Anxiety  63  58  35/65  No  −5  −0.913  No  NC 
Hostility  43  56  35/65  No  +13  2.373  Yes  Significant change 
Depression  68  51  35/65  Yes  −17  −3.104  Yes  Meaningful clinical change 
Self-consciousness  68  60  35/65  Yes  −8  −1.461  No  Nonsignificant change 
Impulsiveness  49  53  35/65  No  +4  0.730  No  NC 
Vulnerability to stress  73  66  35/65  No  −7  −1.278  No  NC 
Warmth  37  43  35/65  No  +6  1.096  No  NC 
Gregariousness  56  42  35/65  No  −14  −2.556  Yes  Significant change 
Assertiveness  29  37  35/65  Yes  +8  1.461  No  Nonsignificant change 
Activity  42  47  35/65  No  +7  1.278  No  NC 
Excitement seeking  43  50  35/65  No  +7  1.278  No  NC 
Positive emotions  37  37  35/65  No  0.000  No  NC 
Fantasy  34  37  35/65  Yes  +3  0.548  No  Nonsignificant change 
Esthetics  47  47  35/65  No  0.000  No  NC 
Feelings  43  46  35/65  No  +3  0.548  No  NC 
Actions  37  40  35/65  No  +3  0.548  No  NC 
Ideas  42  40  35/65  No  −2  −0.365  No  NC 
Values  43  40  35/65  No  −3  −0.548  No  NC 
Trust  57  57  35/65  No  0.000  No  NC 
Straightforwardness  73  60  35/65  Yes  −13  −2.373  Yes  Meaningful clinical change 
Altruism  58  47  35/65  No  −11  −2.008  Yes  Significant change 
Compliance  63  57  35/65  No  −6  −1.096  No  NC 
Modesty  50  57  35/65  No  +7  1.278  No  NC 
Tender mindedness  31  44  35/65  Yes  +13  2.373  Yes  Meaningful clinical change 
Competence  57  44  35/65  No  −13  −2.373  Yes  Significant change 
Order  53  49  35/65  No  −4  −0.730  No  NC 
Dutifulness  57  43  35/65  No  −14  −2.556  Yes  Significant Change 
Achievement striving  53  50  35/65  No  −3  −0.548  No  NC 
Self-discipline  34  47  35/65  Yes  +13  2.373  Yes  Meaningful clinical change 
Deliberation  60  53  35/65  No  −7  −1.278  No  NC 
Social desirability  63  46  35/65  No  −17  −3.104  Yes  Significant change 
Symptomatic scales for depression      L, M, G#           
BDI  30  10, 19, 30  Yes  −22  −6.987  Yes  Meaningful clinical change 
HAM-D  28  12  8, 14, 19  Yes  −16  −12.531  Yes  Meaningful clinical change 

Cutoff points for DBI and HAM-D: M, mild depression; MD, moderate; G, severe.


RCI values >1.96 or ≥1.96 are statistically significant, p<.05. The higher the RCI value the lower the probability observed change is due to chance.

Informed consent was obtained from the patient both for the assessment and the treatment wit the ECT.

Evaluation methodology: reliable change index

Reliable change indexes are a set of statistical procedures used in many fields of psychological assessment to find if the scores of a given individual in a psychometric test have changed compared to a previous assessment with the same instrument.4 Reliable change techniques were initially described by Jacobson and Truax1 and were designed to help in the decision making of significant therapeutic change in patients who had experience a disorder or a clinical intervention.

Jacobson and Truax1 presented a definition of a clinically significant change in therapy. In order to obtain this index, standardized scales with norms and cutoff points to define functional and dysfunctional ranges are needed. In this case, the Spanish norms of the scales were used: Hamilton Depression Scale6 with a .82 Cronbach's Alpha, mean 5.8, and a SD of 2.2; Beck Depression Inventory7 with a .83 Cronbach's Alpha, mean 5.4, and a SD of 5.4, and the NEO PI-R.8 Cutoff points for the three instruments are presented in Table 1.

Additionally, these authors and others (see Iraurgi4 for a review) suggested that the most effective way to estimate if an individual score has reliably and significantly changed is to express the magnitude of the effect. To do so, the magnitude of the change (difference between pretest and posttest) is related to a measurement estimation error calculated on the basis of the reliability of the instrument used and also from the variability of the scores in the normative group. This estimate is the Reliable Change Index (RCI) and the calculations needed are as follows:

  • -

    Step 1. Calculate the standard error of the mean: SE=Sx·(1rxx) ½, or SE=Sx·√(1rxx)

  • -

    Step 2. Calculate the standard error of the difference: SEdif=(2SE2)1/2, or SEdif=SE·√2

  • -

    Step 3. Calculate the reliable change score: RCI=(x2x1)/SEdif

where Sx is the standard deviation of scores of the test in the normative group; rxx is the reliability coefficient of the test; x2 is the posttest score and x1 is the pretest score obtained. The resulting RCI is expressed using standardized z scores: values ≤−1.96 or ≥1.96 are statistically significant with a probability below 5% (p.05). Table 1 shows the RCI for the scales and subscales of the three instruments used with this patient, resulting from the algorithm described and using the reliability indexes and SDs of the scales. An application using Excel was design for the calculations.

Finally, Jacobson and Truax1 define a clinical significant change as one that on the one hand, reaches the cutoff point in terms level of functioning and, on the other, the magnitude is statistically significant. Fulfillment, or not, of one or both criteria results in the following categories: no change (both criteria unfulfilled), nonsignificant change (above the clinical cutoff score but without statistical significance); significant change (a statistically significant result without surpassing the clinical cutoff point) and clinically significant change (both criteria are fulfilled). Table 1 shows the category reached in this case for the scales and subscales of the instruments used.

In order to illustrate the results take as an example the Hamilton Depression Scale (last row in Table 1). The patient obtained, before the ECT treatment a score of 28 far above the cutoff point for severe depression (CP=19). Therefore, the patient presented high intensity and frequency of symptoms that corresponded to a 99 percentile in the norms. Following the intervention, the HAM-D score was 12 somewhere between the cutoff point for mild depression (CP=8) and moderate depression (CP=14) though she still reached a percentile of 95. Magnitude of change is 16 points associated with a RCI of −12.5, a value indicating a statistically very significant change (associate p value is below .0001). However, in this case, the scores have not reached the cutoff point indicating functionality (PC 8) but level of symptoms has been reduced indicating an improvement in intensity and frequency of symptoms that has not occurred by chance. Therefore, this is a clinically meaningful and statistically significant change.


A total of 38 variables were used to evaluate change in a depressed patient treated with ECT. Six variables were statistically and clinically meaningful, three of them clearly associated with the goals of the intervention: the HAM-D, the DBI and the depression facet of the NEO-PI-R.

There were no changes in the five NEO-PI-R factors, an expected finding given the underlying assumptions regarding its character as broad personality traits and stability. Trait measures are sensitive to long term changes. In the same lines, change was not identified in 18 of the NEO-PI-R facets but it is noteworthy to mention some statistically significant changes observed and other not. For instance, in the self-consciousness facet, the patient moves form a 68 pretest score to a score of 60 at posttest, a nonsignificant change. A dysfunctional score is reduced to a functional level, and this could be interpreted as an improvement. However, such an improvement could respond to chance given the fact that the magnitude of change is in the expected range of measurement error. Therefore, having RCI helps us in evaluating change and cautiously interprets it.

On the other hand, we also found results that can be meaningful from a clinical perspective even though criteria strictly based on norms could lead us to disregard them. This is the case of results in the hostility facet (change difference of 13, RCI=2.37) or Gregariousness (CD −14, RCI 2.55). In both cases, RCI values indicated a statistically significant change with a p<.008, this is, in 8 out of 1000 cases we will err when considering that the magnitude of change was due to chance. However, pretest and posttest scores are in the range of functionality and would not be interpreted. Despite this fact, an increase of 13 points in Hostility and a reduction of 14 points in gregariousness may be susceptible of clinical interpretation and viewed as changes attributable to the effect of the intervention.

Finally, consider another score that can have clinical implications: vulnerability. A 73 pretest score changes to 66 at posttest in this patient. Both scores are in the dysfunctionality range and the RCI value (−1.28) does not grant the conclusion of a statistically significant change which leads to no change. However, had the pretest score be of 76, with a 10 point change, the RCI value would have been −2.10, meaning a significant change though not meaningful from a clinical perspective given that the posttest score is in the functionality range. However, such a result will be of interest for the clinicians informing that the change observed is in consonance with the goals of symptom reduction and therefore it would be another criterion to guide future prescriptions.


Data presented in the clinical case discussed show how the reliable change index can be of interest for health providers in guiding decision making regarding observed health results in individual cases. A sound tool is available that goes beyond subjective judgments and offers data to support decisions, controlling for the chance effect in expected results and changes. Besides, it offers a judgment system bases in two levels: clinical and statistical, which allow for the consideration of other results that might have substantial clinical implications. Specifically, in an E.C.T. case it could be a useful tool to asses the efficacy of a continuation and maintenance strategy, as recommend by Martínez-Amoros et al.5

Conflict of interest

The authors declare no conflict of interest.

N.S. Jacobson, P. Truax.
Clinical significance: a statistical approach to defining meaningful change in psychotherapy research.
J Consult Clin Psychol, 59 (1991), pp. 12-19
R. Fernández-Ballesteros.
Evaluación Psicológica Conceptos, métodos y estudio de casos.
Pirámide, (2011),
L.I. Sederer, B. Dickey.
Outcomes assessment in clinical practice.
Williams & Wilkins, (1995),
I. Iraurgi.
Evaluación de resultados clínicos III: Estimación del cambio clínicamente significativo utilizando los índices de cambio fiable.
Norte de Salud Mental, 36 (2010), pp. 105-122
E. Martínez-Amoros, N. Cardoner, V. Gálvez, M. Urretavizcaya.
Eficacia y patrón de uso de la terapia electroconvulsiva de continuación y mantenimiento en el trastorno depresivo mayor.
Rev Psiquiatr Salud Mental, 5 (2012), pp. 241-253
J.A. López-Pina, J. Sánchez-Meca, A.I. Rosa-Alcázar.
The Hamilton rating scale for depression: a meta-analytic reliability generalization study.
Int J Clin Health Psychol, 9 (2009), pp. 143-159
J. Sanz, C. Vázquez.
Fiabilidad, validez y datos normativos del inventario de depresión de Beck.
Psicothema, 10 (1998), pp. 303-318
P.T. Costa, R.R. McCrae.
NEO PI-R, Inventario de Personalidad NEO revisado. Manual Profesional.
TEA Ediciones, (2008),

Please cite this article as: Iraurgi I, Gorbeña S, Martínez-Cubillos M-I, Escribano M, Gómez-de-Maintenant P. Evaluación de resultados clínicos individuales: a propósito de un caso tratado con terapia electroconvulsiva. Rev Psiquiatr Salud Ment (Barc.). 2015;8:11–16.

Copyright © 2014. SEP y SEPB
Article options
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos

es en pt
Política de cookies Cookies policy Política de cookies
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here. Utilizamos cookies próprios e de terceiros para melhorar nossos serviços e mostrar publicidade relacionada às suas preferências, analisando seus hábitos de navegação. Se continuar a navegar, consideramos que aceita o seu uso. Você pode alterar a configuração ou obter mais informações aqui.