Buscar en
Ansiedad y Estrés
Toda la web
Inicio Ansiedad y Estrés Systematic Client Feedback: A naturalistic pilot study
Información de la revista
Descargar PDF
Más opciones de artículo
Original article
DOI: 10.1016/j.anyes.2019.04.005
Acceso a texto completo
Systematic Client Feedback: A naturalistic pilot study
Feedback sistemático del cliente: un estudio piloto naturalístico
Alberto Gimeno-Peóna,
Autor para correspondencia

Corresponding author.
, Javier Prado-Abrilb, Felix Inchaustic, Anxo Barrio-Nespereirad, María Teresa Álvarez-Casariegod, Barry L. Duncane
a Private Practice, Gijón, Spain
b Servicio Aragonés de Salud, Zaragoza, Spain
c Servicio Riojano de Salud, Logroño, Spain
d Servicio Navarro de Salud-Osasunbidea, Pamplona, Spain
e Better Outcomes Now, Jensen Beach, FL, USA
Recibido 09 abril 2019. Aceptado 30 abril 2019
Información del artículo
Texto completo
Descargar PDF
Figuras (2)
Tablas (2)
Table 1. Sample characteristics and status at most recent session.
Table 2. Descriptive statistics and outcome classifications on the Outcome Rating Scale by study.
Mostrar másMostrar menos
Material adicional (1)

Routine outcome monitoring or systematic client feedback (SCF) continues to garner empirical support. The Partners for Change Outcome Management System (PCOMS) is one application of SCF with significant research support but no studies have been conducted in Spain. This investigation describes the effects of PCOMS in routine practice via a comparison to published PCOMS studies.

Material and methods

In a retrospective observational study, PCOMS was implemented with 42 clients treated in routine psychotherapy. Outcomes were measured by the Outcome Rating Scale (ORS) and descriptively compared to PCOMS studies included in a meta-analysis and a PCOMS benchmarking study regarding reliable and/or clinically significant change, no change, and deterioration.


The current study achieved similar rates of reliable and clinically significant change (73.8%) to the benchmarking study (65.6%) and the RCTs (67.5%). Regarding no change rates, the current study (23.8%) mirrored results of the RCTs (25.6%) and was better than the benchmarking study (35.4%). The current study incurred a low 2.4% deterioration rate compared to a 10.7% and 7.0% rate of the benchmarking study and RCTs, respectively.


While not without flaws, this pilot study offers some evidence that the improved outcomes associated with PCOMS may also occur in psychotherapy settings in Spain.

Routine outcome monitoring
Systematic client feedback
Partners for Change Outcome Management System
Evidence-based practice

La monitorización sistemática de resultados o recolección sistemática del feedback del cliente (SFC) continúa acumulando apoyo empírico. El Partners for Change Outcome Management System (PCOMS) es una aplicación de la SFC cuya eficacia no ha sido investigada en España. Este estudio describe los resultados tras implementar PCOMS en un contexto de práctica clínica rutinaria, y los compara con los estudios de referencia disponibles en la literatura especializada.

Material y métodos

En un estudio observacional retrospectivo, PCOMS se implementó con 42 clientes tratados en psicoterapia rutinaria. Los resultados se midieron mediante la Escala de Calificación de Resultados (SRO) y descriptivamente en comparación con los estudios de PCOMS incluidos en un metaanálisis y un estudio de evaluación comparativa de PCOMS con respecto a cambios confiables y/o clínicamente significativos, sin cambios y deterioro.


Se encontraron tasas similares de cambio fiable y cambios clínicamente significativos (73.85) a las obtenidas en el estudio de referencia (65.6%) y ECA (67.5%). En cuanto a las tasas de ausencia de cambios, el presente estudio (23.8%) mostró los mismos resultados que los ECA (25.6%) y mejores que los del estudio de referencia (35.4%). La tasa de deterioro (2.4%) fue inferior al 10.7 y al 7.0% obtenidos en el estudio de referencia y en los ECA, respectivamente.


Con algunas limitaciones que se describen, este estudio piloto ofrece evidencia preliminar de que la mejora de resultados asociada a PCOMS puede darse también en población española.

Palabras clave:
Monitorización sistemática de resultados
Recolección sistemática del feedback del cliente
Partners for Change Outcome Management System
Práctica basada en la evidencia
Texto completo

Over a decade ago, the American Psychological Association (APA) Taskforce on Evidence Based Treatments (2006) recommended that clinicians routinely collect and utilize client-report data to inform treatment. Since then, growing research support of Routine Outcome Monitoring (ROM) or Systematic Client Feedback (SCF) (e.g., Delgadillo et al., 2018; She et al., 2018) has not only empirically validated the Task Force recommendation but also increased its use in varied psychotherapy settings. SCF refers to the continuous monitoring of client perceptions of progress throughout therapy and a real-time comparison with an expected treatment response (ETR) to gauge client progress and signal when change is not occurring as predicted (Duncan & Reese, 2015). With this alert, clinicians and clients have an opportunity to shift focus, revisit goals, or alter interventions before deterioration or dropout (Duncan, 2014; Gimeno-Peón, Barrio-Nespereira, & Prado-Abril, 2018; Prado-Abril, Sánchez-Reales, & Inchausti, 2017).

The need for SCF is based on findings that many clients quit therapy prematurely (Swift & Greenberg, 2012) or have negative outcomes (Reese, Duncan, Bohanske, Owen, & Minami, 2014). The need is also reinforced by the poor outcomes reported in real world settings (e.g., Hansen, Lambert, & Forman, 2002). Client feedback systems aim to identify treatment failures before they occur, allowing time for clinicians to restore therapy to a positive trajectory. More than simply monitoring and collecting data (Kendrick et al., 2016), practitioners must use the client feedback to identify and solve the obstacles of the psychotherapeutic process to improve treatment effectiveness (Delgadillo et al., 2018; Duncan, 2014; Gimeno-Peón et al., 2018).

Several feedback systems have emerged (Castonguay, Barkham, Lutz, & McAleavey, 2013), but only two have randomized clinical trial (RCT) support and have been designated as evidence based practices. First is the Outcome Questionnaire 45.2 (OQ; Lambert, 2015) System. Lambert is the pioneer of SCF, evolving outcome measurement to a “real-time” feedback process with a proven track record of improving outcomes (Lambert & Shimokawa, 2011). The other SCF intervention is the Partners for Change Outcome Management System (PCOMS; Anker, Duncan, & Sparks, 2009). Emerging from clinical practice and designed with the front-line clinician in mind, PCOMS employs two, four item scales, one focusing on outcome given at the beginning of the session (the Outcome Rating Scale [ORS]; Miller, Duncan, Brown, Sparks, & Claud, 2003) and the other assessing the therapeutic alliance administered toward the end of the encounter (the Session Rating Scale [SRS]; Duncan et al., 2003). PCOMS directly involves clinicians and clients in an ongoing process of measuring and discussing both progress and the alliance, the first system to do so. Both PCOMS and the OQ System are a-theoretical, can be integrated into any approach, and can be used across client diagnoses and from a transdiagnostic approximation (e.g., González-Blanch et al., 2018).

PCOMS evolved from “client directed, outcome-informed” clinical work (Duncan & Miller, 2000). Building on common factors research (Lambert, 1986), Duncan, Solovey, and Rusk (1992) proposed a “client directed” psychotherapy that privileged client perspectives regarding the causes and solutions of problems (the client's theory of change) and depended on client resources in any intervention effort. “Outcome-informed” (Duncan & Miller, 2000) began with the use of Lambert's OQ and the Working Alliance Inventory (Tracey & Kokotovic, 1989). From these origins, and concerns regarding the feasibility of longer measures, the Outcome Rating Scale and Session Rating Scale were co-developed (Miller, Duncan, & Johnson, 2002; Miller & Duncan, 2000). Thereafter, based on 2 years of private practice as well as multiple team supervisions in a family therapy community clinic, Duncan created the clinical process of using the ORS and SRS, detailed first in Duncan and Sparks (2002). The need for an operationally defined process for conducting RCTs that spoke only to the clinical use of the ORS and SRS resulted in the name, “Partners for Change Outcomes Management System” or PCOMS (Anker et al., 2009; Duncan & Reese, 2015).

There are seven RCTs that support the efficacy of PCOMS over treatment as usual (TAU) in individual (Brattland et al., 2018; Reese, Norsworthy, & Rowlands, 2009; She et al., 2018), couple (Anker et al., 2009; Reese, Toland, Slone, & Norsworthy, 2010), and group (Schuman, Slone, Reese, & Duncan, 2015; Slone, Reese, Mathews-Duvall, & Kodet, 2015) therapy. In addition, benchmarking studies have found outcomes comparable to RCTs in a public outpatient setting that implemented PCOMS (Reese et al., 2014) as well as with a psychiatric inpatient unit (Reese et al., 2018).

Dissemination of SCF has also been enhanced by the development of new technologies for data collection and analysis in everyday clinical settings (Prado-Abril, 2016). Widespread data collection enables the creation of algorithms for normative trajectories that guide both clinician and client in all decisions that affect treatment (Delgadillo et al., 2017), thus creating more consistent quality of care across clients. For example, technological application of PCOMS across the US and 20 other countries has resulted in over 1.5 million administrations of the ORS and SRS in the data base and predictive algorithms (Duncan & Sparks, 2018).

The empirical support for PCOMS is promising but questions remain about cross-cultural applications. For example, are these findings generalizable to clinical practice in Spain? Pragmatic studies offer a way to evaluate effectiveness of any given intervention (e.g., Weinfurt et al., 2017). Unlike clinical trials, pragmatic studies assess effectiveness in naturalistic contexts, prioritizing external and ecological validity over internal validity. Pragmatic studies, often called practice oriented research (POR; Fernández-Álvarez & Castonguay, 2018), enable the evaluation of target populations in target settings and promote collaboration between clinicians and researchers (Weinfurt et al., 2017).

To this end, this pilot study explored the effects of PCOMS in routine practice via a descriptive comparison of the results with a published benchmarking study of PCOMS (Reese et al., 2014) and three PCOMS RCTs included in the Lambert and Shimokawa meta-analysis (2011). We hypothesized that outcomes generated in this setting by this provider would favorably compare with these published studies of PCOMS regarding reliable and/or clinically significant change, no change, and deterioration.


The sample was recruited from the clinic of a clinical psychologist (AGP) with seven years of clinical experience who practices a systemic integrative theoretical orientation. Eligible candidates were those treated between February 2016 and June 2018, older than 16 years, who completed the PCOMS scales throughout all sessions, and who attended a minimum of two sessions.

Out of 75 candidates, 42 individuals (19 males and 23 females) met the selection criteria. The mean age of participants was 33.41 years (SD=11.17, range 17–49). Reasons for service included: anxiety disorders (31.0%); mood disorders (23.8%); and the remaining 45.2% consisted of couple problems, family issues, vulnerable personality traits, and substance misuse, etc.). Table 1 shows the socio-demographic and clinical characteristics of the sample.

Table 1.

Sample characteristics and status at most recent session.

Socio-demographic characteristics
Females  23 (54.7%) 
Mean age (SD)  33.41 (11.17) 
Singles  29 (69%) 
Labor assets  25 (59.5%) 
Secondary  4 (9.5%) 
High School-Vocational  16 (38%) 
University  18 (43%) 
Clinical characteristics
Anxiety disorders  13 (31%) 
Mood disorders  10 (23.8%) 
Other problems of clinical relevance  19 (45.2%) 
Status at most recent session
Continue in psychotherapy  11 (26.2%) 
Premature termination  14 (33.3%) 
Planned discharge  17 (40.5%) 
Average of psychotherapeutic sessions (SD)  7.5 (6.8) 

The Outcome Rating Scale (ORS; Miller et al., 2003). The first part of the PCOMS feedback intervention is the administration of the ORS at the beginning of each session and the discussion of client progress. The ORS (see Fig. 1) is a Visual Analog Scale consisting of four 10 centimeter lines, corresponding to four domains (individual, interpersonal, social, and overall), allowing for the client's idiosyncratic rendering of his or her life circumstance. Clients place a mark on each line to represent their perception of their functioning in each domain if using a paper and pencil version or touch or click on an iPad or other device if using a web based application. A centimeter ruler is used to measure the distance to the nearest millimeter from the left end of the scale to the client's mark on each of the four lines, or it is automatically scored by the web system. The scores range from 0 to 40, with lower scores signaling higher distress. The web version of the PCOMS (betteroutcomesnow.com) was used in this study.

Fig. 1.

The Outcome Rating Scale (ORS) and Session Rating Scale (SRS). Copyright 2000, 2002, respectively by B. L. Duncan and S. D. Miller. For examination only. Free working copies for individual use in 28 languages are available at https://www.betteroutcomesnow.com/.


Multiple psychometric studies and RCTs have validated the reliability and validity of the English ORS (Duncan & Reese, 2015), and hundreds of thousands of administrations have determined cutoff and reliable change norms (Duncan & Sparks, 2018; Jacobson & Truax, 1991). The reliable change index of the ORS is 6 points; the clinical cutoff is 25. Scores under 25 reflect the distress level of persons seeking psychotherapy. Clinically significant change is denoted by a change of 6 points and the crossing of the cutoff at termination. The Spanish version of the ORS used in the current study has demonstrated good internal consistency (α=.96) and strong concurrent validity (r=.70) with the Clinical Outcomes in Routine Evaluation – Outcome Measure (CORE-OM; Moggia, Niño-Robles, Miller, & Feixas, 2018), and is consistent with other psychometric studies of the English-based ORS.

The Session Rating Scale (Duncan et al., 2003). The second component of the PCOMS feedback intervention is the administration of the SRS toward the end of each session and the discussion of client perceptions about the alliance. The SRS is also a four-item visual analog scale based on Bordin's (1979) classic delineation of the components of the alliance: the relational bond and the degree of agreement between the client and therapist about the goals and tasks of therapy (see Fig. 1). Clients place a mark or touch on a 10-cm line nearest the pole that best describes their felt experience with their therapist. The client's marks on the four items are either measured by a centimeter ruler and totaled or automatically scored for a total ranging from 0 to 40.

The English SRS also has evidence of generating reliable and valid scores in both psychometric and RCT investigations (Duncan & Reese, 2015). Regarding the cutoff score for the SRS, a conservative estimate derived for clinical purposes with descriptive statistics (score at which the majority of clients are above) is 36 (Duncan & Sparks, 2018). The Spanish SRS has shown similar properties to the English SRS (α=.94; Moggia, Niño-Robles, Feixas, & Miller, 2017).


The clinician incorporated the clinical process of PCOMS (Duncan & Sparks, 2018) in his routine practice (based in a brief non-manualized systemic-integrative approach). The client's progress in therapy and perspective of the alliance was monitored and discussed in each session allowing treatment to be individually tailored to the needs of the client (Duncan, 2014). Fig. 2 depicts the web application of PCOMS that plots the client scores against the expected treatment response, allowing ongoing discussion with the client about the benefit, or lack thereof, of services. Such discussions promote psychotherapy as a vehicle for change and open a collaborative space for shared decision-making about all aspects of treatment.

Fig. 2.

The betteroutcomesnow.com graph with Outcome Rating Scale (ORS) scores, Session Rating Scale (SRS) scores, and the expected treatment response (ETR). Progress meter shows client to be less than 50% of ETR, suggesting a conversation about changing therapeutic directions.

Data analyses

An inspection of Table 2 reveals the current study descriptively compared to the Reese et al. (2014) benchmarking investigation conducted in the US (N=5168) and the RCTs (Anker et al., 2009; Reese et al., 2009; N=558) conducted in Norway and the US, respectively, used in the Lambert and Shimokawa (2011) meta-analysis. As depicted in Table 2, the three studies were grouped into four categories according to the magnitude of the differences in the ORS scores between the first and the last session (Bauer, Lambert, & Nielsen, 2004). The first category, reliable change, included those cases that obtained a pre-post difference of 6 points or higher (reliable index of change of the ORS). The second category, clinically significant change, reflects a pre-post difference in the ORS scores of 6 points and, at post-test, the client transcended the clinical cutoff of the ORS (25 points). The third category (no change) refers to those clients who neither worsen nor improve in a significant way; i.e., improvements or deteriorations that do not reach 6 points of difference. Finally, the category of deterioration includes those whose ORS scores decrease 6 or more points with respect to the first session.

Table 2.

Descriptive statistics and outcome classifications on the Outcome Rating Scale by study.

  N=42aMean or # (SD or %)  N=5168b  n=299c 
ORS score at first session  13.59 (8.03)  19.38 (8.17)  18.26 (8.19) 
ORS score at last session  25.48 (9.63)  25.18 (9.05)  27.65 (9.39) 
Pre-post ORS scores difference  11.88  5.8  9.39 
Effect sized  1.48  .71  1.15 
Reliable change  14 (33.3)  1173 (22.7)  N/A 
Clinically significant change  17 (40.5)  1618 (42.9)  N/A 
RCSC  31 (73.8)  2791 (65.6)  203 (67.5) 
No change  10 (23.8)  1830 (35.4)  77 (25.6) 
Deterioration  1 (2.4)  555 (10.7)  21 (7.0) 

Note: #=number of clients in category; ORS=Outcome Rating Scale; N/A=not available; RCSC=reliable and/or clinically significant change.


Current study.


Reese et al. (2014); N=5168 but available for achieving clinically significant change n=3774.


Anker et al. (2009) and Reese et al. (2009); N=558 but n=299 in feedback condition.


Pretreatment–posttreatment effect sizes were calculated by dividing the mean difference by the pretreatment standard deviation.

We also tracked client status at the most recent session: the percentage of clients who were still in active treatment (continue in psychotherapy); those who prematurely terminated (unilateral resolution of the process); and those who had a planned discharge (consensual resolution of the process).


The 42 clients in the current study received 7.5 psychotherapy sessions (SD=6.8, range 2–32): 26.2% remained in active treatment, 33.3% terminated therapy prematurely, and the remaining 40.5% were discharged by consensual decision (see Table 1). Table 2 provides descriptive statistics and compares the outcome categories of the three studies. The first session mean of the current study was 13.59, indicating significant distress. The mean at the end of the treatment was 25.48, a gain of 11.88 points, and slightly above the clinical cutoff. Clients, on average, achieved clinically significant change as measured by the ORS; i.e., they changed more than the reliable change index of the ORS (6 points) and transcended the clinical cutoff (25). Regarding the ORS outcome categories, 40.5% achieved clinically significant change, 33.3% changed reliably, 23.8% experienced no change, and 2.4% of the clients deteriorated. Adding the first two categories, 73.8% or 31 participants demonstrated reliable and/or clinical significant improvements. Comparison with the other studies demonstrated more similarities than differences.

Results compared with another published PCOMS studies

Comparing these results to the RCTs included in the Lambert and Shimokawa (2011) meta-analyses demonstrated that this study produced similar results. Clients, on average, in those studies changed 9.39 points and crossed the clinical cutoff. Results exceeded those attained in the benchmarking investigation in public behavioral health with impoverished clients (Reese et al., 2014) regarding average change (5.8 points), although clients here also crossed the clinical cutoff. The first session means, however, were noticeably different. The current study first session ORS mean (13.59) was substantially lower (indicating higher distress) than both the benchmarking first session mean (19.38) and the RCT first session mean (18.26). The small size of the sample is perhaps the best explanation but more research is needed with clinical populations in Spain.

Regarding the outcome categories, the three data sets look very similar with rates of reliable and clinically significant change of 73.8% (current study), 65.6% (benchmarking study), and 67.5% (the RCTs). These findings corroborate the effects reported in other PCOMS trials (see Duncan & Reese, 2015). Similarly, meta-analyses of psychotherapy efficacy typically find between 56.7% and 67.2% improvement; in contrast, naturalistic studies without SCF have reported improvement as low as 20% (Hansen et al., 2002). Regarding the no change rates, the current study (23.8%) mirrored results of the RCTs (25.6%) and was better that the benchmarking study (35.4%). The biggest difference among the outcome categories occurred in deterioration rates. The current study incurred a very low 2.4% deterioration rate compared to a 10.7% and 7.0% rate of the benchmarking study and RCTs, respectively. Deterioration rates are often reported to be 8–10% (Cuijpers, Reijnders, Karyotaki, de Wit, & Ebert, 2018; Prado-Abril et al., 2017). Again, the small sample size is likely the best explanation.


This study examined the effects of a systematic client feedback system in routine psychotherapy practice. An analysis of reliable and clinically significant change revealed that 73.8% of participants benefitted from treatment. Clients, on average, improved by 11.88 ORS points and crossed the clinical cutoff of 25, thereby achieving clinically significant change. It is notable that clients who ended treatment prematurely are included in the analysis.

A higher rate of premature termination (33.3%) occurred compared to the rate reported (19.7%) in a recent meta-analysis (Swift & Greenberg, 2012). Depending on the type of analyses and how dropouts are defined, however, the variability of premature termination rates can range from 20 to 60% (Swift & Greenberg, 2012). A closer case-by-case look of the current study revealed that 6 (42.8%) of the 14 clients who had prematurely terminated had a pre-post improvement of 6 or more points in the last session. Duncan (2014) asserts that so called “drop outs” should only be counted if a reliable change has not occurred, arguing that the termination may be unplanned from the clinician's perspective but it isn’t “premature” from the client's point of view. Finally, the private healthcare context may also have influenced the rate in the current study. Services are unfunded and expediency is highly valued – clients want rapid intervention and quick results. More research is needed of premature termination that is consistent in definition, complete in analysis, and that includes the client's perspective.

As noted, PCOMS is not only a monitoring system to inform the therapist but also requires discussion and collaboration with clients (Sparks & Duncan, 2018). Such a process creates a higher demand on the therapist to incorporate the feedback which requires specialized training, credentials, and clinical expertise (Gimeno-Peón et al., 2018; Inchausti, Prado-Abril, Sánchez-Reales, Vilagrà-Ruíz, & Fonseca-Pedrero, 2018; Prado-Abril et al., 2017). Successful implementation also requires an organizational commitment to data collection, timely identification of not-on-track clients, and dissemination of the data to clinicians and supervisors, as well ongoing attention to adherence and data integrity (Duncan, 2014).

The results of this study are promising but should be interpreted with caution given the limitations. First, the study includes only a single clinician and therefore no control of therapist effects (Prado-Abril et al., 2017). There was also no baseline information with this clinician nor a control group to establish whether PCOMS had any relationship to the reported results. Part of these limitations are due to the fact that the sample was obtained during a limited period of time and in the context of the private practice of a single clinician. The objective was to show the effectiveness of psychotherapy when integrating SCF, comparing the results with benchmark studies. Future research should include a control group that allows to clarify to what extent PCOMS can improve outcomes in routine practice of clinical psychologists from Spain. The use of only one outcome measure in the pre–post treatment analysis also limits the conclusions that may be drawn.

The empirical support for SCF continues to grow. Until this study, it was unknown whether the feedback effect could be replicated in Spain. It was also unknown to what degree cultural differences might influence the effect. While preliminary and not without flaws, this pilot study offers some evidence that the documented improved outcomes associated with PCOMS may also occur in psychotherapy settings in Spain.

Ethics approval and consent to participate

The study was conducted in compliance with local regulations and internationally established principles of the Declaration of Helsinki (64th World Medical Association General Assemble, Fortaleza, Brazil, 2013).


The authors have no funding to report.

Conflict of interest

Barry L. Duncan is a co-holder of the copyright of the Partners for Change Outcome Management System (PCOMS) instruments. The measures are free for individual use but Duncan receives royalties from licenses issued to groups and organizations. In addition, the web-based application of PCOMS, BetterOutcomesNow.com, is a commercial product and he receives profits based on sales.

Appendix A
Supplementary data

The following are the supplementary data to this article:

[Anker et al., 2009]
M.G. Anker, B.L. Duncan, J.A. Sparks.
Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting.
Journal of Consulting and Clinical Psychology, 77 (2009), pp. 693-704
[APA, 2006]
APA Presidential Task Force on Evidence-Based Practice.
Evidence-based practice in psychology.
American Psychologist, 61 (2006), pp. 271-285
[Bauer et al., 2004]
S. Bauer, M.J. Lambert, S.L. Nielsen.
Clinical significance methods: A comparison of statistical techniques.
Journal of Personality Assessment, 82 (2004), pp. 60-70
[Bordin, 1979]
E. Bordin.
The generalizability of the psychoanalytic concept of the working alliance.
Psychotherapy, 16 (1979), pp. 252-260
[Brattland et al., 2018]
H. Brattland, J.M. Koksvik, O. Burkeland, R.W. Gråwe, C. Klöckner, O.M. Linaker, …, V.C. Iversen.
Effects of Routine Outcome Monitoring (ROM) on therapy outcomes in the course of an implementation process. A randomized clinical trial.
Journal of Counseling Psychology, 65 (2018), pp. 641-652
[Castonguay et al., 2013]
L. Castonguay, M. Barkham, W. Lutz, A. McAleavey.
Practice-oriented research: Approaches and applications.
Bergin and Garfield's handbook of psychotherapy and behavior change, 6th ed., pp. 85-133
[Cuijpers et al., 2018]
P. Cuijpers, M. Reijnders, E. Karyotaki, L. de Wit, D.D. Ebert.
Negative effects of psychotherapies for adult depression: A meta-analysis of deterioration rates.
Journal of Affective Disorders, 239 (2018), pp. 138-145
[Delgadillo et al., 2018]
J. Delgadillo, K. de Jong, M. Lucock, W. Lutz, J. Rubel, S. Gilbody, …, D. McMillan.
Feedback-informed treatment versus usual psychological treatment for depression and anxiety: A multisite, open-label, cluster randomised controlled trial.
Lancet Psychiatry, 5 (2018), pp. 564-572
[Delgadillo et al., 2017]
J. Delgadillo, K. Overend, M. Lucock, M. Groom, N. Kirby, D. McMillan, …, K. de Jong.
Improving the efficiency of psychological treatment using outcome feedback technology.
Behavior Research and Therapy, 99 (2017), pp. 89-97
[Duncan et al., 2003]
B. Duncan, S. Miller, J. Sparks, D. Claud, L. Reynolds, J. Brown, L. Johnson.
The Session Rating Scale: Preliminary psychometric properties of a “working” alliance measure.
Journal of Brief Therapy, 3 (2003), pp. 3-12
[Duncan et al., 1992]
B. Duncan, A. Solovey, G. Rusk.
Changing the rules: A client-directed approach.
Guilford, (1992),
[Duncan and Sparks, 2002]
B. Duncan, J. Sparks.
Heroic clients, heroic agencies: Partners for change.
Author, (2002),
[Duncan and Sparks, 2018]
B. Duncan, J. Sparks.
The partners for change outcomes management system: An integrated eLearning manual for everything PCOMS.
Author, (2018),
[Duncan, 2014]
B.L. Duncan.
On becoming a better therapist: Evidence-based practice one client at a time.
2nd ed., American Psychological Association, (2014),
[Duncan and Miller, 2000]
B.L. Duncan, S.D. Miller.
The heroic client: Doing client-directed, outcome-informed therapy.
Jossey-Bass, (2000),
[Duncan and Reese, 2015]
B.L. Duncan, R.J. Reese.
The Partners for Change Outcome Management System (PCOMS): Revisiting the client's frame of reference.
Psychotherapy, 52 (2015), pp. 391-401
[Fernández-Álvarez and Castonguay, 2018]
H. Fernández-Álvarez, L.G. Castonguay.
Investigación orientada por la práctica: Avances en colaboraciones entre clínicos e investigadores. Introducción.
Revista Argentina de Clínica Psicológica, 27 (2018), pp. 107-114
[Gimeno-Peón et al., 2018]
A. Gimeno-Peón, A. Barrio-Nespereira, J. Prado-Abril.
Monitorización sistemática y feedback en psicoterapia.
Papeles del Psicólogo, 39 (2018), pp. 174-182
[González-Blanch et al., 2018]
C. González-Blanch, O. Umaran-Alfageme, P. Cordero-Andrés, R. Muñoz Navarro, P. Ruíz-Rodríguez, L.A. Medrano, …, Grupo de Investigación PsicAP.
Tratamiento psicológico de los trastornos emocionales en Atención Primaria: El manual de tratamiento transdiagnóstico del estudio PsicAP.
Ansiedad y Estrés, 24 (2018), pp. 1-11
[Hansen et al., 2002]
N.B. Hansen, M.J. Lambert, E.M. Forman.
The psychotherapy dose–response effect and its implications for treatment delivery services.
Clinical Psychology: Science and Practice, 9 (2002), pp. 329-343
[Inchausti et al., 2018]
F. Inchausti, J. Prado-Abril, S. Sánchez-Reales, R. Vilagrà-Ruíz, E. Fonseca-Pedrero.
El trastorno de personalidad por evitación: Una propuesta de tratamiento especializado en la sanidad pública española.
Ansiedad y Estrés, 24 (2018), pp. 144-153
[Jacobson and Truax, 1991]
N.S. Jacobson, P. Truax.
Clinical significance: A statistical approach to defining meaningful change in psychotherapy research.
Journal of Consulting and Clinical Psychology, 59 (1991), pp. 12-19
[Kendrick et al., 2016]
T. Kendrick, M. El-Gohary, B. Stuart, S. Gilbody, R. Churchill, L. Aiken, …, M. Moore.
Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults.
Cochrane Database of Systematic Reviews, 7 (2016),
[Lambert, 1986]
M.J. Lambert.
Implications of psychotherapy outcome research for eclectic psychotherapy.
Handbook of eclectic psychotherapy, pp. 436-462
[Lambert, 2015]
M.J. Lambert.
Progress feedback and the OQ-system: The past and the future.
Psychotherapy, 52 (2015), pp. 381-390
[Lambert and Shimokawa, 2011]
M.J. Lambert, K. Shimokawa.
Collecting client feedback.
Psychotherapy, 48 (2011), pp. 72-79
[Miller et al., 2002]
S. Miller, B. Duncan, L. Johnson.
The Session Rating Scale.
Author, (2002),
[Miller and Duncan, 2000]
S. Miller, B.L. Duncan.
The Outcome Rating Scale.
Author, (2000),
[Miller et al., 2003]
S.D. Miller, B.L. Duncan, J. Brown, J. Sparks, D. Claude.
The Outcome Rating Scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure’.
Journal of Brief Therapy, 2 (2003), pp. 91-100
[Moggia et al., 2017]
D. Moggia, N. Niño-Robles, G. Feixas, S.D. Miller.
Psychometric properties of the Outcome Rating Scale (ORS) and Session Rating Scale 3.0 (SRS 3.0) in a Spanish clinical sample.
Conference presented at 4th Joint Conference Society for Psychotherapy Research UK & European Chapters, http://dx.doi.org/10.13140/RG.2.2.26691.30243
[Moggia et al., 2018]
D. Moggia, N. Niño-Robles, S. Miller, G. Feixas.
Psychometric properties of the Outcome Rating Scale (ORS) in a Spanish clinical sample.
The Spanish Journal of Psychology, 21 (2018), pp. E30
[Prado-Abril, 2016]
J. Prado-Abril.
Facilitadores y obstáculos de una intervención psicoterapéutica asistida por ordenador para la depresión en Atención Primaria: Un estudio cualitativo.
Universidad de Zaragoza, (2016), http://dx.doi.org/10.13140/RG.2.2.28792.55044
[Prado-Abril et al., 2017]
J. Prado-Abril, S. Sánchez-Reales, F. Inchausti.
En busca de nuestra mejor versión: Pericia y excelencia en Psicología Clínica.
Ansiedad y Estrés, 23 (2017), pp. 110-117
[Reese et al., 2018]
R.J. Reese, B. Duncan, J. Kodet, H. Brown, C. Meiller, M. Farook, …, R. Bohanske.
Patient feedback as a quality improvement strategy in an acute care, inpatient unit: An investigation of outcome and readmission rates.
Psychological Services, 15 (2018), pp. 470-476
[Reese et al., 2014]
R.J. Reese, B.L. Duncan, R.T. Bohanske, J.J. Owen, T. Minami.
Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy.
Journal of Consulting and Clinical Psychology, 82 (2014), pp. 731-742
[Reese et al., 2009]
R.J. Reese, L.A. Norsworthy, S.R. Rowlands.
Does a continuous feedback system improve psychotherapy outcome?.
Psychotherapy: Theory, Research, Practice, & Training, 46 (2009), pp. 418-431
[Reese et al., 2010]
R.J. Reese, M.D. Toland, N.C. Slone, L.A. Norsworthy.
Effect of client feedback on couple psychotherapy outcomes.
Psychotherapy: Theory, Research, Practice, & Training, 47 (2010), pp. 616-630
[Schuman et al., 2015]
D.L. Schuman, N.C. Slone, R.J. Reese, B. Duncan.
Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment.
Psychotherapy Research, 25 (2015), pp. 396-407
[She et al., 2018]
Z. She, B.L. Duncan, R.J. Reese, Q. Sun, Y. Shi, G. Jiang, …, A.L. Clements.
Client feedback in China: A randomized clinical trial in a college counseling center.
Journal of Counseling Psychology, 65 (2018), pp. 727-737
[Slone et al., 2015]
N.C. Slone, R.J. Reese, S. Mathews-Duvall, J. Kodet.
Evaluating the efficacy of client feedback in group psychotherapy.
Group Dynamics: Theory, Research, and Practice, 19 (2015), pp. 122-136
[Sparks and Duncan, 2018]
J.A. Sparks, B.L. Duncan.
The Partners for Change Outcome Management System: A both/and system for collaborative practice.
Family process, 57 (2018), pp. 800-816
[Swift and Greenberg, 2012]
J.K. Swift, R.P. Greenberg.
Premature discontinuation in adult psychotherapy: A meta-analysis.
Journal of Consulting and Clinical Psychology, 80 (2012), pp. 547-559
[Tracey and Kokotovic, 1989]
T. Tracey, A. Kokotovic.
Factor structure of the Working Alliance Inventory.
Psychological Assessment: A Journal of Consulting and Clinical Psychology, 1 (1989), pp. 207-210
[Weinfurt et al., 2017]
K.P. Weinfurt, A.F. Hernandez, G.D. Coronado, L.L. DeBar, L.M. Dember, B.B. Green, …, L.H. Curtis.
Pragmatic clinical trials embedded in healthcare systems: Generalizable lessons from the NIH Collaboratory.
BMC Medical Research Methodology, 17 (2017), pp. 144
Copyright © 2019. Sociedad Española para el Estudio de la Ansiedad y el Estrés - SEAS
Opciones de artículo
Material suplementario
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos