Background/Objective: The study evaluated two variations of Parent Management Training (PMT) for children referred to treatment for oppositional, aggressive, and antisocial behavior. The goal was to evaluate the impact of multiple enhancements to optimize common and placebo factors to augment therapeutic change. Method: The families of all children (N=138, 39 girls and 99 boys, ages 6-13) received PMT. One half of the families were assigned to receive an enhanced version with multiple additions designed to increase bonding of the parent to the therapist, professionalism of treatment and setting, credibility of the intervention, and expectancies for therapeutic change. Assessment included multiple treatment outcome measures of the child (problem behaviors, psychiatric symptoms, social competence, and adaptive functioning) and parents (depression, stress, and family relations) showed marked improvements over the course of treatment, and several process measures (therapeutic alliance, credibility of the procedures, expectancy for change). Results: The results indicated that children and parents made marked improvement in all the treatment outcome measures. The vast majority of children fell within the normative range at posttreatment on problem and prosocial behaviors. The two treatment conditions were no different in outcomes for children or parents. Conclusion: PMT led to marked changes in treatment outcome.
Antecedentes/Objetivo: Se evalúan dos versiones del Parent Management Training (PMT) para tratamiento de conducta oposicionista, agresiva y antisocial en niños. El objetivo fue evaluar el impacto de múltiples mejoras para optimizar los factores comunes y de placebo con el fin de incrementar el efecto terapéutico. Método: Las familias de los niños (N = 138, 39 niñas y 99 niños de 6 a 13 años) recibieron PMT. La mitad fue asignada a una versión mejorada con múltiples adiciones para aumentar la vinculación de los padres con el terapeuta, la profesionalidad del tratamiento y el entorno, la credibilidad de la intervención y las expectativas de cambio terapéutico. La evaluación incluyó medidas del efecto del tratamiento en el niño (problemas de comportamiento, síntomas psiquiátricos, competencia social y funcionamiento adaptativo) y los padres (depresión, estrés y relaciones familiares) y medidas del proceso (alianza terapéutica, credibilidad de los procedimientos y expectativa de cambio). Resultados: Niños y padres mejoraron notablemente en todas las medidas del efecto del tratamiento. La gran mayoría de niños se situaron dentro del rango normativo en el post-tratamiento. Las dos condiciones de tratamiento no fueron diferentes para niños ni padres. Conclusión: PMT produjo cambios significativos en el resultado del tratamiento.
In the history of psychotherapy, two broad influences have been proffered to explain why patients improve. The first of these has been the specific intervention techniques (e.g., cognitive therapy for depression) how those techniques lead to therapeutic change. The second broad influence proposed to explain the effects of psychotherapy has focused on factors that are generated or fostered by the techniques and the broad context in which these are provided. These broader influences are more readily conveyed in the context of placebo effects and the treatment of medical disorders. Generally stated, placebo effects are those outcome results that are due to a variety of influences that relate to the beliefs and expectations of the patient and those with whom they interact as well as contextual cues that can promote change (Carvalho et al., 2016; Kaptchuk & Miller, 2015). The power of these effects of treatment have been accorded a major role historically to explain why and how many interventions work and, with recent advances, underlying processes (e.g., in the brain) (Benedetti, 2014; Dodd, Dean, Vian, & Berck, 2017).
In the context of psychotherapy, effects analogous to those of “placebo” effects have been studied, but less extensively than placebo effects in the context of medication and medical treatments. Among the reasons is the difficultly in identifying an inactive, inert treatment (e.g., sugar pill or injection of saline solution). In psychotherapy, we have little idea of what “inert” would mean, given that we do not know the mechanism(s) of change, that is, how and why treatments work. Psychotherapy techniques depend on many interpersonal, intrapersonal, and contextual processes (e.g., learning, persuasion, social influence, cognitive change, alliance with a therapist) that overlap with those same processes leading to placebo effects.
In the psychotherapy literature, those factors that are associated with treatment delivery but not specific to a given treatment have been referred to as common factors of therapy (e.g., Duncan, Miller, Wampold, & Hubble, 2010; Rosenzweig, 1936). These factors span the majority of the several hundred forms of therapy and include coming to treatment sessions, meeting with a professional healer, developing a therapeutic relationship or alliance, hearing a persuasive story line about one's malady and its treatment, engaging in procedures designed to help, having one's expectations and hope mobilized for change, and other contextual or interpersonal influences that could promote expectations in the client or therapist that treatment will be effective. Common factors play a critical role in understanding psychotherapies and may explain in whole or in part why therapeutic change occurs (Frank & Frank, 1991; Wampold & Imel, 2015). Multiple findings support the critical role of common factors, and especially the role of expectations for therapeutic change and credibility of the treatment. We know that “fake” or made-up activities and various control conditions can lead to change if the clients believe the treatment is real and if that treatment is believable as a viable intervention and that viable treatments and control conditions are not likely to be different in treatment outcome if expectancies and credibility are controlled (e.g., Boot, Simons, Stothar, & Stutts, 2013; Gould, Coulson, & Howard, 2012; Kazdin, 2017b; Palpacuer et al., 2017).
While common factors and specific technique factors can be separated experimentally, in clinical work and patient care they operate together. In this approach, common factors of treatment that might promote change are not given instead of treatment factors and are not artifacts to be controlled. Rather, the challenge is to mobilize any factor that can reliably improve patient care. This might well include all sorts of influences in relation to the person who receives the treatment as well as those in their environment who interact and view the client in ways that might promote and encourage change (Grelotti & Kaptchuk, 2011).
The present study tested the effects of enhancing multiple factors related to the delivery of an evidence-based treatment. Several additions were combined to convey to clients the strength of the treatment in effecting change, to increase bonding both to the therapist and the treatment, to increase the professional presentation of the treatment and the setting, and to enhance expectations for therapeutic change. A randomized controlled clinical trial was conducted for the treatment of children referred clinically for oppositional, aggressive, and antisocial behaviors. The treatment included Parent Management Training (PMT) and was provided to all families referred for treatment and included in the study. We have developed this treatment in several controlled trials with children varying in degrees of severity of dysfunction (e.g., children seen in inpatient and outpatient settings) (see Kazdin, 2017a for a review). Through random assignment, as provided by a computer generated random numbers table, half of the families received an additional component that included multiple enhancements that were explicitly designed to augment key common factors that were likely to impact credibility and expectations of the treatment and augment therapeutic change. Our expectation was that our standard treatment (PMT only) and enhanced version (enhanced PMT) treatments would be effective in producing therapeutic change but that the enhanced would produce even greater change. Apart from evaluating child and parent treatment outcome, we evaluated emergent processes and reactions to treatment including credibility and expectancies of treatment, the therapeutic alliance, perceived barriers, and treatment acceptability.
MethodParticipantsParticipation was initiated by families who contacted the Yale Parenting Center, an outpatient treatment service for children and families. After self-referral, caregivers of the referred child completed an initial evaluation to assess child, parent, and family functioning and then began treatment. Informed consent was solicited and obtained from all families who completed treatment. The study met all university approved and government required ethical standards for the conduct of research and protection of patient privacy. To be included in this study symptoms of oppositional defiant or conduct disorder needed to be the primary basis for referral. These symptoms were formally assessed at diagnostic screening. In addition, the children were required to be between the ages of 6-13 years old. The study included 138 children (39 girls and 99 boys) and their caregivers who completed treatment.
Not all participants who met screening criteria and began treatment, completed the program (see Figure 1). Of the sample that began treatment (N=164), 26 (15.9%) of the families dropped out before completing treatment. The drop-out rates were no different as a function of treatment condition or therapist (χ2 for each, < 1.0, ns) and did not differ from completed subjects on demographic variables or measures at pretest designed to evaluate child and parent functioning (see Kazdin, 2017a for our research on dropouts vs completers). In the present study, dropouts were replaced. That is, new cases were randomly assigned to conditions to achieve a minimal number of participants who completed treatment.
The 138 children included in the study ranged in age from 6 to 13 years (M = 8.59, SD = 1.79); 73.2% of the children were European American, 13.0% were African American, 2.9% were Asian American, 7.2% multiracial, and 2.9% identified as other race, based on parent identification of race and ethnicity. 13% of the children identified as Hispanic. Diagnoses of the children were obtained from the Research Diagnostic Interview (Kazdin, Siegel, & Bass, 1992, a structured parent interview that assesses the presence, absence, and duration of child symptoms using criteria of the Diagnostic and Statistical Manual of Mental Disorders (4th ed. TR, American Psychiatric Association, 2000). Reliability of diagnoses, assessed by independent observers, for approximately 10% of randomly selected children was high (kappa =.95 across all diagnoses).
Principal diagnoses of the children were oppositional defiant disorder (55.1%) and conduct disorder (44.4%). Most children (92.8%) met criteria for more than one disorder (M = 2.96 disorders, SD = 1.18). The primary caretakers of the children included biological (73.2%), adoptive or foster mothers (9.4%), biological (8.0%) or adoptive (3.6) fathers, grandmothers (4.3%) or other relatives (1.4%). iPrimary caregivers ranged in age from 29 to 71 years (M =43.07, SD = 7.31); 23.9% of the children came from single-parent families. Median monthly household income was US $6,001 to 8,000 (range: < $2,000 to > $16,000); 3.0% of the families received social assistance.
AssessmentThe overall goals of assessment were to evaluate five domains: treatment outcome, emergent processes during treatment (therapeutic alliance, perceived barriers), evaluation of the treatment manipulation in generating expectancies, treatment integrity and adherence, and treatment acceptability. Assessments included multiple assessment formats (e.g., clinical interviews, parent-report questionnaires) and informants (i.e., parents and therapists). Measures were completed at three points, before treatment began, halfway through treatment, and after treatment ended (pre-, mid-, and posttreatment, respectively). Most parent- and therapist-report measures were collected online using a secure web-based survey program protected for privacy with unique links for each client.
Treatment outcome. To evaluate child outcomes, multiple measures of symptom and adaptive functioning were included. First, to assess a broad range of both internalizing and externalizing symptoms, parents completed the Child Behavior Checklist (CBCL; Achenbach, 1991). The total problem score was evaluated to assess severity of dysfunction across a broad range of symptom domains. The total social competence scale was evaluated to assess participation in activities (e.g., athletics, clubs), social interactions (e.g., number of friends, amount of contact with friends), and academic performance. Multiple forms of reliability and validity of the CBCL have been studied extensively in clinic and non-clinic samples (e.g., Achenbach, 1991). Second, parents completed the Interview for Antisocial Behavior (IAB; Kazdin & Esveldt-Dawson, 1986), a semi-structured parent-report form that measures multiple overt (e.g., fighting) and covert (e.g., lying) antisocial behavior of the child. This measure reflects primary symptoms for which children are referred to the clinic (i.e., conduct problems). Each of the 30 items is rated on a 5-point scale for severity of dysfunction (1 = not a problem at all, 5 = very much a problem) and a 3-point scale for duration (1 = recent or new problem [6 months], 3 = always). Total antisocial behavior is obtained by summing severity and duration scores. Internal consistency, convergent and discriminant validity, and overall construct validity have been supported in multiple studies (e.g., Kazdin & Esveldt-Dawson, 1986; Kazdin et al., 1992). Third, the therapist responsible for the case completed the Child Global Assessment Scale (CGAS; Shaffer et al., 1983) to provide an overall measure of impairment and adaptive functioning. This measure summarizes the child's psychiatric, adaptive, and social functioning in everyday life and consists of a single score between 1 and 100 (with a higher number indicating better functioning). High interrater reliability and concurrent and discriminant validity have been demonstrated for the CGAS (e.g., Bird, Canino, Rubio-Stipec, & Ribera, 1987). Finally, we counted the total number of symptoms present across all diagnoses at pre- and posttreatment. This number represents DSM-IV-TR symptoms for which there was significant impairment in everyday life and in the present sample correlates with (but is distinguishable from) number of diagnoses (r(138) = .76, p < .001).
Therapeutic changes of the parent and the family were assessed by measures that focused on parent depression, perceived stress, parenting practices, and family relationships. First, parents completed the Beck Depression Inventory (BDI; Beck, Steer, & Garbin, 1996) before and at the end of treatment. For each of 21 items, the parent selected one of several statements that best described their experience of a depression symptom; a higher score on the measure indicates more symptoms and greater severity of depression. The psychometric properties of the BDI have been studied extensively (e.g., Dozois, Dobson, & Ahnberg, 1998). Second, parent perceptions of stress were assessed through the Parenting Stress Index (PSI; Abidin, 1995; Lloyd & Abidin, 1985), which was completed before and at the end of treatment. The PSI consists of 120 items, most rated on a 5-point scale, that reflect multiple areas of stress related to the parents’ views of their own functioning. The measure assesses perceived sources of stress, delineates perceived stress from life events, and distinguishes sources of stress from the child (e.g., subscales such as adaptability, demandingness, and child mood) and sources of stress related to the parent functioning (e.g., subscales such as restrictions of role, social isolation, and relations with others). Diverse types of reliability and validity for the PSI have been reported (Abidin, 1995). Finally, a shortened version of the Family Environment Scale (FES; Moos, 1990) was completed by parents before and at the end of treatment. The three subscales of the FES relationship domain (cohesion, expressiveness, and conflict) were used in this study. Parents responded to 27 true-false items that assess quality of interpersonal relationships, support, and family functioning. The FES has been studied extensively, and multiple types of reliability and validity have been demonstrated, as reviewed in the prior citation.
Emergent processes during treatment. To measure the alliance, we administered the Working Alliance Inventory to both parents and therapists. The scale includes 36 items rated on a 7-point scale (1 = never; 7 = always) and yields a total score to reflect a higher quality of the therapeutic alliance (Horvath & Greenberg, 1989). Items focus on the therapist–parent agreement on the tasks and their relevance in therapy; the mutual endorsement of the goals or outcomes of treatment; and the extent to which there is a positive personal attachment, acceptance, and confidence in the relationship. To evaluate perceived barriers to participation in treatment, parents and therapists independently completed the Barriers to Treatment Participation Scale (Kazdin, Holland, Crowley, & Breton, 1997). The scale consists of 44 items rated on a 5-point scale (1 = never a problem, 5 = very often a problem) that relate to stressors and obstacles that interfere with participating in and coming to treatment, treatment demands and issues, perceived relevance of treatment, and parent relationship with the therapist. Prior validation work has shown that parents given similar treatments can perceive the barriers associated with those treatments quite differently (Kazdin, 2000; Kazdin et al., 1997).
Reactions to and evaluation of treatment. We assessed the extent the manipulations designed to distinguish the two treatment conditions and included expectancies and credibility of the treatment. First, we assessed Parent Expectations for Treatment before treatment began and midway through treatment. This scale consists of 17 items, rated on a five-point Likert scale, most from strongly agree to strongly disagree. Questions assessed parents’ belief that the presented treatment was valid, worthwhile and would improve a variety of behavioral difficulties in their child. We used a total score (Cronbach's alpha = .94) to assess total expectations for treatment, a higher score indicating higher expectations for improvements in therapy. Second, parents completed the Credibility/Expectancy Scale. This 6-item measure was adapted from a frequently used measure to evaluate the extent to which a given intervention generates positive expectancies for change (Devilly & Borkovec, 2000). Parents rate items on an anchored scale from one to nine. A summary score is derived where a higher score indicates a higher believed credibility about the likely impact of treatment (Cronbach's alpha =.91). Credibility was assessed at pretreatment assessment and midway through the treatment intervention.
Treatment acceptability. Treatment acceptability refers to the extent to which consumers of treatment (e.g., parents, mental health professionals) view the treatment procedures as reasonable, justified, fair, and palatable (Kazdin, 2000). We evaluated acceptability to ensure that overall the treatments were viewed positively and to explore whether the two versions of treatment differed in acceptability. Parents and therapists evaluated the acceptability of treatment and the progress made in treatment through the Parent Evaluation Inventory (PEI) and the Therapist Evaluation Inventory (TEI), respectively (Kazdin et al., 1992). These measures each include 19 items and assess the extent to which the treatment was viewed positively by the parents. Parents rated their own perception of the treatment, and therapists rated the parents’ perception of treatment acceptability.
Therapists and treatmentTwo therapists (women, European American) participated. Each had a master's degree and training in family treatment or counseling. The therapists had 18 and 23 years of experience administering PMT to families and in training and supervision of other mental health professionals in the treatment. PMT was the intervention provided to all families included in the study. For treatment, the parents are the clients and attend individual sessions with an assigned therapist; they practice and develop several skills and apply these in the home to change child behavior. The focus is on developing concrete behavior changes that emphasize antecedents, behaviors, and consequences. Practice, modeling, role playing, feedback, shaping, and specific behavior-change programs for use outside of the session were used to develop concrete parenting skills (e.g., use of multiple antecedents, ways of crafting behavior, and consequences) (see Kazdin, 2009). We and others have shown that improvement in parenting skills are reflected directly in therapeutic changes of the children (e.g., Eddy & Chamberlain, 2000; Henderson, Rowe, Dakof, Hawes, & Liddle, 2009).
The treatment included a core set of sessions to convey central content areas, themes, and skill areas but were individualized to address child dysfunction at home, at school, and in special family circumstances (e.g., living conditions, custody issues, involvement of extended family members). Treatment lasted for approximately six weekly sessions (weekly basis, administered individually) plus the option of up to two additional session as needed. The treatment manual for the intervention and a review of several controlled trials of the treatment are presented elsewhere (Kazdin, 2009, 2017a).
Parents were randomly assigned to receive the standard treatment, as highlighted previously, or standard treatment with several enhancements. The enhancements, highlighted in Table 1 were designed to increase credibility and expectations for therapeutic change and to improve the effectiveness of treatment. The manipulation included efforts to increase the professional statue of the setting (special certificates and credentials of the therapists in the rooms for the enhanced treatment), more pleasant accoutrements in the session (e.g., fresh flowers in the treatment room), an orientation video by the clinic director touting the favorable outcomes in the past, testimonials, a special binder with materials about the treatment and its effects, a sham ‘delivery’ interruption to the therapist from an important client during a session, a mailed book on PMT written by the clinic director along with a welcome letter, and more attractive and color materials throughout the treatment program (e.g., appointment reminders, handouts, and charts). During treatment, a variety of special expectancy statements, special emails, and or handwritten notes were interspersed that were designed to increase expectations for improvement and bonding between therapist and client.
Enhancements added to the standard PMT.
| Enhancement | When | Brief description |
|---|---|---|
| Intake Intro | Intake | An explanation of the treatment. The therapist introduced the program to the participant as a treatment plan that would produce lasting changes to the family; standard treatment participants were given a more general description. |
| Intake Interview | Intake | Enhanced participants were asked different questions compared to the standard participants. These questions asked about the various strengths of the participant's family, as well as a support question that is revisited in Session 3. |
| Welcome Packet | Intake | Enhanced participants received a Yale-branded binder in which to place their materials for each session. The binder also included a notepad, a YPC magnet, a YPC pen, a sheet detailing different activities to do in New Haven, CT, articles on the effectiveness of the program and a card explaining their therapists’ experience and qualifications. |
| Welcome Video | Intake | A welcome video was played, featuring an introduction by the clinic director and testimonials by parents who previously experienced the therapy. |
| Book Mailing | After Intake | The Everyday Parenting book was mailed to enhanced participants with a welcome letter from the clinic director. |
| Email Appointment Reminders | Before each PMT session | To confirm a participant's next appointment, an email was sent out with the date and time of the appointment as well as various contact information if the appointment needs to be moved or canceled. After the Intake reminder, each appointment reminder was enhanced with a colored YPC logo as well as an italicized expectancy statement reiterating the effectiveness of the treatment. |
| Delivery of Package | PMT1 or PMT2 | An announcement about a fake delivery from a major movie company; another person at the YPC knocks on the door, requesting the therapist's signature for the “nice package”. The therapist left the room for a few minutes after the interruption. |
| Bonding Emails | After PMT2 and PMT4 | Therapists emailed enhanced participants, praising their progress during therapy and encouraging them to continue the program, citing the effectiveness of the treatment. |
| Expectancy Check-In | PMT3 | Therapists congratulated the participant on making it halfway through the program, praising the parent's dedication and hard work as well as citing specific changes in the child's behavior. Therapist assured the participant that the skills learned would be effective in changing behavior in the future. |
| Support Check-In | PMT3 | Therapists followed up about the support question asked during the enhanced Intake. This was brought up again to reinforce the different types of support the participant is getting (e.g., the therapy itself plus an outside source). Questions about the strength of the support and changes to the support were asked. |
| Bonding Notecard | After PMT5 | A handwritten notecard composed by the therapist was sent to the participants, praising them for their hard work and progress. |
| Room Set Up | Throughout | The enhanced room included a colorful poster displaying accolades on one of the walls, bogus awards and a plaque given to the therapist hung on another, and fresh flowers placed on the table. |
| Full-Color Materials | Throughout | Handouts given to the enhanced participants during each session were professionally printed in full-color. Handouts also included expectancy statements from the sessions. These materials were styled differently and in color rather than the standard black and white materials. In addition, point charts printed from Session 2 onward were printed in color. |
| Expectancy Statements | Throughout | A total of thirteen enhanced statements were given over the course of six PMT sessions. Each statement was designed to illustrate the effectiveness of the therapy. For example, in Session 5, before discussing the “shaping” skill, the therapist says, “I think you are really going to like this skill”. Most sessions contained two expectancy statements, except for Session 3 which contains three. |
Note. YPC stands for Yale Parenting Center, a clinical service where the project was conducted.
To maintain the integrity of treatment, (a) therapists followed a treatment manual; (b) materials (checklists, notes, and outlines) were used within each session to foster correct execution of the treatment; (c) postsession documentation summarized what transpired; (d) videotaping of all treatment sessions, some of which were reviewed weekly to provide feedback to the therapists; (e) review of randomly selected sessions by independent observers who rated the integrity of standard and enhanced PMT and (f) weekly case progress and challenge reviews to evaluate the clinical status of parent and child functioning while also maintaining high treatment integrity and adherence to the treatment manual.
In addition, we evaluated the extent to which parents implemented the intervention techniques in the home. To do this, we administered the Treatment Adherence Inventory (TAI) separately to both parents and therapists at the end of treatment. The TAI included 16 items, with each question rated on 5-point scale (1 = almost never, 5 = almost always), and that evaluated how well or how often parents used certain skills. A higher total score indicates a greater level of adherence to the treatment method. Second, we evaluated treatment integrity from the perspective of therapists’ implementation of the treatment. Sessions were videotaped and randomly selected to assess whether in-session enhancements and treatment components were completed as indicated. Across both therapists and all rated sessions, therapist integrity to the treatment method was 99% across 206 (23.5%) of the sessionsiii.
ResultsPreliminary analysesPreliminary analyses using t-and chi-square tests for continuous and categorical variables, respectively, revealed no differences between the two intervention groups in demographic variables including child or parent age, child or parent race or ethnicity, parental marital or employment status, and educational level or family income or in pretreatment clinical characteristics. Also, there were no pretreatment differences among child outcome measures or parent outcome measures, as described previously. There were no therapist differences in relation to the child or parent and family treatment outcome measures, either at pretreatment or at posttreatment. Overall, at the beginning of treatment, no differences on characteristics of the groups that we measured were evident between treatment conditions or therapists.
Parent adherence and treatment integrityThere were no differences between standard and enhanced PMT conditions in parents’ report of adherence (t(136) =1.64, p = .10) or therapist (t(127) = < 1.00) evaluations of parent adherence. Combining both groups, the means on the measures for both parent and therapist versions (M = 67.34 and 64.53, respectively, were above 64, which was the cutoff for good adherence on the measure (4s on all 16 items of the 5-point scale, where a 4 indicates a skill was used ‘most of the time’) indicating good adherence to the PMT procedures characterized both groups. For the enhanced treatment, 97.2% of the cases met criteria for correct execution of enhancements. For the standard treatment, 100% of the cases met criteria for standard treatment, i.e., they did not receive the described enhancements.
Reactions to and evaluation of treatment enhancementsMeasures were included to check on the extent to which the manipulations designed to distinguish the two treatment conditions were reflected in critical facets of the manipulation. Two other measures (expectancy, credibility) were administered to parents before intake and later again at midtreatment (after most manipulations were completed but there was still the opportunity for further change). Separate analyses of covariance for each measure indicated no significant differences between the two treatment groups before intake when those measures were first administered or at midtreatment when they were administered on a second occasion. Over the course of treatment, parents in both conditions (and not differentially) increased significantly in their expectations for change and viewed the treatment as much more credible than they did before the treatment began. At the second assessment, both standard and enhanced PMT expectations and treatment credibility were very high in absolute values on the scale. The data were negatively skewed (bunched toward the high end of the scale) indicating high levels of expectancies and credibility for both treatments.
Therapeutic change of children and parentsChild treatment outcomes. A series of analyses were conducted to determine if children improved over the course of treatment. Within-group t-tests were computed for the entire sample (N =138) to evaluate these changes over time (i.e., from pre- to posttreatment). The means for pretreatment and posttreatment scores on child, parent, and family outcome measures are presented in Table 2 for the sample. The t-tests, also in Table 2, show that children improved over the course of treatment, as reflected in significantly lower total problems (CBCL), improved social competence (CBCL), reduced aggressive and antisocial behavior (IAB), improved overall adaptive functioning (CGAS), and fewer psychiatric symptoms (RDI) (all ps < .001). The magnitude of the improvements (effect size, calculated by d), as noted in Table 2, was large for all but one measure, using Cohen's (1988) criteria for such designations, as noted in the table.
Measures of child, parent, and family treatment outcomes for the entire sample (N = 138).
| Pretreatment | Posttreatment | |||||
|---|---|---|---|---|---|---|
| Measure | M | SD | M | SD | t(138) | d |
| Child Functioning | ||||||
| CBCL total problem score | 66.69 | 6.85 | 57.33 | 10.11 | 15.41* | 2.64 |
| CBCL total social competence | 36.06 | 9.01 | 39.69 | 9.17 | 2.46* | 0.46 |
| IAB total score | 121.05 | 29.90 | 100.84 | 31.06 | 11.37* | 1.95 |
| CGAS | 53.90 | 8.83 | 71.40 | 13.54 | 15.10* | 2.61 |
| RDI total symptoms | 33.75 | 9.23 | 24.96 | 11.36 | 12.05 | 2.07 |
| Parent and family functioning | ||||||
| BDI total score | 8.39 | 7.07 | 5.30 | 5.39 | 6.56* | 1.14 |
| PSI total score | 260.28 | 45.44 | 237.53 | 47.07 | 8.80* | 1.51 |
| FES relationship total score | 8.51 | 4.79 | 10.46 | 2.40 | 3.84* | 0.66 |
Notes. CBCL = Child Behavior Checklist, IAB = Interview for Antisocial Behavior, CGAS = Child Global Assessment Scale, RDI = Research Diagnostic Interview, BDI = Beck Depression Inventory, PSI = Parenting Stress Index, FES = Family Environment Scale. Improvements in most functioning measures are based on reduction of scores (e.g., lower symptoms). However, improvements on the CBCL Total Social Competence, CGAS and FES reflect an increase in scores (e.g., improved functioning, improved relationships). All t-tests are positive to reflect improvements independent of the scaling. Cohen's d is used to estimate magnitude of effect. An arbitrary but widely accepted standard is to consider .20, .50, and .80 as small, medium, and large effect sizes respectively (Cohen, 1988).
Additional analyses evaluated if there were differences in these outcome measures for the two treatment groups (i.e., standard versus enhanced PMT). A series of analyses of covariance (ANCOVAs) were run for each of the child outcome measures, with treatment group as the main factor and pretreatment scores for measures as the covariate. The means for pre- and posttreatment scores on child outcome measures are presented in Table 3 for each treatment group along with F-ratios (for the group effect). As evident in the table, no differences were found at posttreatment between the two treatment conditions. Overall, the consistent finding is improvements for both groups but no support for one version of treatment surpassing the effects of the other.
Measures of child, parent, and family treatment outcomes for each treatment group.
| Standard PMT | Enhanced PMT Contact PMT | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Pretreatment | Posttreatment | Pretreatment | Posttreatment | ||||||
| Measure | M | SD | M | SD | M | SD | M | SD | F |
| Child Functioning | |||||||||
| CBCL total problem score | 66.21 | 7.41 | 57.09 | 10.41 | 67.16 | 6.28 | 57.57 | 9.87 | <1 |
| CBCL total social comp | 35.42 | 8.93 | 39.95 | 8.95 | 36.75 | 9.11 | 39.44 | 9.44 | <1 |
| IAB total score | 121.71 | 30.27 | 99.25 | 29.10 | 120.41 | 29.74 | 102.36 | 32.97 | 1.32 |
| CGAS | 53.99 | 8.74 | 70.98 | 13.73 | 53.83 | 8.86 | 71.80 | 13.45 | <1 |
| RDI total symptoms | 32.31 | 10.28 | 24.19 | 11.95 | 35.16 | 7.90 | 25.71 | 10.79 | <1 |
| Parent and family functioning | |||||||||
| BDI total score | 9.25 | 7.09 | 5.68 | 5.58 | 7.58 | 7.00 | 4.94 | 5.22 | <1 |
| PSI total score | 263.01 | 47.85 | 236.68 | 50.56 | 257.64 | 43.16 | 238.35 | 43.82 | 1.39 |
| FES relationship total score | 7.18 | 5.32 | 10.62 | 2.72 | 9.79 | 3.84 | 10.31 | 2.06 | <1 |
Notes. The F tests reflect results from analyses of covariance on posttreatment scores using pretreatment scores as the covariate. CBCL = Child Behavior Checklist, IAB = Interview for Antisocial Behavior, CGAS = Child Global Assessment Scale, RDI = Research Diagnostic Interview, BDI = Beck Depression Inventory, PSI = Parenting Stress Index, FES = Family Environment Scale.
A critical question is the extent to which the treatment produced clinically important changes. Although there is no standardized way to assess clinical significance in outcome research, one means is to evaluate the extent to which treatments brought child behavior within the nonclinical (normative) range of functioning (Kazdin, 2017b). We drew on data from the well-studied CBCL, mentioned previously, which provides cutoff scores that fall above and below a normative range, as devised from comparisons of clinic and non-referred samples separately for boys and girls (e.g., Achenbach, 1991). We examined whether children placed within the normative range for both the total symptom score and the total social competence score. For these analyses, we excluded children who began treatment in the nonclinical range or borderline range. Of the remaining cases, we examined the proportion in each treatment condition that fell within the normative range at posttreatment. For the total behavior problem scale, there was no difference in the proportion of children who fell within the nonclinical range for standard and enhanced PMT (χ2 < 1.00). Clearly, there was the equivalent of a ceiling effect: 97.6% of children fell within the normative range in the total sample at posttreatment. Similarly, the total social competence scale showed no significant group difference in the proportion of children in who fell within the normative range at posttreatment (χ2 = 1.62, p >.20). Here again, across both conditions 89.4% of the children fell within the normative range at posttreatment. Overall, there were no group differences and, in many ways, differences would be difficult to detect if present. The treatment, provided to each group, whether enhanced or not, appeared to produce effects that placed most of the cases within the normative range.
Parent and family outcomes. We also evaluated outcomes related to parent and family functioning. As with the child outcome data, within-group t-tests were computed for the entire sample (N = 138) to evaluate these measures changes over time (i.e., from pre- to posttreatment) (see Table 2). By the end of treatment, parents reported significantly fewer symptoms of depression, reduced stress levels, and improved relationships among family members (all ps < .001). Effect sizes (d), also in the table, reflect large effects for two of the three parent and family measures.
To evaluate differences in parent and family outcomes based on treatment group, ANCOVAs were conducted, parallel to those reported in the evaluation of child outcomes. In each case, treatment group (standard versus enhanced PMT) was evaluated with pretreatment on a given measure as the covariate. The results (summarized in Table 3) show no differences between the two treatment groups on depression, stress, or family relationships at the end of treatment.
Emergent processes during treatment and acceptabilityUsing 2 x 2 analyses of variance (treatment group x therapist), we evaluated the therapeutic alliance and perceived barriers to treatment. There was no main effects of treatment nor interactions for either the therapeutic alliance or perceived barriers. Overall, the two emergent processes in treatment did not vary as a function of treatment group.
Parents and therapists completed their respective versions of the acceptability measure. Of primary interest was whether standard and enhanced PMT were viewed differently in relation to treatment acceptability. Yet, the two treatment were no different on either parent (t(135) = 1.38. p < .20) or therapist ratings of acceptability (t(< 1.0). The distribution of the scores for both parent and therapist ratings of acceptability were negatively skewed (-.89 and -1.06, respectively), indicating that ratings were “bunched up” toward the high end of the scale. Thus, parents in both groups tended to evaluate the treatment as highly acceptable and as such it would be difficult to discriminate among the treatments (ceiling effect) if, in fact, there were a different reaction in treatment acceptability.
DiscussionChildren and families in both intervention groups received PMT and improved markedly over the course of treatment whether they received standard or enhanced version of the treatment. By the end of treatment, children showed fewer total behavioral problems, fewer and less severe antisocial behaviors, fewer symptoms across a range of psychiatric diagnoses, and improvements in prosocial adaptive functioning at home and at school. The children who began treatment within the clinical range on the CBCL, by the end of treatment, fell within the normative (nonclinical) range on total behavioral problems and total social competence (97.6% and 89.4% of the children, respectively). Those indices, often used to measure clinical significance or importance of the change, convey that the impact of both treatments was large. Apart from child outcomes, significant improvements were also evident in their families: parents reported fewer symptoms of depression and reduced stress and improved family relationships. Parenting practices, the primary focus of training, also improved with both treatments. This was evident in both parent and therapist evaluations of parent adherence to the intervention techniques trained in the sessions.
The main impetus for the study was to test whether PMT with many enhancements would lead to greater therapeutic change than standard PMT. Both groups received the standard treatment we have been investigating for years; additionally, one group received a large set of enhancements that were designed to increase bonding of the parent to the therapist and treatment, increase credibility and professionalism of the treatment and setting, and increase expectations for therapeutic change (Table 1). Despite the variations in procedures, no differences were found between the two treatments on any of the child or parent treatment outcome measures. In addition, no differences were found on emergent processes in treatment (therapeutic alliance, perceived barriers to treatment) or overall reactions to the treatment (treatment acceptability).
No difference findings invariably are subject to many interpretations. The initial interpretation of course is that there in fact is no difference and the results of this study reflect the true state in the world. Yet, there are many other interpretations available that would also lead to a no-difference finding. For example, whether the comparison was statistically powerful enough to detect differences if they existed, whether there were special biases or artifacts such as diffusion of treatment, whether variability related to implementation (sloppiness) introduced error that made any differences difficult to detect, and perhaps most substantively whether the manipulation was implemented in a strong enough way.
There is one explanation that we consider to be plausible to explain no differences between the two treatments in producing therapeutic change. This explanation focuses on the strength of the standard PMT and therapeutic changes resulting from that. Consider these points from the data analyses: large improvements were made from pretreatment to posttreatment on child and parent outcome measures. As we mentioned, most children in both treatment conditions fell within the normative (nonclinical) range of total behavioral problems and social competence at posttreatment. Of course, the marked changes that we measure at the end of treatment are occurring during the treatment. Parents can see these changes over the course of treatment as the intervention focuses on concrete behavioral problems at home and at school. The strength of the standard treatment and the palpable results parents often see early in treatment would contribute to high levels of credibility and expectancies even without special enhancements. These visible changes were perhaps stronger than the many manipulations and enhancements we engineered as a part of the enhanced intervention.
Data from several measures also support the interpretation that the core treatment was sufficiently strong to make demonstration of special enhancements difficult to show. On several measures (e.g., credibility, expectancy, treatment acceptability), the data at the end of treatment were negatively skewed, which means that many scores were bunched up at the very high end of the scale. With cases in both treatment conditions seeing treatments as highly credible, as generating high expectancies for change, and as highly acceptable, finding any differences between treatments would be difficult.
Our study is one of the few that focuses on experimental manipulation of elements that are part of the nonspecific treatment factors that have been accorded great attention in the history of psychotherapy research. Our focus on treatment of children is unique as well. Our no-difference finding has a plausible explanation that we outlined, namely, the equivalent of ceiling effects on many measures. This could readily be due to using a well-studied and developed treatment where additions would be difficult to show made a difference in outcome.
There is a huge literature both in the context of psychological and biological treatments that client beliefs in treatment and its effects make a major difference in treatment outcome. The strength of that requires much further attention because the basic findings have not been harnessed in a way to enhance routine clinical practice. What can be done to enhance treatments that might be integrated into clinical practice and treatment that would improve outcomes and that are not specific to any specific form of psychotherapy? Much of the treatment literature has focused on developing treatment techniques, with much less attention to how existing treatments can be improved and what facets can be modified that might be applicable to many different types of treatment. Improving credibility, expectancies for improvement, and mobilizing the forces within the individual to augment outcomes remain to be exploited in both research and clinical practice. Our one investigation gives cues as to domains that might be evaluated further, namely, a range of enhancements and might stimulate more novel ways to improve treatment. Also, the effect of enhancements might well vary on a wide range of characteristics of the clinical problem, treatment, and the context in which treatment is provided. The area warrants much more attention because of the long history of non-technique factors exerting enormous impact on physical and mental health.
This project was supported by the Blue Guitar Foundation which is gratefully acknowledged here. The authors are grateful to Jacob Ward who assisted with integrity assessment and data related tasks.
- Autonomic dysfunction and exercise intolerance in post-COVID-19 - An as yet underestimated organ system?
- Loneliness among single and coupled individuals in response to the COVID-19 pandemic: A cross-country analysis
- The impact of COVID-19 and socioeconomic status on psychological distress in cancer patients
- Dreaming during a pandemic: Low incorporation of COVID-19-specific themes and lucidity in dreams of psychiatric patients and healthy controls
