Patient, family and clinician preferences for the intensity and implementation of patient and family participation in healthcare design and delivery in psychiatry

Abstract

Background and objectives

Patient and family participation is when the views and experiences of patients and family are sought and taken into account in healthcare. This research identifies patient, family and clinician preferences for the intensity and implementation of patient and family participation in healthcare design and delivery in an adult outpatient psychiatry service.

Methods

A three round Delphi Technique Experiment (DTE) with 31 participants from an outpatient psychiatry service. Patients, family members and clinicians were included.

Results

There was a strong level of agreement reached between patients, family members and clinicians with consensus achieved in 60/65 questions. Patient participation to the point of being involved in the decision making process (87%; n = 27/31) at all times (73%; n = 22/30) was supported. Family participation to the point of being involved in discussions (66%; n = 23/35) as problems arise (67%; n = 21/31) was supported. Participants agreed on the use of patient and family representatives with factors such as ability to dedicate time considered during selection. Personal experience, discussions with wider populations and pre-identified priorities should contribute to representative input. Training was identified as necessary for all stakeholders, including clinicians.

Conclusions

This is the first time that patients, family members and clinicians have all been included in a DTE on this topic. The inclusion of these groups is essential for the development of long-lasting sustainable participation. The focus on preferences for family participation is also novel. Given the role of family members in care and management of chronic illnesses, identification of their role is key.

Introduction

Participation is where people are encouraged to partake in identifying issues, making decisions, designing, developing and implementing policies and delivering services that concern them.¹ This can occur at the patient level (e.g. decisions about care for individual patients), the service level (e.g. decisions about planning services) and the national level (e.g. decisions about national health policy).² Examples include patient or family inclusion in guideline development³ or consultation in setting priorities.⁴

The importance of participation in healthcare has been established globally.² Potential benefits from participation include better health and treatment outcomes,⁵ increased quality of life,⁵ greater self-dignity and esteem^5,6 and more responsive, higher quality services.⁷ However, a mismatch has been identified between rhetoric and action. This is linked with a lack of consensus on the best ways to achieve strong and lasting participation.⁸

Existing research is weak due to heterogeneity in the interventions tested, the patient groups employed and underlying health systems. Interventions tested have included participation through priority setting in primary care services in Canada, participation through an evaluation of mental health services in Ireland⁹ and participation through citizens’ juries investigating e-health in the United Kingdom.¹⁰ There is a divide with research focusing on either participation at the patient¹¹ or national level⁶ with limited focus on participation at the service level. A focus on participation at the service level is of particular importance for psychiatric services where patients have expressed that only those who have experience of mental illness can understand the emotions and consequences which set them apart from the rest of society.¹²

Stemming from the lack of consensus and gap between rhetoric and action the need to focus on defining participation and its components has been acknowledged.¹³ This study focuses on patient and family participation at the service level. The aim is to identify the most appropriate level of intensity and type of intervention to encourage participation in an outpatient psychiatry service through a Delphi Technique Experiment (DTE).

Material and methods

Setting

An adult outpatient psychiatry service in a deprived suburb of Dublin, Ireland. Approximately 2200 patients attend. The service is run by a multidisciplinary team (MDT) including psychiatrists, nurses, psychologists and allied health professionals.

Delphi Technique Experiments

A three-round DTE was conducted with 31 participants from an outpatient psychiatry service.

DTEs are a method of achieving consensus through multiple iterations of a questionnaire completed anonymously by a panel of experts.¹⁴ In each round, participants complete a questionnaire that is returned to the researcher. The researcher removes questions in which stable group consensus has been achieved. The remaining questions are returned to the participants with controlled feedback, including an overview of the answers provided by the panel as a whole, with the option to change their answer or leave it the same¹⁴ (see Supplementary file). This process continues until a stable group consensus is obtained, determined by predefined stopping criteria.

Conventionally, a 15% change or lower between rounds¹⁵ with levels of agreement above 51%¹⁶ are deemed acceptable stopping criteria. The stopping criteria employed for this study were less than 15% change in opinions between rounds and more than 65% of panellists in agreement. Likert scales with four answer categories were employed. For scoring and analysis answer categories were collapsed with ‘1’ and ‘2’ together and ‘3’ and ‘4’ together to give two answer categories.

Sampling

The selection of participants for DTEs depends on the expertise of the panellists in the area being examined.¹⁴ Panellists were experts in the disease and service attended, not patient and family participation, which is the topic of the DTE. Previous research in the same service has illustrated that patient and family participation in healthcare design and delivery does not occur beyond the provision of information.¹⁷ Similar research has also included participants who had not yet taken part in participation.¹⁸

This study included 31 participants: 13 patients, 9 family members, and 9 clinicians. This is reflective of previous research which supports a maximum panel size of 30 as larger panels seldom improve results and can result in low response rates.¹⁹ The breakdown of the sample by stakeholder groups is also within the convention of having 5–10 members in a panel from the same grouping.¹⁹

Patients and family members

Family members were defined as those who attended the service with patients in a caring or supportive capacity. The sampling frame for the patient and family member sample consisted of participants from previous studies in the same research that consented to be invited to take part in future research. Further detail on this sampling frame is provided in a previous publication.¹⁷ A random sample of patients and family members was selected. To allow for refusals oversampling was conducted. Oversampling was based on response rates for a previous study in the same service.¹⁷ Oversampling has been identified as a useful recruitment approach when working with a list of names, particularly within healthcare research.²⁰ This process is illustrated in Fig. 1.

Figure 1.

DTE sample flow chart. * = Numbers for family members are based on estimates from pilot data collection. + = Further information on the collation of the sampling frame is provided in Chapter 5. # = Round three family member sample size increased as 2 Round 2 non-responders took part in Round 3.

Clinicians

All members of the MDT who were in employment within the service for at least three months were invited to take part (see Fig. 1).

Materials

Round 1 questionnaire

The Round 1 questionnaire was a structured questionnaire informed by a literature review, Arnstein’s Ladder of Participation,²¹ the CLEAR Framework²² and interviews with patients (n = 20), family members (n = 20), clinicians (n = 24) and policy leaders (n = 15) from the same services. It is an acceptable and common modification in the DTE process to use a structured questionnaire in Round 1 based upon an extensive literature review or prior qualitative data collection.^3,14,16 A patient and family member representative provided feedback on the questionnaire. A pilot was conducted using a ‘Think Aloud’ methodology²³ with a separate sample of patients (n = 8) and family members (n = 4) and clinicians (n = 3) external to the service.

There were 65 questions in the Round 1 questionnaire. Topics included opinions on participation, intensity and frequency of participation and implementation (see Supplementary file). To ensure all relevant answer options were provided non-standardised questions included an ‘other’ category.

Round 2 and Round 3 questionnaires

Questions that reached a stable group consensus were removed.^15,16 All ‘other’ options were removed. Where an ‘other’ option was suggested by two or more participants this was added in as a new answer option in Round 2. For each question controlled feedback was provided. The answers of the group as a whole was depicted in a histogram, the participant’s answer was circled in red and participants were given the opportunity to change their answer or leave it the same. If a participant chose to remain outside of the group consensus they were asked to explain why (see Supplementary file).

Procedures

Data collection was conducted by the author (LW) between June and August 2016 with a two-week turnaround time between rounds.

Unique identifier numbers (UIN) were employed for anonymity. Questionnaires took ten to fifteen minutes to complete. Informed written consent was obtained. Ethical approval for this research was granted by the St James’ Hospital/Adelaide and Meath Incorporating the National Children’s Hospital Research Ethics Committee, Reference Number 2015-03.

Patients and family members

The research and DTE process was explained over the phone with an invitation to participate. When a patient or family member agreed to participate, they were sent a Round 1 pack in the post. This included an invitation letter, information sheet, consent form, DTE instructions, Round 1 questionnaire and a stamped addressed return envelope. Instructions included a step-by-step guide on how to complete the questionnaire, how the DTE process works, definitions of key words used in the questionnaire and a ‘frequently asked questions’ section. Follow-up phone calls were made 3 days later to ensure the pack had been received and to provide participants with an opportunity to ask questions. A reminder text message was sent, or phone call made to those who had not returned a questionnaire after one week.

Clinicians

A presentation on the research and DTE process was made to the full MDT with an invitation to participate. Clinicians who expressed an interest in taking part were provided with the Round 1 pack. A reminder text message was sent to those who had not returned a questionnaire after one week.

The same procedures were followed for Round 2 and Round 3.

Findings were presented to clinicians who participated and distributed via the post to patient and family participants. No contradictory feedback was provided on the findings.

Analysis

Descriptive statistics with percentages and counts are presented. All analysis was completed using SPSS (version 22; SPSS Inc; Chicago, IL, USA).

Response and attrition rates

The refusal rate was 14% (n = 4) for patients, 12% (n = 2) for family members and 0% (n = 0) for clinicians. The response rate for Round 1 was 72% (n = 39/54), 79% (n = 31/39) for Round 2 and 82% (n = 31/37) for Round 3. The attrition rate between Round 1 and Round 3 was 23% (n = 8) (see Fig. 1).

Results

Demographic description

A total of 42% (n = 13) of the sample were patients, 55% (n = 17) were female and 42% (n = 13) had a level of education of degree or above (see Table 1).

Table 1.

Sample description by DTE round.

Demographic	Round 1		Round 2		Round 3
	N = 39	100%	N = 31	100%	N = 31	100%
Stakeholder
Patient	17	44	14	45	13	42
Family member	11	28	7	23	9	29
Clinician	11	28	10	32	9	29
Missing	–	–	–	–	–	–
Gender
Male	18	46	15	48	14	45
Female	21	54	16	52	17	55
Missing	–	–	–	–	–	–
Age
18–39	14	36	13	42	10	32
40–59	20	51	14	45	16	52
60+	4	10	4	13	4	13
Missing	1	3	–	–	1	3
Education
Junior Cert^a or less	11	28	8	26	10	32
Leaving Cert^a/non degree	12	31	9	29	7	23
Degree or above	15	38	14	45	13	42
Missing	1	3	–	–	1	3
Duration of attendance/employment
≤5 years	16	41	13	42	13	42
6–9 years	5	13	5	16	^b	^b
≥10 years	16	41	12	39	13	42
Missing	2	5	1	3	1	3

– not applicable.

a

State examinations. Junior certificate at aged 15 and leaving certificate at aged 18.

b

Less than 5 participants.

Consensus

A total of 65 questions were asked. Stable consensus was achieved in 50/65 questions in Round 1, 53/65 questions in Round 2 and 60/65 questions in Round 3. The 65% level of agreement for consensus was met in the remaining 5 questions (see Tables 2 and 3).

Table 2.

Preferences for patient participation.

Question	Answer	N (%)
Should patients be involved in planning services?	Yes	38/39 (97.4)
In planning services patients should be…
Told about changes	Yes	37/39 (94.9)
Asked what they think	Yes	37/39 (94.9)
Involved in discussions	Yes	32/39 (82.1)
Have a vote	Yes	27/31 (87.1)
Make decisions on their own	No	30/39 (76.9)
How often should patients be involved?	Always	22/30 (73.3)
How important is it for patients to be involved in the following stages of planning services…
Suggestion of ideas for change	Important	34/39 (87.2)
Before decisions are made	Important	30/35 (85.7)
After decisions are made	Important	30/39 (78.9)
Who should represent patients?	Patient reps	26/31 (83.9)
How should patients be picked for participation?	All interested picked	22/32 (68.8)
Which of the following factors are when picking patients to participate in planning services…
Health	Important	38/39 (97.4)
Diagnosis	Important	32/39 (82.1)
Level of education	Not important	12/19 (63.2)^a
Age	Not important	29/39 (74.4)
Length of service attendance	Important	20/31 (64.5)^a
Frequency of service attendance	Not important	17/31 (54.8)^a
Ability to dedicate time	Important	34/39 (87.2)
Which of the following are important for patient representatives to take into account when providing input to planning services…
Their own experience	Important	35/39 (89.7)
Discussions with other patients	Important	35/39 (89.7)
A list of what is identified as important by patients	Important	37/39 (94.9)
Which of the following are important in training patients for participation…
How the service works	Important	37/39 (94.9)
How changes are made	Important	36/39 (92.3)
How to work with others	Important	37/39 (94.9)
Why patients should be involved	Important	36/39 (92.3)
Communication skills	Important	37/39 (94.9)
Which of the following are important in training clinicians for patient participation…
How to work with others	Important	39/39 (100.0)
Why patients should be involved	Important	37/39 (94.9)
Communication skills	Important	38/39 (97.5)
Decisions need to be justified if different from patient’s opinions	Important	33/39 (84.6)
Patient opinions need to be passed on to those in the decision-making process	Important	39/39 (100.0)
Patients need to be updated on how their opinions were included	Yes	31/39 (79.5)

a

Consensus was not reached in this answer as the level of agreement is below 65%.

Table 3.

Preferences for family participation.

Question	Answer	N (%)
Should family members be involved in planning services?	Yes	33/39 (84.6)
In planning services family members should be…
Told about changes	Yes	32/35 (91.4)
Asked what they think	Yes	32/35 (91.4)
Involved in discussions	Yes	23/35 (65.7)
Have a vote	Yes	19/31 (61.3)^a
Make decisions on their own	No	30/35 (85.7)
How often should family members be involved?	As problems arise	21/31 (67.7)
How important is it for family members to be involved in the following stages of planning services…
Suggestion of ideas for change	Important	31/39 (79.5)
Before decisions are made	Important	30/39 (76.9)
After decisions are made	Important	23/32 (71.9)
Who should represent family members?	Family reps	23/32 (71.9)
How should family members be picked for participation?	All interested picked	24/31 (77.4)
Which of the following factors are important when picking family members to participate in planning services…
Level of education	Not important	21/31 (67.7)
Age	Not important	28/29 (71.8)
Length of service attendance	Not important	21/31 (67.7)
Frequency of service attendance	Important	17/31 (54.8)^a
Ability to dedicate time	Important	33/39 (84.6)
Which of the following are important for family representatives to take into account when providing input to planning services…
Their own experience	Important	30/39 (76.9)
Discussions with other family members	Important	33/39 (79.5)
A list of what is identified as important by family members	Important	36/39 (92.4)
Which of the following are important for training family members for participation…
Mental health illnesses	Important	35/39 (89.7)
How the service works	Important	34/39 (87.2)
How changes are made	Important	34/39 (87.2)
How to work with others	Important	35/39 (89.7)
Why family should be involved	Important	33/39 (84.6)
Communication skills	Important	33/39 (84.6)
Which of the following are important in training clinicians for family member participation…
How to work with others	Important	36/39 (92.3)
Why family members should be involved	Important	35/39 (89.7)
Communication skills	Important	34/39 (87.2)
Decisions need to be justified if different from family member opinions	Important	31/39 (81.5)
Family member opinions need to be passed on to those in the decision-making process	Important	37/39 (94.9)
Family members need to be updated on how their opinions were included	Yes	28/39 (71.8)

a

Consensus was not reached in this answer as the level of agreement is below 65%.

Support for and preferred intensity of participation

The vast majority of participants supported patient participation in healthcare design and delivery (97%; n = 38). The preferred amount of intensity was for patients to be involved to the point of being involved in the decision making process, having a ‘vote’ when changes are being made (87%; n = 27) at all times (73%; n = 22). Family participation was also supported by the majority of participants (85%; n = 33) with a preference for family participation as problems arise to the point of involvement in discussions.

Process

There was agreement that patient representatives should represent patients (84%; n = 26) and family members representatives should represent family members (72%; n = 23). Characteristics identified as important to take into account when selecting representatives included: capacity to dedicate time for all representatives, patient’s health and length of service attendance for patient representatives and frequency of attendance for family representatives.

Strong consensus supported training for patients, family members and clinicians. It was agreed that this should include: how to work with others, communication skills and why patients/family members should be involved. Patients and family members also need training in how the service works and how changes are made.

Patients and family members should be involved at all stages of the decision-making process from suggestions for changes being put forward, to decisions being made and implemented. The individual experience of the representative, conversations with other patients and family members and pre-identified lists of what is important to patients and family members should all be taken into account when representatives are providing input.

There was agreement that it is important for patient and family member opinions to be communicated to those involved in the decision-making process, decisions contrary to patient or family member opinions need to be justified and patients and family members updated on how their opinions were included.

Discussion

This is the first DTE to include patients, family members and clinicians in the identification of the intensity and implementation of participation in healthcare design and delivery. Previous DTEs have included a variety of combinations of groups including: policy leaders, representatives, clinicians, experts, consumers and advocates.^3,18 Patients, family members and clinicians are the three key groups that would be involved in participation at the service level. As such, the inclusion of their preferences is key to ensuring feasible and sustainable participation. This is in line with Medical Research Council (MRC) Guidelines for developing complex health interventions and the Matrix of Service User Participation, which highlights that appropriate participation methods may differ by service and issue.^7,24 It also reflects the triangle of care. This is a holistic methodology that brings together families, service users and professionals to promote safety and recovery for people with mental health issues and to encourage their wellbeing by including and supporting families.⁸

Arnstein’s Ladder of Participation plots the intensity of participation from being provided with information to taking on responsibility with more meaningful participation at the top of the ladder.²¹ Using this ladder there is a preference for patients in a psychiatric service to be involved to the point of being involved in the decision making process, but not taking on full responsibility for decisions. There is a preference for family members to be involved to the point of taking part in discussions. This is reflective of work conducted in England that points towards a preference for a consultative role for patients. Research with members of the public reported a 68% preference for public involvement at the national level but with decision-making responsibility retained by clinicians.¹⁰ Greater levels of participation, such as patients taking responsibility for decisions made, do not seem to be supported. This may be due to a recognised limitation of the DTE process ‘watering down’ preferences.²⁵ It may also be related to the level of participation within these services, which is currently limited to the provision of information.¹⁷

A key concern within participation is tokenism where patients or family members become involved in healthcare design and delivery but they are not listened to in implementation.²⁶ There was strong agreement that patients and family members need to be updated on how their opinions are used and if the final decision differs from patient or family member opinions this needs to be justified. Agreement was also reached that when providing input patient or family member representatives should use their own experience alongside discussions with other patients and pre-identified lists of what is important to patients or family members. Representatives being associated with their peer groups is supported within the literature as this can boost their competency and participation.²⁷

The CLEAR Framework of Factors Driving Public Participation is a process improvement tool to assess encouragement of public participation and recommendations to tackle problems. Part of the argument on which this is based is the idea that people engage depending on the resources that they have access to as well as the support they are provided with.²² High levels of consensus for training to be given to patients, family members and clinicians for participation are supportive of this. Acknowledged barriers in participation include ‘unspoken rules’ of decision making, use of technical language ‘jargon’ and group and power dynamics.²⁸ Irish research in mental health services identified the need for support for participation to include information on how committees work and the structure of decision-making.²⁹ Separately, a review of strategies to involve patients concluded that clinicians needed opportunities to develop their competencies in this area, particularly their communication skills.³⁰ This is supported by this research which agreed that training for all stakeholders for participation needs to include how the service and decision making-process works, how to work with others and communication skills.

This is also the first DTE to focus on the role of family members in healthcare design and delivery. Family participation was favoured through family member representatives to the point of being involved in discussions. Families play a diverse number of roles including providing emotional, social and financial support, monitoring health and well-being, providing professionals with information about the individual as well as advocating for individuals.⁸ As a result it is argued by some in the literature that families should be considered at all stages of care-planning, decision-making and delivery, with the same thought and attention as for patients.⁸ Despite this, data about ‘carers’ contact with health systems is not routinely collected⁸ and previous research on this topic has not focused on the role of family members.³¹

Strengths and limitations

A key strength of this research is the DTE methodology employed, which has been shown to be suitable for addressing real-world issues, such as participation. This methodology overcomes issues associated with group process research. Data collection was conducted anonymously through the post, with controlled feedback reducing the effects of noise that occurs in a group process.¹⁴ The attrition rates of 23% is similar or lower than attrition rates reported for DTEs conducted on similar topics of 25.2%,¹⁸ 41% and 23%.³ This research adds to the existing literature with a focus on participation at the service level, the role of family members and identification of intensity and implementation of participation from patients, family and clinicians together. Guidelines for the reporting of DTE research to ensure all necessary information is provided have been employed.¹⁶

Stable consensus was not achieved for all questions. Four questions remained below the required 65% level of agreement. All are above 50% and most are close to the pre-defined criteria for consensus of 65%. These questions primarily relate to the criteria that should be employed in the selection of those that should represent patients and family members and as such point towards an area requiring further research. However, all questions not achieving consensus has been reported in other similar DTEs.¹⁸ The sample is highly educated (see Table 1). This is due to the inclusion of a cohort of clinicians. This is important to note due to the reported association between a high level of education and support for participation¹⁰ and the potential for the DTE process to mould opinions.¹⁴ Being aware of this the author (LW) took time in each round to discuss the importance of honesty, acceptability of staying outside the consensus and included an opportunity for participants to explain why they felt it was important to stay outside the consensus. In addition, information on further demographic factors of the sample are presented in Table 1 and based on available information are in line with the characteristics of the wider population such as age and gender. However, information on previous experiences, expectations and the impact of stakeholder power differences identified as impacting on opinions on participation were not captured.³²

Conclusion

There is strong support for patient and family participation in healthcare design and delivery within a psychiatric service with patients involved in the decision-making process and family members being involved in discussions. All stakeholders, including clinical, should receive training. DTEs are a successful method of achieving consensus on the level of intensity and implementation of participation with a sample of patients, family members and clinicians.

Further research needs to be conducted to develop and test behaviour change interventions to encourage participation based on the consensus gained from stakeholder groups. Research needs to be expanded to additional service settings to establish if similar patterns of preferences are reflected in other service areas. Services need to assess preferences and implement standardised, effective and meaningful processes and procedures to involve patients and family members in healthcare design and delivery. This needs to be supported by policy to provide necessary essential resources and leadership required for sustainable and effective participation, avoiding tokenism. All stakeholders, clinical as well as service users, should be involved in every stage of this process.

Conflict of interest

The authors declare that there is no conflict of interest.

Funding

This work was funded by the Irish Research Council [EPBPPG/2014/19]; and the Adelaide Health Foundation.

Acknowledgements

The advice and expertise that Dr. Emer Barrett provided for this research is gratefully acknowledged. Thank you to the patient representative Carol Cooney for feedback provided. In particular, we would like to thank all of the participants, patients, family members and clinicians, for their time and input without whom this work would not be possible.