Assessing the implementation of a comprehensive quality management system for cross-sectoral psycho-oncology in Germany

Abstract

Introduction

Quality management in healthcare is essential for safe, effective, and patient-centered services. Quality management systems (QMS) monitor and improve healthcare quality. Integrating QMS is crucial for optimal quality of care, but previous studies show gaps in integration. This study aims to assess program adherence to a QMS in cross-sectoral psycho-oncological care and to develop strategies for better integration, ultimately improving healthcare quality.

Materials and methods

The study used a utility analysis to assess the program adherence of a cross-sectoral psycho-oncology care program using a 5-point scale. The evaluation process involved breaking down the program into distinct areas, and used key figures and developed indicators to assess adherence. Descriptive statistics were used.

Results

The study conducted a comprehensive assessment of program adherence in a complex care program, analysing 4460 evaluation cases based on 128 quality indicators. The results showed a score of 4.2 out of 5 points (84%), indicating a highly effective implementation of the QMS. Notably, the study observed successful implementation of top-down elements, while encountering more challenges in integrating bottom-up aspects.

Conclusion

The study demonstrates effective implementation of a comprehensive QMS. Successful integration was observed in areas such as care concept, care management, quality assurance, and IT-based documentation, while challenges remain in quality development and indicators. Active leadership involvement, staff training, data collection, and a learning culture are essential for successful implementation. Future research should assess the impact and cost-effectiveness of QMSs and develop tailored approaches to sustain healthcare professionals’ motivation in quality improvement efforts.

Resumen

Introducción

La gestión de la calidad en la asistencia sanitaria es esencial para prestar servicios seguros, eficaces y centrados en el paciente. Los sistemas de gestión de la calidad (SGC) controlan y mejoran la calidad de la asistencia sanitaria. La integración de los SGC es crucial para una calidad óptima de la atención, pero estudios anteriores muestran lagunas en la integración. Este estudio tuvo como objetivo evaluar la adherencia de los programas a un SGC en la atención psicooncológica intersectorial y desarrollar estrategias para una mejor integración, mejorando en última instancia la calidad asistencial.

Materiales y métodos

El estudio utilizó un análisis de utilidad para evaluar la adherencia al planteamiento de un programa de atención psicooncológica intersectorial utilizando una escala de 5 puntos. El proceso de evaluación consistió en desglosar el programa en áreas diferenciadas, y se utilizaron ratios e indicadores desarrollados para evaluar la adherencia. Se utilizaron estadísticas descriptivas.

Resultados

El estudio realizó una evaluación exhaustiva de la adherencia al programa en un programa asistencial complejo, analizando 4.460 casos de evaluación basados en 128 indicadores de calidad. Los resultados mostraron una puntuación de 4,2 sobre 5 puntos (84%), lo que indica una implementación muy eficaz del SGC. En particular, el estudio observó una implementación satisfactoria de los elementos descendentes, mientras que encontró más dificultades en la integración de los aspectos ascendentes.

Conclusiones

El estudio demuestra la implementación eficaz de un SGC exhaustivo. Se observó una integración satisfactoria en áreas como el concepto de atención, la gestión de la atención, la garantía de calidad y la documentación basada en tecnologías informáticas, mientras que sigue habiendo dificultades en el desarrollo y en los indicadores de calidad. La implicación activa de la dirección, la formación del personal, la recogida de datos y una cultura de aprendizaje son esenciales para el éxito de la implementación. Las investigaciones futuras deberán evaluar el impacto y la rentabilidad de los SGC y desarrollar enfoques adaptados para mantener la motivación de los profesionales sanitarios en los esfuerzos de mejora de la calidad.

Introduction

Quality management in healthcare plays a pivotal role in ensuring the delivery of safe, effective, and patient-centered services.^1–4 Quality management systems (QMS) can be defined as a dynamic set of interrelated activities, methods and procedures designed to monitor, regulate and improve the quality of healthcare, such as board-led quality monitoring, the establishment of quality policy documents, formal protocols, and the in-depth analysis of key figures.^5,6

In the context of psycho-oncology, where addressing the psychosocial and emotional needs of cancer patients is of utmost importance and complexity, the effective implementation of quality assurance tools becomes even more challenging.^7,8 Cross-sectoral psycho-oncological services in Germany, encompassing diverse healthcare sectors, necessitate a systematic approach to quality management to ensure seamless care and improved patient outcomes.⁹ The effective implementation of quality assurance tools and QMS is a critical aspect of ensuring optimal patient outcomes and delivering high-quality care.¹⁰ However, studies have indicated that planned quality assurance tools or management systems are often not fully integrated into clinical everyday practice, leading to potential lapses in quality and compromised patient care.^11,12 The absence of complete implementation and presence of inefficient, error-prone processes and systems can have far-reaching consequences, including suboptimal patient experiences, increased healthcare costs, and compromised patient safety.^11,13

‘Adherence’ refers to the degree to which individuals or participants follow or comply with the prescribed components, guidelines, or protocols of a particular program or intervention.¹⁴ In the context of this study, program adherence relates to the extent to which the implemented care program meets pre-defined requirements and how well caregivers adhere to the recommended quality assurance tools. Such evaluations can help identify barriers and facilitators to implementation, allowing for targeted interventions and improvements to ensure comprehensive integration.¹⁵

This study aimed to evaluate the level of program adherence of a QMS within the context of a cross-sectoral psycho-oncological care program. For this purpose, strategies and assessment tools were developed to analyze the comprehensive QMS in clinical workflow. The results of the work are described and evaluated in this paper.

Material and methods

The methodological framework employed in this study involves a utility analysis to assess the internal program adherence of the ‘new form of care integrated, cross-sectoral psycho-oncology (nFC-isPO)’.^16,17 Designed, implemented, and evaluated in Germany between 2017 and 2022 across four care networks (CN), the nFC-isPO program aimed to establish a comprehensive stepped psychosocial and psychotherapeutic care approach for adult cancer patients. The primary objective was to reduce anxiety and depression levels while simultaneously promoting treatment adherence among this patient population. The program's translational nature ensures the provision of high-quality psycho-oncological care, enabling effective and tailored support for cancer patients throughout their journey.⁹ Detailed information for a deeper understanding of the structure of nFC-isPO and its QMS can be found in Kusch et al. (2022)⁹ and Supplementary Material A.1. This approach aims to systematically evaluate the implementation of the program and is underpinned by the principles of fragmentation, de-emotionalization, and objectivization, facilitating rational assessments and evidence-based decision-making.¹⁶ The complexity of cross-sectoral services demands a structured and systematic approach to evaluation. Fragmentation involves breaking down the program into smaller, manageable segments, allowing for in-depth analysis and evaluation of each component. De-emotionalization entails distancing from subjective biases, ensuring an objective assessment of the program's effectiveness. By avoiding emotional involvement, the evaluation remains rooted in factual evidence and impartiality. Objectivization emphasizes the use of measurable and quantifiable indicators to assess program adherence.¹⁶ This enables a rigorous and transparent evaluation process, reducing ambiguity and enhancing reliability. The scoring process employed direct ranking for all evaluation cases, except for the quality indicators (QI), whose significance was assessed using the analytical hierarchy process (AHP) as an inspiration.^16,17 In this context, evaluation cases are understood as specific instances, structures or situations that are studied and analyzed for different purposes and evaluated or compared according to different aspects or criteria related to a particular subject or problem. Evaluation cases were formulated based on developed indicators and quality characteristics derived from relevant requirements and quality aspects of the nFC-isPO. The evaluation cases were organized into eight distinct areas, representing the different evaluation domain. These domain were:

• 1.

  Joint agreements

• 2.

  Concept of care

• 3.

  Care management

• 4.

  Quality management

• 5.

  Quality assurance and development

• 6.

  IT-supported documentation and assistance system

• 7.

  Human resources

• 8.

  Quality indicators

A total of 200 evaluation cases were defined to ensure a comprehensive evaluation of the nFC-isPO program's various aspects. The thematic blocks of evaluation domain were coded with unique identifiers (IDs) to facilitate tracking and organization during the evaluation process.

Each evaluation case was categorized into one of the three quality dimensions according to Donabedian.² To ensure the implementation of recognized psycho-oncological quality standards, evaluation items were linked to eight binding quality criteria based on the ‘Joint Commission on Accreditation of Healthcare Organizations (JCAHO)’ recommendations.¹⁸Supplementary Material A.2 provides comprehensive definitions of terms used throughout the evaluation process, ensuring clarity and consistency in the analysis.

The utility analysis procedure included the following steps to calculate the total value from the weighted sum of individual values:

• •

  Definition of evaluation criteria: Rigorous evaluation criteria were established to measure the various aspects of the nFC-isPO program's quality.

• •

  Weighting of evaluation criteria: Each evaluation criterion was assigned a defined point value, reflecting its relative significance in the overall assessment.

• •

  Summation of factor values: The factor values of individual criteria within each evaluation domain were aggregated to derive a total value, providing a holistic evaluation of that particular dimension.

The evaluation criteria can be seen in Table 1. These criteria provide benchmarks against which the performance of each evaluation case can be measured. A predetermined assignment of specific point value regarding quantities, ratings, and intervals to certain scale values was established prior to the evaluation. Answers were scored on a five-point scale. This standardization ensures consistency and comparability in the assessment process, enhancing the reliability and validity of the results.¹⁶

Table 1.

Definition of the evaluation criteria.

Evaluation criteria	Explanation
ID	Each evaluation item has an individual identifier by category and subcategories.
Evaluation domain	Each evaluation case was assigned to one of the following categories: Joint agreements, care concept, care management, quality management, quality assurance and quality development, IT-supported documentation and assistance system, human resources and quality indicators.
Care networks	Each evaluation case was assigned to one of the four care networks or to the overarching care program.
Quality dimension^2	Each evaluation case was assigned to a quality dimension: structural quality, process quality, outcome quality
Quality criterion^18	Each evaluation case was assigned to quality criterion: accessibility, appropriateness, continuity, effectiveness, efficiency, patient perspective, safety, timeliness
Assessability	Is the evaluation case assessable, e.g., whether there is information or data that can be used as a basis for evaluation: Yes/NoExample: For quality Indicator No. 81021, no data from the structured quality reports of the care networks are available for quarter 2 in 2020. Consequently, this evaluation criterion is to be recorded as ‘not assessable’.
Degree of implementation	Is the evaluation case fully implemented e.g., theoretically developed, implemented in care practice:- 5 points = Fully implemented in care practice (> 80%)- 3 points = Partly implemented in care practice (80–51%).- 1 point = Not implemented in care practice (< 50%).Example: 3/8 (37.5%) quality workshops conducted were evaluated. As a result, the evaluation item is awarded one point (37.5% < 50% = Not implemented).
Quarter	Each evaluation case was assigned to one of the eight corresponding quarter (3 months) in the period from 01/01/2019 to 31/12/2020.
Quarterly value	Capture of the value associated with the evaluation case and quarter on a metric scale level with absolute numbers.Example: In care network 1, 60 patients were enrolled in Q1 2019 for “QI 81001 New enrolment: enrolment in contract according to 140a SGB V”. (Quarterly value = 60)
Trend over time	The change of the evaluation case over time was assessed, if applicable:‘The collected data shows a positive trend over the last 2 years’. Values or data series contain no more than one outlier and/or show a clear “positive trend,” i.e., are continuously improving (e.g., shortening of waiting times).‘The collected data is stable and/or largely meets the targets’. The values or data series contain a maximum of two outliers, but on the whole meet the requirement for the quality aspect.‘The collected data is stable, but has potential for optimization’. The specified values or data series contain several ‘red flag’ outliers or are subject to a ‘negative trend’, i.e., deteriorate continuously but without major outliers (e.g., prolongation of waiting times).‘The collected data is unstable’. The values or data series given are subject to strong fluctuations or contain large outliers that cannot be logically explained.‘No (sufficient) data available’. No or insufficient data or information is available as a basis for evaluation.
Influence of Covid-19	Does the Covid-19 pandemic appear to affect the series or specific values? Yes/No

Statistical analysis

The assessment was conducted through descriptive statistics. Various analyses were performed using IBM SPSS Statistics. Metrics were scored using a ‘red flag’ threshold, where a score below the mean of 3 (rounded mean minus one standard deviation) was considered to indicate potential areas of concern.

Results

In this study, 4460 individual assessment decisions were made based on 128 quality indicators across eight quarters and four care networks, along with 72 quality characteristics with varying degrees of specification (see Supplementary Material A.3).

The majority of evaluation cases were related to quality (91.8%, n = 4096), quality management (3.2%, n = 144), joint agreements (1.4%, n = 64) and quality assurance and development (1.3%, n = 56). The process quality gained the highest distribution of evaluation cases (85.2%, n = 3800), followed by structural quality (11.2%, n = 500), and outcome quality (3.6%, n = 160). Fig. 1 shows that the quality criteria “timeless” was the most prevalent area with 38.5% (n = 1728). “Continuity” of care comprised 27.1% (n = 1216), and appropriateness of care made up 15.7% (n = 704) of the evaluations. Process quality evaluations primarily involved assessment items related to timeliness, safety of the care environment, continuity, and appropriateness of care. On the other hand, evaluation cases focusing on effectiveness and patient perspective were more commonly associated with outcome-oriented evaluations. The majority of structural quality evaluation cases related to access to care, or in some cases no quality criterion could be assigned (see Fig. 1).

Figure 1.

Distribution of quality dimensions and criteria in evaluation cases.

The overall assessment indicated that 99.2% (n = 4423) of the evaluation cases were assessed as at least implemented. Summing the factor values, the assessment of the nFC-isPO program adherence resulted in the following ratings (see Table 2).

Table 2.

Results of the scoring procedure.

Evaluation domain	Assessment
	%	Point value
Joint agreements	86	4.3/5
Concept of care	100	5/5
Care management	100	5/5
Quality assurance	98	4.9/5
Quality development	70	3.5/5
IT-supported documentation and assistance system	100	5/5
Human resources	82	4.1/5
Quality indicators	76	3.8/5
Overall	84	4.2/5

The results shed light on the implementation status and effectiveness of different components.

Joint agreements were mostly in place, with 86% of the necessary contracts being successfully established. However, there were three aspects, including the internal quality report for quality assurance and the external quality report for quality workshops that were implemented in only one care network due to data protection regulations. Additionally, the withdrawal of patients under the § 630 BGB IV Contract for Special Care was only present in one care network.

Regarding the care concept, all its components were developed and implemented successfully across the different care networks.

Care management demonstrated its efficiency, as a well-structured management system for nFC-isPO was established and operational within the organization of the care networks.

The quality assurance achieved implementation rates of up to 98%. Notably, the quality policy of nFC-isPO was effectively communicated and agreed upon by collaborating with all stakeholders. The definition and systematic implementation of clinical pathways further exemplified the commitment to quality management. The program also provided necessary knowledge and competences to relevant roles, fostering efficient quality assurance and development actions. The assessment of 87.5% implementation degree was achieved through 28 out of 32 quality circles and 7 out of 8 quality workshops. Despite the challenges posed by the Covid-19 pandemic, where some events had to be initially canceled, alternative solutions such as telephone conferences and digital platforms were introduced to ensure continuous progress.

The evaluation of quality development aspects revealed that 70% of the requirements were successfully implemented. Notably, 81% of the quality circles underwent systematic evaluation within the network, though 38% of the quality workshops were unable to do so due to the interim digital circulation process. The program's commitment to continuous quality improvement was evident through the meticulous management and implementation of quality workshop-related action plans in the participatory quality development process.

IT-supported documentation and assistance system played a crucial role in the program's success. All IT applications and additional modules, such as the quality management module, were effectively developed and integrated into the organization of the care networks. Regular updates and releases ensured the system's functionality and relevance.

Human resources allocation for nFC-isPO reached 82%, with most positions being appropriately filled according to the contract. However, in one care network, the staffing for isPO-onco-guide was missing, while case managers and isPO-onco-guide were absent in two other care networks. One care network managed to cover both roles with a single person, which was considered with half a point value.

The evaluation of quality indicators using performance measurements demonstrated the comprehensive nature of the program. A structured quality report incorporated 128 quality indicators and key figures. Over eight quarters and four care networks, a total of 4096 evaluation cases were analyzed regarding their development and target achievements (see Fig. 2). On average, the results of the quality indicators achieved a score of 3.75 (SD = 0.7).

Figure 2.

Results of the assessments of quality indicators.

Discussion

The overall results of this study demonstrate a high level of implementation and program compliance of the nFC-isPO QMS. The developed strategy, designed evaluation concept and quality assurance tool were tested on the basis of scientific methods and quality criteria. They are usable and applicable for the evaluation and optimization of a comprehensive QMS for cross-sectoral psycho-oncology care.

Numerous studies have focused on developing or implementing quality management and quality assurance tools in healthcare settings. However, there remains a significant gap in our understanding of the effective implementation and program adherence of such measures.¹⁹ Upon analyzing the results, it is evident that the effective implementation of care concept, care management, quality assurance, and IT-supported documentation and assistance system has taken place. A closer look reveals that these aspects are mainly based on theoretical foundations and that the integration of manuals, documents or concepts into health care practice is a focus of the (project) leadership. While various stakeholders contributed to the implementation of individual concepts or documents by management, providers played a crucial role in the actual execution in clinical practice. To effectively implement a QMS, robust leadership support at various levels within the organization is crucial.^6,20 The introduction of such a system brings about significant changes in established mechanisms and practices, leading to structural modifications and fostering streamlined organizational processes.²¹

In contrast, in the domains of quality development, where only 70% of measures were put into action, and quality indicators (76%), the responsibility for executing the measures fell on the caregivers. Introducing regular local quality circles and inter-institutional quality workshops aimed at implementing the PDC/SA cycle and fostering learning effects for all participants. Therefore, lower implementation degrees can be attributed to missing staff participation. These findings are somewhat surprising given the fact that other research indicate the significance of QMS as supportive structural feature in fostering and enhancing teamwork, work satisfaction and safety climate.^6,21 However, in accordance with the findings of this study, Groene et al. (2013)¹⁹ observed that formal structures and responsibilities were implemented to a greater extent than other aspects, such as patient involvement in designing quality improvement interventions or addressing the needs of chronically ill patients.

Though the overall QMS of this study is well-established, systematic reviews suggest that the consistency with quality improvement interventions is insufficiently researched and effects may be context-dependent.^22,23 The study by Taylor et al. (2014)²² reviewed the Plan-Do-Study-Act (PDSA) method as a quality improvement tool, and only 2.7% of the studies met all the method's characteristics and principles. In contrast, while this study methodologically implemented all aspects of the PDSA cycle, only 50% of participants attended meetings regularly. These finding further support the idea that the success of quality improvement interventions, especially, relies on their compatibility with caregivers’ motivation and work practices to achieve maximum effectiveness in implementation. Comparison with other studies confirms that certain factors positively influence successful quality improvement. Trapper and colleagues discovered that effective quality improvement interventions encompass managing clinical workflow and empowering dedicated staff to coordinate specific aspects of patient care, resulting in consistent positive outcomes.²⁴ Based on the findings from relevant studies, this study applies several drivers for the successful implementation and effectiveness of QMS and quality improvement. These key drivers include active leadership involvement, staff training, empowering practices, data collection and analysis, and cultivating a learning culture.^20,24,25

Active leadership involvement throughout the entire project period, providing guidance, resources, and fostering a culture of quality improvement, is crucial for successful QMS implementation. Staff trainings and empowering practices are essential drivers for QMS adoption among healthcare professionals, enhancing their understanding and motivation to effectively implement QMS principles, and were mandatory for caregivers and project partners.^20,24 Nurturing a learning culture and conducting regular data collection and analysis play vital roles in fostering continuous improvement and enabling evidence-based decision-making. The QMS compounded regular quality circles and workshops, as well as a digital helpdesk for caregivers in case of urgent questions.^20,25

While successful implementation relies on various drivers, challenges can impede their effectiveness. Challenges such as resistance to change, time constraints, and ineffectively implemented tools were also encountered during this study and should be addressed to optimize the benefits of QMS.^20,25 A significant hurdle was the resistance to new initiatives and the complexity of QMS, as healthcare professionals may be hesitant to adopt changes they perceive as burdensome. Overcoming this resistance demands open communication, stakeholder involvement in planning, and ample support during implementation.^20,25 Time constraints and competing tasks were recognized as challenges for healthcare staff in integrating the QMS, potentially leading to incomplete or suboptimal implementation; careful planning, considering staff members’ realistic time commitments, is essential to address this issue.^20,25

The study under discussion has several limitations. One key limitation is the absence of a standardized method for assessing the implementation of QMS for new forms of care in health care in line with the predefined requirements and their impact. However, implementation in the practical setting and methodological selection of a utility analysis enabled a systematic and robust approach to evaluate the implemented QMS with predefined requirements and their application for the nFC-isPO.

Nevertheless, there are avenues for further exploration. The impact of the quality management tools employed in the study warrants more in-depth investigation. Numerous studies have consistently indicated that the implementation of quality-assured interventions yields various positive effects on aspects of healthcare delivery and patient outcomes.^6,21 In this respect, the benefit of the nFC-isPO QMS should be examined. Additionally, to assess the overall effectiveness, it is essential to evaluate the cost-effectiveness of this QMS e.g., using the cost-of-quality concept. This concept takes into account costs associated with error prevention, testing, and both internal and external errors.²⁶ Furthermore, to assess the construct validity, it is crucial to test using alternative approaches, e.g., the ‘Quality Management Systems Index’.^27,28

Even though nFC-isPO has been implemented to a high degree, there is still room for improvement in its implementation, especially with regard to its integration into the daily clinical routine of staff. The recently published study by Lee et al. impressively demonstrates that quality improvement measures can be successfully translated into practice in a minimally invasive way.²⁹ In Lee et al.’s study (2022), a randomized clinical trial was conducted to evaluate the efficacy, feasibility and acceptability of a communication-priming intervention on electronic health records (EHRs) documented goals-of-care discussions in hospitalized patients with serious illnesses. Similar to the quality management in this intervention, Lee et al. successfully developed an automated and IT-based intervention that did not require much more active involvement, but was based on personal patient- and clinician-specific preferences through integration into the given conditions. The QMS presented here also attempted to involve all stakeholders in the continuous process of quality management and improvement with as little effort as possible for the care providers. However, more use could have been made of existing structures, such as automated patient information based on patient needs derived from IT-based documentation.

Conclusion

In conclusion, the study highlights the effective implementation of a comprehensive QMS for cross-sectoral psycho-oncology care in Germany. The findings indicate a high level of adherence to the quality management program, with successful implementation observed in areas like care concept, care management, quality assurance, and IT-supported documentation. However, challenges remain in areas like quality development and quality indicators, which necessitate targeted interventions to improve integration into clinical workflows. The study provides valuable insights into the drivers and challenges of implementing QMS in healthcare and underscores the importance of active leadership involvement, staff training, empowering practices, data collection, and fostering a learning culture for successful implementation. Future research should take into account the assessment of the impact and cost-effectiveness of QMS. Additionally, a tailored approach, based on the needs of key stakeholders and informed by prior assessments identifying facilitating factors for successful implementation, should be developed to sustain the motivation and engagement of healthcare professionals in quality improvement efforts. The findings from this study contribute to the advancement of quality healthcare practices and offer opportunities for continuous improvement in patient care.

Authors’ contributions

Conceptualization, LD and CL.; methodology, LD and CL; validation, LD, CL and MK; formal analysis, LD; investigation; LD; writing—original draft preparation, LD; writing—review and editing, CL; supervision, CL and SSt; project administration, LD; funding acquisition, SST and MK. All the authors have read and agreed to the published version of the manuscript.

Ethics approval and consent to participate

The study was registered in the German Clinical Trials Register (DRKS) (DRKS-ID: DRKS00021515) on 3rd September 2020. The main project was registered on 30th October 2018 (DRKS-ID: DRKS00015326). The project was approved by the Ethics Commission of the Faculty of Medicine of Cologne University (18–092) on 15th October 2018.

Consent for publication

Not applicable.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary files.

Funding

This work was supported by the Innovation Fund of the Joint Federal Committee of Germany [grant number: 01NVF17022].

Conflict of interest

The authors have no competing interests.