Impact of the NIHSS scale in the Stroke Unit of the Ramón y Cajal University Hospital: A tool to improve the quality of care

Abstract

Introduction

Stroke is a time-dependent pathology which requires a rapid and universal neurological assessment tool. Since October 2021, following training of nursing staff, the NIHSS scale has been implemented in order to increase comparability, concordance and improve communication with the medical team.

Objectives

1. To examine the concordance among scores of the NIHSS scale implemented by nursing staff and medical team in Neurology Service of the Ramón y Cajal Hospital. 2. To describe the subjective perception of using the NIHSS scale from the nursing staff in the Neurology Service of the Ramón y Cajal Hospital.

Method

Cross-sectional observational study that included 100 patients distributed in two periods of three months each in the Stroke Unit of the Ramón y Cajal Hospital. The NIHSS scale scores given by nursing and neurology were collected, as well as the discordant items of each assessment. Statistical analyzes included frequencies, percentages, Cronbach's alpha and the Intraclass Correlation Coefficient. A survey was conducted among the 22 professionals to assess the degree of acceptance.

Results

A high concordance (68% and 70% in each period) was achieved between the nursing and neurology. A total of 94% of professionals reported to get used to the NIHSS scale.

Conclusions

The implementation of the NIHSS scale is a clear opportunity for improvement in the stroke unit, facilitating communication between nursing and neurology teams, due to agreement between scores and increasing quality of medical care.

Resumen

Introducción

El ictus es una patología tiempo-dependiente que precisa de una herramienta de valoración neurológica rápida y universal. Desde octubre de 2021, tras la formación del personal de enfermería, se implantó la escala NIHSS para mayor comparabilidad, mejor concordancia y mejor comunicación con el equipo médico.

Objetivos

1. Estudiar la concordancia entre las puntuaciones otorgadas en la escala NIHSS por el personal de enfermería y el equipo médico del servicio de Neurología del Hospital Ramón y Cajal. 2. Describir la percepción subjetiva respecto al uso de la escala NIHSS entre el personal de enfermería del servicio de Neurología del Hospital Ramón y Cajal.

Método

Estudio observacional transversal que incluyó 100 pacientes distribuidos en dos períodos, cada uno de tres meses, en la Unidad de Ictus del Hospital Ramón y Cajal. Se recogieron las puntuaciones en la escala NIHSS otorgadas por parte de enfermería y de neurología, así como los ítems discordantes de cada escala. El análisis estadístico realizado incluye frecuencias, porcentajes, alfa de Cronbach y el índice coeficiente de correlación intraclase. Se realizó una encuesta propia a los 22 profesionales de enfermería para valorar el grado de aceptación.

Resultados

Se alcanzó una concordancia alta (87% y 90% en cada periodo) entre las puntuaciones de enfermería y neurología. El 94% de los profesionales refieren haberse adaptado a la escala.

Conclusiones

La implantación de la escala NIHSS demuestra que es una clara oportunidad de mejora en la unidad de ictus, facilitando la comunicación entre enfermería y neurología debido a la concordancia entre las puntuaciones y aumentando la calidad asistencial.

Introduction

Stroke is a time-dependent pathology, and therefore requires universal and rapidly applicable assessment tools to standardise neurological assessment¹ on the recommendations of the World Health Organisation.² Stroke scales reflect the heterogeneity of stroke patients and the consequent difficulties in reliably assessing outcome with respect to disability or neurological deficit. Several neurological rating scales exist,³ for example, the Canadian scale⁴ and the National Institute Health Stroke Scale (NIHSS).⁵ These scales consist of scoring various modalities of an individual's neurological function and then adding the scores up to obtain an index of neurological status.³ These scales were developed for several reasons, such as quantifying the deterioration of neurological status⁴ and adjusting the final score to the initial severity of the stroke.⁶ Therefore, for a common neurological assessment of patients it is important to use the same scale among healthcare staff (i.e., medical and nursing staff) within neurology services.

In particular, in the neurology service of the Hospital Ramón y Cajal, the staff made differentiated use of the scales, i.e., the medical staff used the NIHSS scale and the nursing staff the Canadian scale. Although the Canadian scale has traditionally been more widely used, the NIHSS scale is more useful and complete,^4,5 and its clinical use has been implemented both internationally and nationally.^1,7,8 Thus, for proper patient monitoring, the use of the NIHSS scale was implemented among the nursing staff, bringing the neurological assessment tool into line with the medical team. This implementation was carried out in October 2021, after training the nursing staff in the application of the NIHSS9 scale, for greater comparability, concordance and better communication between the medical team and the nursing staff.^10,11

Following the implementation of the use of the scale by the nursing staff, a study was proposed to examine the comparability of the application of the NIHSS scale between the medical team and the nursing staff, and the acceptability of the use of the scale by the nursing staff of the Neurology Department of the Hospital Ramón y Cajal. The starting hypotheses were: a) the concordance of application of the NIHSS scale between medical staff and nursing staff would be high, and b) nursing staff would show a high acceptability to the use of the NIHSS scale.

As a result, the study objectives were to:

• 1

  Study the concordance between the scores given on the NIHSS scale by the nursing staff and the medical team of the Neurology Service of the Hospital Ramón y Cajal.

• 2

  Describe the subjective perception of the use of the NIHSS scale among the nursing staff of the Neurology Department of the Hospital Ramón y Cajal.

Methodology

Study design

A cross-sectional observational study was carried out including 100 patients admitted to the Stroke Unit of the Ramón y Cajal Hospital.¹² For the inclusion of patients, convenience sampling was used (beds 1 and 5 of the unit were selected) and patients were taken consecutively over time with the following inclusion criteria: age >18 years, diagnosis of ischaemic or haemorrhagic stroke (of any location) that met the criteria for admission to the Stroke Unit, simultaneity at the time of assessment of the NIHSS scale between the neurologist and the nursing staff (on the same shift). In addition, exclusion criteria were defined as: diagnosis of a transient ischaemic attack (TIA) or any other non-cerebrovascular pathology as the primary cause. The study was divided into two waves, each lasting three months; the first period was from 1 November 2021 to 1 February 2022, and the second period was from 2 February 2022 to 2 May 2022, and 50 patients were included in each.

Furthermore, to assess the subjective perception of the use of the NIHSS scale, 22 nursing professionals answered a survey.

NIHSS scale

The NIHSS¹³ scale consists of 11 items that allow us to quickly explore cortical functions, superior cranial pairs, motor function, sensitivity, coordination and language.⁸ Thus, it is composed of: 1) assessment of the level of consciousness (also includes response to questions and motor commands), 2) conjugate gaze, 3) visual fields, 4) facial paresis, 5) upper limb paresis, 6) lower limb paresis, 7) limb ataxia, 8) sensitivity, 9) language, 10) dysarthria and 11) extinction-neglect-inattention. Each item is given a score according to whether it is preserved or impaired. This scale scores the severity of the stroke numerically: the higher the score, the greater the severity, with 42 being the maximum score and 0 the minimum, which corresponds to the absence of neurological focality.

Both nursing and neurology staff administered the NIHSS scale to the same patients, and the total NIHSS score (range 0–42) and the individual score for each of the different items given by the nursing and neurology staff were recorded.

Data collection was carried out by members of the research team by recording the NIHSS scale in the patients' electronic medical records (EMR), as this scale is used in the routine examination of patients admitted to the Stroke Unit. Data collection had no impact on treatment or care, and the anonymity and confidentiality of the data obtained was maintained in compliance with Organic Law 3/2018, of 5 December, on the protection of personal data and guarantee of digital rights. The study, with human participants, was reviewed and approved by the Ethics Committee of the Hospital Ramón y Cajal.

The main variable analysed was the concordance between the total NIHSS scores of neurology and nursing, and secondary variables were: the percentage of patients in which there is a valid discordance (between 1−3 points difference) between neurologist and nurse; the number of failures in the assessment of the different items by nurses (and percentage with respect to the total number of assessments), and the percentage of patients in whom concordance could not be analysed due to lack of recording in the clinical history.

Survey of subjective perception of the NIHSS scale use

The subjective perception of the nursing staff was assessed by means of a survey to evaluate the degree of acceptance and the difficulties in carrying out and recording the NIHSS scale. This survey was carried out 10 months after the inclusion of the NIHSS scale to the 22 nursing professionals on the staff of the unit who were present at the time of transition from the Canadian scale to the NIHSS scale with varying experience in the Stroke Unit. The survey was presented in digital format via a QR consisting of 12 closed single-choice questions, 7 of which were dichotomous (yes/no), asking: 1 whether they were present at the training session, 2) whether they considered the NIHSS scale more comprehensive than the Canadian scale, 3) whether they would prefer to use the Canadian scale again, 4) whether they felt they had adapted to the new NIHSS scale, 5) whether they thought they knew how to correctly pass the NIHSS scale, 6) whether they had recorded the NIHSS scale in the patient's electronic medical record, and 7) whether they thought that the NIHSS scale provided greater patient security. Another 4 multiple-choice questions: 1 on which item was it most difficult to score the examination performed, 2) which item was the easiest to score, 3) the reason why the recording was not made in the patient's electronic medical record, and 4) whether there was concordance with the score given by the medical team. Finally, there was a question in a rating scale format to evaluate from 0 to 10 (where 0 is not at all and 10 is excellent) separately the simplicity, speed and usefulness of the scale.

Statistical analysis

The scores collected were categorised into: a) total agreement (same score); b) valid disagreement due to inter-observer variability (between 1 and 3 points difference), and c) severe disagreement (more than three points difference).

Descriptive statistical analysis was performed using IBM SPSS Statistics for Windows, version 29.0, Armonk, NY: IBM Corp. to calculate the frequencies expressed as absolute number and percentages of the total of all variables, as well as the agreement between continuous quantitative variables through Cronbach's alpha (considered a value > .70 acceptable) and the intraclass correlation coefficient index (considering a result < .40 poor, between .41 and .59 sufficient, between .60 and .74 good and >.75 excellent) with a confidence interval of 95%.

Results

A total of 100 patients were included. Of these, 22 had no NIHSS total score data recorded by nursing staff, so the total study sample included 78 patients (median age: 69 years; 41 males and 37 females) with complete NIHSS total score data by both nursing staff and clinicians, as well as individual scores on each of the 11 items.

Fig. 1 summarises the results of neurologist-nurse concordance in both periods. During the first period, between 1 November 2021 and 1 February 2022, the concordance (same score) between neurologist and nursing staff was 31%, with a discordance of between 1−3 points in 56% of cases and a discordance of more than 3 points in 13% of cases. In other words, taking into account that the inter-observer variability of the scale assumes a difference of between 1 and 3 points, in 87% of the cases the agreement was very good. During the following period, between 2 February 2022 and 2 May 2022, 44% of the scales received the same score by neurologist and nursing staff. Discordance between 1−3 points was 46%, while discordance greater than 3 points was 10%. Here again, good agreement was maintained in 90% of the cases. The items that generated the greatest discordance in the first period were “facial paresis”, “upper limb paresis” and “dysarthria”, while in the second period they were “visual fields”, “facial paresis”, “upper limb paresis” and “sensitivity”. Table 1 shows the percentage of failures according to the different items in the two periods.

Figure 1.

Percentage of concordance between scores on the NIHSS scale between nursing and neurology in both periods in a sample of 78 patients in total, 39 in each period.

Table 1.

Percentage of failures in the different items of the NIHSS scale in each of the two periods.

	First period	Second period
Items
1a. Level of awareness	3.17%
1b. Verbal questions	3.17%
1c. Motor commands	11.11%
2. Conjugate gaze	4.76%	9.30%
3. Visual fields	1.58%	11.62%
4. Facial paresis	14.28%	20.93%
5. Upper limb paresis	17.46%	11.62%
6. Lower limb paresis	9.52%	6.97%
7. Ataxia of the limbs	1.58%	6.97%
8. Sensitivity	11.11%	11.62%
9. Language	1.58%	4.65%
10. Dysarthria	12.69%	9.30%
11. Extinction-Negligence-Inattention	7.93%	6.97%

In addition, Table 2 shows the inter-observer agreement. Cronbach's alpha coefficient was .893 in the first period, and .888 in the second period. The interobserver correlation coefficient index was .957 and .941 in the first and second periods, respectively.

Table 2.

Intraclass correlation coefficient index in each of the periods studied.

First period			Second period
ICC	95%CI	P	ICC	95%CI	P
.957	.92–.977	P < .001	.941	.891–.969	P < .001

95% CI: 95% confidence interval.

ICC: Intraclass correlation coefficient.

Survey of subjective perception of NIHSS scale use

With regard to the assessment of the nursing staff in this service, and their degree of satisfaction with this new way of working, 19 of the 22 surveys were collected, i.e. 86% of the surveys submitted. Of these 19 professionals, 26% had professional experience in the Stroke Unit of less than 5 years, 69% between 5 and 15 years, and 5% had more than 15 years of experience in the Stroke Unit.

Ninety-four percent of the professionals reported having adapted to the NIHSS scale, compared to 6% who denied adaptation, although only 60% considered that they knew how to apply the NIHSS scale correctly, compared to 40% who did not. Twenty per cent would have preferred to re-apply the Canadian scale and 80% preferred to apply the NIHSS scale. Regarding the concordance perceived by the nursing staff with respect to the score given by the medical team, 73.3% said that most of the time they agreed, 15.8% said that most of the time they did not, and 10.9% considered that the scores agreed completely.

Regarding the difficulty encountered by the nursing staff in applying the scale, 26.5% reported that the most complicated aspect was the assessment of limb ataxia, 26.5% the exploration of visual fields, and 21.4% the conjugated gaze, followed by sensitivity (5.1%), verbal questions (5.1%), facial paresis (5.1%), lower limb paresis (5.1%) and extinction-negligence (5.1%).

Discussion

The results obtained show an increase in agreement between the scores of the NIHSS scale between the medical team and the nursing staff of the Neurology Department of the Hospital Ramón y Cajal, after the initial trial period, which justifies the importance of implementing a tool that enhances communication in the multidisciplinary team.

There are several neurological assessment scales,³ for example, the Canadian scale⁴ and the National Institute Health Stroke Scale (NIHSS).⁵ The Canadian scale has traditionally been the most widely used, although it is more limited and simpler than the NIHSS^4,5 and only assesses mental functions (level of consciousness, orientation and language) and motor functions (face, arm and leg). However, the NIHSS scale is also not exempt from limitations, such as: a) not adequately assessing strokes in vertebro-basilar territory, as it does not include a detailed assessment of the cranial pairs⁹; b) not including assessment of gait, praxia and agnosias, and c) giving a higher score to left hemispheric strokes because the involvement of cortical functions, such as aphasia, are highly represented in the scale. However, the NIHSS scale has numerous advantages that make it a more useful and comprehensive scale,¹⁰ such as, for example: a) determine stroke severity¹¹ (<5 mild; 5–15 moderate, 16–20 severe, >20 very severe); b) assess further neurological deficits, such as sensation, confrontation visual fields, extinction, dysmetria or ataxia; c) show greater prognostic value¹⁴ (NIHSS < 7 has better recovery); d) predict in-hospital complications; e) indicate the need for revascularisation treatment (NIHSS score 4–25), and f) be related to infarct volume, which provides objective evolution (an increase of 4 points or more determine a neurological impairment). Therefore, the common use of the NIHSS scale by the medical team and nursing staff of the Neurology Service of the Hospital Ramón y Cajal was implemented.

The scores given by the nursing staff and the medical team on the NIHSS scale show that a high percentage of agreement is maintained, although there is some variability in some scores, but less than 3 points on the scale, which could be related to inter-observer variability, and we understand that this does not represent a relevant discordance. However, severe discordance (greater than 3 points on the scale) decreases. Likewise, there is a tendency for nurses to give higher scores, perhaps because they associate greater severity with the symptoms, or because of inexperience, and this decreases in the second period of the study. As a result, periodic certification of the NIHSS by professionals may be advisable.

Regarding study limitations, a non-negligible percentage of the scale was not recorded in the clinical history, justified in most cases by lack of time. Undoubtedly, the workload involved in working with scales is a matter of controversy. Furthermore, with regard to sample size, we were limited to the number of patients attended by a hospital unit. Carrying out analyses with larger sample sizes would be necessary, ideally extending the study to other neurology units. It is also necessary to closely monitor the degree of compliance and recording of the scale, as we have detected professionals who are not sufficiently rigorous when recording them, or do so incompletely.

Conclusions

The existence of a care plan in stroke units with the NIHSS scale improves communication between the medical team and the nursing staff within the stroke unit, thereby detecting any clinical changes or complications early, and allowing for faster action to be taken, consequently increasing the quality of care and patient safety.

The use of the same scale (i.e., NIHSS scale) between neurologists and nurses reduces inter-individual variability and increases the quality of care provided to patients. We have shown that the nursing staff is capable of performing the NIHSS with an acceptable concordance percentage, and that it is necessary to carry out recurrent continuous training courses in the unit in order to provide the nursing staff with competencies.

During the study we were able to confirm that both the reception of the new scale and the implementation of the scale by the nursing staff were generally satisfactory, as it is a reliable and positive tool that provides greater patient information. However, familiarisation with the scale is necessary, as it is much more exhaustive than the Canadian scale.

The success of stroke units lies in the monitoring and protocolisation of stroke patient care. The main purpose of stroke units is to ensure close and continuous patient monitoring, with specific and frequent care and close follow-up by the neurologist.

The implementation of the NIHSS scale is a clear opportunity for improvement in a stroke unit. This implies designing a training plan and a simple strategy to carry it out. It would be beneficial for other stroke units to join the initiative in order to use the same assessment tool, facilitating communication between units and improving the quality of care.

Conflict of interests

The authors have no conflict of interests to declare.

Acknowledgements

Our thanks to Irene Esteban and Juan Luis Chico for their help with the research.