Delirium is frequently underdiagnosed in patients with dementia. The Repérage Actif du Delirium Adapté à la Routine (RADAR) can be adapted to nursing routines for delirium screening. We validated the Spanish RADAR version and determined the best time of day for its administration.
MethodsAll dementia patients admitted to a postacute care centre on one day were independently assessed by nurses using the RADAR at the morning and midday and by geriatricians with the Delirium Diagnostic Tool-Provisional (DDT-Pro) reference standard for delirium and subsyndromal delirium (SSD). We evaluated the test–retest temporal stability of the morning and midday RADAR assessments, the RADAR validity considering these two time points and then, the suitability of the DDT-Pro for diagnostic confirmation.
ResultsOf 34 dementia patients included, 47.1 % had delirium, and 83.3 % had behavioural, mental or neurological disturbances that made diagnostic assessment difficult. The test–retest temporal stability of the RADAR was moderate, which is consistent with the fact that the diagnostic accuracy of the midday assessment for delirium (79.4 %) was better than that of the morning (73.5 %). The screening accuracy when also considering SSD, accounting for either assessment time, was 79.4 %. Several correspondence and correlation analyses support the use of DDT-Pro for confirmation and assessment of delirium severity after RADAR screening.
ConclusionThe RADAR is useful for the screening of delirium and SSD by nurses in dementia patients and midday assessments have greater diagnostic validity than morning assessments. Screened patients need subsequent diagnosis confirmation before starting therapeutic measures.
Delirium is a syndrome with important clinical implications, increasing the risk of cognitive impairment, dementia, institutionalisation and death,1 as well as increasing economic costs during hospitalisation and postdischarge care.2 Moreover, delirium is a highly prevalent entity regardless of the clinical setting, with a particularly high risk for those with a previous diagnosis of dementia.3
Despite the high prevalence and clinical importance of delirium, some reports have found frequencies of undiagnosed or unreported delirium of over 80 %.4–7 On the other hand,3 some clinicians tend to diagnose delirium when they identify symptoms of the cognitive core domain (such as inattention and disorientation) and noncore symptoms (affective and psychotic symptoms), paying less attention to symptoms of the other two core domains (circadian cycle and higher-order thinking).8 This helps to explain why only some delirium cases are diagnosed and why errors are made in the differentiation of the disorder from other diagnoses that also involve cognitive alterations and psychotic or affective symptomatology, such as dementia.
The detection of delirium in postacute care units is particularly challenging for a diversity of reasons. Although many patients in these facilities are elderly and have dementia, both of which are known risk factors for delirium,9 clinicians often focus on resolving the specific medical-surgical reasons leading to admission and prioritise the diagnosis of dementia over that of delirium—the opposite tendency of which happens in acute care settings, where priority is given to ruling out delirium. Moreover, as patients in postacute care settings typically have had health problems for some time and many are referred from hospitals or nursing homes, it is difficult to obtain information regarding the onset and evolution of their mental alterations.10,11 Behavioral and psychological disturbances are also common in patients with dementia, affecting approximately 90 % of people admitted to postacute care centres, further increasing the difficulty in the assessment of possible delirium.12,13 Finally, the ratio of doctors and nurses to patients is lower, making mental status assessment just one aspect of many other clinical and administrative responsibilities.14
Multiple screening instruments have been developed to facilitate delirium identification by healthcare professionals who are not experts in psychopathology.15 Among these instruments, the four-item Confusion Assessment Method (CAM) algorithm is the most widely used; this algorithm has strengths such as rapid administration but has documented limitations such as poor performance when administered by untrained personnel,16 due to the difficulty in determining the presence of its four items (acute change, inattention, disorganised thinking and level of consciousness)17 and low sensitivity despite being administered by experts, especially in patients with dementia,16,18 where disturbances in the tests of attention and organised thinking are frequent, even in the absence of delirium.19
Our team validated two recently developed screening instruments, the 4 A's test (4AT) and the Delirium Diagnostic Tool-Provisional (DDT-Pro), which were administered by geriatricians during routine clinical practice in a postacute health care centre.11 The three items of the DDT-Pro performed well, including in patients with dementia. Conversely, two of the four 4AT items had limitations in relation to the population studied, as in the case of the attention item, which showed poor diagnostic performance, probably due to the complexity and multipathology of our patients, and the acute onset and fluctuating course item, probably due to the difficulties in determining the onset and evolution of symptoms in postacute care patients. Moreover, those two items obtained a very low positivity in patients with delirium, which is not desirable for a screening tool, and have dichotomous 0/4 scoring options making them mathematically less useful. The three items of the DDT-Pro and their graded severity also allow for the diagnosis of subsyndromal delirium (SSD) according to the definition of the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR).20 For these reasons, the DDT-Pro is useful for the diagnosis of delirium in complex populations, including those with baseline dementia admitted to postacute care wards.
Nurses have a central role in observing and recording changes in patients’ mental state. However, they have limited time to perform multiple tasks. Therefore, there is a need for simple screening instruments that are quick to administer and adaptable to nurses’ routine activities. The Repérage Actif du Delirium Adapté à la Routine (RADAR) was designed to be used as a 6th vital sign assessed by nurses during the administration of medication and is based on patient observation. In the validation of the original version of the RADAR, the sensitivity and the specificity were 73 % and 67 % with respect to delirium according to the DSM-IV.21 The Spanish RADAR version has not been validated.
Our aim was to validate the Spanish version of the RADAR for the screening of delirium and SSD in patients with dementia by nursing staff, and to determine the best time of the day for its administration. The instrument was administered in the morning and at midday, allowing to determine its reliability and validity at each of the two time points and when both administrations were considered together.
Materials and methodsThis was a cross-sectional study conducted at the Monterols Postacute Care Centre in Reus (Spain). All the patients who were in the centre at midweek (Wednesday) with baseline dementia were eligible to be included and assessed on the same day.
ProceduresThe study was conducted by the geriatrics, nursing and psychiatry teams of the centre. Three geriatricians administered the DDT-Pro in the morning to the patients assigned to their care (over a period of no more than 4 h), and nursing staff independently administered the RADAR while administering medication and recording vital signs, both in the morning (between 8:00 and 9:00 h) and at midday (12:30–13:30 h). The RADAR assessment was considered positive for delirium if the answer to any of the three questions (items) about the patient's current status was “yes”: 1) at the morning, 2) at the midday, OR 3) at either time (morning or midday). Psychiatrists collected demographic and clinical information and scored the motor items (#7 and #8) of the Delirium Rating Scale-Revised-98 (DRS-R98) to determine the motor subtype of delirium. The assessments made by each team were conducted independently.
Establishing the presence of dementia and comorbiditiesTo determine whether patients had dementia, the centreʼs physicians accessed all available sources, such as electronic health records, neuroimaging results and clinical assessments. The physicians also collected information on the comorbid cause of dementia that led to the need for admission to the unit.
InstrumentsDemographical and clinical characteristics were collected and recorded in a standardised way.
Reference standardThe DDT-Pro was the reference standard for the diagnosis of delirium. This tool is based on phenomenological delirium research that led to the description of its three core domains, enables easy assessment by nonpsychiatric health personnel and provides a continuous score that allows for the detection of delirium, with different cut-off scores for both delirium and SSD. Each item is scored on a Likert scale between zero (maximum severity) and three (no disturbance) and contains anchored instructions for the clinical disturbances corresponding to each level of severity, with nine as the maximum possible score and zero as the worst possible score. Two items are scored by direct patient assessment (comprehension and attention/vigilance), and the third is scored by assessment over the last 12–24 h using any available means (sleep-wake cycle). The DDT-Pro was validated in the same centre of the current study, with a sensitivity of 77.2 % and specificity of 84 % for the recommended cut-off score of ≤6 for delirium and a better sensitivity of 84.8 % for the cut-off of ≤7, recommended for also identifying patients with SSD.11 The tool was also validated in other settings with patients difficult to diagnose, obtaining good validity values: the original study in patients with acquired brain injury (sensitivity 100 % and specificity 94.4 %,)22 in elderly with a high prevalence of dementia from an internal medicine ward (88.0–90.0 % sensitivity and 85.3–81.2 % specificity)23 and elderly patients from different departments of a general hospital (80–84 % and 82.4–94.1 %).24
Instrument to validateThe RADAR (in English, recognising acute delirium as part of your routine) was designed to be used as a 6th vital sign by nurses during medication administration and is based on patient observation. In the validation of the original version in Canada, when the instrument was “positive” at any of four potential 12-h medication administration times (i.e., a “yes” answer for any item without differentiating the administration time: 08:00 h, 12:00 h, 15:00 h OR bedtime), the sensitivity was 73 %, and the specificity was 67 % for screening DSM-IV delirium.21 The RADAR has two original French and English versions.
The original RADAR items in English are “When you gave the patient his or her medication…1) Was the patient drowsy? 2) Did the patient have trouble following your instructions? 3) Were the patientʼs movements slowed?”. We performed a simple translation into Spanish, which was revised and approved by the author and published on the RADAR website (www.philippevoyer.org/outil-radar).
The RADAR was designed with two important intentions,21 to be easily administered by non-expert nurses based on their cross-sectional interaction with the patient and to favour the screening of hypoactive delirium which is often underdiagnosed.25 These features, relevant for delirium screening, deserve otherwise some clarification regarding the instrument: the most non-specific RADAR item addresses current somnolence, which is not the same as sleep-wake circadian alteration (e.g., when the patient is awake in the morning may be somnolent without necessarily having a circadian sleep-wake alteration), and another one targets observed hypoactivity to the detriment of hyperactivity. Item 2 (following instructions) has some specific characteristics of the core delirium symptoms of attention (cognitive domain) and comprehension (higher order thinking). During the initial validation studies, after iterating 45 questions, items specifically measuring agitation or hyperactive delirium were not retained, but all of these cases were positive on item 2.
Other assessments for the sample and delirium characterisationThe Charlson Comorbidity Index determined the severity of medical burden and has prognostic value. A 1, 2, 3 or 6 score is assigned to 19 clinical conditions. The sum of the individual scores determines the total score, where a 0 to 1 value indicates no comorbidity, 2 indicates low comorbidity and ≥3 high comorbidity.26
Regarding the motor items of the DRS-R98, we used Items #7 (agitation) and #8 (retardation), where a positive score for only Item #7 indicated hyperactive delirium, a positive score for only Item #8 indicated hypoactive delirium, and a positive score for both indicated mixed delirium; a score of 0 for both items indicated normoactive delirium.27
Statistical analysisThe data were analysed in SPSS v. 21 and in a spreadsheet. Although continuous variables were not normally distributed, are reported with means ± standard deviations rather than as mean ranges, and were analysed with the Mann–Whitney U test. Discrete variables are reported as frequencies and percentages and analysed with the chi-squared test or Fisherʼs exact test.
Patients' clinical and demographic variables were compared according to the presence or absence of delirium by the DDT-Pro ≤6 cut-off. To verify that the DDT-Pro (reference standard) correctly differentiates delirium from non-delirium and to provide a description of the clinical characteristics and severity of delirium in the sample, we compared the frequency of positive cases regardless their severity (scores 0–2) and of moderate-severe cases (scores 0–1) for each of the three items, as well as the total scores of the tool, between the groups with and without delirium.
We used the morning and midday assessments to evaluate the reliability of the RADAR (test-retest temporal stability to assess the impact of the time of administration), considering that the test result was positive for a “yes” response to any item in the morning (test) compared to a “yes” response to any item in the afternoon (retest). Kappa coefficients >0.4 indicated moderate concordance, >0.6 substantial concordance, and >0.8 almost perfect concordance.
Regarding the validity of RADAR assessment in the morning, at midday OR in combination, we report the diagnostic accuracy for each scenario (percentage of patients correctly classified) for delirium (cut-off score ≤6 of the DDT-Pro) and for delirium and SSD (cut-off score ≤7), as well as sensitivity, specificity, positive and negative predictive values (PPV and NPV) and positive and negative likelihood ratios (LR+ and LR-). All these discriminant measures are presented with 95 % confidence intervals (95 % CIs).
Since the DDT-Pro can be used for the provisional diagnosis of delirium,23 we performed two different analyses of the relationship of the RADAR and its items with the DDT-Pro. First, bivariate analyses of the correspondence of the RADAR items and of the RADAR delirium status with the DDT-Pro delirium diagnosis. Second, the correlation of the RADAR and its items with the total DDT-Pro score was assessed with one-tiled point-biserial correlations. These two types of analyses were performed for the morning and midday RADAR assessments.
Correspondence analysis reports the correspondence/closeness of the chi-squared score distance of a categorical variable (i.e., the RADAR and its individual items with DDT-Pro ≤6 cut-off delirium status), while including the number of dimensions explaining the correspondence (i.e., describing the construct validity of the correspondence, similar to factor analysis), the dimension's inertia, which is a variance measure (the higher the better), and the proportion of inertia explained by each possible response of the variables of interest (i.e., positive/negative RADAR and items), with the mass of each possible response (i.e., relative weight of response frequency). A p-value for chi-squared comparisons is reported for the significance of each bivariate correspondence model.
ResultsThe distribution of patients within the study is shown in Fig. 1. There were 103 potential patients in the centre the day of the study. Twenty-one of them were excluded for various reasons and 48 did not have dementia. Therefore, 34 (41.5 %) patients with dementia were included in the study and 16 (47.1 %) had delirium according to the DDT-Pro ≤6 cut-off.
According to the DRS-R98, five (31.3 %) delirium patients had the hyperactive subtype, four (25 %) the hypoactive, three (18.8 %) the mixed, and four (25 %) were normoactive.
Clinical and demographic characteristicsWe found no differences in any of the clinical or demographic variables analysed between patients with and without delirium (Table 1). Where the dementia type was available, the most common was Alzheimerʼs followed by vascular. Among all participants, 15 (83.3 %) had behavioural, mental or neurological disturbances complicating their mental status assessment.
Clinical and demographic variables of the 34 patients with dementia, differentiated by the presence or absence of delirium according to a cut-off score ≤6 on the DDT-Pro. The results are reported as frequencies (percentages) for discrete variables and means ± standard deviations for continuous variables.
The mean total DDT-Pro score for patients without delirium was 8.72 ± 0.575, and that for delirium patients was 3.69 ± 1.922 (p < 0.001).
Table 2 shows the percentage of positivity for each of the three DDT-Pro items and the presence of moderate to severe impairment in patients with and without delirium, where the prevalence of positivity was significantly greater for comprehension and vigilance in delirium patients. Moderate to severe disturbances, including sleep-wake cycle disturbances, were significantly more common in patients with delirium. Notably, all patients with delirium had moderate to severe disturbances in vigilance.
The frequency of positivity or moderate to severe impairment for each item of the DDT-Pro in 34 patients according to the diagnosis of delirium.
The test-retest kappa reliability of the RADAR was 0.509 (p = 0.002).
The diagnostic accuracy for delirium considering both morning and midday assessments was 76.4 % (95 % CI 58.4–88.6 %), with 12 out of 16 delirium patients being correctly screened. One patient received a score of seven on the DDT-Pro (SSD), and when this patient was included among the cases, the accuracy of the RADAR for SSD or delirium was 79.4 % (95 % CI 61.6–90.7 %), with 13 out of 17 patients correctly screened.
When considering only the morning assessment, the accuracy of the RADAR was 73.5 % for delirium (95 % CI 55.4–86.5 %), with nine patients correctly screened, and 76.47 % (95 % CI 58.4–88.6 %) for SSD or delirium, with 10 patients correctly screened.
The midday assessment correctly screened 79.4 % (95 % CI 61.6–90.7 %) of the patients with delirium (12 correctly screened) and 76.5 % (95 % CI 58.4–88.6 %) of those with SSD or delirium (12 patients correctly screened).
Table 3 shows all validity indicators of the RADAR, for which the sensitivity of the midday assessment was clearly better than that of the morning and similar to that when both assessments were considered. In concordance, the likelihood of a given patient not being delirious when the RADAR is negative (i.e. post-test likelihood for a lower -LR) was higher for the midday.
Validity of the RADAR for the diagnosis of delirium according to the DDT-Pro. Data are reported for each of the two assessments and for both (Total). Values are shown with 95 % confidence intervals in parentheses for the DDT-Pro cut-off scores ≤6 (delirium) and ≤7 (SSD and delirium).
PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio, LR-: negative likelihood ratio.
The bivariate correspondence models of the RADAR variables (positiveness of the instrument and of each individual item) with DDT-Pro delirium were all within a unidimensional construct and significant at p < 0.05 except the item 1 (drowsiness).
The description of all the models is found in Table 4, the best performing midday models are presented first. RADAR item 2 (following instructions) had the highest inertia for the correspondence with DDT-Pro delirium, with a high proportion of explained inertia (76.5 %) when answered as positive. Although significant, the item 3 (slowed movements) had lower inertia than both the item 2 and the whole RADAR. The whole RADAR had the smallest distance to DDT-Pro delirium diagnosis, 0.05 difference between the chi-squared scores at midday, with the highest mass.
Correspondence analysis of the RADAR and its items with the delirium diagnosis according to the reference standard, DDT-Pro ≤6 cutoff. Analyses correspond to both the midday and morning RADAR assessments and all their dimensional solutions were one dimensional. Since the midday performance was the best, it is reported first. The model for item 1 (drowsiness) had the lower inertia and was not significant. The models for the other items and for the whole RADAR were significant, with item 2 (following instructions) and the whole RADAR models having the higher inertias. Positive rather than negative response to any item or for the RADAR explained most of the dimension inertia, with item positivity for item having the highest explained inertia among the midday significant models.
There was a significant correlation pattern for the RADAR items 2 and 3 and for the whole RADAR with the DDT-Pro total score span. Point-biserial coefficients for midday assessment were better: for item 1, −0.266 (p 0.064); for item 2, −0726 (p < 0.001); for item 3, 0.437 (p 0.005), and for the whole RADAR −0.615 (p < 0.001). Those of the morning where: item 1, 0.101 (p 0.286); item 2, −0.629 (p < 0.001); item 3, −0.456 (p 0.003); whole RADAR −0.593 (p < 0.001).
DiscussionWe evaluated the performance of the RADAR which assesses behavioural and interactional disturbances easily identifiable by nursing staff, for delirium and SSD screening in patients from a postacute care centre with well-known diagnostic difficulties, namely, baseline dementia and a high frequency of neuropsychiatric/behavioural symptoms or sequelae of stroke. The RADAR administered by nursing staff as part of their regular patient care identified patients with delirium or SSD with good diagnostic accuracy, although the time of day at which was administered for cross-section status assessment influenced its screening ability.
The assessment of the onset of delirium symptoms in different care settings is a major diagnostic challenge, as there is often no reliable source of information, which prevents the appropriate use of many screening instruments, such as the CAM,28 the 4AT,29 or even the DSM-5-TR criterion B for determining the onset/course of a delirium episode.20 Moreover, in dementia patients with behavioural disturbances, it is difficult to identify the time of delirium onset. Conversely, the study of postacute care patients shows that the assessment of the symptoms of the three delirium core domains allows the delirium diagnosis, even though it was not possible to determine the moment of onset in patients with mild cognitive impairment or dementia.30 In this study, the DDT-Pro differentiated delirium from non-delirium by five score points and the RADAR, which does not ask about symptoms onset or course, also performed well.
Many patients in the current study came from general or psychiatric hospitals or residential facilities (64.7 %), for whom the clinical information contained in the referral reports was often brief and limited to the pathological process motivating the admission, often without specifying symptoms indicative of delirium.11,31 The frequency of comprehension and vigilance alterations (Table 2) differed between patients with and without delirium, even from its mild presentation. Sleep-wake cycle disturbances were more severe in patients with delirium but similar to those in controls when they were mild. The high frequency of sleep disturbances in patients with dementia,32 as in many of this study patients, may explain these findings. On the other hand, moderate-severe disturbances in the sleep-wake cycle of our patients with delirium involved more episodes of daytime sleepiness, awakenings with confusion or severe fragmentation with rapid alteration of the 24-h cycle (DDT-Pro item #3 score 0–1 instructions).
The RADAR had good diagnostic validity, both when considering the results of either of the two assessments (morning or midday) and when analysing each assessment separately. However, the midday assessment had better sensitivity than the morning assessment, which is a key aspect for an instrument such as this, with a greater number of patients correctly screened. Compared to the use of two daily assessments, the use of only a midday assessment would have little reduction in sensitivity and equal specificity, with less burden of nurses. The test-retest reliability between the morning and midday assessments was moderate.
It is possible that the greater nursing workload in the morning (due to the number of medications and clinical or administrative procedures required) and the particularity that at this time patients are expected to be just waking up, hinder the correct administration of the RADAR items and the isolated assessment of sleepiness, which would alter the screening capacity.14 Although it is important to assess the presence of delirium during each patient contact, our results suggest that emphasis should be placed on midday assessments. Such a finding could also be related to the moderate-severe sleep-wake cycle disturbances (i.e., more evident over the course of the day) observed in our postacute care patients with delirium and baseline dementia, which would favour the identification of the disorder at midday assessments.
Future work could determine whether the RADAR administration during the night shift implies difficulties or advantages. Two aspects to consider for the nocturnal administration of the instrument are the proximity to bedtime and the number of nursing staff, which is usually lower than that in the daytime.
A previous study in the same centre indicated that it is important to also consider SSD cases,11 which has similar prognostic implications to delirium33 and whose diagnostic importance is also reflected in the incorporation of specific criteria in the new American Psychiatric Association's DSM-5-TR.20 The RADAR allowed us to identify the only SSD patient in our sample, although this was determined during the morning assessment. A larger sample size would be helpful to establish whether SSD cases are also better identified at a specific time of day.
In contrast to what has been suggested in some previous studies reporting a higher prevalence of hypoactive delirium in patients with baseline cognitive impairment,34 in our sample, there was a greater percentage of patients with hyperactive delirium. The RADAR items facilitate the identification of hypoactive delirium, which may be more easily missed by healthcare staff25; however, although the number of patients did not enable a subgroup analysis, our study suggest good performance of the instrument regardless of the delirium motor subtype.
The correlation and correspondence patterns of the RADAR with the DDT-Pro permit recommending DDT-Pro assessment after delirium screening (Table 4). There is a unidimensional correspondence construct of the RADAR with the true delirium status, and the distance of the RADAR to the DDT-Pro standard warrants diagnosis confirmation with the last, especially when the positive RADAR item is the more distant item 1. Furthermore, there was a correlation of two RADAR items as well the whole RADAR with the DDT-Pro score, indicating that the screening result is in line with the DDT-Pro score span, from no delirium to SSD and mild or severe full-delirium, deserving severity quantification with the DDT-Pro.
The RADAR item most correlated and close/correspondent with the DDT-Pro was the assessment of instruction-following item 2, which is relatively the most straightforward with delirium core domains, cognitive (attention) and higher order thinking.8 The correlation and correspondence of the less specific item 2 (current drowsiness) with the DDT-Pro were nonsignificant.
Since the sleep-wake cycle item of the DDT-Pro assesses the evolution of the circadian core domain during the previous 12–24 h, delirium status can be verified with it at any day or night time. The strategy of using a brief screening instrument followed by a more specific SSD and delirium diagnosis with severity assessment seems highly advisable for the busy postacute care facilities, where a psychiatrist is not always available for diagnostic assessment.
One study limitation was that we did not administer the screening instrument during the night shift. However, in contrast to most studies of delirium clinical instruments, our study considered the time of day when the RADAR was administered. We found that the midday assessment is very important, but assessments at other times of the day or at night would allow us to identify the few patients not detected at that time. Another limitation of this study is the relatively small number of participants, as we selected a group of patients who represent a challenge for delirium diagnosis and for nurses screening. Finally, the RADAR screening instrument is non-specific for delirium, therefore before any therapeutic measure it is utterly necessary to confirm the delirium diagnosis. We provide data on the usefulness of DDT-Pro for delirium diagnosis confirmation and severity assessment.
In conclusion, when administered by nursing staff the RADAR is useful for screening delirium and SSD in patients with dementia admitted to postacute care facilities. The RADAR performance is better when administered at midday rather than in the morning, with little change in screening capacity when the two evaluation times are considered together.
It is recommended that the RADAR be administered during each nursing shift according to the availability of professionals and workload, or with preference at the midday, and that positive cases be assessed using the DDT-Pro or undergo a psychiatric assessment when available before starting treatment. Future studies could inform on the usefulness of the RADAR in other clinical settings.
Philippe Voyer: author of the original version of the RADAR, but no conflicts to declare regarding this study. José Franco: co-owner of the copyright of the DDT-Pro scale but does not charge for non-profit use. All other authors declare no conflicts of interest.
