Clinical utility of new cut-off scores for the world health organization ADHD self-report scale among adolescents in psychiatric outpatient care

Abstract

Previous analyses of the proposed cut-off score for the 18-item World Health Organization ADHD Self-Report Scale (ASRS-18) among psychiatrically referred adolescents have shown limited clinical utility. This prospective study examined the diagnostic accuracy and clinical utility of new cut-off scores of the ASRS-18 in a consecutive sample of 111 Swedish adolescent psychiatric outpatients. Using the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS) as the reference standard and based on predefined sensitivity and specificity criteria, multiple new general and sex-specific cut-off scores were evaluated. Results showed that clinical utility was greater for sex-specific cut-off scores than for general cut-off scores. The greatest change in pre-test to post-test probability of ADHD diagnosis was observed with a balanced, high-specificity diagnostic cut-off score for girls, where the probability of ADHD increased from 40% pre-test to 82% post-test. The proposed new cut-off levels for the ASRS-18 are useful for the detection and identification of ADHD among adolescents in general psychiatric outpatient settings.

Introduction

Attention-deficit hyperactivity disorder (ADHD) has a lifetime prevalence of 4–11% among young people with high persistence rates into adulthood (Danielson, Visser, Chronis-Tuscano & DuPaul, 2018; Polanczyk, Willcutt, Salum, Kieling & Rohde, 2014; Sayal, Prasad, Daley, Ford & Coghill, 2018; Shaw & Sudre, 2021; van Lieshout et al., 2016). When left untreated, ADHD is associated with multiple negative outcomes, including educational underachievement, impaired social interactions, risky behaviours, poor physical health, and increased risk for suicide, addiction, antisocial behaviours and injuries (Becker, Luebbe & Langberg, 2012; Flory, Molina, Pelham, Gnagy & Smith, 2006; Howard et al., 2019; Impey & Heun, 2012; Mohr-Jensen, Muller Bisgaard, Boldsen & Steinhausen, 2019; Pham, 2016; Prasad, West, Sayal & Kendrick, 2018; Pulgaron, 2013; Shaw et al., 2012; Thompson, Molina, Pelham & Gnagy, 2007). Accordingly, early identification and treatment are important in the prognosis of affected youth. However, among adolescents, recognizing ADHD can be challenging. One reason for this is that symptoms of ADHD may be more subtle in adolescents compared to younger children, making it more difficult to detect the disorder. Another reason is that adolescents with ADHD are more likely to have co-occurring mental health disorders which can further complicate identification (Sultan, Liu, Hacker & Olfson, 2021). Identification of ADHD in adolescent girls can be even more difficult, due to a typical symptom presentation of inattention and co-occurring internalizing symptoms rather than hyperactivity and impulsivity and use of compensatory strategies that may disguise ADHD symptoms (Hinshaw, Nguyen, O'Grady & Rosenthal, 2022; Young et al., 2020). Hence, girls with ADHD are less likely to be disruptive and their symptoms can be overlooked by teachers and other informants (Coles, Slavec, Bernstein & Baroni, 2012).

Self-rating scales are a time- and cost-efficient method to collect information on symptoms of ADHD and can be helpful in the detection and identification of impairing levels of inattention and hyperactivity in adolescents. However, to be useful in clinical settings self-report scales must have sound psychometric properties established in settings that allow for generalization to equivalent clinical populations.

One self-report scale commonly used to assess ADHD is the 18-item World Health Organization Adult ADHD Self-Report Scale (ASRS-18) (Kessler et al., 2005, 2007). The ASRS-18 is a Likert-type self-report questionnaire designed to identify symptoms of ADHD. Respondents indicate the frequency of occurrence of 9 attention-deficit symptoms and 9 hyperactivity symptoms during the last 6 months on a 5-point scale, ranging from “never” to “very often”. Kessler and colleagues suggested a summation of scores based on dichotomized item-ratings, yielding a maximum score of 18, and then use a cut-off score of 9 to identify elevated levels of ADHD symptoms. However, alternative scoring methods yielding different total scores for the ASRS-18 have been used in a range of studies based on either adult psychiatric samples or adolescent community samples (Corbisiero, Hartmann-Schorro, Riecher-Rössler & Stieglitz, 2017; Hagen et al., 2017; Kim, Lee & Joung, 2013; Somma, Borroni & Fossati, 2019; van de Glind et al., 2013).

Originally developed for the assessment of ADHD symptoms in adults, the ASRS-18 has been increasingly used among adolescents. Reports from a limited number of evaluations of the ASRS-18 among adolescents have shown adequate psychometric properties in community samples (Adler et al., 2012; Green et al., 2019; Somma et al., 2019). Only one study has examined the performance of the ASRS-18 among adolescent psychiatric outpatients (Sonnby et al., 2015). In their study, Sonnby and colleagues analysed diagnostic accuracy and post-test probability of ADHD based on the dichotomous summation system and the proposed cut-off of 9 (Kessler et al., 2005). The results showed 79% sensitivity (girls 83%; boys 75%) and 60% specificity (girls 60%; boys 59%). However, the cut-off score of 9 showed limited clinical utility because results at or above this score yielded a post-test probability of an ADHD diagnosis of about 50%, leaving clinicians with a high level of uncertainty about the presence of a diagnosis after ASRS-18 administration among adolescent psychiatric outpatients (Sonnby et al., 2015). Furthermore, the ASRS-18 showed better psychometric properties among girls than boys.

The present study provides follow-up analyses on the clinical sample in Sonnby et al.´s study. The overall aim was to determine new, tailored cut-off scores for the ASRS-18 for ADHD assessment purposes among psychiatrically referred adolescent boys and girls. The specific aims were to (a) examine the reliability and known-groups validity of the ASRS-18 and (b) evaluate the diagnostic accuracy and clinical utility of new general and sex-specific cut-off scores.

Method

Participants and setting

This prospective study was carried out in two child and adolescent psychiatry (CAP) outpatient units in the county of Västmanland, Sweden. Data collection was conducted during 63 predefined recruitment weeks between August 2011 and June 2013. During the study period, 37,494 children and adolescents were living in urban, sub-urban and rural regions within the CAP units’ catchment area. Patients were eligible for study inclusion if they met the age criteria (13–17 years at the time of giving consent to participate), irrespective of reasons for help-seeking. Exclusion criteria were intellectual disability and inadequate Swedish-speaking skills.

During the study period, 202 consecutive patients were eligible for inclusion. Of these, 77 were excluded: 28 declined to participate, 45 were missed as eligible and four did not return after their first appointment. Of the remaining 125 patients, 14 were excluded from analysis because information was missing from their ASRS-18 (Fig. 1). There were no significant differences in sex, age, or proportions of ADHD between those excluded because of missing data and the remaining participants.

Fig. 1.

Flow Chart of Participant Recruitment and Exclusion.

Note. ASRS-18 = The World Health Organization ADHD Self-Report Scale, 18 items; K-SADS = Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children

The study was approved by the Ethical Review Board in Uppsala, Sweden, in concordance with the Declaration of Helsinki. Written informed consent to participate was obtained at each participant's first CAP appointment.

Measures

The World Health Organization ADHD Self-Report Scale for Adolescents (ASRS-18) (Adler et al., 2012). The ASRS-18 is an 18-item, Likert-type (0 = ‘never’, 4 = ‘always’) questionnaire designed to assess symptoms of ADHD in adolescents. The questionnaire is designed in accordance with DSM-IV diagnostic criteria for ADHD and explores how often symptoms occurred in the last six months. In the current study, a total score of the ASRS-18 was computed by summing up individual participant scores on the 18 items, yielding a total maximum score of 72. The study used the Swedish version of the ASRS-18, with minor adaption of the wording to suit adolescents (Sonnby et al., 2015).

Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime Version 2009 (K-SADS)( Ambrosini, 2000; Kaufman et al., 1997). The K-SADS is a semi-structured diagnostic interview designed to guide clinicians in the assessment of psychiatric disorders in children and adolescents aged 6–18 years. The K-SADS consists of an introductory interview, a screening interview and eight diagnostic supplements covering affective disorders, psychotic disorders, anxiety disorders, behavioural disorders, substance use disorders, eating disorders, tic disorders and autism spectrum disorders. Clinicians score the symptom severity and determine diagnoses based on all available information. Widely used as a diagnostic tool in child and adolescent mental health research, it has been shown that psychiatric diagnoses based on the K-SADS interview conducted at a single visit have excellent validity (Ambrosini, 2000; Fitzpatrick et al., 2012; Jarbin, Andersson, Rastam & Ivarsson, 2017).

Procedure

The ASRS-18 was administered to adolescents as part of a regular assessment procedure at the CAP units. Diagnostic assessment was conducted by clinicians who used the K-SADS interview to gather information on history of psychiatric symptoms and age of onset, current symptoms, and functional impairment in school, at home, with friends and during leisure-time. Clinicians were unaware of the results of the ASRS-18. Adolescents and parents were interviewed together. Diagnoses were based on DSM-IV criteria using information gathered during the diagnostic interview. The K-SADS interview was conducted within 14 days of ASRS-18 completion (M = 3.65, SD = 3.62), with interviewers unaware of ASRS-18 results. The ASRS-18 was re-administered to 14 adolescents (12.6%) directly after the interview because the time frame of 14 days was violated. These reassessments replaced initial assessments in all analyses.

Interviewer training and inter-rater reliability

All interviewers were experienced CAP clinicians with extensive K-SADS interview training before data collection. A free-marginal multi-rater kappa (multi-rater κfree) (Brennan & Prediger, 1981; Randolph, 2005) was chosen to calculate inter-rater reliability (IRR). IRR was calculated before data collection, based on interviewers’ ratings on videotaped model interviews, and showed adequate agreement: multi-rater κfree = 0.88, percent overall agreement (Po) = 0.94. Average multi-rater κfree and Po reliability during data collection was multi-rater κfree = 0.94 (Po = 0.97) for all diagnoses, and multi-rater κfree =0.73 (Po = 0.87) for the composite diagnostic category of any ADHD.

Statistical analysis

Statistical analyses were conducted using IBM Statistical Package for the Social Sciences (SPSS, version 26). A p-value < 0.05 was considered significant. To explore differences between groups for continuous and ordinal data, independent t tests were performed and validated using the non-parametric Mann–Whitney U test. A chi-squared test was used for dichotomous and categorical variables. The internal consistency of the ASRS-18 total score was calculated with Cronbach's alpha.

The diagnoses of ADHD, ADD, and ADHD not otherwise specified were combined into one category: any ADHD. Receiver operating characteristic (ROC) analyses were conducted to examine the ability of the ASRS-18 total score to predict the presence or absence of a diagnosis of any ADHD in the sample. The strength of the prediction is indicated by the area under the curve (AUC), which ranges from 1.0 (perfect association) to 0.5 (no association) (Hanley & McNeil, 1982). Tailored cut-off scores both for the total sample and specifically for boys and girls were determined using the coordinates of the ROC curve plot (sensitivity vs. 1-specificity) with the K-SADS as the reference standard. Four cut-off scores were identified for each of three groups: the total sample, and for boys and girls separately. The following criteria were used to identify cut-off scores: the screening cut off represents the highest score with a minimum sensitivity of 0.90; the diagnostic cut off represents the lowest score with a minimum specificity of 0.90; the balanced screening cut off represents the score that maximizes sensitivity when specificity is set at minimum 0.50; the balanced diagnostic cut off represents the score that maximizes specificity when sensitivity is set at minimum 0.50.

Dichotomized variables were used for calculating sensitivity, specificity, and accuracy (i.e., proportion of participants correctly classified as either having or not having a diagnosis of ADHD). Post-test probabilities for different cut-off scores were calculated using the methods of positive (PPV) and negative (NPV) predictive values and likelihood ratios (LRs). LRs range from 0 to infinity and indicate how much a test result will alter the pre-test probability of diagnosis. LRs above 1 indicate the presence of a diagnosis, whereas LRs below 1 indicate the absence of a diagnosis. LRs equal to 1 lack diagnostic information (McGee, 2002). Results are reported in accordance with the recommendations of the Standards for Reporting of Diagnostic Accuracy (Bossuyt et al., 2015)

Results

Description of sample

The study group consisted of 111 psychiatrically referred adolescents (67 girls, 60.4%) aged 12–18 years (M = 15.7, SD = 1.5), predominantly ethnic Swedes (98%) and their parents. The most common causes for help-seeking were symptoms of depression (28.8%), ADHD (27.9%) and anxiety (20.7%). According to the K-SADS, the number of diagnoses per adolescent ranged from 0 to 8 (median = 2, interquartile range = 1–3). The majority of adolescents, 70.3%, had at least one comorbid disorder. Nine (8.1%) had no diagnoses. Fifty-seven (51.4%) adolescents (40.3% of all girls, 68.2% of all boys) were diagnosed with ADHD. Several group differences were observed. First, there was a higher proportion of boys among adolescents with ADHD than among adolescents without ADHD. Second, when girls and boys were analysed separately, mean age differences were observed among girls only; girls with ADHD had a lower mean age than girls without ADHD. Third, among adolescents diagnosed with at least one psychiatric disorder, differences in the mean number of disorders were observed among girls only, with more concurrent disorders among girls with ADHD than girls without ADHD.

In the total sample, the mean ASRS-18 score was 34.3 points (median = 36.0, SD = 13.5) with no statistically significant sex differences (girls: median = 34.6, SD = 14.4; boys: median = 33.8, SD = 12.3) (not shown in tables). There were no differences in total mean scores between those who completed the ASRS-18 before the K-SADS and those who were re-administered the questionnaire after the K-SADS. Table 1 shows socio-demographic and diagnostic characteristics of girls and boys with and without ADHD.

Table 1.

Socio-demographic and Clinical Characteristics Among Adolescent Psychiatric Outpatients by ADHD Status and Sex.

	All		p-value	Girls		p-value		Boys				p-value
				ADHD	Non-ADHD			ADHD		Non-ADHD
	ADHD	Non-ADHD		n = 27	n = 40			n = 30		n = 14
	n = 57	n = 54
Sex
Boys, n (%)	30 (52.6)	14 (25.9)	.006
Age, mean (SD)	15.5 (1.5)	15.9 (1.4)	.122	15.4 (1.4)	16.2 (1.3)	.017		15.6 (1.6)			15.1 (1.5)	.335
Prevalence of ADHD by type, n (%)
Any ADHD disorder	57 (100)			27 (100)				30 (100)
ADHD, combined type	23 (40.4)			13 (48.1)				10 (33.3)
ADHD, inattentive	25 (43.9)			10 (37.0)				15 (50)
ADHD, hyperactive-impulsive	1 (1.8)			1 (3.7)				0
ADHD, not otherwise specified	8 (14.0)			3 (11.1)				5 (16.7)
Number of disorders, n (%)^a			.275^b			.211^b						.441^b
1	11 (19.3)	13 (28.9)		4 (14.8)	9 (24.3)			7 (23.3)			4 (50.0)
2	22 (38.6)	14 (31.1)		8 (29.6)	12 (32.4)			14 (46.7)			2 (25.0)
3	6 (10.5)	11 (24.4)		2 (7.4)	9 (24.3)			4 (13.3)			2 (25.0)
4	6 (10.5)	3 (6.7)		4 (14.8)	3 (8.1)			2 (6.7)			0
≥5	12 (21.1)	4 (8.8)		9 (33.3)	4 (10.8)			3 (10.0)			0
Number of disorders, mean (SD)^a	2.9 (1.7)	2.4 (1.3)	.125	3.5 (1.9)	2.5 (1.4)	.038		2.3 (1.2)			1.8 (0.9)	.213
ASRS-18, mean (SD)	40.2 (12.5)	28.1(11.7)	<0.001	43.6(13.8)	28.5(11.3)	<0.001		37.0(10.5)			26.9(13.4)	.009

a

Estimates among adolescents diagnosed with at least one psychiatric disorder, n = 102.

b

Fischer's exact test, two-tailed.

Note: K-SADS = Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children; ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items.

Reliability and known-group validity

High internal consistency was observed for the ASRS-18, both in the total sample (Cronbach's α = 0.91) and separately among girls (α = 0.92) and boys (α = 0.89). Known-group validity was examined by analyses of between-group differences in mean ASRS-18 scores. In the total sample, ASRS-18 scores of adolescents with a diagnosis of ADHD were significantly higher, with a large effect size as determined by Cohen's d (Cohen, 1988), compared with the ASRS-18 scores of adolescents without ADHD (d = 0.99). Similarly, analyses of known-group validity by sex showed that the ASRS-18 scores of girls and boys with a diagnosis of ADHD were significantly higher, with large effect sizes, than the ASRS-18 scores of girls and boys without ADHD (d = 1.2 and d = 0.84 for girls and boys, respectively). Means and standard deviations for the ASRS-18 by ADHD status and sex are presented in Table 1.

Diagnostic accuracy and clinical utility of new general and sex-specific cut-off scores

ROC analyses were used to examine the overall ability of the ASRS-18 to predict a diagnosis of any type of ADHD as determined using the K-SADS. The predictive ability, measured as AUC, fell in the fair to good range both in the total sample (AUC = 0.74, 95% CI = 0.65–0.83, p < .001) and in separate analyses for girls (AUC = 0.80, 95% CI = 0.69–0.91, p < .001) and boys (AUC = 0.71, 95% CI = 0.53–0.89, p = .027) (Fig. 2). There were no significant differences between the AUCs of girls and boys (AUC difference = 0.10, Z = 0.88, p = .378).

Fig. 2.

Receiver Operating Characteristic (ROC) Curves for the ASRS-18 Total Score (Range 0–72 Points) in a Clinical Sample of Adolescent Psychiatric Outpatients (Total Sample: n = 111; Girls: n = 67, Boys: n = 44).

Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items

The clinical utility of multiple new general and sex-specific cut offs, derived from ROC analyses using predefined criteria of sensitivity and specificity, were examined. Overall, the diagnostic cut off for girls (ASRS-18-score = 43; sensitivity = 56%; specificity = 90%) and balanced diagnostic cut off for girls (ASRS-18-score = 44; sensitivity = 52%; specificity = 93%) showed the best performance, yielding a moderate increase in the likelihood of a diagnosis of ADHD. An ASRS-18 score above these cut-off scores raised the probability of diagnosis by 39 and 42 percentage points, respectively, i.e., from 40% to 79% and 82%. In contrast, the corresponding diagnostic cut-off scores for boys lacked clinical value because the 95% confidence interval for the LRs for these cut-offs crossed the value of 1 (diagnostic cut off for boys: ASRS-18-score = 45; sensitivity = 17%; specificity = 93%; balanced diagnostic cut off for boys: ASRS-18-score = 38; sensitivity = 56%; specificity = 90%).

In the total sample, all four new general cut-off scores showed some clinical utility and yielded post-test probabilities above 50%, ranging from 60% to 80%, when ASRS-18 scores fell above these cut offs.

The general and sex-specific high-sensitivity screening cut-off scores representing the highest score with a minimum sensitivity of 90% (total sample: ASRS-18-score = 24, sensitivity = 91%, specificity = 35%; girls: ASRS-18-score = 25, sensitivity = 93%, specificity = 35%; boys: ASRS-18-score = 24, sensitivity = 90%, specificity = 43%) all lowered the probability of diagnosis to a similar degree (∼30 percentage points) when ASRS-18 results fell below these cut offs.

The accuracy of the general cut-off scores was fairly consistent at just above 60%, whereas accuracies for the sex-specific cut offs were more varied. The highest accuracy was observed for the balanced diagnostic cut-off score for girls (76%) and the two screening cut-off scores for boys (75%). Estimates of diagnostic accuracy and clinical utility for the new cut-off scores are presented in Tables 2 and 3.

Table 2.

Diagnostic Accuracy of Four New Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.

ASRS-18	Base rate%	AUC (95% CI)	Sens.%	Spec.%	Accuracy%	PPV%	NPV%	LR+ (95% CI)	LR– (95% CI)
Total sample (n = 111)	51.35	0.75^⁎⁎⁎(0.66–0.84)
Screening cut off (≥24 points)^a			91.23	35.19	63.96	59.77	79.17	1.41 (1.14–1.74)	0.25 (0.10–0.62)
Diagnostic cut off (≥44 points)^b			35.09	90.74	62.16	80.00	56.98	3.79 (1.53–9.38)	0.72 (0.58–0.88)
Balanced screening cut off (≥30 points)^c			77.19	51.85	64.86	62.86	68.29	1.60 (1.18–2.19)	0.44 (0.26–0.76)
Balanced diagnostic cut off (≥40 points)^d			50.88	81.48	65.77	74.36	61.11	2.75 (1.49–5.08)	0.60 (0.45–0.81)

a

cut off with minimum 90% sensitivity.

b

cut off with minimum 90% specificity.

c

cut off with maximized sensitivity at minimum 50% specificity.

d

cut off with maximized specificity at minimum 50% sensitivity.

⁎⁎⁎

p < .001.

Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items; Base rate: rate of adolescents with a diagnosis of ADHD/ADD; AUC: area under the curve; Sens: Sensitivity; Spec: Specificity; Accuracy: correctly classified; PPV: Positive Predictive Value; NPV: Negative Predictive Value; LR+: Positive Likelihood Ratio; LR–: Negative Likelihood Ratio.

Table 3.

Diagnostic Accuracy of Eight New Sex-Specific Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.

ASRS-18	Base rate%	AUC (95% CI)	Sens.%	Spec.%	Accuracy%	PPV	NPV	LR+ (95% CI)	LR– (95% CI)
Girls (n = 67)	40.30	.80^⁎⁎⁎ (0.69–0.92)
Screening cut off (≥25 points)^a			92.59	35.00	58.20	49.01	87.50	1.42(1.11–1.83)	0.21(0.05–0.86)
Diagnostic cut off (≥43 points)^b			55.56	90.00	72.12	78.94	75.01	5.56(2.07–14.94)	0.49(0.32–0.76)
Balanced screening cut off (≥30 points)^c			81.50	50.00	62.68	52.37	80.01	1.63 (1.14–2.33)	0.37(0.16–0.87)
Balanced diagnostic cut off (≥44 points)^d			51.90	92.50	76.12	82.35	74.01	6.91(2.19–21.78)	0.52(0.35–0.78)
Boys (n = 44)	68.18	.71^⁎⁎ (0.53–0.89)
Screening cut off (≥24 points)^a			90.00	42.86	75.00	77.14	66.67	1.58(0.99–2.52)	0.23(0.07–0.80)
Diagnostic cut off (≥45 points)^b			16.67	92.86	40.91	83.33	34.21	2.33(0.30–18.15)	0.90(0.72–1.11)
Balanced screening cut off (≥26 points)^c			83.30	57.10	75.00	80.64	61.54	1.94 (1.04–3.64)	0.29(0.12–0.73)
Balanced diagnostic cut off (≥38 points)^d			56.70	71.40	61.63	80.95	43.48	1.98(0.82–4.80)	0.61(0.36–1.03)

a

cut off with minimum 90% sensitivity.

b

cut off with minimum 90% specificity.

c

cut off with maximized sensitivity at minimum 50% specificity.

d

cut off with maximized specificity at minimum 50% sensitivity.

⁎⁎

p < .01.

⁎⁎⁎

p < .001.

Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items; Base rate: rate of adolescents with a diagnosis of ADHD/ADD; AUC: area under the curve; Sens: Sensitivity; Spec: Specificity; Accuracy: correctly classified; PPV: Positive Predictive Value; NPV: Negative Predictive Value; LR+: Positive Likelihood Ratio; LR–: Negative Likelihood Ratio.

Discussion

This study examined psychometric properties and the clinical utility of new, empirically derived screening and diagnostic cut-off scores for the ASRS-18. We present follow-up analyses based on our previous work on the ASRS-18 using data from an adolescent psychiatric sample aged 12 to 18 years old (n = 111) to examine the ability of tailored ASRS-18 cut-off scores to predict a K-SADS-based diagnosis of any ADHD in a psychiatric outpatient setting.

Consistent with previous reports, the ASRS-18 showed good psychometric properties in our sample, with high reliability and validity (Adler et al., 2012; Somma et al., 2019; Sonnby et al., 2015). The examination of known-groups validity in the present study demonstrated that the ASRS-18 can distinguish between adolescents with and without an ADHD diagnosis in a general CAP outpatient setting.

In our prior evaluation of the ASRS-18 among psychiatrically referred adolescents, we found that the proposed general cut-off score of 9 (score range 0–18 based on the dichotomized scoring method) was of limited clinical value in a CAP setting because it yielded a post-test probability of ADHD of 50%, leaving clinicians with a high degree of uncertainty after ASRS-18 administration (Kessler et al., 2005; Sonnby et al., 2015). In this follow-up study, we conducted ROC analyses in the total sample and separately among girls and boys to obtain four new ASRS-18 cut-off scores for each of these groups, using predefined criteria: a screening cut off with 90% sensitivity with no restraints on specificity, a balanced screening cut off with maximum sensitivity and minimum 50% specificity, a diagnostic cut off with 90% specificity with no restraints on sensitivity, and a balanced diagnostic cut off with maximum specificity and minimum 50% sensitivity.

Overall, most of the new cut-off scores presented in this study were of greater clinical value than the cut-off of 9 proposed by Kessler et al. The majority of the new cut-off scores yielded post-test probabilities above or below chance level with the strongest performances observed for general and sex-specific diagnostic and balanced diagnostic cut-off levels.

To our knowledge, only one other study has explored alternative cut-off scores for the ASRS among adolescents (Somma et al., 2019). For their study, Somma et al. recruited adolescents attending professional education in Italy, but only those reported by their teachers to have externalizing problem behaviours were included in the analyses of diagnostic accuracy. This procedure resulted in a high ADHD prevalence (33%) in the sample, which makes it more akin to CAP outpatient settings, which are characterized by high ADHD prevalence. Using the same ASRS-18 scoring and summation method as the present study, Somma et al. suggested a general ASRS-18 cut-off score of 31 points for the detection of ADHD, yielding a sensitivity of 78%, a specificity of 72%, and post-test probability of ADHD of 58% (25 percentage point increase from pre-test probability) (Somma et al., 2019). We identified two ASRS-18 cut-off scores close to that of the Italian study, namely the general and girl-specific balanced screening cut-offs, both representing a score of 30. However, whereas the sensitivity and specificity of the Italian cut-off were fairly balanced, we observed substantially different estimates with higher sensitivities (77% and 82%, respectively) than specificities (50% and 52%, respectively) for both cut-offs and smaller changes in pre-test to post-test probability of ADHD (∼12 percentage points). The Italian study did not explore sex-specific cut-off scores; however, results of the current study indicate that the clinical usefulness of sex-specific cut-off scores is greater than that of general cut off levels.

It is important to note that the prevalence of ADHD varies greatly between population-based and clinical samples. The high prevalence of ADHD in our sample leads to a higher PPV and a lower NPV for the ASRS-18, compared with community-dwelling adolescent samples. However, the PPV and NPV values are not intrinsic to the ASRS-18 and are influenced by the high ADHD prevalence in outpatient settings. Hence, we included likelihood ratios, a prevalence-independent and clinically useful measure of test performance, in our analyses (Kent & Hancock, 2016).

Strengths and limitations

The present study has several methodological strengths. The sample consisted of consecutively referred unselected adolescent outpatients who were included in the study regardless of the cause of referral to CAP, which decreases spectrum bias and increases the generalizability of our results to general CAP settings. Our study can be considered representative of adolescent populations with high ADHD prevalence rates who are usually referred and treated in CAP across Europe and worldwide (Danielson et al., 2018). Another methodological strength is the assessment procedure where experienced clinicians gathered information on ADHD diagnostic criteria from both adolescents and their parents with the aid of a semi-structured diagnostic interview, the K-SADS interview, which strengthened the accuracy of diagnoses. Finally, our study sample had a predominance of adolescent girls, which supported the identification of cut-offs for girls. Although ADHD is more prevalent among boys than girls (Nussbaum, 2012), and sex is associated with different symptom profiles (Solberg et al., 2018), both groups experience the same negative consequences (Halmoy, Fasmer, Gillberg & Haavik, 2009) and benefit from early diagnosis (Hamed, Kauer & Stevens, 2015). However, ADHD is often underrecognized in girls because of differences in symptom profile; it tends to be characterized by more inattentive and fewer hyperactive/impulsive symptoms compared to ADHD in boys (Gershon, 2002). Therefore, there is an urgent need for suitable screening and diagnostic tools to identify affected girls. We believe our study can contribute to more successful recognition and diagnostic accuracy for ADHD in girls, raising awareness of the need for specific cut-off scores by sex.

Some limitations of the current study must be acknowledged. The most important limitation is the small sample size which may have reduced power to identify sex-specific cut off scores, especially among boys. Moreover, data-driven selection of cut-off levels may lead to overestimation of sensitivity and specificity, with smaller sample sizes increasing the amount of bias (Whiting, Rutjes, Westwood & Mallett, 2013). This study used DSM-IV (American Psychiatric Association, 2000) diagnostic criteria for ADHD diagnosis. Although core diagnostic criteria are unchanged in DSM-5 (American Psychiatric Association, 2013), a change of the minimum number of symptoms required for diagnosis among older adolescents (age 17 or older) may have implications for assessment with the ASRS-18. Furthermore, high rates of comorbidity were noted among participants, especially among girls with ADHD. Comorbidity may imply a symptom overlap, influencing the ratings of the ASRS-18. However, the internal consistency of the ASRS-18 was high in both sexes, indicating adequate reliability for both girls and boys.

Directions for future research

Despite the ASRS-18 being one of the most widely used self-report ADHD measures in psychiatry (Gray, Woltering, Mawjee & Tannock, 2014), there is a lack of studies exploring the clinical utility of alternative cut-off scores for the detection and identification of ADHD in outpatient settings. To our knowledge, our study is the first to analyze alternative cut-off scores for the ASRS-18 among psychiatrically referred adolescents.

The results of the current study suggest that the two new diagnostic cut-off scores for girls showed the best performance among all new cut-off scores, yielding a moderate increase in the probability of an ADHD diagnosis. Further studies in larger samples are warranted to validate these findings.

Conclusion

The ASRS-18 is a valid tool for assessing ADHD among adolescents in CAP settings. Results of this study showed that the ASRS-18 can distinguish between psychiatrically referred adolescents with and without an ADHD diagnosis. The clinical utility of the ASRS-18 in CAP settings can be enhanced by using the total score summation method with the new general and sex-specific cut offs proposed in this study. Multiple sources of information need to be carefully considered when assessing adolescents with ADHD symptoms. No single screening or diagnostic tool can replace a full clinical assessment. The ASRS-18 can be a useful tool for the detection and identification of ADHD among adolescents but must always be used together with other adequate indicators and information sources in the context of ADHD diagnostic assessment (Austerman, 2015).