Design and validation of a scale to measure worry for contagion of the COVID-19 (PRE-COVID-19)

Abstract

Objective

The increase in COVID-19 cases is generating fear and concern in society, which generates an emotional response that influences the adoption of health-related behaviors. The objective of the study was to design and validate the Scale of Worry for Contagion of COVID-19 (PRE-COVID-19).

Method

The study had a descriptive cross-sectional design. The population were young people and adults who resided in the cities of Lima and Callao during the declaration of the national health emergency due to the COVID-19 pandemic. 816 young people and adults from Lima and Callao (200 men and 616 women, Medad = 28.40; SD age = 7.10) participated during the period from March 16 to 27, 2020, who were recruited through a non-probability sample. The PRE-COVID-19, the WHO-Five Well-Being Index, the Generalized Anxiety Disorder Scale-2 and a single item were applied to measure the general perception of health. The scales were shared using a Google form through social networks. An internal consistency reliability analysis and structural equation models were performed, specifically confirmatory factor analysis (AFC). The recommendations of the Declaration of Helsinki and the principles that guide the ethical practice of online studies were followed.

Results

The results show a one-dimensional model with satisfactory goodness-of-fit indices (χ2 (9) = 52.00; CFI = .99; RMSEA = .09 [.07, .12]; WRMR = .85). The λ of the model were higher than 0.50 and the reliability had an excellent value (ω = .90). Likewise, the convergent and discriminant validity is evident between PRE-COVID-19 and measures of anxiety, well-being and self-reported health.

Conclusion

The results indicate that the PRE-COVID-19 is a valid and reliable instrument to measure concern about the spread of COVID-19 and the emotional impact on people.

Resumen

Objetivo

El incremento de los casos de la COVID-19 está generando miedo y preocupación en la sociedad, lo que genera una respuesta emocional que influye en la adopción de comportamientos relacionados con la salud. El objetivo del estudio fue diseñar y validar la Escala de Preocupación por el Contagio de la COVID-19 (PRE-COVID-19).

Método

El estudio tiene diseño descriptivo transversal. La población han sido jóvenes y adultos residentes en las ciudades de Lima y Callao durante la declaración de la emergencia sanitaria nacional por la pandemia del COVID-19 durante el periodo del 16 al 27 de marzo de 2020 quienes fueron reclutados a través de un muestreo no probabilístico. Se aplicaron la PRE-COVID-19, la WHO-Five Well-Being Index, Generalized Anxiety Disorder Scale-2 y un ítem único para medir la percepción general de salud. Las escalas fueron compartidas mediante un formulario de Google a través de redes sociales. Se realizó un análisis de fiabilidad por consistencia interna y modelos de ecuaciones estructurales, específicamente el análisis factorial confirmatorio (AFC). Se siguieron las recomendaciones de la Declaración de Helsinki y los principios que guían la práctica ética de los estudios vía online.

Resultados

Participaron 816 jóvenes y adultos de Lima y Callao (200 varones y 616 mujeres, M_edad = 28.40; DE_edad = 7.10). Los resultados muestran un modelo unidimensional con índices de bondad de ajuste satisfactorios (χ2 (9) = 52.00; CFI = .99; RMSEA = .09 [.07, .12]; WRMR = .85). Las λ del modelo fueron superiores a 0.50 y la fiabilidad tuvo un valor excelente (ω = .90). Asimismo, se evidencia la validez convergente y discriminante entre la PRE-COVID-19 y medidas de ansiedad, bienestar y salud autoinformada.

Conclusión

Los resultados indican que la PRE-COVID-19 es un instrumento válido y fiable para medir la preocupación por el contagio de la COVID-19 y el impacto emocional en las personas.

What is known?

Worldwide statistics indicate that the propagation of COVID-19 has important psychological, economic, social and political consequences. The rapid rise in COVID-19 cases also causes an increase in fear and worry in the community. This is understandable, given that individuals are worried about their health and do not want to be infected by a virus which causes numerous health problems and has caused millions of deaths around the world. Evaluation of the worry associated with COVID-19 may be important, as it leads to the adoption of forms of behaviour that promote health. Nevertheless, to date few instruments with suitable psychometric properties have been available to measure the level of worry about COVID-19 transmission in the Spanish language.

What does this paper contribute?

This study developed and evaluated the psychometric properties of a short tool to assess worry about COVID-19 transmission and its impact at an emotional level on personal health. This tool is known as the Scale of Concern about COVID-19 Transmission (PRE-COVID-19). The results indicate that there is evidence that PRE-COVID-19 is a valid construct that is both convergent and discriminating, as well as its excellent reliability in measuring the level of worry about COVID-19 transmission. The PRE-COVID-19 tool will make it possible to know and understand worry about the transmission of this disease and its impact on people’s behaviour, as well as generating evidence for intervention in health emergencies.

Introduction

SARS-CoV-2 is a type of coronavirus that first appeared in December 2019 in Wuhan (China), and it causes the disease known as COVID-19.¹ From the moment it appeared, COVID-19 has spread rapidly to different countries, putting the lives of millions of people at risk.² Due to this, the World Health Organisation declared an international public health emergency before raising this to the level of a pandemic. This situation has given rise to global and local responses to contain the outbreak, such as the recommendation to frequently wash the hands, obligatory social isolation (or quarantine) and the isolation of infected individuals and those with a high probability of infection.³ In spite of these measures, statistics worldwide indicate that the propagation of COVID-19 seems to be unstoppable,⁴ with important psychological, medical, economic, social and political consequences.^5,6

Another result of the rapid increase in COVID-19 cases is the rise in social worry. This is understandable, given that people worry about their health and do not want to be infected by a virus that that causes negative consequences for health and even a relative risk of death.⁷ Different studies have therefore been conducted on behavioural changes during outbreaks of infectious diseases. They evaluate the perception of risk, as a cognitive process based on the perceived probability of personal infection or the perceived severity of the disease, or as an emotional process based on evaluation of worry and anxiety.⁸ Both dimensions of the risk (cognitive and emotional) are important in gaining an understanding of individual behaviour in connection with infectious diseases.⁹

More specifically, the worry about the disease is defined as an emotional response to an ailment that is an important aspect of managing it.¹⁰ This emotion influences the adoption of forms of behaviour that promote health.¹¹ Studies of infectious respiratory diseases indicate that during their early phases, when their characteristics, treatment and prevention are unclear, emotional responses to risk may be better predictors of protective behaviours.¹² For example, recent studies have shown the effect of worry on the prevention and diagnosis of Middle East Respiratory Syndrome.¹⁰ In a similar way, it is important to know and understand the levels of worry in connection with the disease, specifically the worry about the transmission of COVID-19, given that evidence may be generated for intervention in medical emergencies.

The scientific literature describes many instruments that are used to measure worry about different aspects of health, such as the Cancer Worry Scale,¹² which has been adapted to Peruvian culture,¹³ or single item measurements to gauge worry about the transmission of respiratory viruses.¹⁰ Nevertheless, specific instruments are needed with evidence of their validity and reliability to measure worry associated with the transmission of COVID-19. The general objective of this study is therefore to design and validate the PRE-COVID-19. Its specific objectives are to evaluate the validity of the content of the items of the PRE-COVID-19 and to examine its internal structure, estimating its reliability based on its internal consistency. It also analyses the comparative convergent and discriminatory validity of the PRE-COVID-19 and other measurements of psychological variables (well-being, health and anxiety).

Method

Design

A descriptive transversal study was undertaken, focussing on the design and analysis of the psychometric properties of the PRE-COVID-19 scale.

Study population and scope

The population was composed of young people and adults who live in the cities of Lima and Callao. Both cities are in the centre and western-centre of Peru, and the study took place during the declared national health emergency.

Inclusion and exclusion criteria

The inclusion criteria were: individuals over the age of 18 years who live in Lima or Callao, who had given their informed consent and who had access to the online questionnaire. All of the individuals who did not fulfil these criteria were excluded from the study.

Sample

The final sample was selected using non-probabilistic intentional sampling and its size was calculated based on the recommendation for studies which use a structural equation model.^14,15 The number of observed (6) and latent variables in the model was therefore considered (there was one latent variable: worry about transmission), together with the size of the anticipated effect (0.10), the desired probability (0.05) and the level of statistical power (0.95). The minimum required sample size was therefore 328 participants, while more than double the recommended amount were recruited in ensure better representativeness.

Variables and instruments

Worry about the transmission of COVID-19

The PRE-COVID-19 was prepared, composed of 6 items which evaluate worry about the transmission of COVID-19 and the impact this worry may have on the everyday functioning of individuals, more specifically on their mood and capacity to perform their everyday activities. The 6 items have 4 Likert-type options for answering (from 1 = never or rarely to 4 = almost always), where the higher scores indicate more frequent worry about transmission.

Well-being

The WHO-Five Well-Being Index¹⁶ was used. This is composed of 5 items with 4 alternative Likert-type answers (from 0 = never to 3 = always). The total score is obtained by adding the scores for all of the items, and high scores indicate a high level of well-being. The reliability of this study here is considered to be appropriate (ω = 0.87).

Anxiety

The Generalized Anxiety Disorder Scale-2¹⁷ was used. This is composed of 2 items that measure emotional behaviour (feeling nervous) and cognitive behaviour (worry) regarding generalised anxiety during the previous 2 weeks. The items have 4 possible answers on a Likert-type scale (from 0 = never to 3 = almost every day), where a higher score indicates greater generalised anxiety. In this study the Generalized Anxiety Disorder Scale-2 was found to have an appropriate level of reliability according to the Angoff-Feldt coefficient (r_AF = 0.88).

General perception of health

For the purposes of the study the first question was selected (“In general, how would you describe your health over the last week?”) from the Short Form-8 Health Survey Questionnaire.¹⁸ This question is a general measurement of perceived state of health. It uses a 5 point Likert-type scale for answering (where 1 = very poor and 5 = very good). A higher score indicate a better general perception of health.

Procedure

The Cancer Worry Scale¹² which evaluates worry about developing cancer and how this affects the everyday functioning of individuals was adapted for the PRE-COVID-19 scale. Some terms were modified in this process, such as the replacement of “the last month” with “the last week” or “possibility of developing cancer” with “possibility of catching coronavirus”. Subsequently the content of the PRE-COVID-19 was analysed. Thus 14 expert judges (professionals with experience in clinical and health psychology), were contact through their email accounts and evaluated the consistency and relevance of the items. Each criterion was scored from 0 (not relevant/consistent/clear) to 3 (totally relevant/consistent/clear). In the same way, 25 individuals with similar characteristics to those of the final sample evaluated the degree of comprehension and clarity of the items on a scale from 0 (not clear at all) to 3 (totally clear). The final version of the PRE-COVID-19 was prepared on the basis of these evaluations. Table 1 shows the items in the Peruvian adaptation of the Cancer Worry Scale and the final version of the PRE-COVID-19.

Table 1.

Spanish version of the Cancer Worry Scale and the PRE-COVID-19.

Cancer Worry Scale	PRE-COVID-19
Item 1: During the last month, how often have you thought about your probabilities of developing cancer?	Item 1: During the last week, how often have you thought about the probability of catching coronavirus?
Item 2: During the last month, has thinking about the possibility of developing cancer affected your mood?	Item 2: During the last week, has thinking about the possibility of catching coronavirus affected your mood?
Item 3: During the last month, has thinking about the possibility of developing cancer affected your capacity to perform your “everyday activities”?	Item 3: During the last week, has thinking about the possibility of catching coronavirus affected your capacity to perform your “everyday activities””?
Item 4: To what degree does the possibility of developing cancer one day worry you?	Item 4: To what degree does the possibility of catching coronavirus worry you?
Item 5: How often do you worry about the possibility of developing cancer?	Item 5: How often do you worry about the possibility of catching coronavirus?
Item 6: Is being worried about developing cancer an important problem for you?	Item 6: Is being worried about the possibility of catching coronavirus an important problem for you?

The final version of the PRE-COVID-19 was shared using a Google document through social networks (Facebook and WhatsApp). This method of gathering data was used for 2 reasons: (a) the application of social isolation measure in Peru hindered face-to-face data gathering, and (b) this is a valid procedure which follows Internet Based Methodology.¹⁹ The ethical and methodological recommendations for research using Internet were followed.²⁰ Data were gathered during the first week after the declaration of the medical emergency in Peru (from 16 to 27 March 2020). Completing the questionnaire took approximately 20 min.

Data analysis

The RStudio interface (version 1.1.463) of the R statistical program (version 3.6.0) was used for the calculations. Analyses were performed sequentially. Aiken’s V and its 95% confidence intervals were calculated first; this coefficient makes it possible to analyse content validity based on the evaluation of the clarity, consistency and relevance of the items in the PRE-COVID-19. V values of ≥ 0.70 and a lower limit of the 95% confidence interval  ≥ 0.59 express a positive evaluation of the items at sample and population levels, respectively.²¹ Descriptive statistics were calculated in second place, including the arithmetical mean, standard deviation, asymmetry and kurtosis, as well as the response rates for the Likert-type answer options, as they are ordinal measures. Thirdly, confirmatory factorial analysis (CFA) was selected as it uses a theoretical presumption of the internal structure of the instrument, in which case CFA is the technique to use par excellence, to explain how each item affects a single factor. The CFA was estimated using the lavaan library of the RStudio interface. Due to the ordinal nature of the items the Diagonally Weighted Least Squares with Mean and Variance corrected estimator was used, based on the polychoric correlation matrix. This matrix calculates a lineal association between the latent variables which underlie ordinal observed variables.²² Goodness of fit measurements were used to consider a factorial structure as good²³: the comparative fit index (CFI) ≥ 0.95, the root mean square error of approximation was used (RMSEA) < 0.06 with the weighted residual mean root (WRMR) < 1. Factorial loads (λ) higher than 0.50 were considered to be appropriate. Likewise, the average variance extracted (AVE) was calculated, and a higher value than 0.50 was considered to be appropriate and indicates that more than 50% of construct variance is due to its indicators.²⁴ Fourthly, reliability was estimated based on internal consistency using the ω coefficient, as this is the appropriate option for factorial models.²⁵ Fifthly, the evidence was examined based on the relationship with other converging and diverging variables. Thus correlations were used based on the structural equations models between the PRE-COVID-19 score and the score of the WHO-Five Well-Being Index, the Generalized Anxiety Disorder Scale-2 and the single item for measuring the general perception of health. To determine convergence the criterion that factorial loads were above 0.60 or 0.70 was used, while for discriminatory capacity the AVE of each latent variable must be higher than the square of the correlations between the latent variable in question and each one of the others.²⁶ Additionally, it was considered that the magnitudes of the effect would be r ≥ 0.20: the recommended minimum; r ≥ 0.50: moderate; and r ≥ 0.80: strong.²⁷

Ethical considerations

The research protocol was reviewed and approved by the Ethics Committee of the Universidad Privada del Norte (Registration No.: 20203001). The Helsinki Declaration recommendations were also followed, together with the principles that guide the ethical practice of online research.²⁸ All of the participants gave their informed consent before answering the online questionnaire. The consent form described the confidentiality and anonymity of the their data and the study objective and procedure, as well as the option to take part in the study and withdraw from it at any time, and the possibility of communicating with the research team to clarify any doubts. A protected file folder was also used to protect the stored data by means of a password. Care was taken to ensure that no unauthorised person had access to the data, and it was guaranteed that the data would not be used for any purpose other than those of the study.

Results

Description of the participants

The participants were 816 young people and adults in Lima and Callao (Peru), ranging in age from 18 to 50 years old, with an average of 28.40 years (standard deviation 7.10). 200 of the total number of participants were men and 616 were women; 427 of them live with an older adult. Respecting the medium by which they obtain information about COVID-19, the majority said it was television, radio and the press (n = 421), followed by official government sources (n = 224) and social networks (n = 153). 594 participants stated that they had received enough information about COVID-19 to take decisions in their everyday life, while 222 considered that this was not the case.

Content validity

Table 2 shows that all of the items in the PRE-COVID-19 are clear, relevant and consistent, at sample level (V ≥ 0.79) as well as at population level (the lower limit of the 95% confidence interval  ≥ 0.59).

Table 2.

Aiken’s V for the evaluation of the clarity, consistency and relevance of the items in the PRE-COVID-19.

items	Clarity				Consistency				Relevance
	Av	SD	V	CI95%	Av	SD	V	CI95%	Av	SD	V	CI95%
Expert judges
1	2.71	.61	.90	.75−.97	2.64	.63	.88	.72−.96	2.86	.36	.95	.81−.99
2	2.64	.63	.88	.72−.96	2.43	.65	.81	.64−.91	2.43	.76	.81	.64−.91
3	2.71	.47	.90	.75−.97	2.93	.27	.98	.85−1.0	2.86	.53	.95	.81−.99
4	2.50	.76	.83	.66−.93	2.36	.74	.79	.61−.90	2.50	.65	.83	.66−.93
5	2.57	.65	.86	.69−.94	2.71	.61	.90	.75−.97	2.79	.43	.93	.78−.98
6	2.43	.65	.81	.64−.91	2.86	.36	.95	.81−.99	2.71	.73	.90	.75−.98
People in general
1	2.86	.36	.95	.81−.99
2	2.36	.74	.79	.61−.90
3	2.86	.53	.95	.81−.99
4	2.43	.65	.81	.64−.91
5	2.93	.27	.98	.85−1.0
6	2.79	.43	.93	.78−.98

SD: Standard Deviation; CI95%: 95% Confidence Interval; Av: Average.

Preliminary analysis of the items

Table 3 shows the descriptive statistics of the scale; item 4 can be seen to have the highest arithmetical mean, and the deviation in all of the items is lower than 0.96, with a minimum of 0.75. The asymmetry shows that in general there is a tendency towards low scores; moreover, the kurtosis indicates that in item 3 there is a greater accumulation of data in the centre of the distribution.

Table 3.

Descriptive statistics of the items.

Items	Percentage of answer options				Av	SD	g1	g2
	1	2	3	4
1	19.85	50.25	22.92	6.99	2.17	.82	.43	−.26
2	49.51	36.15	9.93	4.41	1.69	.82	1.10	.62
3	59.56	27.33	9.44	3.68	1.57	.81	1.34	1.06
4	5.88	54.17	30.02	9.93	2.44	.75	.49	−.19
5	25.00	48.90	20.47	5.64	2.07	.82	.48	−.23
6	21.32	40.56	25.00	13.11	2.30	.95	.30	−.81

SD: Standard Deviation; g1: Asymmetry; g2: Kurtosis; Av: Average.

Internal structure and reliability

The CFA was used to verify the internal structure, modelling a unidimensional structure as proposed in previous publications. The goodness indexes reveal a satisfactory fit: χ² (9) = 52.00; CFI = 0.99; RMSEA = 0.09 [0.07, 0.12]; WRMR = 0.85. The λ of the model were higher than 0.50 and are shown in Fig. 1. The average λ was 0.77. In the same way, the value of the AVE is appropriate (0.61) and it indicates that the latent factor is suitably explained by its observed variables. The reliability was calculated using the ω coefficient, giving a value of 0.90, which may be considered excellent.

Figure 1.

Factorial structure of the PRE-COVID-19.

Convergent and discriminatory validity

According to the study objectives converging and discriminatory validity were evaluated. For this purpose a CFA was used to specify and evaluate a model with 4 latent variables: worry about transmission, perception of health, well-being and generalised anxiety. Analysis of this type makes it possible to estimate the correlations between constructs at a latent level. This model had an excellent fit: χ²₍₁₄₂₎ = 176.77; RMSEA = 0.07; WRMR = 0.88; CFI = 0.98. Fig. 2 shows the estimation of standardised parameters for this model.

Figure 2.

Estimation of CFA model standardised parameters for validity in association with other variables.

The correlations between worry about COVID-19 transmission and the other constructs were as had been hypothesised: it was positive with generalised anxiety (ρ = 0.74, P < .01) and negative with well-being (ρ = −0.43, P < .01) and the perception of health in general (ρ = −0.58, P < .01). Convergence was also demonstrated by the fact that the factorial loads were higher than values of 0.70 (see Fig. 2). On the other hand, discriminatory validity (Table 4) was also demonstrated as the AVE was found to be higher than the square of the interfactorial correlations.²⁴

Table 4.

Discriminatory validity analysis.

	Worry	Well-being	Anxiety	General health
Worry	.62	.19	.55	.33
Well-being	.19	.66	.33	.62
Anxiety	.55	.33	.90	.38
General health	.33	.62	.38	.69

The extracted average variance is shown in the diagonal.

Discussion

Worry about the disease plays an important role in the psychological reactions and behaviours that promote health help to reduce the risk of developing a disease.^10,11 Thus in Peru and other Spanish-speaking countries, appropriate instruments are required to measure psychological aspects associated with the worldwide medical emergency of COVID-19, such as worry about transmission.

The CFA showed that the unidimensional model of the PRE-COVID-19 had good fit indexes. Moreover, the factorial loads of the items revealed a strong relationship between them and the latent construct denominated “worry about the transmission of COVID-19”. More specifically, item 2 (“During the last week, has thinking about the possibility of catching coronavirus affected your mood?”) is the one that has the greatest factorial load and therefore best explain worry about the transmission of the disease. This result is expected, as the social isolation measures imposed by the governments of countries affected by the COVID-19 pandemic keep a large number of people confined to their homes, which may trigger a set of psychological problems such as panic, anxiety and depression.²⁹ In turn, reliability analysis also gave satisfactory results, making it possible to state that the PRE-COVID-19 is internally consistent.

The evidence for converging validity showed that greater worry about transmission is associated with higher levels of anxiety. Although worry is associated with a high level of anxiety, they are conceptually different constructs as anxiety is characterised by the presence of feelings of stress and excitement.³⁰ This result must be interpreted cautiously, as the measurement of anxiety covers 2 items which only measure the frequency of anxiety symptoms. Future studies will be able to evaluate the evidence for converging validity by adding items that measure the severity and duration of anxiety.

On the other hand, higher levels of worry about transmission are associated with a fall in well-being and a negative perception of health in general. This shows that worry here may be associated with reported health, as it also affects the evaluation of experienced symptoms.³¹ Likewise, the associations between worry, perceived health and well-being would be consistent with the hypothesis of the perception of symptoms. In this, individuals with lower levels of well-being and positive emotions are more likely to pay attention to bodily sensations and interpret them as indicators of disease, leading them to express these worries explicitly.³²

This study is not free of limitations. Firstly, although the number of participants was large enough to properly perform a CFA, the difference between the numbers of men and women make it impossible to carry out an invariance measurement analysis according to sex. However, it must be said that invariance analysis was not one of the study objectives. Future studies may resolve this problem by ensuring that an equal number of individuals of both sexes take part. Secondly, the variability in terms of age, health profiles or the presence of comorbidities, among other aspects, could affect participants’ perception of their vulnerability to COVID-19. Thirdly, the absence of a second measurement makes it impossible to calculate test-retest reliability, temporal stability or longitudinal invariance. Given this, future studies should satisfy this aspect by using longitudinal designs to identify the psychological factors that may predict worry about COVID-19 transmission. Moreover, the transversal design does not enable identification of whether anxiety and negative self-perception of health and well-being may be risk factors or consequences of worry about catching COVID-19. Fourthly, people from Lima and Callao took part in this study, 2 cities in the centre and eastern centre of Peru. They were selected by convenience-based sampling and are not a proper representation of the general population of Peru. This may induce a selection bias that would restrict generalisation of the findings to samples from other regions of Peru. It is therefore necessary to have a more representative and diverse sample (with different educational levels, work situations, levels of income, a history of or current chronic diseases, etc.) to compare and generalise the results. Fifthly, the duration of data gathering was limited, and this may introduce bias into the sample that was included. Subsequent studies must broaden the time period for gathering data of this type.

In spite of the limitations of the study, the results shown indicate that the PRE-COVID-19 is a useful instrument for gaining a better understanding of how people react and adapt to information on the risk of COVID-19 transmission. In turn, the shortness of the instrument prevents overloading the examinees due to fatigue and it may be of use in large scale studies that evaluate psychological aspects in medical emergencies. Knowing the level of worry about transmission based on the PRE-COVID-19 may also be useful when planning and evaluating the impact of proposals for psychological intervention which seeks to reduce the level of worry associated with the presence of COVID-19. It is also useful in the identification of the most vulnerable individuals who may develop anxiety, which is associated with worry about transmission.

To conclude, the results indicate that the PRE-COVID-19 is a valid and reliable unidimensional instrument to the measure worry about COVID-19 transmission and its emotional impact on individuals.

Conflict of interests

The authors have no conflict of interests to declare.