The diabetes education material on diabetes for website: Results of a validation process

Abstract

Objective

To validate an educational material on diabetes through an expert consensus for its implementation into a web site.

Material and methods

An observational study was carried out in a group of health professionals, for which an educational material was developed for patients with diabetes. Topics included nutrition, physical exercise, control indicators, complications, pharmacological treatment, among others. The language, text and figures were focused on easy comprehension, additionally, a section of didactic activities to be answered by the patient with diabetes at the end of each module was included. To evaluate the educational material by health professionals, an instrument was designed and validated. Once all the educational material was available, each of the modules was sent by e-mail to at least three clinical experts in the assigned topic, as well as the instrument for the evaluation of the module.

Results

Thirty-seven experts were included in the study, 76% rated the educational modules evaluated as highly adequate, while only 24% rated them as adequate. The instrument used obtained a good level of internal consistency, with a Cronbach's alpha coefficient of 0.92. In the dimensions of the instrument, the lowest Cronbach's alpha score was that of “call-to-action”, with a value of 0.71.

Conclusion

The diabetes educational material was rated as highly appropriate by the clinical experts. The developed instrument has an adequate content validity, as well as a good level of internal consistency.

Resumen

Objetivo

Validar un material educativo en diabetes a través de un consenso de expertos para su implementación en un sitio web.

Material y métodos

Se realizó un estudio observacional en un grupo de profesionales de la salud, para ello se desarrolló un material educativo para pacientes con diabetes. Se incluyeron temas de nutrición, ejercicio físico, indicadores de control, complicaciones, tratamiento farmacológico, entre otros. El lenguaje, texto y figuras fueron enfocadas a una fácil comprensión, además de una sección de actividades didácticas a responder por el paciente con diabetes al finalizar cada módulo. Para evaluar el material educativo por los profesionales de la salud, se diseñó y validó un instrumento. Una vez que se contó con todo el material educativo, se envió cada uno de los módulos por correo electrónico al menos a tres expertos clínicos en el tema asignado, así como el instrumento para la evaluación del módulo.

Resultados

Se incluyeron 37 expertos en el estudio, 76% calificó los módulos educativos evaluados como altamente adecuados, mientras que solo 24% lo evaluó como adecuados. El instrumento utilizado obtuvo un buen nivel de consistencia interna, con un coeficiente alfa de Cronbach de 0,92. En las dimensiones del instrumento, la puntuación del alfa de Cronbach más baja fue la de «llamada a la acción», con un valor de 0,71.

Conclusión

El material educativo en diabetes fue calificado como altamente apropiado por los expertos clínicos. El instrumento desarrollado tiene una adecuada validez de contenido, así como buen nivel de consistencia interna.

Introduction

The worldwide prevalence of type 2 diabetes in the adult population was 9.3% (463 million adults) in 2019, and it is estimated that by 2045 it will increase up to 700 million.¹ In Mexico, this clinical condition affects 13.7% of the population, and 68% of them have poor glycemic control.²

The main objective of diabetes education is to provide the patient enough knowledge to learn how to take assertive decisions, and to develop skills for the self-management of the disease.³ Diabetes education is also known to improve metabolic control by reducing glycosylated hemoglobin.⁴ The evidence suggests that the Information Communication Technologies (ICTs) promote patient active participation in the self-management of the disease, and improve lifestyle (physical activity and nutrition), as well as glucose self-monitoring.⁵

Information about the impact of the ICTs on other metabolic control indicators is less solid.⁶ There are several mobile applications offered to provide diabetes education, however the lack of endorsement from clinical experts has been documented.⁷ There are certain limitations of these educational tools such as (difficulty) ease of use, complex information architecture, as well as lack of learning objectives. In addition, reports on ICTs strategies linked to an educational theory to provide diabetes education are scarce.⁸

In addition, educational apps or websites generally do not meet the standards suggested by associations of diabetes educators and clinical experts.^9,10 Moreover, it is important to consider the characteristics of the population, such as education level, social environment, health professional–patient interaction relationship, among other aspects that influence the provision of education through the use of ICTs.

The objective of this study was to validate an educational diabetes material for a website through expert consensus and development of instrument.

Materials and methods

Study design

This is a cross-sectional study to validate the educational material content by experts’ opinion prior to the implementation of an educational website on type 2 diabetes, as part of the first phase of the multicentric clinical study called: effect of education through a mobile application with medical reinforcement to improve metabolic control of patients with type 2 diabetes. Multicentric Clinical Trial. This study was approved by the National Research Committee of the Instituto Mexicano del Seguro Social (IMSS) with registry number R-218-758-100.

Instrument construction and validation

A literature search for validated instruments for health professionals was conducted in order to evaluate the proposed educational material for patients with diabetes or chronic diseases. We identified the need to develop an instrument for this purpose, due to the lack of instruments used to evaluate diabetes education programs directed to websites or mobile applications in Mexico or Latin America. Therefore, an instrument was developed for the purpose.

Items and internal consistency

The logical and content validity of the instrument was evaluated by an expert panel.¹¹ This panel was composed by three nutritionists, four physicians specialized in primary care medicine and a doctor in epidemiology, all with more than 10 years of experience in diabetes care. This panel of experts was selected from IMSS clinics in Mexico City, Mexico. The expert panel evaluated the instrument in a blinded and independent.

The first version of the instrument included 22 items with six dimensions. After review by the expert panel, it was agreed to eliminate 4 items, in order to make the instrument simple, understandable and not redundant. Two items were eliminated from the attraction dimension, one item from the call-to-action dimension and one item from the educational content dimension. Finally, the questionnaire included 18 items with six dimensions.

The internal consistency of the instrument was obtained by evaluating the educational material content, which was done through the Cronbach's alpha test, as described in the section on statistical analysis.

We used a five-point Likert scale to score the instrument. The categories were from slightly agree (1 point) to strongly agree (5 points), resulting in a scale of 18–90. For the classification we formed tertiles from the total value of the instrument and the following categories were obtained: (a) low ≤ 30, (b) medium 31–60, and (c) high ≥ 61 points.

The six dimensions evaluated by this instrument were: Attraction (items 1–5), comprehension (items 6–9), motivation (items 10–12), acceptance (items 13–14), call-to-action (items 15–16), and educational content (items 17–18).

To rate each dimension, the scale was: (a) Attraction low ≤ 8 points, medium 9–16 points, and high ≥ 17 points. (b) Comprehension: low ≤ 7, medium 8–13, and high ≥ 14 points. (c) Motivation: low ≤ 5, medium 6–10 points, and high ≥ 11 points. (d) Acceptance, call-to-action and educational content: low: 1–3 points, medium 4–6 points, and high ≥ 7 points. The instrument used to evaluate the educational program and to validate it by expert consensus is shown in Appendix 1.

Validation of diabetes education material

In the first phase of the validation of the educational content, we reviewed and updated the topics included in the online diabetes course previously used in another diabetes clinical trial conducted by our research group.¹² The educational materials were updated, and the final topics included were: understanding of the disease, nutrition, physical exercise, metabolic control indicators, myths and facts about diabetes, complications, depression, and family support. Subsequently to this review, our research group decided to include two new topics, pharmacological treatment and oral health. The new version of the educational topics and its goals are shown in Fig. 1.

Figure 1.

Objectives of diabetes education material.

In each topic, a sociocultural-designed character induces the user through the contents, with brief short messages to reinforce specific aspects to be retained. The educational material contains illustrations as well as multimedia components. At the end of each topic, a multi-option or a true or false question, or any didactic activity is presented to reinforce learning. In order to assure learning, to provide motivation and feedback and generate a call-to-action for the patient, the research team implemented the theory of Gagné.¹³ This educational theory starts with getting the attention of users, setting learning objectives, reviewing prior knowledge, providing guidance, assessing performance and finally, providing feedback.

To assess diabetes knowledge, items from the instrument called “Development of Diabetes Knowledge Questionnaire in Spanish DKQ-24” were included in the didactic activities of the educational modules.¹⁴ We invited to participate to clinical experts in diabetes care from IMSS and physician from outside the institute. All of them have more than five years of clinical experience in the care of patients with type 2 diabetes. When they agreed to participate and according to their clinical training, they received by e-mail the educational topic to evaluate it as well the link to the instrument to be filled out online. Each topic of the educational material was evaluated by at least three experts. The panel consisted of psychologists, diabetes educators, nutritionists, endocrinologists, family doctors and dentists. Each member of the expert panel conducted an independent and blinded evaluation of the assigned educational topic.

Fifteen days were given for the experts to review and complete the instrument online. The educational material developed and validated by the consensus of experts, is the basis of the educational site “Entiendo mi diabetes”, which can be visited at: http://entiendomidiabetes.com/.

Statistical analysis

Data are expressed as mean ± standard deviation, median with interquartile range according to the distribution of data for the total score of the instrument and its dimensions.

Frequency measures and percentages were used to obtain the category in the classification of the educational material reviewed by the experts.

The internal consistency of the instrument for each dimension was obtained by Cronbach's alpha reliability coefficient. Statistical analysis was performed in the IBM Statistics SPSS V25.0 statistical software.

Results

Internal consistency

The internal consistency of the instrument used for the evaluation of the educational material obtained a Cronbach's alpha value of 0.922. For the dimensions of the instrument, the “comprehension and motivation dimension”, obtained the highest value with a Cronbach's alpha of 0.88, followed by the “educational content” with 0.87; while the lowest score was obtained for the “call-to-action” dimension of 0.71 respectively. These data are shown in Table 1.

Table 1.

Reliability analysis of the instrument based on the evaluation of the educational material on diabetes, n = 37.

Topics	Mean ± SD	Cronbach's alpha analysis
Total score	66.7 ± 7.1	0.92
Dimension
Attraction	18.6 ± 1.8	0.74
Comprehension	14.3 ± 2.4	0.88
Motivation	11.3 ± 2.1	0.88
Acceptance	7.3 ± 0.9	0.75
Educational contents	7.3 ± 1.0	0.87
Call-to-action	7.7 ± 0.5	0.71

SD, standard deviation.

Validation of educational diabetes material

Table 2 shows the average assessment and classification of the diabetes education program by the expert consensus. A total of 37 clinical experts participated in the validation of at least one educational topic. In summary, 76% of the topics were rated as very adequate and only 24% as adequate. The average rate was of 66.7 ± 7.1, out of 90 points, with a median of 69 (interquartile range 62–72).

Table 2.

Total score of educational material in diabetes by expert consensus, n = 37.

Category	n	%
Low	0	0
Medium	9	24
High	28	76
	Instrument score
Mean ± SD	66.7 ± 7.1
Median	69
Quartile 25	62
Quartile 75	72

Score: Low rate (<30 points), medium rate (31–60 points), high rate (>61). SD, standard deviation.

Table 3 shows the scores for the ten topics evaluated. Knowing about diabetes, metabolic control indicators, pharmacological treatment, and diabetes complications were all rated high or adequate. Nutrition and oral complications topics were rated medium (67%); none of the topics were rated low of inadequate.

Table 3.

Score of educational material by consensus of experts, n = 37.

Topics	Expertsn (%)	Category
		Lown (%)	Mediumn (%)	Highn (%)
1. Knowing about diabetes	5 (13.5)	0	0	5 (100)
2. Depression and family support	5 (13.5)	0	1 (33)	4 (80)
3. Metabolic control indicators	4 (11)	0	0	4 (100)
4. Pharmacological treatment	3 (8)	0	0	3 (100)
5. Nutrition 1	3 (8)	0	2 (67)	1 (33)
6. Nutrition 2	4 (11)	0	2 (50)	2 (50)
7. Diabetes complications	3 (8)	0	0	3 (100)
8. Physical exercise	4 (11)	0	1 (25)	3 (75)
9. Myths and facts about diabetes	3 (8)	0	1 (33)	2 (67)
10. Oral complications	3 (8)	0	2 (67)	1 (33)

The data are presented as frequency and percentage.

After review of the educational material by the expert panel, it was found that the dimensions with the highest proportion of high ratings were information (95%), acceptance (89%) and attraction (84%). The dimensions of understanding and motivation were rated low by 3% of the experts. These data are shown in Table 4.

Table 4.

Score in the dimensions of the instrument by the consensus of experts of the educational diabetes material, n = 37.

Dimensions of instrument	Category
	Low		Medium		High
	n	%	n	%	n	%
Attraction	0	0	6	16	31	84
Comprehension	1	3	8	21	28	76
Motivation	1	3	10	27	26	70
Acceptance	0	0	4	11	33	89
Call-to-action	0	0	8	22	29	78
Educational content	0	0	2	5	35	95

The data are presented as frequency and percentage.

Discussion

Diabetes education is an essential component to achieve the goals of control and self-care attitudes and promotion a better lifestyle in the patient. The use of ICTs can be useful to link health professionals with the population with diabetes.^15,16

This study is a report that involves the validation of educational content by expert consensus. Our results show the validating process of the educational program in diabetes prior to its incorporation into a mobile app and a website. The educational program was highly approved by 76% of clinical experts.

The evidence shows that strategies to deliver education in diabetes through mobile applications have been positive. Currently, few strategies achieve educational materials that have been designed following a pedagogical theory and that are easy to use and to understand by the target population.^7,17

In Mexico, there are just a few educational websites on diabetes that accompany patients with accurate information, easy access, appropriate design and content, as well as being supported by clinical scientists. These features of a diabetes education website have been previously evaluated by other authors, which increases the likelihood of its recommendation for use by healthcare professionals in patients with diabetes and their primary caregiver.¹⁸ In addition, the benefit of knowing about their disease as well as the social support of online education interventions in patients with diabetes or cardiovascular disease has been reported.¹⁹

We consider our report to be one of the first to show the development and validation of diabetes educational content by clinical experts, which strengthens the recommendation of its use by other health professionals. Subsequently, the usability of the previously validated educational website in patients with diabetes will have to be measured. This is an important element to increase the acceptance rate and compliance of the online educational course.²⁰

Despite the acceptance of the use of ICTs to provide diabetes education reported in the literature, a low proportion of patients perform a complete review of the topics to be addressed for disease management. There is currently a large supply of mobile applications and websites related to wellness, education, diet planning, physical exercise, as well as self-management, among others.²¹ Nevertheless, few these websites and mobile applications are derived from a process of development and validation from clinical diabetes experts to sustain its development.²²

In this study, a validated instrument was used to measure the level of knowledge about the disease as part of the didactic activities. In addition, it is important to evaluate self-management of the disease, lifestyle and health beliefs, before and after an educational intervention with the use of ICTs.²³

We must to highlight the need to validate educational materials in diabetes, prior to their exposure to patients, both by clinical experts and supported with a theoretical educational model. Therefore, it is necessary to create tools for clinicians and, in a second phase, for patients with diabetes. The aim of diabetes education is to ensure knowledge acquisition, understanding and motivation of patients to learn about the disease and self-management.^22,24

The effect that the use of mobile applications can have on metabolic control is not yet conclusive.²⁵ Mobile applications or web sites can be tools to provide diabetes education and promote a healthy lifestyle to patients.²⁶ One of the main objectives of ICTs for patients is to achieve adequate self-management of their disease, in this sense, it has been identified success in achieving metabolic control of patients when there is a link between the use of this type of tools along with the recommendations of health professionals.²⁷

Providing education with the use of ICTs increases access to this type of strategies to the target population. In Mexico there are many rural areas, where it is difficult to access digital platforms. Finally, it will be necessary to adapt this type of strategy, such as the use of telemedicine, to provide health education to the population with diabetes in areas where there is no access to the Internet or to devices such as computers or cell phones.²⁸

There are limited tools for validating educational content by clinical researchers, diabetes educator and primary care clinical professionals. Together for the validation of the educational program, in the present research, an instrument was developed to evaluate the main characteristics of the educational content in diabetes, which had an adequate reliability.

In this sense, the instrument was conformed with items aimed at evaluating the attraction, comprehension, motivation, and acceptance, among other aspects. In the same way, transculturation processes of instruments have been carried out to evaluate behavioral change in patients participating in diabetes education programs.²⁹

This instrument is proposed as a tool for the evaluation of educational content in diabetes, allowing the evaluation of aspects that facilitate the improvement of knowledge in diabetes and to promote action for diabetes self-care. It has been reported the usefulness of the use of ICTs to carry out disease care actions and a better communication with the health professional.^30,31

The educational program topics of the present study, only the dimensions of comprehension and motivation were rated as low. However, the rating of the educational content was overall high. The ICTs to provide education or monitoring of the patient with diabetes, should keep on being assessed, considering culture, race, literacy and socioeconomic levels.³²

The limitations of the present study were that the evaluation of the experts was done with a specific topic according to their clinical training, so that they could think that some topic was not included when it was reviewed by another expert. One limitation is that the educational material was evaluated only by the consensus of experts, but we insist that in future studies the level of acceptance in patients with diabetes should measure. In addition, it is important to measure the perception of quality of the web site by the patient with diabetes in terms of motivation to use, ease of use, comprehension, and attraction, which ultimately defines the usability and acceptance of the educational web site. Diabetes education is a substantial and important component of treatment, in this way the patient can adopt a better lifestyle and to acquire self-management skills. The use of educational web sites or mobile applications can facilitate the education to reach a larger population and can be provided according to the personal schedule of patients.

It is necessary for any educational program to be evaluated by diabetes experts, this will allow evaluation of the understanding and usefulness of the information included in these diabetes education programs. Thus, it is very important to highlight the clinical implication of having validated instruments for health professionals with the objective of evaluating the educational content aimed at the use of. It is also important to evaluate the usability of these educational sites or mobile applications for the end users, who are patients with diabetes.

Conclusion

The validation by expert consensus of an educational program to be implemented on a web site, obtained a high or good score. This validation provides the certainty of implementing a web-based educational strategy validated by health professionals.

The instrument designed to evaluate the proposed educational material had a high reliability. This is proposed as a tool to evaluate educational programs, which aim to implement educational strategies for patients with diabetes as well as other chronic diseases.

Conflicts of interest

The authors declare that they have no conflicts of interest.

Acknowledgements

The authors are grateful with the expert consensus made up by psychologists, diabetes educators, nutritionists, endocrinologists, family physicians, and dentists, for the evaluation of the educational program in diabetes.