metricas
covid
Buscar en
Journal of Healthcare Quality Research
Toda la web
Inicio Journal of Healthcare Quality Research Investigation the success rate of hospital information system (HIS): Development...
Journal Information
Vol. 36. Issue 2.
Pages 103-112 (March - April 2021)
Share
Share
Download PDF
More article options
Visits
580
Vol. 36. Issue 2.
Pages 103-112 (March - April 2021)
Original Article
Full text access
Investigation the success rate of hospital information system (HIS): Development of a questionnaire and case study
Estudio de la tasa de éxito del sistema de información hospitalaria: desarrollo de un cuestionario y estudio de un caso
Visits
580
Z. Ebnehoseinia, M. Jangib, M. Tarac, H. Tabeshc,
Corresponding author
TabeshH@mums.ac.ir

Corresponding author.
a Medical Informatics, Psychiatry and Behavioral Sciences Research Center, Mashhad University of Medical Sciences, Mashhad, Iran
b Health Information Technology Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
c Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (4)
Show moreShow less
Tables (3)
Table 1. Participants’ characteristics.
Table 2. Results of factor analysis.
Table 3. Mean of hospital information system (HIS) success rate based on Information System Success Model (ISSM).
Show moreShow less
Additional material (1)
Abstract
Introduction and objectives

The implantation of hospital information systems (HISs) has grown dramatically in recent years. Understanding the success rate of HIS is key in health organizations. In this study, a validated questionnaire for HISs evaluation based on the Information System Success Model (ISSM) has been provided. In addition, the HIS success rate was determined.

Materials and methods

The current study was conducted at one of the largest teaching hospitals in eastern Iran. The 44-items questionnaire was developed for data gathering. The questionnaire covered ISSM dimensions, which include analysis of quality system, quality of information, quality service, system use, usefulness, satisfaction, and net benefits. Content validity, constructs validity, and reliability of the ISSM questionnaire was measured. HIS success rate has been determined and categorized based on users’ perspective as follows: appropriate (75%HIS success rate), moderate (50%HIS success rate<75%), low coverage (25%HIS success rate<50%), and poor (coverage rate<25%).

Results

In total, 253 users participated in the study. The ISSM questionnaire was validated by an expert panel with CVI: 85.12% and CVR: 88.22%. The overall Cronbach's alpha value of the instrument was determined as 92.2%. Nine factors with eigenvalues greater than 1.00 were identified, jointly accounting for 66.91% of the total variance. The value of KMO was.866 showed that the sample size was adequate for factor analysis. The highly significant Bartlett's test (p<0.000) indicated that variables were correlated and the factor analysis was appropriate. Our results demonstrated that the total mean of HIS success was “moderate” base on the users’ point of view.

Conclusions

The findings of the current study, provide valuable scientific evidence for key affecting factors on hospital EHR in Iran as a developing country.

Keywords:
Health information systems
Electronic health records
Information theory
Resumen
Introducción y objetivos

La implantación de los sistemas de información hospitalaria (HIS) ha crecido drásticamente en los últimos años. Comprender la tasa de éxito de un HIS es clave para las organizaciones sanitarias. En este estudio se aportó un cuestionario validado para la evaluación de HIS, basado en el Information System Success Model (ISSM). Además, se calculó la tasa de éxito del HIS.

Materiales y métodos

El estudio actual se llevó a cabo en uno de los mayores hospitales docentes del este de Irán. Se desarrolló un cuestionario de 44 ítems para recabar los datos. Dicho cuestionario cubrió las dimensiones del ISSM, que incluyen el análisis del sistema de calidad, la calidad de la información, la calidad del servicio, así como el uso, utilidad, satisfacción y beneficios netos del sistema. Se midieron la validez del contenido, la validez de los constructos y la fiabilidad del cuestionario ISSM. Se determinó y categorizó la tasa de éxito del HIS sobre la base de la perspectiva de los usuarios, de la manera siguiente: adecuada (75%≤ tasa de éxito de HIS), moderada (50%≤ tasa de éxito de HIS<75%), baja cobertura (25%≤ tasa de éxito de HIS<50%), y mala (tasa de cobertura <25%).

Resultados

En total participaron 253 usuarios en el estudio. El cuestionario ISSM fue validado por un panel de expertos, donde CVI=85,12% y CVR=88,22%. El valor general de alfa de Cronbach del instrumento fue del 92,2%. Se identificaron nueve factores con Eigen valores superiores a 1, que representaron en conjunto el 66,91% de la varianza total. El valor de KMO fue de 0,866, lo cual reflejó que el tamaño de la muestra fue adecuado para el análisis factorial. La prueba de Bartlett altamente significativa (p<0,000) indicó que las variables estaban correlacionadas, y que el análisis factorial era adecuado. Nuestros resultados demostraron que la media total del éxito de HIS era «moderada», basándose en el punto de vista del usuario.

Conclusiones

Los hallazgos del presente estudio aportan evidencia científica valiosa para los factores clave que afectan a los registros sanitarios electrónicos (EHR) en Irán, como país en desarrollo.

Palabras clave:
Sistemas de información sanitaria
Registros sanitarios electrónicos
Teoría de la información
Full Text
Introduction

In the past several decades, hospital information systems (HISs) have played an important role in health care settings. The HISs has been widely adopted in hospitals. Evaluating the success rate of health information systems to help decision-makers obtain information about the impact of these systems, therefore, system evaluation becomes an important topic to all organizations that aim to implement any health information systems.1

System success refers to the adoption and usage of an information system by a large number of users on a continued basis.2 Success in health information systems refers to “the usefulness of the information system for users and healthcare managers”.3

Information System Success Model (ISSM) is one of the well known and valid models to assess the success rate of information systems. ISSM was developed by DeLone and Mclean in 1990 and was subsequently updated in 2003. ISSM is encompassed of seven dimensions including system quality, information quality, service quality system use, intention to use, satisfaction, and net-benefits.4

ISSM was adopted and modified in many of health care studies for HIS evaluation. For example in the studies by Nunes et al.,5 Ojo,6 Petter,7 Ribière,8 and Yu et al.9 HIS was evaluated using original or a modified version of ISSM. The original version of ISSM is a conceptual framework and does not provide a questionnaire for evaluating the HISs.10 As well, ISSM is a causal mole. The overall system success depends on the interaction of seven dimensions of ISSM.4 In plenty of studies, the most important influencing factors on HISs success were determined based on ISSM.5,11,12 In several studies, the relation between the dimensions of ISSM was examined or ISSM was modified and validated.6,8,9,13,14 For example in a study by Nunes et al.5 influencing factors on satisfaction with the HIS in Brazil was assessed. As far as we know, few studies have focused on suggesting a validated questionnaire for HISs evaluation and investigating of HIS success rate.

The aim of the current study was to create a validated questionnaire for HISs evaluation based on the ISSM model. Secondary aim was to determine and categorize the HIS success rate in a large teaching hospital based on the users’ perspective.

MethodsInstrument

In the current study, a self-administrated structured questionnaire was developed based on ISSM for data gathering. To ensure validity, questions assessing each dimension were adapted from the validated questionnaires of the previous studies.

The questionnaire was divided into two sections. The first section was included sociodemographic information of the participant users. The second section was comprised of 44 questions that covered seven dimensions of ISSM dimensions. A 5-point Likert scale ranging from “completely disagree” to “completely satisfy” was mainly used. The seven dimensions and number of questions in each dimension were as follows: system quality (n=8)12,15–19 information quality (n=11),12,17,20–22 service quality (n=8),23–26 system use (n=1),15 intention to use (n=2),15 satisfaction (n=3), and net benefits (n=11).10,12,20,25,27–30Appendix A demonstrates the questionnaire. The dimension of system quality consisted of three evaluation measures including availability, reliability, and usability. The dimension of system quality, information quality, and service quality encompassed several evaluation measures.

Content validity ISSM questionnaire was measured using a Content Validity Index (CVI) and Content Validity Ratio (CVR) by an expert panel (ten medical informatics specialists).

Construct validity was carried out using factor analysis. Factor analysis was performed based on the principal component analysis (PCA) technique to explore important and appropriate factors within each dimension. The Kaiser-Meyer-Olkin (KMO) was applied to measure sampling adequacy, and Bartlett's Test of Sphericity was used to assess whether factor analysis is appropriate. Varimax rotation was performed to facilitate the identification of uncorrelated factors. The number of factors to be retained was based on eigenvalues>1, with factor loading serving as an indicator of the degree to which each item was associated with the factor. For each of the remained factors, the number of items making a significant contribution was determined by considering a factor loading of the threshold of 0.400 or greater. The reliability of the questionnaire was measured by Cronbach's alpha.

Study settings

The study conducted in a large teaching hospital. The case hospital is the largest teaching hospital in eastern Iran, which has been in operation for more than 70 years. The hospital consists of 29 specialized and sub-specialized departments such as gynecology, midwifery, poisoning, cardiac surgery, ICU, and CCU.

The HIS of the case hospital covers subsystems for different services including admission, discharge, and transfer (ADT), outpatient clinics, pharmacy, nursing, laboratory, radiology, accounting, and insurance.

Participants

All HIS users who worked in administrative and financial departments, clinical departments, laboratory, pharmacy, radiology, social security, nutrition, and physiotherapy were invited to participate in the current study. Given, the high number of users in the clinical departments (n=1450), 10 percent of them were considered as sample size.

Data collection

The current study was a part of the doctorate thesis (research plan code: 951350) and supported by Mashhad University of Medical Sciences. The study was reviewed and approved by the ethical committee of Mashhad University of Medical Sciences. The researches met all HISs’ users in person and asked them to participate in the study. Users who agreed to participate in this study completed the questionnaires. Participation in the study was voluntary. Participants were free to discontinue completing the questionnaire at any time.

Measurement of HIS success rate

HIS success rate in the case hospital was calculated in three steps. First, the rate of HIS success based on the point of view of each user was determined for evaluation measures, dimensions, and total dimensions of the ISSM based questionnaire. In the second step, the mean and confidence interval of the HIS success rate was calculated for evaluation measures, dimensions, and total dimensions of the ISSM based questionnaire by all groups of users. These groups were financial and administrative departments, clinical departments, laboratory, pharmacy, radiology, and nutrition and physiotherapy. For more details about the measurement of the HIS success rate, see our previous paper.31

In the third step, the mean of the HIS success rate was categorized as follows: 1 – Appropriate (75%HIS success rate), 2 – Moderate (50%HIS success rate<75%), 3 – Low coverage (25%HIS success rate<50%), and 4 – Poor (coverage rate<25%).

Statistical analysis

Sociodemographic variables were summarized using frequencies and proportions. Missing data were removed from the analysis. As well, negative statements were reversed so that higher values indicated more score.

To compare the mean of HIS success rate among users’ characteristics the normality of data in seven dimensions of ISSM was assessed. The Shapiro–Wilk test was performed to assess the normality of data. Mean comparisons of the HIS success rate were made among seven user groups and other users’ characteristics using ANOVA for normal variables and Kruskal–Wallis test for non-normal variables. The Tukey Honestly Significant Difference (HSD) test for normal variables and the Mann–Whitney test for non-normal variables was performed to identify significant inter-group differences. Statistical significance for all of the analyses was defined as p0.05. Data analysis was performed using SPSS, version 21 statistical software.

ResultsParticipants

A total of 300 questionnaires were distributed to all of the target users. Finally, 253 valid questionnaires were collected. (Response rate %84.3). Response rates were very high across the departments in the case hospital. Table 1 shows the characteristics of participants.

Table 1.

Participants’ characteristics.

Participants’ characteristics  Sub groups  Frequency (%) 
DepartmentsAdministrative and financial  44 (17.4) 
Clinical  148 (58.5) 
Laboratory  28 (11.1) 
Pharmacy  5 (2) 
Radiology  13 (5.1) 
Social security  10 (4) 
Nutrition – physiotherapy  5 (2) 
SexMale  94 (37.2) 
Female  159 (62.8) 
Age (years)20–29  54 (21) 
30–39  123 (47) 
40–49  72 (28) 
=>50  4 (1.6) 
Education levelHigh School Diploma  15 (5.8) 
Associate's  22 (8.6) 
Bachelor's  192 (74.7) 
Master's  24 (9.3) 
Work experience (years)<4  54 (24.1) 
5–9  39 (17.4) 
10–14  52 (23.2) 
15–19  46 (20.5) 
20–24  21 (9.4) 
≥25  12 (5.4) 
ICDL certificationNo  30 (11.9) 
Yes  215 (85) 
Computer experience (years)≤1  23 (9.1) 
23 (9.1) 
11 (4.3) 
26 (10.3) 
≥5  170 (67.2) 
HIS experience (years)≤1  10 (4) 
7 (2.8) 
14 (5.5) 
11 (4.3) 
≥5  211 (83.4) 
Reliability and validity

The ISSM questionnaire was validated by an expert panel with CVI: 85.12% and CVR: 88.22%. The overall Cronbach's alpha value of the instrument was determined as 92.2%, demonstrating high reliability. The value of Cronbach's alpha was very high among all dimensions of the questionnaire and the range of value of Cronbach's alpha was 70.2% to 93%. Since the dimension of “system use” had only one question, Cronbach's alpha for this dimension was not measured. Nine factors with eigenvalues greater than 1.00 were identified, jointly accounting for 66.91% of the total variance. The value of KMO was.866 showed that the sample size was adequate for factor analysis. The highly significant Bartlett's test (p<0.000) indicated that variables were correlated and the factor analysis was appropriate. “The range” of factor loadings was 0.496 to 0.909. Out of a total of 44 questions, the factor loadings of three questions were lower than 0.3 that were excluded from the questionnaire. The questions were as follows: “interacting with the HIS does not require a lot of your mental effort”, “privileges required to access the HIS restrict accessibility to necessary patient information for daily tasks”, and “you have received appropriate levels of training; so that you can employ HIS features and procedures for your daily tasks”. These questions belonged to dimensions of system quality, information system, and service quality, respectively. The factors 1–2–3 were labeled availability, reliability, and usability. These three factors considered as a dimension of system quality. The factors of 4–9 were labeled according to the ISSM dimensions that were as follows: information quality, service quality, intention to use, system use, satisfaction, and net benefits. Table 2 shows the value of Cronbach's alpha for each factor and the factor loadings of questions.

Table 2.

Results of factor analysis.

Dimensions  Factors  (Cronbach's alpha %)  Questions (factor loading) 
System quality(70.2)  Availability1 (0.724), availability2 (0.635). 
(79.4)  Reliability1 (0.852), reliability2 (0.835). 
(74.7)  Usability1 (0.743), usability2 (0.792), usability3 (0.759), usability4 (-0.042)*. 
Information quality  (72.3)  Security1 (0.645), security2 (-0.006)*, ease of understanding1 (0.703), ease of understanding2 (0.627), ease of understanding3 (0.515), ease of understanding4 (0.689), ease of understanding5 (0.549), completeness1 (0.7), completeness2 (0.695), acurracy1 (0.509), adaptability1 (0.065). 
Service quality  (89.2)  Responsiveness1 (0.645), responsiveness2 (0.734), assurance1 (0.752), assurance2 (0.724), empathy1 (0.761), empathy2 (0.715), empathy3 (0.496), education1 (0.64), education2 (-0.002)*. 
Intention to use  (90.6)  Intention to use 1 (0.89), intention to use 2 (0.909). 
System use    use1 (0.766). 
Satisfaction  (77.8)  satisfaction1 (0.5), satisfaction2 (0.427). 
Net benefits  (93.0)  Net benefits1 (0.802), net benefits2 (0.768), net benefits3 (0.781), net benefits4 (0.703), net benefits5 (0.722), net benefits6 (0.715), net benefits7 (0.738), net benefits8 (0.639), net benefits9 (0.73), net benefits10 (0.695), net benefits11 (0.71). 

Note: * The excluded questions from the questionnaire (factor loadings lower than 0.3).

HIS success rate

On average, the “total HIS success rate” was 63.83% (CI: 62.65%, 65.02%) that were categorized in the “moderate” group. The range of the HIS success rate was 51.15% to 73.86%. The case HIS acquired highest rate in system quality dimension. The dimension of information quality had the lowest rates of HIS success. Table 3 presents the mean success rates for all HIS evaluation dimensions based on the users’ point of view. Fig. 1 shows the total HIS success rate in departments of the case hospital.

Table 3.

Mean of hospital information system (HIS) success rate based on Information System Success Model (ISSM).

Dimensions  Evaluation measures  Mean (95% confidence interval)  HIS success categories* 
System qualityAvailability  58.93 (56.57, 61.29)  Moderate 
Reliability  67.00 (64.76, 69.22)  Moderate 
Usability  72.83 (71.04, 74.62)  Moderate 
Total  73.86 (72.35, 75.37)  Moderate 
Information qualitySecurity  21.74 (20.99, 22.49)  Poor 
Ease of understanding  79.59 (78.13, 81.04)  Appropriate 
Completeness  42.58 (41.29, 43.87)  Low 
Accuracy  22.79 (22.11, 23.48)  Poor 
Compatibility  47.04 (44.26, 49.81)  Low 
Total  51.15 (50.05, 52.24)  Moderate 
Service qualityResponsiveness  59.80 (57.65, 61.96)  Moderate 
Assurance  61.15 (58.97, 63.32)  Moderate 
Empathy  63.29 (61.33, 65.26)  Moderate 
Education  55.42 (53.03, 57.80)  Moderate 
Total  54.42 (52.73, 56.11)  Moderate 
Intention to use68.18 (65.72, 70.64)  Moderate 
System use69.73 (65.83, 73.62)  Moderate 
Satisfaction63.05 (60.83, 65.26)  Moderate 
Net benefits64.48 (62.29, 66.67)  Moderate 
Total HIS success63.83 (62.65, 65.02)  Moderate 

Note: Appropriate (75%HIS success rate), Moderate (50%HIS success rate<75%), Low coverage (25%HIS success rate<50%), and Poor (coverage rate<25%).

Figure 1.

Total HIS success rate.

(0.1MB).
System quality

Based on users’ opinion the dimension of system quality attained the highest mean of HIS success. Fig. 2 shows the mean of HIS success in the factors of system quality dimension. A significant difference between “usability” and the variable of gender was observed (p<0.046). The “usability” in the female was higher than men. The findings of this study showed that there is a significant difference in the mean of HIS success in the “usability” evaluation measure between users in the various departments (p<0.011). The “usability” based on the opinion of users in financial and administrative departments was higher than users in clinical and radiology departments. There was a significant difference between the ICDL certificate and system reliability (p<0.014). Users with ICDL certificate compare to users without ICDL agreed that HIS had lower reliability.

Figure 2.

The mean success rate on the dimension of system quality.

(0.31MB).
Information quality

The dimension of “system quality” the lowest mean of HIS success. Fig. 3 shows the mean of HIS success in various departments in this dimension. Our results showed that there was a reverse relation between the mean of evaluation measure of “completeness” and “education level” (p-value: 0.004). A significant difference was observed between the evaluation measure of “compatibility” and “departments” (p-value: 0.24). The HIS has more compatibility with other computer systems in the financial and administrative departments compare to clinical departments. There was a significant difference between the “ICDL certificate” and “system reliability” (p<0.003). The users with ICDL certificate compare to users without ICDL believed that the HIS had more compatibility with other systems.

Figure 3.

The mean success rate on the dimension of information quality.

(0.28MB).
Service quality

The mean HIS success rate of the “service quality” dimension was ranked 6th. Fig. 4 shows the mean of HIS success in this dimension. A significant relation was observed between the mean of HIS success rate of responsiveness and the variable of “departments” (p-value: 0.019). The mean of HIS success rate of responsiveness in the laboratory department had a significant difference with the financial and administrative department (p-value: 0.009), clinical departments (p<0.001), and nutrition and physiotherapy (p-value: 0.032). Half of the participants in the present study belonged to these three departments. The mean of HIS success rate of responsiveness in the laboratory department was lower than the mentioned departments. There was a positive relationship between the mean of HIS success rate of “responsiveness” and the variables of “age” (p<0.001), “education level” (p-value: 0.008), and “work experience” (p-value: 0.014). As well, there was a significant difference the mean of HIS success rate with the variables of “age” (p-value: 0.008), “education level” (p-value: 0.014), and “work experience” (p-value: 0.019). The results showed that users younger than 40 years old needed more training courses than older users (p-value: 0.010).

Figure 4.

The mean success rate on the dimension of service quality.

(0.29MB).

The mean HIS success in the evaluation measure of “responsiveness” was significantly associated with users’ education level (p-value: 0.008). There was an important difference between the mean of “responsiveness” between users with Master's and High School Diploma (p-value: 0.050) as well as an Associate's degree (p-value: 0.045).

The results showed that there was a significant relationship between the mean of HIS success in the evaluation measure of “assurance” and “work experience”. The mean of assurance in the users with higher “work experience” was larger than users with lower work experience.

Intention to use and system use

The mean of HIS success rate of “intention to use” and “system use” dimensions was ranked 2nd and 3nd, respectively. There was a significant difference between “intention to use” and users’ characteristics. Given, the high mean rate of HIS success in the dimension of “system use”. But, a significant difference between the mean of “system use” in financial and administrative departments and other departments was observed. The mean rate of “system use” in these departments was higher than the others. The lowest mean rate of “system use” belonged to clinical departments. The mean rate of “system use” among users of financial and administrative departments, social security, laboratory, pharmacy, radiology, nutrition and physiotherapy and, clinical departments were 96.13%, 94.00%, 87.14%, 76.00%, 73.85%, 72.00%, and 56.88%, respectively. There was a significant difference between “education level” and “system use” (p<0.011).

There was a significant relation between “system use” and education level ‘(p-value: 0.016). The rate of system use was lower in the Diploma users in compression with Associate's users (p-value: 0.040) as well as Master's users (p-value: 0.05). As well, a positive relationship between “system use” and Master's user and Bachelor's users was observed. Reserves significant relation was observed between “system use” and age. “System use” in older users was lower than younger users.

Satisfaction

Based on users’ experience the mean success rate of the “satisfaction” dimension was 63.05% (CI: 60.83%, 65.26%). The mean HIS success in the evaluation measure of “satisfaction” was not significantly associated with departments in the case study. A positive relationship between satisfaction and education level was observed. There was an important difference between the mean of satisfaction between users with High School Diploma and Associate's degree (p-value: 0.05) as well as a Master's degree (p-value: 0.024). Our results showed that there was a reverse relation between the mean of evaluation measure of “satisfaction” and “ICDL certification” (p-value: 0.012). The users with ICDL certificate comparison with users without ICDL had a higher level of satisfaction.

Net benefits

The “net-benefits” dimension acquired a moderate success rate on users’ viewpoints. A significant difference between the mean success rate of “net-benefits” and users’ characteristics was not observed.

Discussion

In the current study, an ISSM based questionnaire was developed for hospital information systems. The questionnaire had a high degree of validity and reliability. As well, the HIS success rate based on users’ perspective in seven dimensions of ISSM was assessed and categorized. The most important results of the current study will be discussed further in the following paragraphs.

Based on the results of factor analysis nine factors were identified in the ISSM questionnaire. Six factors completely confirmed the original ISSM dimensions. But, three independent factors including availability, reliability, and usability were identified that was considered as a dimension of system quality. The factor of availability refers to “access to the computer resource and infrastructure where and when it is needed”. Reliability is “the dependability and consistency of access and uptime of systems”.24 Usability refers to “the capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfill the specified rang of tasks, within the specified range of environmental scenarios”.32 The current study was conducted in a large teaching hospital in a developing country. Hospitals in developing countries face technological and economic constraints. From our results, it is possible that the factors of availability, reliability, and usability can get different success rates in a hospital. Our results showed that although the case hospital have not sufficient computer resources and computer equipment was subject to frequent unexpected or inconvenient downtimes, the HIS usability had approximately an appropriate success rate. These are important findings in the understanding of the influencing factors on system quality in a developing country. It seems the success rate of availability, reliability, and usability factors may be measured separately. As well, the impact on HIS success of three factors was confirmed in many of the previous studies. The findings of the current study were directly in line with previous findings. For example, the results of a study by Khalifa and Alswailem showed the unavailability of laptop computers and mobile computers to data entry and information retrieval processes at the point of care, system failures, unexpected slowness of HIS, and poor interface are most important influencing factors of HIS success. They believed that this factor can reduce the chance of HIS acceptance by users and success implementation.33

The results of the present of the study showed there was a positive relation between HIS success and users’ characteristics. For example, a significant difference between the success rate of usability and gender as well as users’ department was observed. Also, our results revealed that there was a reverse relation between evaluation measure of completeness and education level. Users with university degrees highlighted that the HIS did not cover the department's workflow and did not offer the information and functions that users need. A similar result was reported by Amin et al.34 The results of this study showed that there are significant differences between different types of users in the quality of HIS interface, quality of HIS function, and quality of HIS performance.34 Nadri et al.35 and Phichitchaisopa et al.36 found out that there is a significant difference between usability and gender. We suggest that HIS developer should pay attention to the important roles of users in HIS success, and the associated characteristics of users when delivering HIS.

System compatibility17 and completeness21,37 are two critical characteristics to effect an individual health care provider's performance and HIS success rate. The participants did not satisfied with the completeness and system compatibility of the case HIS. A significant difference between system compatibility and departments was observed. The results showed that the financial and administrative departments’ workflow has a higher level of HIS coverage in comparison with clinical and laboratory departments in HIS. The results demonstrate two things. First, the case HIS focuses on financial and administrative workflow. Second, HIS has low compatibility with clinical software and equipment. This result ties well with previous studies. The findings of the study by Asadi et al. revealed that information systems in Iran are mainly focused on financial objectives.38 Moghaddasi et al. stated that although more than 80 percent of government hospitals in Iran have HIS, Existing HISs encompasses functions including administrative, financial, admissions, discharge, transfers, para clinical data, and very limited clinical function.39

HIS education acquired the lowest HIS success based on the users’ perspectives. Our results revealed that although the case HIS had been implemented for 20 years, users need to holding HIS training courses. These results confirmed the importance of holding continuous training courses for HIS. A similar pattern of results was obtained in a study by Ajami et al. In this study a hospital information system was evaluated 3 years after implementation. They found out that users still need HIS education.2 The results of a study by Zikos et al. highlighted that many challenges in HIS implementation were directly related to the lack of skilled IT experts and insufficiently trained users.40

In the present study, a reverse relation between HIS education and the variables of age, work experience, and computer experience was observed. The young users with low work experience and computer experience need more HIS education to compare to other users. These are explanations of the reasons for our results. HIS in the case hospital had been implemented 20 years ago. As well, the HIS does not have a built-in user manual. It seems new users with compression older users received lower education training courses and resources. They may be face problems in using the system. Overall these findings are supported by findings reported by Aldosari et al. They found out the variables of gender, age, education level, years of work experience, years of computer experience, and formal computer training were the important factors that had a positive impact on the system acceptance.41 As well, Ajami et al. state that non-trained users fear system changes because they fear would not be able to use the HIS and lose their job.2 So, the hospital should hold regular training courses to educate users on how to use the HIS.

In the study by Jebraeily et al. the quality of IT service was measured in four university hospitals in Iran. The results of this study showed that among all influencing factors of service quality, users had the lowest satisfaction with the responsiveness. Jebraeily et al. believed that the IT staffs in hospitals are responsible for many tasks and many problems are referred to them.42 They stated the high workload led to the low level of responsiveness of IT staff.42 A similar result was observed

In the current study. The mean of HIS success in the evaluation measure of “responsiveness” was lower than 60%. As well, responsiveness had a significant difference in laboratory department comparison with other departments. Laboratory subsystems in the case HIS has been changed dramatically the three months ago that led to a remarkable increase number of laboratory user's requests. An important finding of these results is the perceived importance of the enhancement of HIS support when the system has major changes. Our results provide evidence of the existence of a positive relation between responsiveness and education level. The rate of IT staffs’ responsiveness in the users with Maters’ degree was higher than other users. In a study by Ahmadian et al. was reported that there was a positive relationship between education level and hardware challenges.43 From this evidence, it is clear that there is an important relationship between education, hardware challenges, and IT staffs responsiveness.

Conclusion

Our results indicated the seven of ISSM were found to be applicable to measure the HIS success rate in hospitals. As well, the developed questionnaire provide objective scientific evidence for HIS success rate that can help to judge by comparison HISs success rate in various hospitals.

Conflict of interests

The authors declare that they have no conflict of interest.

Acknowledgments

The current study was a part of the PhD project (research plan code: 951350) and supported by medical informatics department at Mashhad University of Medical Sciences.

Appendix A
Supplementary data

The following are supplementary data to this article:

References
[1]
B. Rahimi, V. Vimarlund.
Methods to evaluate health information systems in healthcare settings: a literature review.
J Med Syst, 31 (2007), pp. 397-432
[2]
S. Ajami, Z. Mohammadi-Bertiani.
Training and its impact on hospital information system (HIS) success.
J Inf Technol Softw Eng, 2 (2012),
[3]
J. Alipour, A. Karimi, S. Ebrahimi, F. Ansari, Y. Mehdipour.
Success or failure of hospital information systems of public hospitals affiliated with Zahedan University of Medical Sciences: a cross sectional study in the Southeast of Iran.
Int J Med Inform, 108 (2017), pp. 49-54
[4]
W.H. Delone, E.R. McLean.
The DeLone and McLean model of information systems success: a ten-year update.
JMIS, 19 (2003), pp. 9-30
[5]
G. Nunes, M. Javier.
Hospital information system satisfaction in brazil: background and moderating effects.
Int J Res Found Hosp Healthc Adminstration, 2 (2014), pp. 1-9
[6]
A.I. Ojo.
Validation of the DeLone and McLean information systems success model.
Healthc Inform Res, 23 (2017), pp. 60-66
[7]
S. Petter, W. DeLone, E. McLean.
Measuring information systems success: models, dimensions, measures, and interrelationships.
Eur J Inf Syst, 17 (2008), pp. 236-263
[8]
V. Ribière, A.J. LaSalle, R. Khorramshahgol, Y. Gousty.
Hospital information systems quality: a customer satisfaction assessment tool.
Proceedings of the 32nd annual Hawaii international conference on systems sciences. 1999. HICSS-32. Abstracts and CD-ROM of full papers,
[9]
P. Yu, S. Qian.
Developing a theoretical model and questionnaire survey instrument to measure the success of electronic health records in residential aged care.
PLOS ONE, 13 (2018), pp. pe0190749
[10]
M.B. Michel-Verkerke, A.M. Hoogeboom.
Evaluation of the USE IT-questionnaire for the evaluation of the adoption of electronic patient records by healthcare professionals.
Methods Inf Med, 52 (2013), pp. 189-198
[11]
J.L. Hsiao, H.C. Chang, R.F. Chen.
A study of factors affecting acceptance of hospital information systems: a nursing perspective.
J Nurs Res, 19 (2011), pp. 150-160
[12]
R.F. Chen, J.L. Hsiao.
An investigation on physicians’ acceptance of hospital information systems: a case study.
Int J Med Inform, 81 (2012), pp. 810-820
[13]
O.G. Otieno, H. Toyama, M. Asonuma, M. Kanai-Pak, K. Naitoh.
Nurses’ views on the use, quality and user satisfaction with electronic medical records: questionnaire development.
J Adv Nurs, 60 (2007), pp. 209-219
[14]
Messeri P, Khan S, Millery M, Campbell A, Merrill J, Shih S, et al. An information systems model of the determinants of electronic health record use. Appl Clin Inform. 2013;4:185-200, https://doi.org/10.4338/aci-2013-01-ra-0005
[15]
V. Venkatesh, M.G. Morris, G.B. Davis, F.D. Davis.
User acceptance of information technology: toward a unified view.
MIS Q, (2003), pp. 425-478
[16]
V. Venkatesh, H. Bala.
Technology acceptance model 3 and a research agenda on interventions.
Decis Sci, 39 (2008), pp. 273-315
[17]
M.I. Mohd Salleh, N. Zakaria, R. Abdullah.
The influence of system quality characteristics on health care providers’ performance: empirical evidence from Malaysia.
J Infect Public Health, 9 (2016), pp. 698-707
[18]
P.J. Hsieh.
Physicians’ acceptance of electronic medical records exchange: an extension of the decomposed TPB model with institutional trust and perceived risk.
Int J Med Inf, 84 (2015), pp. 1-14
[19]
B. Holtz, S. Krein.
Understanding nurse perceptions of a newly implemented electronic medical record system.
J Technol Hum Serv, 29 (2011), pp. 247-262
[20]
D. Garcia-Smith, J.A. Effken.
Development and initial evaluation of the Clinical Information Systems Success Model (CISSM).
Int J Med Inform, 82 (2013), pp. 539-552
[21]
Q. Gan.
Is the adoption of electronic health record system “contagious”?.
Health Policy Technol, 4 (2015), pp. 107-112
[22]
C. Bossen, L.G. Jensen, F.W. Udsen.
Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: approach, results, and success factors.
Int J Med Inf, 82 (2013), pp. 940-953
[23]
M. Top, A. Yilmaz, E. Karabulut, O.G. Otieno, M. Saylam, Bakır, et al.
Validation of a nurses’ views on electronic medical record systems (EMR) questionnaire in Turkish health system.
J Med Syst., 39 (2015), pp. 67
[24]
L.F. Pitt, R.T. Watson, C.B. Kavan.
Service quality: a measure of information systems effectiveness.
MIS Q, (1995), pp. 173-187
[25]
C. Oroviogoicoechea, R. Watson.
A quantitative analysis of the impact of a computerised information system on nurses’ clinical practice using a realistic evaluation framework.
Int J Med Inform, 78 (2009), pp. 839-849
[26]
D.L. Goodhue, R.L. Thompson.
Task-technology fit and individual performance.
MIS Q, (1995), pp. 213-236
[27]
C. Sicotte, G. Pare, K.K. Bini, M.P. Moreault, G. Laverdure.
Virtual organization of hospital medical imaging: a user satisfaction survey.
J Digit Imaging, 23 (2010), pp. 689-700
[28]
K.M. Kuo, C.F. Liu, C.C. Ma.
An investigation of the effect of nurses’ technology readiness on the acceptance of mobile electronic medical record systems.
BMC Med Inform Decis Mak, 13 (2013),
[29]
E.S. Kirkendall, L.M. Goldenhar, J.L. Simon, D.S. Wheeler, S. Andrew Spooner.
Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: expectations and experiences of hospital staff.
Int J Med Inf, 82 (2013), pp. 1037-1045
[30]
M.P. Gagnon, P.K. Talla, D. Simonyan, G. Godin, M. Labrecque, M. Ouimet, et al.
Electronic health record acceptance by physicians: testing an integrated theoretical model.
J Biomed Inform., 48 (2014), pp. 17-27
[31]
Z. Ebnehoseini, H. Tabesh, K. Deldar, S.M. Mostafavi, M. Tara.
Determining the hospital information system (HIS) success rate: development of a new instrument and case study.
Open Access Maced J Med Sci, 7 (2019), pp. 1407-1414
[32]
B. Shackel.
Usability-context, framework, definition, design and evaluation.
Hum Factors Inform Usabil, (1991), pp. 21-37
[33]
M. Khalifa, O. Alswailem.
Hospital information systems (HIS) acceptance and satisfaction: a case study of a tertiary care hospital.
Proc Comput Sci, 63 (2015), pp. 198-204
[34]
I.M. Amin, S.S. Hussein, W. Isa.
Assessing user satisfaction of using hospital information system (HIS) in Malaysia.
People, 12 (2011), pp. 13
[35]
H. Nadri, B. Rahimi, H. Lotfnezhad Afshar, M. Samadbeik, A. Garavand.
Factors affecting acceptance of hospital information systems based on extended technology acceptance model: a case study in three paraclinical departments.
Appl Clin Inform, 9 (2018), pp. 238-247
[36]
N. Phichitchaisopa, T. Naenna.
Factors affecting the adoption of healthcare information technology.
EXCLI J, 12 (2013), pp. 413
[37]
M.M. Yusof, J. Kuljis, A. Papazafeiropoulou, L.K. Stergioulas.
An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit).
Int J Med Inform, 77 (2008), pp. 386-398
[38]
F. Asadi, H. Moghaddasi, R. Rabiei, F. Rahimi, S.J. Mirshekarlou.
The evaluation of SEPAS national project based on electronic health record system (EHRS) coordinates in Iran.
Acta Inform Med, 23 (2015), pp. 369-373
[39]
H. Moghaddasi, A. Mohammadpour, H. Bouraghi, A. Azizi, H. Mazaherilaghab.
Hospital information systems: the status and approaches in selected countries of the Middle East.
Electron Physician, 10 (2018), pp. 6829
[40]
D. Zikos, A. Mitsios, J. Mantas.
Assessment of hospital information systems implementation: a case study.
EFMI-STC, (2011),
[41]
B. Aldosari, S. Al-Mansour, H. Aldosari, A. Alanazi.
Assessment of factors influencing nurses acceptance of electronic medical record in a Saudi Arabia hospital.
Inform Med Unlock, 10 (2018), pp. 82-88
[42]
M. Jebraeily, B. Rahimi, Z. Zare Fazlollahi, H. Lotfnezhad Afshar.
Using SERVQUAL model to assess hospital information system service quality.
Hormozgan Med J, 23 (2019), pp. e86977
[43]
L. Ahmadian, N. Dorosti, R. Khajouei, S.H. Gohari.
Challenges of using hospital information systems by nurses: comparing academic and non-academic hospitals.
Electron Physician, 9 (2017), pp. 4625
Copyright © 2020. FECA
Article options
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos

Quizás le interese:
10.1016/j.jhqr.2022.11.003
No mostrar más