Buscar en
Revista de Contabilidad-Spanish Accounting Review
Toda la web
Inicio Revista de Contabilidad-Spanish Accounting Review Is it worth it to consider videogames in accounting education? A comparison of a...
Información de la revista
Vol. 20. Núm. 2.
Páginas 118-130 (01 Julio 2017)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
2633
Vol. 20. Núm. 2.
Páginas 118-130 (01 Julio 2017)
Open Access
Is it worth it to consider videogames in accounting education? A comparison of a simulation and a videogame in attributes, motivation and learning outcomes
¿Merece la pena considerar los videojuegos en la enseñanza de contabilidad? Comparación de una simulación y un videojuego respecto a atributos, motivación y resultados de aprendizaje
Visitas
2633
Jordi Carenysa, Soledad Moyab, Jordi Perramonc,
Autor para correspondencia
jordi.perramon@bsm.upf.edu

Corresponding author.
a EADA Business School, c/ Aragó 204, 08011 Barcelona, Spain
b ESADE Business School, Avenida de la Torre Blanca, 59, 08172 Barcelona, Spain
c Barcelona School of Management – Universitat Pompeu Fabra, Balmes, 132, 08008 Barcelona, Spain
Este artículo ha recibido

Under a Creative Commons license
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Tablas (8)
Table 1. Game attributes.
Table 2. Comparison of attributes.
Table 3. Gender and programme distribution.
Table 4. Results by variables and components.
Mostrar másMostrar menos
Abstract

The objective of this study is to assess the effectiveness of videogames in comparison to simulations in a higher education environment and with regard to their attributes, motivation, and learning outcomes, as three of the main dimensions that play a role in the effectiveness of digital game-based learning. Results demonstrate significant differences between the attributes and motivation dimensions, while no significant differences were found for the learning outcomes. This would imply that although both instructional tools lead students to the desired level of knowledge acquisition, the motivation generated, together with the set of features provided by the games complement each other, leading to a superior learning experience. These results support the inclusion of videogames as a complement to simulations in higher education accounting and business environments and allow us to propose a blended approach that provides the learner with the ‘best of both worlds’.

Keywords:
Student learning
Student motivation
Learning outcomes
Learning and teaching
Masters degrees
JEL classification:
M40
M10
O31
Resumen

El objetivo de este estudio es evaluar la eficacia de los videojuegos en comparación con las simulaciones en un ambiente de educación superior por lo que respecta a sus atributos, motivación y resultados de aprendizaje como 3 de las principales dimensiones que desempeñan un papel clave en la eficacia del aprendizaje basado en juegos digitales. Los resultados revelan diferencias significativas entre los atributos y las dimensiones de la motivación, si bien no se encontraron diferencias para los resultados de aprendizaje. Esto implicaría que, aunque ambas herramientas instructivas dirigen a los estudiantes al nivel deseado de adquisición de conocimientos, la motivación generada junto con las características que facilitan los juegos se complementan entre sí, lo que permite una experiencia de aprendizaje superior. Estos resultados respaldan la inclusión de los videojuegos como un complemento a las simulaciones en estudios superiores de contabilidad y entornos empresariales y permiten que se proponga un enfoque mixto que provee al aprendiz con «lo mejor de ambos mundos».

Palabras clave:
Aprendizaje de los estudiantes
Motivación de los estudiantes
Resultados de aprendizaje
Aprendizaje y enseñanza
Másteres
Texto completo
Introduction

The enhancement of the use of games in higher education settings is currently promoted by reputable institutions like the Massachusetts Institute of Technology (MIT, 2014), who include as a recommendation in their annual final report the exploration and promotion of game-based learning as part of the extension of new pedagogical approaches. They justify this recommendation with a statement supporting further analysis of the engagement generated by games, which suits the new millennial generation (Prensky, 2007) who are closer to technology and familiar with gamification.

Although the use of games has long been present in higher education classrooms (Faria, Hutchinson, Wellington, & Gold, 2009), there remain many unresolved questions (Carenys & Moya, 2015; Girard, Ecalle, & Magnant, 2012, Henderson, Selwyn, & Aston, 2015) in relation to its effectiveness. In this regard, and although the literature devoted to digital game based learning (DGBL) has developed quite strongly in the last decade, there remains a need for research that answers questions such as: (1) the definition of a conceptual framework for the use and implementation of digital games (Mayer et al., 2014), (2) assessment of the impact of different dimensions of game based learning on effectiveness (Huang, Johnson, & Han, 2013), and (3) the differences evident in the learning process depending on the instructional tool employed (Carenys & Moya, 2015; Faria, 2001).

In this paper, we focus on the effectiveness of DGBL and, in particular, on videogames, as a relatively new instrument for educational use in higher education environments. If we examine the history of game use in higher education, we realize that simulations (Faria, 2001; Faria & Wellington, 2004) have been the most popular instructional tools used thus far. However, videogames have entered the classroom as a complementary tool for learning that may bring additional outcomes, although these outcomes are still in need of validation (Girard et al., 2012).

Regarding the differences between simulations and videogames, it could be said that they sometimes overlap (de Freitas & Oliver, 2006), as there are videogames that include elements of simulations while some simulations are intended to be ‘played with’. Additionally, they may present differences regarding educational potential, as simulations have long been considered a support in education and training (in particular in military training and business and medical education) in comparison to videogames. Therefore, the distinction is often blurred and still lacks a proper taxonomy (Tobias & Fletcher, 2012).

Assessing the effectiveness of digital games lies in the definition and measurement of different dimensions that have been considered in the literature. Previous studies have focused on the attributes of games (Bedwell, Pavlas, Heyne, Lazzara, & Salas, 2012; Wilson et al., 2009), on motivation (Huang, Huang, & Tschopp, 2010; Huang et al., 2013), or on learning outcomes (Ranchhod, Gurau, Loukis, & Trivedi, 2014), some from an individual and some from a multidimensional perspective. In our study, we consider these dimensions relevant and useful for the assessment and comparison of the effectiveness of our chosen tools. The inclusion of the attributes is of particular relevance as, based on the literature, it could be the dimension that discriminates and, therefore, supports, the differences between simulations and videogames.

Therefore, the objective of this study is to assess the effectiveness of videogames in comparison to simulations with regard to their attributes, motivation, and learning outcomes, as three of the main dimensions that play a role in the effectiveness of DGBL. To achieve our objective, we have conducted several experiments in an accounting and business education setting where we introduced a simulation and a videogame that tackle the same concepts but from a very different perspective.

To our knowledge, no prior studies have assessed and compared the effectiveness of simulations and videogames except for the study by Merchant, Goetz, Cifuentes, Kenney-Kennicutt, and Davis (2014) where the authors compared the use of different instructional tools in K.12 and higher education environments. However, their research comprised a meta-analysis that compared studies using either simulations or videogames while our study rests on an experiment that combines both tools in the same environment and with an identical purpose.

Our results demonstrate significant differences between the attributes and motivation dimensions of the videogame compared to the simulation, while no significant differences arise for the learning outcomes. This would imply that although both instructional tools lead students to the desired level of knowledge acquisition, the motivation generated, together with the set of features provided by the games as perceived by students, complement each other, leading to a superior learning experience. These results support then the inclusion of videogames as a complement to simulations in higher education accounting business environments. The rationale for such a statement is that, while in some attributes or motivational dimensions, simulations perform better than videogames, in others, videogames are perceived as superior. Thus, a blended approach could provide the learner with the ‘best of both worlds’.

The remainder of this paper is structured as follows: Section “Literature review, theory, and hypothesis development” deals with the literature review and the research questions, Section “Data and methodology” explains both the simulation and videogames employed together with the data and methodology, section “Results” presents our results, section “Discussion” includes the discussion and, finally, in section “Conclusions” we summarize our conclusion.

Literature review, theory, and hypothesis development

The literature on DGBL has evolved substantially over time (Faria et al., 2009). In a review of the research presented at the Association of Business Simulation and Experiential Learning (ABSEL) for the period 1974–1999, Faria (2001) identified three main strands of research. The first strand focused on key performance factors; that is, studies that examined the relationship between the performance of participants in a game and participant characteristics. The second strand focused on the learning effectiveness of simulations versus other methods, basically lectures and cases. Finally, the third strand was dedicated to the learning outcomes of simulations.

Initially, we build on this classification of research provided by Faria (2001) and reconsidered in more recent literature reviews (Carenys & Moya, 2015; Gosen & Washbush, 2004) to focus on the second strand, which is devoted to the comparison of methods when a new player enters a DGBL scenario for the particular environment of higher education – in this case, videogames. Research on videogames may be lacking because this group of DGBL tools is newer than simulations. Given the growth in the use of videogames and virtual worlds, however, understanding which attributes make them more or less effective than other digital games is an important question for future research. Videogames and simulations likely have significantly different attributes; therefore, their learning outcomes could also be substantially different but perhaps complementary.

In a thorough revision of the accounting and business literature on DGBL (Carenys & Moya, 2015), the authors identify that the games that have been incorporated into higher education classrooms have largely been simulations, and, to a lesser extent, videogames.1 However, videogames and simulations (as will be developed later) may have different characteristics and videogames, therefore, may provide students with a new and different learning experience that can be added to current methodologies and games.

One of the challenges facing researchers and educators in DGBL is the wide spectrum of concepts and definitions linked with different learning tools, as well as the lack of a proper taxonomy with which to classify them (Tobias & Fletcher, 2012). Carenys and Moya (2015), for example, suggest that the DGBL literature has used, sometimes interchangeably, the terms simulation, business game, business simulation game, or videogame. Although it is beyond the scope of this research to establish a consensus on the defining features of previous digital instructional tools, for the purpose of this study, we will conceptualize simulations as ‘instruction delivered via personal computer that immerses trainees in a decision-making exercise in an artificial environment in order to learn the consequences of their decisions’ (Sitzmann, 2011).

Additionally, we propose that given the current state of DGBL literature, there are no clearly defined boundaries between simulations and videogames, and that the distinctions between them are blurred (de Freitas, 2006). We suggest that the contrast between simulations and videogames rests more on gradual or incremental dissimilarities in their attributes (see Table 2) than on categorical differences. For instance, most videogames will be based on much more complex and elaborate storylines than simulations and, typically, videogames will include higher levels of fantasy and detachment from reality.

Table 2.

Comparison of attributes.

  Simulation  Videogame 
Challenge  The instructor can customize the level of difficulty (e.g. the amount of the credit line available, accessibility or not of other financing sources). This feature makes it possible to adjust the challenge of the activity to the learners’ starting point and potential capacity.  The videogame includes three pre-defined levels of difficulty.
Starting with the introductory level might help to mitigate the possible frustration of the learners in the early stage of play. 
Competition  Participants compete individually against the game system, as they devote their attention to optimizing their own performance.
The game permits ranking of participants’ performance. 
Participants compete individually against the game system, as the aim of the game is to achieve the best possible results against the environment.
No formal ranking of participants is available. 
Rules  Includes a short embedded video (5min approx.) to explain the rules of the game. It also includes a downloadable PDF file containing a summary of the rules. The game is structured around decisions over a nine-year period, divided into three sub-periods of three years each. Each sub-period represents one run of the game. Participants can stop the simulation to evaluate and assess their decision, so this is a turn-based simulation. Time required to play is 90min approximately.  Includes a short embedded video (5min approx.) to explain the main rules of the game. The game can be played throughout an unlimited number of periods, each period representing one month. However, participants are under permanent time pressure, as once they start to play the game, it is only possible to slow it down, but not to stop it. Therefore, the time required to evaluate decisions is very tight and limited. Consequently, this is closer to a real-time based game. Time required to play is 60min approximately. 
Goal oriented task  Maximize company value while remaining solvent. Goal not explicit in the game. No sub-goals are defined.  Goal is explicit in the game. Maximize company value and accumulate a minimum cash surplus. The videogame includes several sub-goals that can be defined as building blocks to achieving the ultimate goal (e.g. moving from cash to credit transactions or adding one shift to the factory). 
Fantasy  Presents a realistic scenario. The simulation places the participant in a company that retails nutritional products.  Presents a fantasy scenario. The videogame places the participant in a zapper's factory in an outer space planet, completely detached from reality. 
Telling stories  Participants face the situation of a manager coping with a company that is about to go bankrupt and must make the right decisions to turn the company around.  Participants must build up, in a limited timeframe, sufficient cash to upgrade the zapper's factory; otherwise, they will be ejected from the game. 
Engagement  The simulation generated a high level of engagement. During the playtime, participants were totally immersed in the game, and after the experiment, many of them asked for the opportunity to play it again.  The videogame also generated a high level of engagement. From anecdotal evidence, some students reported playing the videogame on their own for hours after the experiment. 
Autonomy/control  Limited autonomy, as participants’ decisions are limited to evaluating, accepting, or foregoing the pre-defined alternative decisions in each turn of the game.  Wider autonomy and control, as participants can choose to pursue a large number of different procurement/manufacturing/credit strategies to accumulate the required cash. 
Multimedia representation  Uses a professional and business like interface, based on data and information structured around three tabs (prepare, analyze, decide) displayed in the form of tables, graphs, and downloadable financial projections.  Uses a friendly and intuitive interface, based on a graphic adventure, music, and sound. 
Feedback  Feedback on performance is shown in the form of financial statements and financial ratios.  Feedback on performance is shown in the form of financial statements, tables, and graphs. The videogame also displays as pop-ups, animated cartoons with sound and voice, chastising, encouraging or congratulating the learner for his/her performance. 
Transfer of skills  Its realistic setting together with a professional and business-like appearance, facilitates understanding of how tactical and strategic decisions related to working capital can affect the firm's financial position.  The videogame aims to develop short-term financial management skills, as participants are exposed to decision making to increase revenues (and profits) as quickly as possible without running out of cash. 

Assessing the effectiveness of a given learning tool relies on different dimensions that should be considered when assessing its validity (Huang et al., 2013; Montagud & Gandía, 2014). For the particular case of digital games, the dimensions that have been considered relevant in previous literature are attributes, motivation, and learning outcomes (Bayart, Bertezene,Vallat, & Martin, 2014; Garris, Ahlers, & Driskell, 2002; Ranchhod et al., 2014; Wilson et al., 2009).

Attributes may be defined (Yusoff, Crowder, Gilbert, & Wills, 2009) as ‘the aspects of a game that support learning and engagement and that are based on the exploration of the established theories of learning and motivation’. Attributes are therefore essential variables in any conceptual framework designed for the assessment of effectiveness. Garris et al. (2002) refer in their study to game attributes (often also referred to as ‘characteristics’ or ‘features’ in the literature) and their relationship to learning outcomes. The authors present an input–process–output model of instructional games with three main steps. In the first step, the instructional programme is designed taking into consideration instructional content and several attributes of the game. In the second step, the attributes promote a cycle that includes behaviour, judgement, and feedback. If these two steps are properly developed and the content is successfully paired with the attributes, the cycle generates recurring and self-motivated game play. In the third step, this engagement in game play leads to the achievement of training objectives and specific learning outcomes. Based on their literature review, the attributes selected are fantasy, rules and goals, sensory stimuli, challenge, mystery, and control.

Wilson et al. (2009) and Bedwell et al. (2012) both similarly attempt to determine the relationship between attributes and learning outcomes. They select a set of attributes based on a review of the literature. In the former paper, the authors refer to adaptation, assessment, challenge, conflict, control, fantasy, interaction, language communications, location, mystery, pieces or players, progress and surprise, representation, rules and goals, safety, and sensory stimuli. They propose a number of propositions by which different attributes are expected to be related to different learning outcomes. Meanwhile, Bedwell et al. (2012) argue that a list of 16 attributes is not easily manageable and that some attributes may overlap; thus, they propose attribute categories based on the card sorts procedure performed by subject matter experts. The final list comprises nine categories, each with associated attributes.

More recently, we find some multidimensional studies that also incorporate attributes (Huang et al., 2013), basing their selection on previous literature. Following the same methodology, we define a list of desired attributes but we include an additional criterion for selection, which is the expected difference between simulations and videogames based on our perception and knowledge of the instructional tools. That is, based on our knowledge of the two tools and the differences between simulations and videogames, we select those attributes that form part of the effectiveness models but, simultaneously, could be perceived as different by students. Our final list of attributes is provided in Table 1.

Table 1.

Game attributes.

Attribute  Definition  Sources 
Challenge  A challenging activity should be perceived as achievable, unpredictable, somewhat vague, and designed to stretch and flex players’ existing knowledge or skill levels. The level of challenge must only minimally exceed learners’ potential competency capacity to overcome the obstacles. Otherwise, this characteristic might frustrate the learners in the early stage of play.  (Csikszentmihalyi, 1975), (Garris et al., 2002), (Malone, 1981), (Malone & Lepper, 1987), (Rieber & Matzko, 2001), (Wilson et al., 2009), (Bedwell et al., 2012), (Huang et al., 2013
Competition  Competition in DGBL can present many forms. Players may compete with themselves, the game system, individual players, or other teams to achieve game objectives.  (Csikszentmihalyi, 1975), (Amory, 2007), (Moreno-Ger et al., 2008), (Rieber & Noah, 2008), (Huang et al., 2013
Rules  Games without rules are meaningless. Rules exemplify problem-solving processes in various forms for learners to follow. Game rules further ensure fair play within the system.  (Garris et al., 2002), (Moreno-Ger et al., 2008), (Westera et al., 2008), (Wilson et al., 2009), (Bedwell et al., 2012), (Calabor et al., 2015
Goal-oriented tasks  The game includes goal-oriented tasks that can be defined as building blocks for players to achieve the winning goal. Incomplete tasks require players to revisit them until the player's performance meets the competency requirement.  (Csikszentmihalyi, 1975), (Malone, 1981), (Bedwell et al., 2012), (Huang et al., 2013), (Calabor et al., 2015
Fantasy  Games can situate players in a world of fantasy that is completely detached from reality. Players can have experiences that are difficult to acquire in the real world and are constantly engaged in the game playing process.  (Garris et al., 2002), (Amory, 2007), (Malone & Lepper, 1987), (Wilson et al., 2009), (Bedwell et al., 2012), (Huang et al., 2013
Telling stories  Games often have distinctive storylines for players to follow. This is often the case in adventure and historical games. Storylines add contextual references and interactional complexity to the game. They help players to relate their personal experiences and common sense to the game goals, tasks, and rules.  (Dickey, 2007), (Moreno-Ger et al., 2008), (Rieber & Noah, 2008; Rieber & Matzko, 2001), (Huang et al., 2013
Engagement  Engagement enables players to immerse themselves cognitively and affectively in the game-based environment. Players consider themselves part of the game. The experience of ‘flow’ is the ultimate outcome of deep engagement in games.  (Csikszentmihalyi, 1975), (Malone, 1981), (Malone & Lepper, 1987), (Moreno-Ger et al., 2008), (Huang et al., 2013
Autonomy/control  Games often allow players to carry out actions autonomously in the process of completing game tasks. Players have great control over what paths to follow in order to solve the problem or perform the task.  (Garris et al., 2002), (Malone & Lepper, 1987), (Azriel, Erthal, & Starr, 2005; Wilson et al., 2009), (Bedwell et al., 2012), (Huang et al., 2013
Multimedia representations  DGBL takes advantage of multimedia representations to embody prior attributes like fantasy, storytelling, or competition. These multimedia representations reduce the cognitive demands on players’ limited capacities and simultaneously develop visual or spatial analysis skills.  (Ang et al., 2007; de Felix & Johnson, 1993; Hays, 2005; Huang et al., 2013
Feedback  Refers to the measurement of achievement during the game. Feedback provides a tool for players to learn from previous actions and adjust accordingly.  (Bedwell et al., 2012), (Azriel et al., 2005), (Wilson et al., 2009), (Calabor et al., 2015
Transfer of skills  After the students have played, they should perceive that what they have learned, i.e. the skills acquired, will be useful for their future work.  (Stainton, Jhonson, & Borodzicz, 2010), (Yusoff et al., 2009), (Huang et al., 2013), (Calabor et al., 2015

Therefore, based on our selected attributes together with the expected differences between simulations and videogames, we pose our first hypothesis.H1

There are no significant differences in the perception of attributes, between the simulation and the videogame.

A second dimension often considered in validating effectiveness is motivation. Motivation can be defined as ‘a theoretical construct used to explain the initiation, direction, intensity, persistence, and quality of behaviour’ (Maehr & Meyer, 1997) and it plays an important role in the satisfaction of the student (Douglas, Douglas, McClelland, & Davies, 2015; Pons, Arquero, & Donoso, 2012). It is multi-dimensional and is generally described as being variable in both level (i.e. the intensity of motivation) and orientation (i.e. the type of motivation; (Ryan & Deci, 2000). When focusing on types of motivation, the literature often relies on the distinction between intrinsic and extrinsic motivation (Calabor, Mora, & Moya, 2015; Garris et al., 2002; Huang et al., 2013; Kapp, 2012). From this perspective, intrinsic motivation refers to the process where a person undertakes an activity for its own sake, for the enjoyment it provides, for the learning it permits, or for the feeling of accomplishment it evokes (Lepper, 1988). Extrinsic motivation, on the other hand, refers to those behaviours that are undertaken to obtain some reward or avoid punishment (Buckley & Doyle, 2014).

Most motivational theories and models display elements of both intrinsic and extrinsic motivation (Kapp, 2012). In education, motivation is considered a key determinant of learning and is used to explain the attention and effort students dedicate to particular learning activities (Brophy, 2013). For this reason, teachers must manage learner motivation, where the objective is to increase motivation levels with a view to engendering positive outcomes, such as increased effort and persistence and enhanced performance (Buckley & Doyle, 2014). For that reason, it can be considered a relevant dimension for our purposes.

If we analyze the motivational theories most widely featured in the DGBL literature, we find the ARCS (Attention, Relevance, Confidence, Satisfaction) model (Huang et al., 2013) and the Integrative Theory of Motivation, Volition, and Performance, or MVP theory (Bulander, 2010; Garris et al., 2002), developed respectively in Keller (1987) and in Keller (2008). The ARCS model bases its development on four different variables: the first is gaining learner attention, the second is the relevance of the target material, the third is participants’ confidence in their capacity to achieve the learning goals, and the fourth, finally, focuses on learner satisfaction in the worth of their effort.

Using the original ARCS model, Keller (2008) more recently proposed the MVP to include learners’ volitional control, cognitive information processing, and final outcome processing (Huang et al., 2013). This theory proposes that the learning process starts with motivational processing, which allows for the definition of goals. At this stage, the ARCS components are incorporated. The output of this first motivational process leads to the next stage, volitional processing, where learners transform their intentions regarding performance into learning actions. The third step is cognitive learning processes, where learners complete learning tasks by interacting with a multimedia learning environment. Finally, in outcome processing, learners have completed the learning task and are ready to assess the effort they have invested and compare it to their learning gain.

Based on the previous paragraphs, we have considered the ARCS model to be the most adequate at the time of analysing students’ perceptions regarding motivation. Furthermore, due to the different characteristics that define simulations and videogames (some of which have already been considered in the attributes dimension), we expect that students’ perceptions in relation to how the game motivates them will be different for each. Thus, we pose our second hypothesis in the following terms:H2

There are no significant differences in students’ perception of motivation between the simulation and the videogame.

A third and also prominent dimension that has been considered in relation to the effectiveness of DGBL is that referred to the learning outcomes. An early classification is that proposed by Hoover and Whitehead (1975), who state that experiential learning occurs when a personally responsible participant cognitively, affectively, and behaviourally processes knowledge, skills, and/or attitudes in a situation of learning characterized by strong active involvement. This definition leads to a very well known classification of learning outcomes often cited in the literature: (a) cognitive learning, (b) behavioural learning, and (c) affective learning. (Faria, 2001) used this classification to make sense of his classification of effective business simulation research.

Following this classification, cognitive learning might be described as the process of developing an understanding of basic concepts and underlying facts so that decision-making can occur (Ranchhod et al., 2014; Vos & Brennan, 2010; Yalabik, Howard, & Roden, 2012). Behavioural learning refers to the process whereby participants incorporate what they have learned into action and make correct decisions or change behaviours based on the content learned. Affective learning, on the other hand, refers to participants’ attitudes towards what they have learned (Ranchhod et al., 2014).

A similar classification, which has been widely used, is that in Kraiger, Ford, and Salas (1993), based on Bloom's taxonomy (Bloom, Englehart, Furst, Hill, & Krathwohl, 1956). The authors try to synthesize previous studies by proposing a classification that includes three broad categories of learning outcome: skill-based, cognitive, and affective. Skill-based outcomes can be identified with behavioural outcomes in the first classification. The DGBL literature (Bedwell et al., 2012; Ranchhod et al., 2014; Wilson et al., 2009) has often built on this classification in both theoretical and empirical research. Some studies focus on specific learning outcomes or only on one of the categories, like, for example, Yalabik et al. (2012) for the cognitive outcome or Fitó, Hernández, and Serradell (2014) for the skills/behavioural outcome, while others approach the learning outcomes dimension from a multidimensional perspective (Ranchhod et al., 2014). Therefore, we pose the following third hypothesis.H3

There are no significant differences in the perception of cognitive learning outcomes between the simulation and the videogame.

Data and methodologyThe choice of games

The digital games selected for this experiment were specifically designed to facilitate participants to learn accounting and financial management principles, in particular working capital management, while running a virtual company. Both games place the participants in the situation of understanding the implications on liquidity and profitability of many operating business decisions and grasping the relevance of a sound management of the working capital needs. They also highlight long term and strategic implications. In order to contrast the hypotheses of this research, two games were selected: a simulation and a videogame.

The simulation used in this research was ‘Working Capital Simulation: Managing growth V.2’, developed by Harvard Business School.2 In this single-player simulation, students act as the CEO of a small company, and decide whether to invest in growth and cash-flow improvement opportunities in three phases over nine simulated years. Each opportunity has a unique financial profile and students must analyze the effects on working capital. Students must understand how the income statement, balance sheet, and statement of cash flows are interconnected and consider the possible effects of each opportunity on the firm's financial position. The company operates on thin margins with a constrained cash position and limited available credit. Students must optimize the use of ‘internal’ and external funds as they balance the desire for growth with the need to maintain liquidity. The simulation aims to develop intuition regarding a firm's cash conversion cycle and to learn about the trade-offs between revenue and EBIT growth while also managing net operating working capital.

The videogame used was ‘Marty Raygun's Fistful of dollars’,3 developed by ViaVivo, where the participants are challenged to make choices about customers, suppliers, and different types of inventory while running a manufacturing business. During the game, participants are exposed, to decision such as selecting among customers (suppliers) who pay (offer) high (low) prices, selecting from the customers (suppliers) who offer to pay on a cash basis (on a credit basis) or the customers that place the biggest orders. ‘Fistful of Dollars’ uses a trial and error approach to problem solving that aims to create pattern recognition skills and financial management intuition. The videogame aims to develop an understanding of how to increase revenues (and profits) as quickly as possible without running out of cash.

These two games were selected because both successfully embody the main features presented in previous literature as attributes of digital games, as presented in Table 1.

Furthermore, the selected games meet other criteria that the researchers found relevant, both from an operational and conceptual level:

  • a)

    Both games fit very well with the academic content of the different courses in which participants in the experiment were enrolled.

  • b)

    Facility of use for the students, so that a very short time was required to master the basic rules of the games, providing high playability in a short time.

  • c)

    Both games are, however, sufficiently complex to avoid students finding quick and easy solutions.

  • d)

    Both games force participants to acquaint themselves with management decision-making processes, regarding objectives and business strategy.

Both games can be played together in one sitting, in this way reducing the risk of high abandonment rates that have been associated with longer gaming experiences (Tao, Cheng, & Sun, 2009).

In Table 2, a comparison of the selected digital games is presented, based on the list of selected attributes considered in our study.

Questionnaire and data collection

Following previous literature on DGBL effectiveness, a questionnaire4 was prepared (Buckley & Doyle, 2014; Chen & Law, 2015; Fitó et al., 2014; Guillén & Aleson, 2012). Firstly, from the analysis of the previous literature on DGBL effectiveness, the research team produced a draft questionnaire. This was then revised and validated by expert simulation instructors, and finally, it was pilot tested with a group of students. The following paragraphs provide more information on this process.

The questionnaire used in this research included four parts (Appendix A includes the final questionnaire). The first section contained questions on the demographic variables of participants, the second section regarded perceived attributes of the games used in this research, the third section regarded motivation, and the final section regarded perceived learning outcomes. The questionnaire used a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly agree).

The questions on attributes were based on a previous instrument used in the literature (Huang et al., 2013). These questions were addressed to identify the participants’ perceptions5 regarding the attributes of the two games. Previous literature has supported the analysis of DGBL effectiveness using learners’ views by arguing that knowledge is not a structure that can be accumulated and that asking the protagonists in the learning process is the best way to assess effectiveness (Ranchhod et al., 2014). Two questions were omitted from the original questionnaire, as they assumed team play, which is not the case for the selected games.

Regarding the motivational aspects, we based our questions on the Instructional Material Motivational Survey (IMMS) derived from the ARCS model (Keller, 1987) to assess learners’ perceived motivational support in Attention, Relevance, Confidence, and Satisfaction. Prior literature in DGBL has adapted the IMMS questionnaire to a DGBL environment. Building on this literature, we employ the ARCS model and the questions adapted from Huang et al. (2013).

Regarding the learning outcomes, these were again assessed based on students’ own perspectives of the learning process, which is the most commonly used alternative found in previous research on the effectiveness of DGBL (Buckless, Krawczyk, & Showalter, 2014; Huebscher & Lendner, 2010; Tao, Chen, & Sung, 2009; Tao, Cheng, & Sun, 2012; Wynder, 2004). The researchers produced an inventory of the learning goals connecting the selected topic (working capital management) with the chosen games.

In order to improve reliability, the questionnaire was revised and validated through in depth personal interviews by three expert simulation instructors. Using the experts’ comments, the researchers decided to omit some questions that were not relevant to the selected games. Finally, the questionnaire was pilot tested on 15 students with very similar profiles to those who were to participate in this study, but that were enrolled in a different course, unrelated to the experiment described here. The questionnaire was then considered perfectly understandable and usable.

Regarding ethical issues, it must be noted that, before any research was conducted, an application for research ethics approval was made at the institution and full research ethics approval was granted for the study. This was a voluntary experiment and students registered for the different sessions freely and gave their consent for their contributions to be used for the research project. Consistent with best practice in research ethics, students were invited to contribute only after all marking aspects had been completed. The research instrument was administered to and collected from students by a third party not involved in the teaching or in the research, and the instruments did not involve asking for names or any identifying information.

Sample

Our sample included students enrolled in an MSc Programme (Finance and Accounting) and an MBA in a Spanish business school. All courses in these programmes are delivered in English. Participants were contacted by the researchers via email, asking them to volunteer in the experiment, and after sending a reminder, they were asked to register for the session. Finally, 132 students agreed to participate and were allocated to one of the four scheduled dates for the experiment. All students enrolled in the experiment had attended different courses on accounting and finance before participating. To control for the effect on the perception of attributes, motivation, and learning outcomes of playing the videogame or simulation first, in two of the sessions, participants were asked to play the simulation first and then the videogame, while in the other two sessions, the order was reversed. Participants were randomly assigned to a specific date. No statistical differences were found between the groups that played the simulation first when compared to the groups that played the videogame first.

All participants were asked to view the instructional videos on the web platforms of the games to understand the basic rules before starting to play. Both games were then tested at the business school IT laboratory and played using the respective web platforms. The length of the session was 3h and playing time was evenly split between simulation and videogame. All participants in the experiment were able to complete both games; thus, this time length was considered appropriate. Finally, they were asked to complete the questionnaire comparing their perceptions on attributes, motivation, and learning outcomes for both the simulation and the videogame. In total, 132 questionnaires were collected, all of which were considered valid.

Table 3 shows the frequency and distribution of our sample in a crosstab showing the distribution of the gender and Master's course of the sample.

Table 3.

Gender and programme distribution.

  Men  Women  Total 
International master in accounting and finance  10  14  24  18% 
International master in finance  35  28  63  48% 
MBA  27  18  45  34% 
Total  75  57  132  100% 
57%  43%  100%   
Data analysis

After collecting data from students for the simulation and videogame experience, we analyzed the surveys and checked for significant differences between the two samples. Our first step was to find out the variables that explained each construct, using Exploratory Factor Analysis. Loading factors and components are shown in Appendix B. In this phase, any item with a loading factor minor than 0.6 was excluded (Loiacono, Watson, & Goodhue, 2002).

Prior literature has used EFA for the summarization of different dimensions of DGBL (Calabor et al., 2015; Huang et al., 2013). EFA is appropriate to summarize the questionnaire variables with minimum loss of information, because the pedagogical effectiveness of the SG may depend on several factors, and, that there are no a priori dependence relationships between them.

The second step was looking for the internal consistency of the data, so we conducted Cronbach's alpha test of reliability. According to Kline (2000) and George and Mallery (2003) the internal consistency of the model depends on the following values: Excellent (>0.9), Good (0.6< >0.7), Poor (0.5< >0.6) and Unacceptable (<0.5). The overall results were satisfactory. Both Kaiser–Meyer–Olkin (KMO) index and Barlett's Sphericity test confirmed that factor analysis was likely to generate satisfactory results as well, according to the parameters of Visauta (1998). The internal consistency of the data is shown in Appendix B.

The last step was to look for significant differences among variables and constructs using a T-test. Results are shown in Table 4.

Table 4.

Results by variables and components.

Variable  Item  Mean Sim.  Mean VG  Sig (p t-test) 
1.3  The game's rules are clearly presented  4.41  4.07  0.000 
1.4  The game's rules are easy to follow  4.41  3.89  0.000 
1.5  The game's goals are clearly presented  4.43  4.36  0.300 
1.6  The game situates me in a fantasy world  2.95  4.16  0.000 
1.8  The game engages me deeply in the playing process  3.93  4.43  0.000 
1.9  The game allows me to full control my actions  3.91  4.26  0.001 
1.11  The game's animations are attractive  3.53  4.42  0.000 
1.12  The game tasks are clearly presented  4.25  4.16  0.329 
1.14  The game provides all information necessary for me during the playing process  4.43  4.16  0.001 
1.15  The game's audio elements are attractive  2.81  3.84  0.000 
1.16  The game provides explanatory feedback on my performance  4.27  3.55  0.000 
1.17  The game provides corrective feedback on my performance  3.84  3.59  0.005 
1.18  I can easily transfer sills I learned from the game to the real world  4.20  3.77  0.000 
1.20  The game provides enough support to help me accomplish the game tasks  4.02  4.11  0.207 
1.21  The game provides enough previews to prepare me for the game playing  3.93  4.20  0.002 
1.22  The game allows me to learn from my mistakes  4.23  4.11  0.188 
1.23  The game keeps me interested throughout the playing process  4.20  4.61  0.000 
1.24  The game is fun to play  4.14  4.82  0.000 
  COMPONENT 1.1  4.22  3.99  0.000 
  COMPONENT 1.2  3.64  4.36  0.000 
2.3  The game was more difficult to understand than I would like for it to be  2.77  2.73  0.603 
2.4  After reading the introductory information, I felt confident that I knew what I was supposed to learn from the game  3.89  3.52  0.001 
2.5  Completing the exercises in the game gave me a satisfying feeling of accomplishment  4.41  4.36  0.529 
2.6  It is clear to me how the content of the game is related to things I already know  4.57  4.50  0.150 
2.10  The quality of the writing in the game helped to hold my attention  4.00  3.86  0.075 
2.11  The content of the game is so abstract that it was hard to keep my attention on it  2.18  2.14  0.529 
2.17  There are explanations of examples of how people use the knowledge in the game  3.28  3.33  0.357 
2.18  The activities in the game were too difficult  2.45  2.43  0.730 
2.22  The content and style of writing in the game convey the impression that its content is worth knowing  4.05  3.57  0.000 
2.23  I learned some things that were surprising or unexpected with the game  3.48  3.55  0.300 
2.27  The style of writing in the game is boring  2.32  2.07  0.006 
2.28  I could relate the content of the game to things I have seen, done or thought about in my own life  4.16  3.93  0.000 
2.30  It felt good to successfully complete the game  4.61  4.52  0.140 
2.31  The content in the game will be useful to me  4.18  4.20  0.693 
2.32  I could not really understand quite a bit of the material in the game  1.91  2.02  0.035 
2.34  It was a pleasure to work on such a well-designed game  4.43  4.41  0.730 
  COMPONENT 2.1  4.39  4.32  0.004 
  COMPONENT 2.2  2.62  2.63  0.807 
  COMPONENT 2.3  3.74  3.56  0.000 
3.1  I learned how working capital management affects cash.  4.32  4.39  0.280 
3.2  I learned how decisions related to other functional areas of the company affect working capital.  4.28  4.26  0.707 
3.3  I understand better the relevance of working capital management.  4.36  4.45  0.045 
  COMPONENT 3.1  4.32  4.37  0.165 
Results

The descriptive statistics of the whole sample and the differences between the simulation and videogame are shown in Table 4 In terms of this study, we consider a difference to be significant when p was less than .05.

Results regarding H1

Our first hypothesis was that there are no significant differences in the perception of attributes between the simulation and the videogame.

After analysing the attributes results, two factors were identified. From the examination of the components obtained following rotation, we observe how factor 1 was loaded with 11 items, including variables like rules, goals, control, tasks clearly presented, information necessary, feedback, transfer skills, enough support, enough preview, and learn from my mistakes. These variables are related to the structure of the game, that is, what are the rules of the game, what steps should participants take, what is the feedback obtained. It could be said that they reflect the “configuration” of the game.

On the other hand, factor 2 was loaded with six variables including items like involvement, fantasy, animation, audio, interest and fun. So, this second factor is closer to the “enjoyment” of the experience.

When we analyze the differences between these two dimensions, we observe how both factors show to be statistically different for the simulation in comparison to the videogame (Table 4). That is, participants perceive that the simulation game and the videogame are significantly different both from the “configuration” perspective and also from the “enjoyment” one.

If we analyze our results at a variable level, we observe that, there are differences in 14 variables (77.78%). These 14 variables correspond to questions related to the following attributes: rules, fantasy, engagement, control, multimedia representations, feedback, and transfer of skills.

Our data show that the participants perceived some differences in the difficulty of the games, in the way that the games prepared them for play, or in the way that the games provided information or feedback. In this sense, participants felt that simulation was easier. Additionally, participants perceived that the simulation facilitated the transfer of skills to the professional world more than the videogame. Therefore, from a game “configuration” point of view (game organization, rules, feedback, and transfer of skills), the participants found significant differences and favoured the simulation in comparison with the videogame.

Regarding the playing time itself, participants noted differences in the way that the games situated them in a fantasy world, in the entertainment experienced when playing the games, and in the way that the games involved them, perceiving that the videogame engaged them deeply in the playing process. In general, participants felt that playing the videogame was more engaging than playing the simulation game. Therefore, from this point of view, the participants favoured the videogame. Finally, and in relation to game appeal (referred to as the animation aspects), our results again demonstrate different perceptions depending on the tool. Specifically, participants experienced significant differences in the attractiveness of the games’ graphics or animation. They felt that playing the videogame was significantly more appealing than playing the simulation game, so they again favoured the videogame in comparison with the simulation.

It is also worth noting that the attributes of both digital games were very favourably evaluated by the participants. In 18 out of 24 variables related to attributes (see Appendix A) the mean values of the simulation and videogame exceed 4, and all the mean values are above 3.

Therefore, we reject H1.

Results regarding H2

Our second hypothesis was that there are no significant differences in the perception of motivation between the simulation and the videogame.

When we analyze the motivation results, we observe that three factors were deemed important. Factor 1 is loaded with six items including variables like satisfaction and accomplishment, feeling good and usefulness. We then denominate this first component as “satisfaction”. Factor 2 is loaded with five items and variables like interest, difficulty, understanding and their perception of feeling confident when playing the game. We then denominate this second component as “confidence”. Finally, Factor 3 is loaded with five items, including explanations, examples clarity and importance, all related with the relevance of the game. We denominate then this third factor as “relevance”.

However, our results show that only factors 1 and 3 present statistically significant differences. That is, participants perceived videogames and simulations as different both from a “satisfaction” and from a “relevance” perception but not from a “confidence” point of view. Our results also show that while the videogame is stronger in satisfaction, the simulation is perceived as better for confidence and relevance (although differences in the former are not statistically significant at a factor level).

At a variable level, we found significant differences in 10 out of the 16 motivational variables (62.5%), related to the participants’ satisfaction and perception of relevance. In general, participants felt more satisfaction and engagement when playing the videogame than the simulation game. In this sense, they significantly enjoyed playing the videogame more than playing the simulation. They also wanted to know more about the topic when they played the videogame.

Regarding confidence, and again according to the participants, the videogame was easier to follow, as the rules of the simulation were more difficult to remember. In fact, students expressed that, when they first examined the game, they also had the impression that the videogame would be easier to follow than the simulation game.

Therefore, we reject H2.

Results regarding H3

Our third hypothesis was that there are no significant differences in perception regarding cognitive learning outcomes between the simulation and the videogame.

When we analyze the learning outcomes results, we observe that there is only one component deemed important and, this component does not show statistically significant differences for the simulation in comparison to the videogame. That is, participants considered the cognitive learning outcome to be the same when they played the videogame and the simulation game. There are no significant differences related to game type.

It emerges that the significant differences detected in the perception of motivation and attributes do not have a significant effect on the perception of learning outcome in the construct level. Thus, our results demonstrate that although there are significant differences between the simulation and the videogame, both are perceived by participants as equally effective in terms of learning outcomes.

Overall, the mean score for the learning outcome perception is 4.35, which is the highest of the model. This means that, regardless of the game selected, students perceived that they learned by means of the digital game experience.

Hence, we accept H3.

Discussion

The present study was designed to assess the effectiveness of videogames in comparison to simulations with regard to attributes, motivation, and learning outcomes, as three of the main dimensions that play a role in the effectiveness of DGBL.

This research found evidence that students perceive the attributes of videogames and simulations differently. We found, by means of factor analysis, that two major factors, “game configuration” and “enjoyment” explain the complex pattern of relationships among variables measuring games’ attributes.

We will firstly discuss “enjoyment”, that participants evaluated more favourably in the videogame than in the simulation. This factor was loaded with variables linked to engagement, fantasy, animation, audio, interest and fun. Previous research has consistently demonstrated the importance of engagement in DGBL, as it enables players to immerse themselves in the game-based environment, with the ultimate goal of achieving the so-called ‘flow state’ (Csikszentmihalyi, 1975; Malone, 1981; Malone & Lepper, 1987). Regarding this “enjoyment” factor, participants perceived that the videogame, engages them more deeply, and generates a more pleasant playing experience when compared with the simulation. Moreover, the videogame maintained participants’ interest more than the simulation throughout the playing process. A likely explanation of the greater “enjoyment” attributed to the videogame is the multimedia resources that it deploys. Graphics, animation, and audio are perceived as far more attractive in the videogame than in the simulation, attributes that previous research (Ang, Zaphiris, & Mahmooh, 2007; Hays, 2005; Huang, Lin, & Huang, 2012) has also reported to reduce the cognitive demands on players’ limited capacities, making the videogame more engaging.

Turning now to “game configuration”, we must first point out that for this factor, the simulation performs better than the videogame. The provision of the information before, during, and after the playing process, clearly favours the simulation. This factor was loaded with variables linked to attributes like rules, goals, control, tasks clearly presented, information necessary, feedback, transfer skills, enough support, enough preview, and learn from my mistakes. Furthermore, the rules of the game are also considered easier to follow in the simulation, while in the videogame, they seem to be learned more on a trial and error basis. A reasonable consequence of this is that the simulation is perceived to give a wider controllability of actions, and control has been reported by a number of authors to be a relevant attribute for assessing the effectiveness of DGBL (Bedwell et al., 2012; Garris et al., 2002; Huang et al., 2013; Malone & Lepper, 1987; Wilson et al., 2009). Moreover, feedback on players’ performance is also perceived as more satisfactory in the simulation. These findings seem to be consistent with other research (Bayart et al., 2014; Garris et al., 2002; Moreno-Ger, Burgos, Sierra, & Fernández-Manjón, 2008; Westera, Nadolski, Hummel, & Wopereis, 2008; Wilson et al., 2009) that accounted for the importance of a clear definition of goals and rules for DGBL to be effective. Finally, the simulation makes it easier to transfer learned skills from the game to the real world, which is consistent with the fact that the videogame storyline occurs in a fantasy world, in contrast with the realistic setting of the simulation.

Taken together, these results suggest that the participants assess very favourably the attributes of both digital games. However, each tool seems to possess some specific attributes that are especially valued. The data reported here support the idea that the more valued attributes of the simulation are related to the establishment of clear rules, the provision of analytic information before, during, and after the playing process, a superior feedback, a large the controllability of actions and a business-like appearance, (“game configuration”). The data also suggest that a larger sensory stimuli due to the extensive use of multimedia resources are perceived as the most distinctive features of the videogame. These attributes keep the player more engaged in the playing process than the simulation (“enjoyment”). This set of findings supports the conceptual premise that blending videogames and simulations could benefit the effectiveness of DGBL by combining the more valued attributes identified in each tool. As pointed out by Kapp (2012), multiple attributes are required to make a game an effective learning experience, and it is the interplay of these elements that makes for the most effective and motivational games.

Turning now to the experimental evidence regarding motivation, the results of this study also confirm differences in motivation between the videogame and the simulation. The ARCS model (Keller, 1987, 2008) offers a comprehensive framework to compare these dissimilarities. The factor analysis conducted in this research bounded together three dimensions of motivation. The first one was loaded with variables as explanations, examples, clarity and relevance (“relevance”). The second factor clumped together variables as understanding, difficulty and perception of feeling confident (“confidence”). Finally, the third factor included variables like satisfaction, accomplishment and feeling good (“satisfaction”).

With regard to “relevance”, defined as the applicability of the learning to the learner's past, present or future knowledge (Keller, 1987), our results demonstrate that the simulation content and style of writing convey the impression that it is more worth knowing it than the videogame. Despite the higher relevance that students attributed to the simulation, the videogame attracted more their attention from the very beginning and seemed to be easier than the simulation. This apparent contradiction may be explained by the fact that the videogame interface design is eye catching while the simulation looks dry and unappealing.

As far as “satisfaction” is concerned, the willingness of the participants to learn more about working capital management is greater for the videogame. This greater readiness to learn more about the topic is likely to be related to the higher level of enjoyment perceived during the playing process for the videogame. With respect to “confidence”, defined by Keller (1987) as the reliance of the learner to achieve the learning goals, the findings suggest that the simulation allows the learners to successfully steer their learning experience more confidently than the videogame. Further, the good organization of the simulation helps the participants to be more confident that they will learn the content (although as mentioned before, that difference was not statistically significant). From all of the above, a relevant finding of this study is that of supporting the inclusion of videogames to complement simulations in environments where additional motivation is required.

Overall, the results have gone some way towards enhancing our understanding of how a blended use of simulation and videogames could produce the ‘best of both worlds’ in terms of increasing motivation throughout the learning process, as each tool demonstrates some specific attributes of the ARCS model that are perceived as superior in terms of their motivational implications.

The results of this experiment confirm that both the simulation and the videogame generate in the participants a favourable perception regarding the achievement of learning outcomes. However, contrary to expectations, no significant difference was found regarding whether one or other tool provides a superior understanding of working capital management. Nevertheless, these results need to be interpreted with caution, as in this study, the evaluation of learning outcomes was limited to a single broad cognitive outcome. It is possible, therefore, that the inclusion in the study of a more detailed inventory of learning goals related to working capital management could produce different results. It may also be the case that if other types of learning outcome (behavioural or affective) were considered in the study, the simulation and videogame could conceivably produce different results. This finding was somehow unexpected, and suggests that more research on this topic needs to be undertaken before we can determine how the different tools contribute to the achievement of different learning outcomes.

Regarding the educational implications of this study, it should also be noted that the benefits of digital games reported in this article are also dependant on the context in which digital games are used. Obviously, some of the features of the context where this study took place are transferable to other settings, but others, probably not. The next paragraphs highlight some of these contextual features that the research team found specially relevant.

Firstly, the participants volunteered to take part in the research and received no compensation for their participation (and with no effect on their grades). This could introduce a bias on the results, as it must be recognized that not everyone is necessarily interested in playing computer games at all. As pointed by Whitton (2010), “there could be a number of reasons for this, including a lack of understanding of what an educational game entails, prior negative experiences with educational games (…) or a feeling that games are frivolous or inappropriate for education”. Consequently, in a different context, making the gaming experience compulsory and widening it to other participants who may perceive digital games in the context of learning as a waste of time, could yield different results. Also related to the profile of the participants, all of them were post-graduated, but still, they were relative young (average age was 26 years), with considerable experience in the use of computers and previously educated in the topic the games were about. Further work could investigate if participants with a different profile, respond similarly to simulations and videogames. It may be the case that elder participants, with less computer literacy and non-familiar with the topic of study, could produce different results.

Secondly, and regarding the organization of the gaming session, several issues must be pointed out. The DGBL experience presented in this article was relatively short (3h). Undoubtedly, this short span of time helped to sustain the engagement and motivation generated and reduced the risk of high abandonment rates that have been associated with longer gaming experiences (Tao et al., 2009). Furthermore, this session was designed to be in a face-to-face teaching context, with the presence of an accounting instructor, as opposite to a fully online context. It can be reasonably assumed that it will be easier for students to get started in such face-to-face context, than if working in a purely online context. In the latter situation they won’t be able to ask for help from the instructor or from other students if they have difficulties, for instance, understanding what they are meant to do with the game. Students finding this type of problems may lose interest and become demotivated. Therefore, ensuring support for students at the start of the games and during its development seems of paramount importance to achieve the expected outcomes of DGBL. Hence, instructors planning to use longer or purely online games, should consider how to back up those students who may have problems with the game.

Finally, regarding environment and technological implications, this experience took place in the business school IT laboratory, where the research team had previously tested both games in terms of suitability of hardware, software and network. Each student had access to a computer with Internet connection. Additionally, during the experiments, an IT technician was present to solve any problem. Without doubt, these actions helped to smooth the gaming experience. However, if on the contrary, students had been asked to work on their own at home, it would have been impossible to ensure that all machines, operating systems and Internet connections were appropriate to use the selected digital games.

In summary, accounting instructors pondering the introduction of digital games in their courses should consider the four areas mentioned above (people involved, organizational issues, the environment of the experience and the technology). These four areas are of particular importance, as they may act as drivers or constraints of the success of the digital learning experience. Depending on the specific combination of these contextual features that each instructor faces, it would be possible to set reasonable expectations on what is sound to await from DGBL.

Conclusions

The most obvious finding to emerge from this study is that of the differences perceived by students for the attributes and motivational effects of simulations and videogames. This indicates that it is worth considering blending simulations and videogames in a single course, as the two tools are complementary in terms of their attributes and motivational effects. While the simulation ensures similarity between the learning situation and a future professional setting, the videogame generates greater engagement. It is possible, therefore, to assume that a course combining videogames and simulations could produce a superior learning experience. The findings from this study also suggest that the choice of using just simulations (or just videogames) in a course or blending them, is likely to affect the evaluation of the learning effectiveness of DGBL. Thus, further research could clarify if different approaches to combine distinct types of digital game yield different learning outcomes.

The practical implications derived from our results are that accounting and business educators can find support for the inclusion of videogames in their courses. Educators considering this option may find, in this study, guidance regarding what they may expect from the introduction of videogames as an additional tool. In particular, educators delivering courses to students who are diverse in terms of their prior knowledge and motivation may find the incorporation of videogames useful in coping with these differences and enhancing the learning experience of all participants. If videogames can attract students to some areas often considered difficult and rough and promote interest, engagement, and, ultimately, a desire for lifelong learning, there is no doubt that considering their inclusion must be worthwhile.

Notwithstanding these contributions, this study is not free of limitations. A first limitation is that the generalisability of the results is subject to a small sample size. Furthermore, the study focuses on the perception of a single cognitive learning outcome and does not include other types of learning outcome identified in previous literature. Consequently, this would be a fruitful area for further research.

Conflict of interests

The authors declare no conflict of interest.

Appendix A
Questionnaire

Questions regarding attributes (adapted from Huang et al., 2013)

Variable  Description 
VAR 1.1  The game is challenging enough for me to play. 
VAR 1.2  The game requires me to compete. 
VAR 1.3  The game's rules are clearly presented. 
VAR 1.4  The game's rules are easy to follow. 
VAR 1.5  The game's goals are clearly presented. 
VAR 1.6  The game situates me in a fantasy world. 
VAR 1.7  The game's storyline is comprehensible. 
VAR 1.8  The game engages me deeply in the playing process. 
VAR 1.9  The game allows me to have full control of my actions. 
VAR 1.10  The game's graphics are attractive. 
VAR 1.11  The game's animations are attractive. 
VAR 1.12  The game tasks are clearly presented. 
VAR 1.13  The game provides all the information necessary for me before the playing process. 
VAR 1.14  The game provides all the information necessary for me during the playing process. 
VAR 1.15  The game's audio elements are attractive. 
VAR 1.16  The game provides explanatory feedback on my performance. 
VAR 1.17  The game provides corrective feedback on my performance. 
VAR 1.18  I can easily transfer skills I have learned from the game to the real world. 
VAR 1.19  The progression of the game task makes sense to me. 
VAR 1.20  The game provides enough support to help me accomplish the game tasks. 
VAR 1.21  The game provides enough previews to prepare me for the game playing. 
VAR 1.22  The game allows me to learn from my mistakes. 
VAR 1.23  The game keeps me interested throughout the playing process. 
VAR 1.24  The game is fun to play. 

Questions regarding motivation (adapted from Huang et al., 2013)

VAR 2.1  When I first looked at the game, I had the impression that it would be easy for me 
VAR 2.2  The was something interesting at the beginning of the game that got my attention 
VAR 2.3  The game was more difficult to understand than I would like for it to be 
VAR 2.4  After reading the introductory information, I felt confident that I knew what I was supposed to learn from the game 
VAR 2.5  Completing the exercises in the game gave me a satisfying feeling of accomplishment 
VAR 2.6  It is clear to me how the content of the game is related to things I already know 
VAR 2.7  The game had so much information that it was hard to pick out and remember the important points 
VAR 2.8  The interface design of the game is eye-catching 
VAR 2.9  Completing activities in the game successfully was important to me 
VAR 2.10  The quality of the writing in the game helped to hold my attention 
VAR 2.11  The content of the game is so abstract that it was hard to keep my attention on it 
VAR 2.12  As I worked on the game, I was confident that I could learn the content 
VAR 2.13  I enjoyed the game so much that I would like to know more about this topic 
VAR 2.14  The design of the game looks dry and unappealing 
VAR 2.15  The content of the game is relevant to my interests 
VAR 2.16  The way the information is arranged in the game helped keep my attention 
VAR 2.17  There are explanations of examples of how people use the knowledge in the game 
VAR 2.18  The activities in the game were too difficult 
VAR 2.19  The game has things that stimulated my curiosity 
VAR 2.20  I really enjoyed learning with the game 
VAR 2.21  The amount of repetition in the game caused me to get bored sometimes 
VAR 2.22  The content and style of writing in the game convey the impression that its content is worth knowing 
VAR 2.23  I learned some things that were surprising or unexpected with the game 
VAR 2.24  After working on the game for a while, I was confident that I would be able to pass a test on the content 
VAR 2.25  The game was not relevant to my needs because I already knew most of it 
VAR 2.26  The variety of reading passages, activities, illustrations… helped keep my attention on the game 
VAR 2.27  The style of writing in the game is boring 
VAR 2.28  I could relate the content of the game to things I have seen, done or thought about in my own life 
VAR 2.29  There are so many words on each game screen/page that is irritating 
VAR 2.30  It felt good to successfully complete the game 
VAR 2.31  The content in the game will be useful to me 
VAR 2.32  I could not really understand quite a bit of the material in the game 
VAR 2.33  The good organization of the content in the game helped me be confident that I would learn this material 
VAR 2.34  It was a pleasure to work on such a well-designed game 

Questions regarding learning outcomes

VAR 3.1  I learned how working capital management affects cash. 
VAR 3.2  I learned how decisions related to other functional areas of the company affect working capital. 
VAR 3.3  I understand better the relevance of working capital management. 

Appendix B
(Loading factors, components and reliability results)

AttributesMotivationLO
Variable  Load  C1  C2  Variable  Load  C1  C2  C3  Variable  Load  C1 
VAR 1.3  0.70    VAR 2.3  0.71      VAR 3.1  0.80 
VAR 1.4  0.86    VAR 2.4  0.81      VAR 3.2  0.84 
VAR 1.5  0.67    VAR 2.5  0.68      VAR 3.3  0.84 
VAR 1.6  0.72    VAR 2.6  0.72           
VAR 1.8  0.67    VAR 2.10  0.87           
VAR 1.9  0.72    VAR 2.11  0.82           
VAR 1.11  0.66    VAR 2.17  0.73           
VAR 1.12  0.72    VAR 2.18  0.75           
VAR 1.14  0.72    VAR 2.22  0.70           
VAR 1.15  0.67    VAR 2.23  0.80           
VAR 1.16  0.68    VAR 2.27  0.82           
VAR 1.17  0.75    VAR 2.28  0.64           
VAR 1.18  0.70    VAR 2.30  0.72           
VAR 1.20  0.76    VAR 2.31  0.76           
VAR 1.21  0.71    VAR 2.32  0.71           
VAR 1.22  0.66    VAR 2.34  0.76           
VAR 1.23  0.77                   
VAR 1.24  0.81                   
KMO  0.709KMO  0.642KMO  0.746
Cronbach's Alpha  0.812Cronbach's Alpha  0.672Cronbach's Alpha  0.891

References
[Amory, 2007]
A. Amory.
Game object model version II: A theoretical model for educational game development.
Educational Technology Research and Development, 55 (2007), pp. 55-77
[Ang et al., 2007]
C.S. Ang, P. Zaphiris, S. Mahmooh.
A model of cognitive loads in massively online role playing games.
Interacting with Computers, 19 (2007), pp. 167-179
[Azriel et al., 2005]
J.A. Azriel, M.J. Erthal, E. Starr.
Answers, questions, and deceptions: What is the role of games in business education?.
Journal of Education for Business, 81 (2005), pp. 9-13
[Bayart et al., 2014]
C. Bayart, S. Bertezene, D. Vallat, J. Martin.
Serious games: Leverage for knowledge management.
TQM Journal, 26 (2014), pp. 235-252
[Bedwell et al., 2012]
W. Bedwell, D. Pavlas, K. Heyne, E. Lazzara, E. Salas.
Toward a taxonomy linking game attributes to learning: An empirical study.
Simulation & Gaming, 43 (2012), pp. 729-760
[Bloom et al., 1956]
B.S. Bloom, N.D. Englehart, E.J. Furst, W.H. Hill, D.R. Krathwohl.
Taxonomy of educational objectives: The classification of educational goals.
Handbook I: Cognitive Domain, David Mc Kay, (1956),
[Brophy, 2013]
J.E. Brophy.
Motivating students to learn.
Routledge, (2013),
[Buckless et al., 2014]
F.A. Buckless, K. Krawczyk, D.S. Showalter.
Using virtual worlds to simulate real-world audit procedures.
Issues in Accounting Education, 29 (2014), pp. 389-417
[Buckley and Doyle, 2014]
P. Buckley, E. Doyle.
Gamification and student motivation. Interactive learning environments, ahead of print.
[Bulander, 2010]
R. Bulander.
A conceptual framework of serious games for higher education-conceptual framework of the game Innov8 to train students in business process modelling: ICE-B 2010.
Paper presented at the Proceedings of the International Conference on e-Business, (2010),
[Calabor et al., 2015]
M. Calabor, A. Mora, S. Moya.
Learning accounting through serious games: Attributes and motivation.
Paper presented at the 38th European Accounting Association Annual Congress, (2015),
[Carenys and Moya, 2015]
J. Carenys, S. Moya.
Digital game based learning in accounting and business education: A literature review.
Paper presented at the BAFA SIG Accounting Education Conference, (2015),
[Chen and Law, 2015]
C. Chen, V. Law.
Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation.
Comput. Hum. Behav., (2015),
(ahead of print)
[Csikszentmihalyi, 1975]
M. Csikszentmihalyi.
Play and intrinsic rewards.
Journal of Humanistic Psychology, 8 (1975), pp. 41-63
[de Felix and Johnson, 1993]
J. de Felix, R. Johnson.
Learning from video games.
Computers in the Schools, 9 (1993), pp. 119-134
[de Freitas, 2006]
S. de Freitas.
Using games and simulations for supporting learning.
Learning, Media and Technology, 31 (2006), pp. 343-358
[de Freitas and Oliver, 2006]
S. de Freitas, M. Oliver.
How can exploratory learning with games and simulations within the curriculum be most effectively evaluated?.
Computers & Education, 46 (2006), pp. 249-264
[Dickey, 2007]
M.D. Dickey.
Game design and learning: A conjectural analysis of how massive multiple online role-playing games (MMORPGs) Foster intrinsic motivation.
Educational Technology Research and Development, 55 (2007), pp. 253-273
[Douglas et al., 2015]
J. Douglas, A. Douglas, R. McClelland, J. Davies.
Understanding student satisfaction and dissatisfaction: An interpretive study in the UK higher education context.
Studies in Higher Education, 40 (2015), pp. 329-349
[Faria et al., 2009]
A.J. Faria, D. Hutchinson, W.J. Wellington, S. Gold.
Developments in business gaming a review of the past 40 years.
Simulation & Gaming, 40 (2009), pp. 464-487
[Faria and Wellington, 2004]
A.J. Faria, W.J. Wellington.
A survey of simulation game users, former-users, and never-users.
Simulation & Gaming, 35 (2004), pp. 178-207
[Faria, 2001]
A.J. Faria.
The changing nature of business simulation/gaming research: A brief history.
Simulation and Gaming, 34 (2001), pp. 97-110
[Fitó et al., 2014]
A. Fitó, A. Hernández, E. Serradell.
Comparing student competences in face-to-face and online business game.
CHB, 30 (2014), pp. 452-459
[Garris et al., 2002]
R. Garris, R. Ahlers, J. Driskell.
Games, motivation, and learning: A research and practice model.
Simulation & Gaming, 33 (2002), pp. 441-467
[George and Mallery, 2003]
D. George, P. Mallery.
SPSS for windows step by step: A simple guide and reference 11.0 update.
4th ed., Allyn & Bacon, (2003),
[Girard et al., 2012]
C. Girard, J. Ecalle, A. Magnant.
Serious games as a new educational tools: How effective are they? A meta-analysis of recent studies.
Journal of Computer Assisted Learning, 29 (2012), pp. 207-219
[Gosen and Washbush, 2004]
J. Gosen, J. Washbush.
A review of scholarship on assessing experiential learning effectiveness.
Simulation & Gaming, 35 (2004), pp. 270-293
[Guillén and Aleson, 2012]
V. Guillén, M. Aleson.
Serious games and learning effectiveness: The case of it's a deal!.
Computers & Education, 58 (2012), pp. 435-448
[Hays, 2005]
R. Hays.
The effectiveness of instructional games: A literature review and discussion.
Naval Air Warfare Center Training System Division, (2005),
[Henderson et al., 2015]
M. Henderson, N. Selwyn, R. Aston.
What works and why? Student perceptions of useful digital technology in university teaching and learning.
Studies in Higher Education, (2015),
[Hoover and Whitehead, 1975]
D. Hoover, C. Whitehead.
An experiential-cognitive methodology in the first course in management: Some preliminary results.
Simulation Games and Experiential Learning in Action, 2 (1975), pp. 23-25
[Huang et al., 2012]
E. Huang, S. Lin, T. Huang.
What type of learning style leads to online participation in the mixed-mode e-learning environment? A study of software usage instruction.
Computers & Education, 58 (2012), pp. 338-349
[Huang et al., 2013]
W.D. Huang, T.E. Johnson, S.H.C. Han.
Impact of online instructional game features on college students’ perceived motivational support and cognitive investment: A structural equation modeling study.
Internet and Higher Education, 17 (2013), pp. 58-68
[Huang et al., 2010]
W.H. Huang, W.Y. Huang, J. Tschopp.
Sustaining iterative game playing processes in DGBL: The relationship between motivational processing and outcome processing.
Computers & Education, 55 (2010), pp. 789-797
[Huebscher and Lendner, 2010]
J. Huebscher, C. Lendner.
Effects of entrepreneurship simulation game seminars on entrepreneurs’ and students’ learning.
Journal of Small Business and Entrepreneurship, 23 (2010), pp. 543-554
[Kapp, 2012]
K.M. Kapp.
The gamification of learning and instruction, game-based methods, and strategies for training and education.
John Wiley & Sons, (2012),
[Keller, 1987]
J. Keller.
Development and use of the ARCS model of instructional design.
Journal of Instructional Development, 10 (1987), pp. 2-10
[Keller, 2008]
J. Keller.
An integrative theory of motivation, volition and performance.
Technology, Instruction, Cognition and Learning, 6 (2008), pp. 79-104
[Kline, 2000]
P. Kline.
The handbook of psychological testing.
2nd ed., Routledge, (2000), pp. 13
[Kraiger et al., 1993]
K. Kraiger, J.K. Ford, E. Salas.
Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation.
Journal of Applied Psychology, 78 (1993), pp. 311-328
[Lepper, 1988]
M.R. Lepper.
Motivational considerations in the study of instruction.
Cognition and Instruction, (1988), pp. 289-309
[Loiacono et al., 2002]
Loiacono, E. T., Watson, R., Goodhue, D. L. (2002) WEBQUAL: A measure of website quality. In: K. Evans & L. Scheer (Eds.), 2002 Marketing educators’ conference: Marketing theory and applications, 13, 432–437.
[Maehr and Meyer, 1997]
M.L. Maehr, H.A. Meyer.
Understanding motivation and schooling: Where we’ve been, where we are and where we need to go.
Educational Psychology Review, 9 (1997), pp. 371-409
[Malone, 1981]
T. Malone.
Toward a theory of intrinsically motivating instruction.
Cognitive Science, 4 (1981), pp. 333-369
[Malone and Lepper, 1987]
T. Malone, M. Lepper.
Making learning fun: A taxonomy of intrinsic motivations for learning.
Aptitude, Learning and Instruction, 3 (1987), pp. 223-253
[Mayer et al., 2014]
I. Mayer, G. Bekebrede, C. Harteveld, H. Warmelink, Q. Zhou, T. van Ruijven, et al.
The research and evaluation of serious games: Toward a comprehensive methodology.
British Journal of Educational Technology, 45 (2014), pp. 502-527
[Merchant et al., 2014]
Z. Merchant, E.T. Goetz, L. Cifuentes, W. Keeney-Kennicutt, T.J. Davis.
Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis.
Computers & Education, 70 (2014), pp. 29-40
[MIT, 2014]
MIT.
Institute-wide Task Force on the future of MIT education. Final Report.
Massachusetts Institute of Technology, (2014),
Available at: http://web.mit.edu/future-report/TaskForceFinal_July28.pdf
[Montagud and Gandía, 2014]
M. Montagud, J. Gandía.
Entorno virtual de aprendizaje y resultados académicos: Evidencia empírica para la enseñanza de la Contabilidad de Gestión.
Revista de Contabilidad, 17 (2014), pp. 108-115
[Moreno-Ger et al., 2008]
P. Moreno-Ger, D. Burgos, J.L. Sierra, B. Fernández-Manjón.
Educational game design for online education.
CHB, 24 (2008), pp. 2530-2540
[Prensky, 2007]
M. Prensky.
Digital game-based learning.
Paragon House, (2007),
[Pons et al., 2012]
D. Pons, J.L. Arquero, J.A. Donoso.
Distance learning and academic performance in accounting: A comparative study of the effect of the use of videoconferencing.
Revista de Contabilidad, 15 (2012), pp. 195-209
[Ranchhod et al., 2014]
A. Ranchhod, C. Gurau, E. Loukis, R. Trivedi.
Evaluating the educational effectiveness of simulation games: A value generation model.
Information Sciences, 264 (2014), pp. 75-90
[Rieber and Noah, 2008]
L. Rieber, D. Noah.
WGames, simulations and visual metaphors in education: Antagonism between enjoyment and learning.
Educational Media International, 45 (2008), pp. 77-79
[Rieber and Matzko, 2001]
L.P. Rieber, M.J. Matzko.
Serious design of serious play in physics.
Educational Technology Research and Development, 41 (2001), pp. 14-24
[Ryan and Deci, 2000]
R.M. Ryan, E.L. Deci.
Intrinsic and extrinsic motivations: Classic definitions and new directions.
Contemporary Educational Psychology, 25 (2000), pp. 54-67
[Sitzmann, 2011]
T. Sitzmann.
A meta-analytic examination of the instructional effectiveness of computer-based simulation games.
Personnel Psychology, 64 (2011), pp. 489-528
[Stainton et al., 2010]
A. Stainton, J. Jhonson, E. Borodzicz.
Educational validity of business gaming simulation: A research methodology framework.
Simulation & Gaming, 4 (2010), pp. 705-723
[Tao et al., 2009]
Y.-H. Tao, C.-J. Cheng, S.Y. Sun.
What influences college students to continue using business simulation games? The Taiwan experience.
Computers & Education, 53 (2009), pp. 929-939
[Tao et al., 2012]
Y.H. Tao, C.-J. Cheng, S.Y. Sun.
Alignment of teacher and student perceptions on the continued use of business simulation games.
Educational Technology & Society, 15 (2012), pp. 177-189
[Tobias and Fletcher, 2012]
S. Tobias, J. Fletcher.
Reflections on a review of trends in serious gaming.
Review of Education Research, 82 (2012), pp. 233-237
[Visauta, 1998]
B. Visauta.
Análisis estadístico con SPSS para windows. Estadística multivariante (II).
McGraw Hill, (1998),
[Vos and Brennan, 2010]
L. Vos, R. Brennan.
Marketing simulation games: Student and lecturer perspectives.
Marketing Intelligence & Planning, 28 (2010), pp. 882-897
[Westera et al., 2008]
W. Westera, R.J. Nadolski, H.G.K. Hummel, I.G.J.H. Wopereis.
Serious games for higher education: A framework for reducing design complexity.
Journal of Computer Assisted Learning, 24 (2008), pp. 420-432
[Whitton, 2010]
N. Whitton.
Learning with digital games: A practical guide for engaging students in higher education.
Routledge, (2010),
[Wilson et al., 2009]
K. Wilson, W. Bedwell, E. Lazzara, E. Salas, C. Burke, J. Estock, et al.
Relationships between game attributes and learning outcomes.
Simulation & Gaming, 40 (2009), pp. 217-266
[Wynder, 2004]
M. Wynder.
Facilitating creativity in management accounting: A computerized business simulation.
Accounting Education, 13 (2004), pp. 231-250
[Yalabik et al., 2012]
B. Yalabik, M. Howard, S. Roden.
The innovation game: Lessons in strategy and managing operations.
International Journal of Operations & Production Management, 32 (2012), pp. 1441-1459
[Yusoff et al., 2009]
A. Yusoff, R. Crowder, L. Gilbert, G. Wills.
A conceptual framework for serious games.
Paper presented at the 9th IEEE International Conference on Advanced Learning Technologies, Riga, (2009), pp. 21-23

There are some other digital games that have been incorporated as Virtual Worlds or Massive Multiplayer Online Games but those are beyond the scope of our study.

There are some other digital games that have been incorporated as Virtual Worlds or Massive Multiplayer Online Games but those are beyond the scope of our study.

The videogame is freely available at http://sims.myej.org/wcgame/.

The questionnaire was delivered in English. Participants were international and with very different origins, and sometimes English was not their native language. However, to join the programme it is necessary to pass a test –Test of English as a Foreign Language (TOEFL) – that guarantees enough level and comprehension of the language.

In this sense, and as stated by Ranchhod et al. (2014), the analysis of effectiveness based on students’ perceptions avoids the limitations of trying to evaluate the successfulness of a specific tool from a narrower and therefore limited perspective as knowledge outcomes.

Copyright © 2016. ASEPUC
Opciones de artículo
Herramientas