1 Introduction

The aim with this comparative study is to attain a deeper understanding of common problems related to design, development and implementation of virtual laboratories. In this article we analyse and present observations gained through the work on two web-based virtual laboratories - the Gas Laboratory (available online at http://esi.stanford.edu/gasesinwater/gasesinwater15.htm) and the Virtual Colour Laboratory (available online at http://dvfl.portal.chalmers.se/). Each laboratory was developed independently within two very different content areas (physics and colour appearanceFootnote 1). Still, they share a considerable overlap in both goals and production circumstances. Thus, valuable experiences can be drawn from a comparison and the sharing of empirical data from the production process and the evaluation results. We describe similarities and differences concerning 1) intentions - aim, target groups; 2) laboratory design, i.e. set-up, functions, and levels of interactivity and abstraction; and 3) whether designed for teacher led debriefings or not. Through a discussion about differences and similarities, a deeper understanding of these complex issues can be reached. A number of observed challenges related to three stages - design, development and use - in the two virtual laboratory projects are described, analysed and thematised. After having defined what production and implementation issues appear to be common between the two virtual laboratory projects, our research question is: What specific design considerations can be drawn from these observations?

1.1 How virtual labs can facilitate learning

A virtual laboratory is considered to be an interactive environment in which simulated experiments can be carried out. A laboratory can be characterized as “a playground for experimentation” (Mercer et al. 1990) providing tools that can be used to manipulate objects relevant to a specific scientific domain (such as chemicals in a chemistry lab). This playground for experimentation concept is relevant both for virtual laboratories and conventional hands-on laboratories. However, there is an important distinction in terminology. A conventional laboratory is an open playground where “the user can expand the laboratory by adding new objects, creating new experiments” (ibid.), while due to technical limitations a virtual laboratory usually do not have that flexibility. A virtual laboratory is often custom-built to perform a limited set of experiments.

Virtual laboratories have become increasingly common as a form of teaching aid in different learning situations (Lewis 2014; Achuthan et al. 2017, 2018). Creating a virtual laboratory for teaching and learning is, however, highly complex, incorporating skills in diverse areas such as interaction design, visualisation, and pedagogy. It involves design and production of texts, images, 3D environments and interactivity, and the production requires programming and animation. The development of a virtual laboratory as well as implementing it as a laboratory exercise for learning requires knowledge in the three domains outlined by the TPACK model (Mishra and Koehler 2006; Koehler and Mishra 2009), i.e. technology, pedagogy, and content knowledge. A laboratory exercise is here defined as a confined, scripted set of experiment procedures intended to be used for teaching purposes. An exercise is performed by the learner herself, while a demonstration is here considered to be performed by the teacher, albeit these two can involve essentially similar laboratory procedures.

To add even more complexity, the notion of virtuality is ontologically and conceptually multifaceted (Eriksson 2016; Hayles 1999; Heim 1993; Nusselder 2009). The virtual is often characterised as being almost as if something actual (Heim 1993). A virtual laboratory is virtual in the sense that its components - glass bottles, chemicals, burners and so on - partially behave as if they were physical, while being simulations based on a model of the actual reality. On the other hand, a virtual laboratory is a learning environment where students can perform learning activities, and from that viewpoint it is not more virtual than a book or a documentary movie. Such ambiguities sometimes lead to confusing arguments concerning the underlying concept for the design work. As illustrated in this article, the complexity of constructing a virtual laboratory often leads to difficulties in reaching the desired goals regarding clarity, usability, technology and design.

Standard arguments in favour of using virtual laboratories include accessibility (it is available online, and can be reached from anywhere anytime), resource economy (there is no physical setting, and no physical supplies), and demonstration possibilities (it can show things that normally cannot be seen, or are difficult to explain) (Carnevale 2003; Lewis 2014). A virtual laboratory is furthermore claimed to facilitate an enhanced learning experience, health and safety concerns, or ethical and legislative issues (Lewis 2014). A thorough understanding of scientific theories is difficult to obtain with traditional teaching methodologies (Achuthan et al. 2018). Achuthan et al. (2017) showed that learning was improved when students could use virtual labs prior to physical ones. Studies also show that students’ learning outcome is equal, or higher, in non-traditional laboratories, such as virtual laboratories, compared to traditional laboratory environments (Brinson, 2015). However, despite these potentials, a virtual laboratory is not unproblematic to design, develop and finally integrate in a learning situation.

The rapid increase of students’ use of online resources over recent decades requires schools to take an expert role and to positively shape students’ exposure to digital technology (Bulfin et al. 2016; Achuthan et al. 2018; Hu 2017). In this undertaking, it is imperative that online learning resources should involve a relationship between the technology used and the subject area and/or grade level at which it is applied (Sorensen 2016). Like all educational tools, digital learning materials involve certain problems, and educational gains from technical innovations cannot be taken for granted. Furthermore, research on instructional technologies often produces results that are not readily adopted by the school system and not easily transformed into education. Reasons for the scarce use of research results for teaching practice could include the fact that several of the findings emanate from short-term interventions or experimental studies, which are problematic to apply in school activities (Arnseth and Ludvigsen 2006; Schrum et al. 2005). Analysing students’ scientific reasoning in their work with discovering scientific concepts in long term, design-based and comparative studies might be a way of unravelling the learning process engendered by digital technologies.

1.2 Interactive learning environments and computer simulations

The usage of interactive computer simulations in science education has been subjected to several studies (e.g., Chao et al. 2016; Olympiou and Zacharia 2012; Sarabando et al. 2014; Son 2016; Whitworth et al. 2018). In a large-scale study Whitworth et al. (2018) made use of computer simulation in biology laboratory work. The interactive learning environment produced concentration profiles of enzyme-substrate complex, and plotted these at a graphic user interface to control reaction parameters. Students had to learn how to use and control altered reaction parameters in this computer simulated biology lab. The result showed that the computer simulation improved conceptual understanding of the reaction type under study. Higher gains for students using computer simulations than for those only using hands-on activities was also shown in a study of learning physics concepts (Sarabando et al. 2014). In this two-year long study, a computer simulation where used where the users could create, see and interact with analytical, analogous and graphical representations of the physical concepts of weight and mass. The simulation also allowed multiple representations to be seen simultaneously. Results indicated that the simulation facilitated the development of students´ scientific understandings about the concepts of weight and mass by allowing students to explore and test predictions. Moreover, Sarabando et al. (2014) found that the total gains obtained depended to a great deal on the teachers’ pedagogy when using the computer simulation to teach the concepts of weight and mass.

Besides, both the above mentioned studies by Sarabando et al. (2014) and Whitworth et al. (2018) indicated that combining computer simulations with hands-on experimentation was as effective, or even more effective for learning conceptual understanding of the topic under study, than using physical experimentation only. Other studies have similarly demonstrated that a combination of computer simulated and physical experiments can enhance students’ conceptual understanding of scientific phenomena more than the use of simulated or physical experiments alone (Chao et al. 2016; Olympiou and Zacharia 2012). This claim is supported by evidence suggesting that interactive learning environments in which multiple representations are presented promote students understanding of scientific concepts (e.g. Adadan 2013; Jornet and Roth 2015).

In a literature review Rutten et al. (2012) concluded that literature provides robust evidence that computer simulations can enhance traditional instruction, especially as far as laboratory activities are concerned. Contrasting the value of using physical and simulated experimentation in science education, in another literature review, de Jong et al. (2013) concluded that simulated experiments can sometimes be more appropriate than hands-on activities, while physical tests are more apt on other occasions.

Furthermore, Son (2016) showed that carefully designed computer simulations have the potential to result in more favorable attitudes towards science among students. The usefulness of simulated experimentation appears to be a complex matter and dependent on several factors, such as teacher support (Furberg 2016; Sarabando et al. 2014; Strømme and Furberg 2015) or integration of physical and virtual practices (Chao et al. 2016; Olympiou and Zacharia 2012; Sarabando et al. 2014; Whitworth et al. 2018). Studying student-teacher interaction in computer-supported lab work, Strømme and Furberg (2015) found that a main concern for teachers was to direct the learners attention to coexisting conceptual perspectives. In a similar study, Furberg (2016) acknowledged the importance of teachers’ support with practical advice about how to solve assignments provided in interactive learning environment. The findings of these studies emphasize that conceptually oriented issues and procedural challenges have consequences for instruction and design of computer simulated interactive learning environments.

1.3 Conveying information in 3D models and virtual environments

The production of virtual media is highly influenced by the digital tools by which it is created. Depending on the possibilities of the tool itself, and the user competence of the designer, this can either create more opportunities or limit the design choices. Balakrishnan et al. (2007) observe that physical objects rather than the spatial experience are emphasised in common digital communication tools for visualisation. In current rendering technologies great achievements are made in representational similarity through increased photorealism. Accordingly, the challenge lies in the experimental concordance with a corresponding real space and how to find the appropriate level of realism and detailing to use for a specific purpose (Balakrishnan et al. 2007). Visualisations have to be able to interpret some issues exactly (Drettakis et al. 2007), while merely sketching others (Lange 2005). In some cases, a small to modest amount of detail is sufficient for obtaining the level of realism needed (Pettit et al. 2006). If the information is too complex, contains too many parameters, or is too abstract, it will be difficult to grasp (San José et al. 2011; Stahre Wästberg et al. 2013; Wissen Hayek 2011).

Interactivity has been a major feature in the debate on how to advance learning technologies. The degree of interactivity ranges from low to high depending on the type of control available to the users. There is a general assumption – often referred to as the interactivity effect – that the higher the interactive level, the greater the degree to which learning should increase when students engage in multimedia technologies (Evans and Gibbons 2007). In line with the proposed interactivity effect – that is, that interactivity can help learners overcome difficulties of perception and comprehension during the learning process – Wang et al. (2011) examined the impact of animation interactivity on students’ learning and found that increased interactivity significantly improved student achievement. A variety of interactive learning packages for instructional purposes in science education are accessible online calling upon research results explaining how they function in classroom practice.

2 Research approach

The analysis is based on a comparison of two studies of design of virtual laboratories; the Gas Laboratory (Karlsson et al. 2013; Eriksson et al. 2010) and the Colour Laboratory (Stahre Wästberg and Billger 2016; Stahre et al. 2009). The Gas Laboratory was developed in a research project aimed at investigating learning in a virtual laboratory. The Colour Laboratory was developed as an information project where researchers sought a way to present research results visually rather than purely textual. Thus, the two laboratories were developed within different contexts and with different purposes. Nonetheless, they both resulted in similar artifacts.

Both laboratories were developed in the area of design research (Cole et al. 2005; Hevner 2007; Hevner and Chatterjee 2010; Dunin-Woyseth and Michl 2001). Design research involves several iterations of the design cycle as well as maintaining a balance between design and evaluation of the developing design artifact (Hevner 2007). In the development process, each virtual laboratory was designed, built, tested, evaluated and then re-designed for further iterations. In each project the development of the virtual laboratory was followed by all the team members from the initial idea to the final stage, combining theory and practice. Both the Gas Laboratory and the Colour Laboratory went through three iterations during the development period (see more details on the development of the laboratories in Section 3).

In the development of the Gas Laboratory the project team used an action research method as proposed by Avison et al. (1999). This implied involving researchers and practitioners (the teachers and students) acting together in a particular cycle of activities throughout the iterative process. Moreover, an ethnographic method was used for observation and documentation of the design and production of the Gas Laboratory. Ethnography is the study of social interactions, practices and events. The observed social expressions are described and to some extent interpreted and assigned a meaning (Atkinson and Hammersley 1989; Clifford 1986; Geertz 1973; Hughes et al. 1994).

The project team for the development of the Colour Laboratory used a design based, abductive research approach (Reichertz 2004, p. 159–165). An abductive approach implies moving between theory and empirics and let the understanding of the research object successively grow. This meant frequently changing perspective, i.e. to switch between reflections on the work and practical elaborations in the design of the laboratory (Stahre 2009, p 47).

Thus, both projects used a mixed method approach, where different methods supported each other, were used in order to provide a deeper and more holistic understanding of the results (Campbell and Fiske 1959; Creswell 2003; Denzin 1978; Flick 2009; Iversen 2005; Jick 1979; Seale 1999). The data involved quantitative tests, qualitative surveys, as well as qualitative, ethnographic observations and interviews.

In the comparative study of the two virtual laboratories the members of the two research teams then conducted a joint evaluation in two phases:

  • Phase 1: Compilation of information from both laboratories in a matrix concerning: 1) intentions, 2) laboratory design, 3) form of guidance, and 4) user evaluation setups.

  • Phase 2: Comparisons of the two virtual laboratory projects in relation to the design and development processes, the setups of the laboratories and the results from the user evaluations.

3 Comparison phase 1: Compilation of content of laboratories

A brief description of the two laboratories is summarised in this section. Table 1 shows similarities and differences between the two laboratories concerning 1) intentions - aim, target groups; 2) laboratory design, i.e. set-up, functions, and levels of interactivity and abstraction; and 3) whether the laboratories are designed for teacher-led debriefings. In the Gas Laboratory study 180 upper secondary school students and 8 teachers participated. In the Colour Laboratory study 25 architecture students and 20 university teachers and professionals participated. The setups of the user studies are further described in Table 2. An interactive model of a traditional laboratory bench was used in the Gas Laboratory (Fig. 1a-b). In the Colour Laboratory free standing stations, one for each exercise, were used (Fig. 2a-c). Both projects were based on a web page.

Table 1 Compiled information on intentions, laboratory design, and guidance for the Gas Laboratory and the Colour Laboratory
Full size table
Table 2 Compiled information of the user studies for the Gas Laboratory and the Colour Laboratory
Full size table
Fig. 1
figure 1

a The introductory view of the Gas Laboratory. b The laboratory setup: A. Laboratory bench, B. Laboratory shelf, 1. Empty bottle, 2. Water tap, 3. Water hose, 4. Gas bottle with oxygen, 5. Gas bottle with carbon dioxide, 6. Laboratory heater and shaker, 7. Bottle cap, 8. Water thermometer (F and C°), 9. Can with NaCl (salt)

Full size image
Fig. 2
figure 2

a The present design of the Colour Laboratory web page, displaying the nine included stations. The language of the demonstrator is Swedish, but a future version in English is planned. b Example of the stations which compare colour perception depending on spatial setting - in this case the effect of daylight on colour appearance outdoors vs indoors. c Example of the stations which demonstrate simultaneous contrast. These stations functioned as real world exercises were the user matched colour samples on differently coloured backgrounds

Full size image

Our intentions with the production of the laboratories were that they should be both pedagogical and experimental in their two different areas. The functions of the Gas Laboratory included three laboratory tests investigating the solubility of gases in water. In the Colour Laboratory, each station was structured around an exercise on a specific colour phenomenon or principle. The level of interactivity was slightly higher in the Gas Laboratory than in the Colour Laboratory, where physical reactions to the activities are simulated. In the Colour Laboratory the user could change the conditions, e.g. choose different paints for a facade or an interior, and the phenomena were illustrated.

Both laboratories aimed for a realistic appearance, and the level of abstraction was fairly similar in both settings. The Gas Laboratory mimicked a real world setting with semi-realistic workbench details, authentic procedure, and a predetermined sequence. The Colour Laboratory used rendered 3D interiors and facades, manipulated photos of 2D facades, and 2D colour samples to create realistic settings, however, the exercise procedures differed in levels of authenticity. The form of guidance was quite similar in both projects. One difference was that the Colour Laboratory had instructions on different information levels for each station. Another difference was that the Gas Laboratory showed alternative actions when wrong choices were made, which the Colour Laboratory did not.

4 Comparison phase 2: Development process, design and user evaluations

In this section, we present the results of our joint reflections on the design and development processes, as well as setups of the laboratories, and the results from the user evaluations. We discuss 1) how to balance ambitions with available resources, 2) intended levels of user interaction with exploratory freedom, 3) how to find appropriate levels of realism depending on target group, 4) how to choose between mimicking real world appearance or enhanced features, and 5) how to find the best learning situation for the virtual laboratory? Definitions of these design challenges are further described in Table 3.

Table 3 The five design challenges which were discerned during the comparison
Full size table

4.1 How to balance ambitions with available resources

A mutual set of challenges for the two project teams concerned ambitions and expectations. The projects were funded on the basis of their relative novelty and their use of advanced digital media. In line with the interactivity effect (Evans and Gibbons 2007) the ambitions with both projects were to provide them with a high degree of interactivity in order to advance students’ learning (Wang et al. 2011). The Colour Laboratory was initially planned as a virtual environmentFootnote 2 using interactive 3D models to mimic the experience of both exterior and interior colour effect illustrations, as if visiting a park or a museum. The user was envisioned to traverse these environments, making discoveries along the way. The Gas Laboratory was initially planned as a virtual environment in the form of a laboratory bench; smaller in room scale than the Colour Laboratory, but with some options to perform various kinds of experiments in different ways. The users were supposed to use it as a self-instructive product, i.e. without the guidance of a teachers or institution.

In both projects, time consuming production challenges gradually changed these ambitions. The central issue was the lack of both high-level professional media production skills and funding resources. Instead of hiring professional media producers, the projects engaged semi-professional in-house resources (that is, the researchers themselves). The argument for this was to retain full creative control over the final product as well as to save expenses. The Colour Laboratory proved to be technically challenging to realise as a three-dimensional environment; there were issues with the technical platform in itself, in the combination of indoor and outdoor environments, and in the re-creation of accurate colour experiences.

The high level of ambition regarding the setting contributed to a focus on technology, rather than on content. The solution to this was to simplify the setting, the result being a technically simpler web based set of separate “rooms”, viewed as more or less interactive scenes. In the Gas Laboratory, there were similar experiences. Creating graphics and coding the interaction for such a setting proved to be very time consuming. Eventually, only one experiment was built. Additionally, the self-instructive usage proved to be challenging from a pedagogical perspective, resulting in a solution with teacher-guided debriefings. Figure 3 illustrates how challenges changed the character of the virtual laboratories during production and evaluation. The diagram shows a general summary of how our originally ambitious intention to create self-instructive and open virtual environments was transformed into closed and guided final products. This shift from intention to implementation is highly similar despite the diverse nature of the two laboratories.

Fig. 3
figure 3

An illustration of how our originally ambitious intention to create self-instructive and open virtual environments was transformed into closed and guided final products. The horizontal axis describes the laboratories’ capacity to function depending on learning support, where the concept standalone means that a laboratory is fully comprehensible in itself and guided means that it depends on some kind of learning support in order to reach its highest capacity. The vertical axis describes the interaction capacity of the laboratories, where the concept open refers to free interaction within a virtual setting, and closed refers to limitations when it comes to interactive choices

Full size image

4.2 How to balance intended levels of user interaction with exploratory freedom

When evaluating the design and development process as well as the user tests, the project team behind the Gas Laboratory found that the virtual laboratory with its look and feel promised to be explorative, but actually had a low level of interactivity. The students started the virtual laboratory exercise, and when interaction was required, the system stopped and waited for the students to interact. Usually only one option (e.g. heating the water) was possible to select. Rather than being open for different choices and interactions, the Gas Laboratory became a linear narrative with a closed chain of events. Interaction is characterised by the agency it gives the user to manipulate the system, creating events with different outcomes (Robinett 1992; Murray 1997; Dreyfus 2001). Since full interactivity could not be realised, due to time and monetary constraints, a narrative solution was used to solve the problems. Narrative is characterised by the narration of already determined actions (Walther 2003; Juul 2005). While a few attempts were made to make it appear more open – such as adding a few more labelled bottles on a shelf – there was no time or resources to rebuild the laboratory.

Even a conventional real world laboratory exercise, were you work with the exercises hands-on, is to a large extent closed and linear, constructed to make students follow a recipe and arrive at one specific observable outcome, which then is discussed in a classroom debriefing. If the Gas Laboratory exercise was hardly interactive at all, did it really require the illusion of openness? This question relates to the purpose of the exercise. The Gas Laboratory was a step-by-step procedure to follow in order to produce an observation that required interpretation and subsequent understanding of causality. It was a demonstration based on a discovery or guided inquiry approach (Karlsson et al. 2012), pretending to be an open laboratory environment. Even though this illusion of openness lead to distinct disappointment among some students, their high understanding of media production circumstances made them forgiving towards this limitation. From a pedagogical and narrative viewpoint, the goal was met; the students observed and discussed the gas bubbles. One possible interpretation is that the illusion of openness neither supported nor impeded the learning.

4.3 How to find appropriate levels of realism depending on target group

The results from both studies highlight the importance of using appropriate levels of realism and accuracy, depending on target group. In the Colour Laboratory project, two problems were identified. The impression of visual realism was affected by 1) the unforeseen spatial differences between the virtual setting and reality, and 2) difficulties in visualising real world colour appearance (Stahre et al. 2009; Billger et al. 2004; Stahre and Billger 2006). Since a realistic colour appearance could not be accurately reproduced, the problem of visualising colour phenomena instead became a problem of how to visually enhance the visualisations to provide the phenomenological experience of these phenomena. In the first iteration of the Colour Laboratory, the two problems mentioned above were addressed by exaggerating spatial aspects of the setting and manipulating the visual depiction of the colours, so that the experience of different colour effects was simulated. In the final version, the two problems were solved by creating simplified settings and by excluding specific information such as NCS codes,Footnote 3 and instead using a more general terminology (such as “light blue”) when describing a colour.

Regarding the Colour Laboratory, it was clearly shown that the more knowledge the user possessed on the subject, the higher their demands for visual precision. A constant challenge therefore was to determine the level of realism necessary when visualising the specific NCS codes. The researchers behind some of the demonstrated material, as well as the group of professionals in the user evaluation, were very sensitive regarding the accuracy in how the colours would appear on screen. Their need for visual accuracy in the presented material proved to be much higher than the need of the students in the other target group. While the professionals aimed for a high level of visual realism regarding the colour reproduction, the students were more accepting towards the fact that it was the principle that was demonstrated rather than the fully realistic colour appearance. It could be argued whether the perception of the visualisation differs between the users depending on their different levels of knowledge in the visualised subject, leading to a more critical position if you know the visualised material well, compared to if you do not. The difficulty in projects of this kind, involving both experts, and learners, is to find the right balance in the design so that both categories will benefit from its usage.

In the Gas Laboratory, neither visual realism nor complex interaction was actually required. The intent of the laboratory was to let students observe the amount of gas bubbles when changing three physical parameters (temperature, pressure and salinity), and to have them discuss and interpret what these gas bubbles actually meant. Research within visualisation clearly suggests that in many circumstances a stylised image is more efficient and easier to understand than one with a higher degree of realism and detail (Ryan and Schwartz 1956). The evaluation of the Gas Laboratory indicates that the students did not request a more realistic representation of the laboratory environment; the students’ concern was rather how to interpret the amount of bubbles in different stages of the experiment. Occasionally students had difficulties in discerning the amount of bubbles. However, that problem was not really due to a lack of visual realism but rather an issue related to screen layout; the virtual gas bubbles should have been a bit bigger on-screen, allowing a closer look at them.

A question regarding both laboratories is if it would not have saved a lot of time and been equally effective to use photos or video recordings instead of animated visualisations. In the evaluation of the Colour Laboratory, photos as real world examples were requested by some participants. Others supported the use of visualisations but called for enhanced detailing in the visual settings in order to make them resemble real world scenarios. The motivation for using visualisation technology instead of photos was the possibility to mimic aspects of a real world setting, such as illumination, in a clearer way than a photo would have done. In the Gas Laboratory the usage of footage was discussed, but an argument against it was that it would be difficult to shoot the bubbling in a clear way; animating the bubbles made it easier to slightly exaggerate the size of bubbles and their clarity. There was also a general assumption that the laboratory bench would be used for other experiments in forthcoming exercises.

4.4 How to choose between mimicking real world appearance and adding enhanced features

One way to build a virtual laboratory is to see it as a real world hands-on laboratory, but in digital format. This is what we choose to call a cloned virtual laboratory. Another option is to add extra features to the laboratory that could not be part of a real world laboratory. This is what we choose to call an enhanced virtual laboratory. In this latter option for building a virtual laboratory we choose the term enhanced, not to confuse the concept with augmented reality. In augmented reality, graphics are added to the actual reality (Milgram et al. 1994). In the enhanced virtual laboratory, it is the virtual environment that is enhanced in relation to a corresponding real world laboratory. However, some of these enhancements can be conceptually very similar to how augmented reality work. The view of molecular events discussed below, for instance, could very well be added as an augmented reality enhancement in a real world laboratory.

Students using the cloned, first iteration version of the Gas Laboratory had trouble understanding what the bubbles meant in relation to gas solubility in water. Since the Gas Laboratory was virtual, different enhancements could be added to clarify the experiments. One suggested enhancement was imagery showing events on the molecular scale, in order to provide the students with a cognitive model for the chemical process (Fig. 4). Eventually it was decided not to show this molecular view, in order to make the enhanced virtual laboratory as realistic as possible compared to a cloned virtual laboratory.

Fig. 4
figure 4

a A cognitive model for the chemical process is shown in this design mockup for the enhanced virtual laboratory. The inserted image to the left shows what is happening on the molecular level in the bottle. b Detail of the gas meter

Full size image

Another enhancement was the gas meter (Fig. 4b), a virtual instrument showing the percentage of dissolved gas. While the molecular view showed things that are not visible in a cloned virtual laboratory, the gas meter supplemented what was already shown by other means; the amount of dissolved gas, but in a numerical way. Students made appreciating comments concerning the gas meter: “You saw the difference [in amount of gas] on the small meter”, and “You could so clearly see it when it was written how many percentages there now is in the water”.

In the Colour Laboratory the stations consisted of a mix between cloned and enhanced exercises. Already in the first iteration, where the setting consisted of a virtual world, the virtual environment could be looked upon as an enhancement in itself. It displayed a dense mixture of spatial colour phenomena - such as showing the impact of distance on colour perception - and abstract ones - such as learning about the NCS colour sphere - along a guided walk, using enhancing aspects such as overblown scale and perspective to deepen the understanding. In the final version the Colour Laboratory used enhancements to for example show comparative examples on colour perception, simultaneously depending on scale or spatial setting (Fig. 2b). The two stations demonstrating simultaneous contrast were examples of a cloned virtual laboratory, i.e. they functioned as a real world exercise were the user matched colour samples on differently coloured backgrounds (Fig. 2c). However, they had one function of an enhanced virtual laboratory, by showing when the tasks were successfully completed.

The choice between a cloned and an enhanced virtual laboratory is largely a choice between utilising the benefits with the digital media format on one hand, and on the other hand attending to authenticity and fidelity. It seemed reasonable to enhance the virtual laboratory, as long as these additions didn’t make it fundamentally different compared to a real world laboratory. Striking this balance needs to be done differently in each case, though.

4.5 How to find the best learning situation for the virtual laboratory

One crucial challenge lied in determining how the virtual laboratories were best put to use. Should the laboratory be open for use to anyone with an interest in the content, only guided by the information on the web page? Or, should it be designed for usage in a specific learning situation, i.e. as a complementing demonstration for a teacher? The two laboratories were used in different settings, the Gas Laboratory was evaluated in a classroom situation, and the Colour Laboratory evaluated by users at home.

The Gas Laboratory was initially intended to be a self-instructive exercise, which proved to be a major difficulty. In the first user tests the students generally did not reach the intended learning outcome; the main problem was interpretation of the gas bubbles, especially since gas bubbles were seen twice; first, during gassing of the water and after that, during the subsequent change of physical parameters. The first bubbles meant that the water was bubbled with gas in order to dissolve gas in the water, while the subsequent bubbles meant that the level of dissolved gas decreased and that it left the water. This switch in meaning was difficult for the learners to understand without guidance. In the second version of the gas laboratory guiding texts were added, especially an explanatory text to be read after the exercise, as well as a gas meter; a graphical element containing a feigned measurement of amount of solved gas. This increased the achieved learning outcome. In the third version a teacher did an in-class debriefing, and both post-test questions, ethnographic observations and post-interviews suggested that most of the students understood the exercise and achieved the intended learning outcome. This result clearly indicates that the Gas Laboratory benefited from being used with teacher guidance. Teachers are needed to take an expert role in students’ use of digital technology (Bulfin et al. 2016; Sorensen 2016; Laurillard 2012).

Regarding the Colour Laboratory, the simplicity in appearance of the different stations sometimes prevented users from actually perceiving the full complexity of the demonstrated colour phenomena, which at least in some cases affected the learning outcome. The interpretation is that the Colour Laboratory would be put to best use in a controlled environment where a teacher could provide a context around it, and fill in any information gaps.

A common demand on virtual laboratories is that they should function as efficiently as self-instructive learning materials. Our difficulties in making students achieving the learning goals without teacher guidance suggest that it cannot be taken for granted that a virtual laboratory can function without a teacher-lead debriefing. This is similar to real laboratories where students need either written guides or a teacher/assistant that go through the process. On one hand the interactive and adaptive discussion between the students and their teacher is unique and cannot be recreated in a self-instructive tool such as a virtual laboratory. On the other hand, well-written texts should be able to recreate the required rhetorical argumentation, as a “discussion” between the text and the student. However, there is no conclusive answer to the debate. In the third version of the Gas Laboratory, the quite extensive guiding texts before, during, and after the actual laboratory experiment were very appreciated by the students. The guiding texts were appreciated also in the Colour Laboratory, in particular the possibility to choose from different levels of information depending on the user’s interest in the station at hand. It would certainly have been possible to perform the exercises in the two laboratories without written text. However, then the exercises would have been harder to interpret correctly, and the overall context might have been lost.

5 Conclusions: Design challenges and considerations

The aim with this comparative study is to gain a deeper understanding of common problems related to design, development and implementation of virtual laboratories. The purpose of this article has been to summarize observations gained through the work on two web-based virtual laboratories, based on the following questions: Which production and implementation issues appear to be common when comparing the two different virtual laboratory projects? Which design considerations can be drawn from these observations? A common concern in both projects was how to choose a suitable level, kind, and purpose, of both interactivity and visual realism in the laboratories. Due to technical problems and lack of resources both projects produced virtual laboratories that were far simpler than the initially expected result. A conclusion drawn from this is that the development of successful virtual laboratories requires a huge amount of resources and time. However, results indicate that despite the technical production issues, the users actually did achieve most of the intended learning outcomes.

Five design challenges of particular interest have been identified: 1) Ambitions and expectations versus lack of resources, 2) Explorative interaction versus linear narrative, 3) Appropriate levels of realism and accuracy, 4) Cloned versus enhanced laboratory, and 5) Self-instructive versus teacher-guided debriefing. (Table 3).

Based on the analysis and interpretation of these observations, a research contribution is provided in the form of the following design considerations:

  • Be very clear about the purpose of the virtual laboratory, and in what context you intend it to be used. Media consumers, especially teenagers and young adults, are highly media literate and can quite easily see through attempts where for example a linear demonstration pretends to be an interactive laboratory. Consider which type of media you intend to build - simulation, laboratory, demonstration, and so on. Indicate clearly for the user what they are interacting with.

  • Strive to use the simplest possible design and technology, still meeting the demands efficiently. In some cases advanced technology such as virtual environments or even virtual reality might be needed, but a technology-minimalistic strive will lower the risk that a too advanced technology is used for its own sake. The most eye catching techniques might not always correlate with what is relevant to show.

  • Adapt levels of realism and accuracy to the intended target group as well as to the intended learning outcome.

  • Continuously consider enhancements of the virtual laboratory to increase the learning outcome. It can be profitable to provide help when needed and visualise things that are not possible in a real laboratory. Balance this potential against possible advantages of having a virtual laboratory that closely mimics real-life laboratory exercises.

  • Regard a virtual laboratory as an illustrative playground that requires external support in the form of guiding, explanatory texts or teacher debriefing. The virtual laboratory provides the students with experience and observations, but does not always necessarily provide understanding on its own. Guidance is often necessary to help the students to understand the illustrated scientific phenomena.

Designing and implementing a virtual laboratory is a highly complex process with different requirements and solutions depending on each projects unique circumstances. The similarities between the two virtual laboratory projects suggest that these design considerations can be helpful for the production of similar projects. In this article, we have defined five challenges and from these drawn five design considerations for virtual laboratories. The next step would be to implement and evaluate these design considerations in new virtual laboratories, and thereby further develop knowledge and design strategies.