Creative students in self-paced online learning environments: an experimental exploration of the interaction of visual design and creativity

Abstract

Creativity is an increasingly recognized construct in technology-enhanced learning. However, our understanding of how creativity interacts with the design of online learning environments to affect learning experiences is still limited. For example, do creative students benefit from different learning environment designs than those benefitting their less creative peers? This experimental study (N = 187) explores this question by investigating the visual design of a self-paced online learning environment, specifically the degree of visual structure, in relation to students' creativity. Creativity was measured in different ways, along the lines of vocational/study choice, self-reported personality and behavior, and creative production. Students were randomly assigned to either a visually unstructured (experimental group) or a visually highly structured (control group) learning environment. They reported their preference, impulse for activation, and situational motivation after the learning experience. Results indicate interaction effects consistent with the role of creativity in perception and learning. More specifically, creative students reported more motivation after learning in an unstructured environment, whereas non-creative students reported relatively better learning experiences in the highly structured condition. These results contribute to resolving previous conflicting findings from separated studies, yet some ambiguities remain. Results and implications are discussed, and recommendations for future research are laid out.

Introduction

Creativity is considered an important twenty-first century skill as well as an increasingly explicit goal of formal learning that may help students navigate an uncertain future (Beghetto, 2005, 2010; Craft, 2011). At the same time, knowledge of how to accommodate and incorporate creativity into the design of learning experiences is still emerging (Chan & Yuen, 2014; Dalke et al., 2007; Richardson & Mishra, 2018). Moreover, only scarce attention has been paid to considering creativity not only as an outcome of learning experiences but as an individual difference variable that should be accounted for in deliberate instructional design (Rosar et al., 2018).

With information and communication technologies increasingly commonplace in all aspects of human endeavor and, similarly, usage of digital technologies for educational purposes being ubiquitous (Seaman et al., 2018; Garrett et al., 2019), there is a need for understanding how these online-based learning experiences may cater to individual differences, like creativity, in a pedagogically sound way. A central aspect of pedagogically sound learning design is the recognition of the role of the learning environment itself, the space in which learning activities take place. Just as for physical learning spaces (Ellis & Goodyear, 2016; Merriënboer et al., 2017), the characteristics of online spaces in which learning is to occur are consequential for learning processes (Weidlich & Bastiaens, 2019; Flynn et al., 2018). Besides general-level best practices, there is also a need for understanding specific interactions in technology-enhanced learning, for example, how design features of online learning environments interact with creativity as an individual difference, such that engaging and effective learning experiences may be fostered.

The present study attempts to provide a first step toward this goal. Through a randomized online experiment, we investigated the role of creativity as an individual difference with different measurement approaches and its interaction with a specific design variable, the visual structure of the learning environment. Importantly, we do not attempt to elicit creativity through our learning environment but instead investigate the effects of two different learning environment designs. Based on the literature, we hypothesize that these effects may be different for students that differ in trait creativity, thus suggesting an interaction effect.

In the next section, we first outline the literature on trait creativity, then situate this with respect to the educational domain before more specifically reviewing the literature on the design of online environments and potential indicators of creativity. From this, we derive several research questions. Then, we describe the method for addressing these questions, our experimental design, the materials, our sample, measures, and analyses. After reporting our results, we discuss our findings in light of the literature and the limitations of our study.

Literature review

Creativity

The concept of creativity has initiated a long-running historical debate about its key factors. Up to now, there is no widely accepted standard definition of creativity (Runco & Jaeger, 2012), although most scholars agree that creativity involves the development of a novel product, idea, or problem solution (Hennessey & Amabile, 2010). Early one-factor-theories of genius, giftedness or talent (Galton, 1978; Taylor, 1975) were replaced by multifactorial and multiprocedural concepts of creativity (Guilford, 1950, 1975; Torrance, 1975). With the exception of the widely explored visual–figural domain (DeYoung et al., 2007; Han et al., 2018; Reuter et al., 2006), there is no final agreement about the domain-specific nature of creativity (Baer, 1993; Kaufman & Baer, 2004). The distinction of everyday creativity and, highly creative giftedness, what is frequently called Little c and Big C, respectively, constitutes another considerable challenge (Guilford, 1950; Stein, 1987; Runco & Richards, 1997; Kaufman and Baer, 2004; Silvia et al., 2014).

The assumption that creative individuals perceive their environment more holistically, which in turn underlies creative environment-assimilating and creative perceptual processes, is a popular assumption but has not been demonstrated conclusively (Lewin, 1935; Schachtel, 1959), although more recent research suggests that certain aspects of creativity are indeed associated more strongly with global attentional processes compared to local hemispheric processes (Razumnikova & Volf, 2015). Worrying about what has become a booming but increasingly fragmented area of research, Hennessey and Amabile (2010) proposed seven major levels from which creativity may be investigated: (1) neurological, (2) affect/cognition/training, (3) individual/personality, (4) groups, (5) social environment, (6) culture/society, and (7) systems approach. The present study is concerned with the interaction of level 2 “affect/cognition/training” with level 3 “individual/personality.”

Creativity and education

It is a popular notion that schools and formal education stifle creativity in students (Berliner, 2011; Runco, 2014). Explanations put forward for this assumption are the one-size-fits-all nature of formal education, the prevalence of standardized testing, as well as the inability of the system to handle displays of non-conformity, a frequent behavioral correlate of creativity (Kettler et al., 2018; Longo, 2010; Runco et al., 2017; Van Hook & Tegano, 2002). Increasingly, the recognition of creativity as a desirable goal of learning processes to be fostered, or at the very least, not to be stifled, has led to calls for improving classrooms in this regard (Beghetto & Kaufman, 2013; Starko, 2013). Regarding creativity-conducive factors in education, research has increasingly focused on the classroom context and the learning environment in general (Beghetto & Kaufman, 2014; Chan & Yuen, 2014; Davies et al., 2013; Richardson & Mishra, 2018). Recommendations for learning environments supporting student creativity include: open-ended tasks involving choice, authentic and relevant tasks, support for risk-taking, atmosphere of care, tolerance, and respect. A weakness of this line of research is that delineated guidelines are frequently not specific to creativity but instead apply to most well-designed learning environments. Moreover, as the empirical studies underlying these recommendations frequently do not define creativity as an individual difference variable but as a broad class-level outcome, potential differences in efficacy between students remain hidden.

Focusing on online learning environments more specifically, we find a dearth of research acknowledging creativity as a variable to be considered in the design of online learning environments. In fact, there appears be no empirical investigation assessing the efficacy of certain design characteristics with respect to fostering creativity or supporting creative students in online learning settings. To identify a starting point to explore these relationships, we turn to established studies to identify foundational design characteristics that may interact with trait creativity.

Design of online environments for creative students

Vision and visual perceptual processing are integral to cognitive processing. For example, the visual interpretation of presented stimuli is governed by processes which infer the whole, e.g., a diagram explaining photosynthesis, through the relationship of its elements, e.g., lines, arrows, colors (Arnheim, 1974). As a result, the visual arrangement of elements presented to students affects a student’s ability to discriminate and organize them for cognitive processing (Guilford, 1975). Thus, in general, well-structured material is expected to be beneficial for learning processes. Turning to research from educational psychology, we find an evidence demonstrating the efficacy of well-structured learning material from different lines of research. One example of a relevant line of research are findings from the signaling principle in multimedia learning (Mayer and Moreno, 2003; Mayer, 2017). As per this principle, students learn better when their visual attention is guided through structuring signals, for example, by highlighting essential information through colors (Richter et al., 2016; Scheiter & Eitel, 2015). Importantly, this line of research considers individual difference moderators of this effect, the most prominent being prior knowledge. Faced with more knowledgeable students, the design recommendation are reversed, highly structured visual design through signals would then hamper learning (Richter et al., 2016). Another line of research pertaining to the relevance of visual design are emotional design effects. Here, learning is facilitated through positively valenced emotional design of the learning material, e.g., colorful and expressive aesthetics (Heidig et al., 2015; Schneider et al., 2016; Um et al., 2012). As a mechanism for these learning benefits, increased motivation and, thus, higher mental effort has been put forward (Um et al., 2012).

Conversely, there is also evidence that rich and unorganized stimuli may be beneficial. For example, creative individuals like artists may prefer this to organized, balanced, regular, symmetrical patterns (Arnheim, 1947; Guilford, 1975). This preferences for complexity (Barron, 1953) could constitute a considerable factor to drive the interest and motivation of creative students. In fact, more recent empirical research has provided evidence for this. In Goldschmidt and Smolkov (2006), design students working on creative tasks in a rich and visually unorganized environment yielded better results than their peers without these stimuli. In another study, highly complex photographic stimuli was rated to be more conducive to creative behavior (McCoy & Evans, 2002). Evidence from cognitive psychology further supports the notion of individual differences in creativity leading to differences in the processing of visual stimuli (Chamberlain & Wagemans, 2015).

As online and distance learning becomes increasingly commonplace (Seaman et al., 2018), it is an important goal to better understand how instruction in these settings may be designed for diverse populations to facilitate effective and engaging learning experiences. Self-paced courses, frequent in informal learning settings, allow students to learn at their own pace (Lim, 2016; Southard et al., 2015). Without instructors and/or peer students, students are uniquely reliant on their own faculties, their emotions and motivations during learning, making the issue of appropriately designed environments even more relevant (Cho & Heron, 2015; Hu, 2008). Although there is a body of research into design principles for certain student populations like nurses, pilots, and physicians, our review has yielded only a single study empirically investigating online learning design principles with relation to the population of creative students. Rosar et al. (2018) analyzed differences in preference for inquiry-based versus guided learning between students enrolled in a creative domain of study and students who weren’t. Results suggest an interaction between these factors such that the non-creative population strongly preferred guided instruction, whereas the creative population was mostly indifferent to the degree of guidance, with a slight descriptive advantage of non-guided design. However, as a weakness, this study measured creativity solely by self-reported study program (creative or non-creative). Also, the sample size of this study was relatively small. In order to represent the complex nature of creativity, more ambitious measurement of creativity should be attempted while sampling from a larger population to arrive at better estimates of effects and reduce the likelihood of false-negatives. Finally, Rosar et al. (2018) solely investigated student preference as dependent variable, thus neglecting potential downstream effect like increased motivation. To extend this, further outcome variables that may result from student preferences and are conducive to student learning, like impulse for activation and situational motivation, should also be considered (see Fig. 1).

Fig. 1

Theoretical model displaying how creativity and design may interact to affect outcome variables

Given this preliminary state of research, we find it important to better understand how design features of self-paced online learning environments fit the creativity of the students that are the target groups of these learning experiences. As the multimedia learning literature suggests highly structured and streamlined learning material (e.g., Richter et al., 2016) while the literature on creative individuals conversely suggests complex and unorganized stimuli (e.g., Goldschmidt & Smolkov, 2006), we derive from this the potential of an interaction effect. This interaction may be such that (1) creative students report better learning experiences in a design that is rich and visually unorganized compared to a more organized learning environment, whereas for (2) relatively less creative students the opposite is true. They may report preferring learning in a more structure environment compared to an unorganized design.

Indicators of creativity

As observable characteristics of creative individuals, openness and curiosity are frequently put forward (Guilford, 1975; Schachtel, 1959), a notion has been supported by more recent research (Silvia et al., 2014). Brain research has also shown that among the Big Five personality, openness to experience significantly mediates high gray matter volume and trait creativity (Li et al., 2015), whereas Tan et al. (2019) demonstrate an association between openness to experience and intrinsic motivation, which in turn contributed to self-rated creative personality. Another main observable predictor for creativity was seen in creative behavior (Torrance, 1975), usually associated with artistic individuals demonstrating this behavior, for example, by using unconventional approaches and breaking with routines and conventions. As an example, in entrepreneurship, it was shown that constructive rule-breaking and other non-conforming behavior was associated with creative entrepreneurial behavior (Obschonka et al., 2013). More generally, it is reasonable to assume that individuals higher on trait creativity will demonstrate and report more creative behavior in trait-evoking situations (George & Zhou, 2001).

It is commonly assumed that both of these indicators, personality and behavior, although strongly associated with original and highly creative production, are not themselves sufficient. Already in 1975, Torrance showed that actual creative output is not solely explained by personality factors. Research since has shown there is an aspect of raw ability to be considered such that creative dispositions and motivations affect creative performance via creative ability which itself a product of divergent thinking (Choi, 2004). In the classic literature, Guilford (1975) understood this divergent thinking ability as an important component within the wider construct of intelligence, as formulated in his broad structure-of-intellect model. As a central facet of divergent thinking, ingenuity may be seen as the flow of ideas prompted by a stimulus and resulting a relatively large and varied number of figural ideas, word associations, or solution approaches (Runco & Jaeger, 2012).

Summing up the literature on observable indicators, we find that there are three main types to be considered in order to tap the construct of creativity, (1) creative personality, (2) creative behavior, and (3) creative production.

Research questions

Although, as presented in the literature review, there is both a classic literature indicating to the importance of considering creativity in designing learning experiences as well as an emerging body of research connecting this to technology-enhanced learning, dedicated research systematically exploring creativity as an individual difference variable in relation to design features is virtually nonexistent. Thus, our overarching research question is:

How does creativity interact with learning environment design features to affect learning experiences?

Previous research into creativity as individual difference factor operationalized the construct through enrollment in educational programs related to art and creativity and found a significant interaction in the preference for inquiry-based versus guided instruction in a self-paced online learning environment (Rosar et al., 2018). As a conceptual replication, the present study seeks to extend this beyond enrolment choice to also include vocational choice (creative vs non-creative occupation).

Thus, our first research question is: How predictive is vocational and study choice (creative vs. non-creative) for understanding variation in learning experiences in self-paced environments?

Addressing the dearth of online learning design recommendation for creative student populations, this study attempts to further explore relevant design features of the online learning environment for students scoring high on creativity. More specifically, this study varies the visual structure, i.e., the degree to which elements are displayed in an ordered versus relatively unordered fashion in the interface.

Thus, our second research question is: How relevant is the visual structure for eliciting differential learning experiences in relatively creative versus relatively non-creative students?

Recognizing creativity as a complex and multi-faceted construct, this study operationalizes the construct in different ways that, together, may help form a nuanced picture of the dependencies between this largely unexplored individual difference factor and design features of the learning environment. Specifically, besides operationalizing creativity through vocational and enrolment choice, we sought to measure it through two further methods, (1) self-reported visual-figurative creativity and (2) visual-figurative creative production. These two additional methods are in line with what Batey (2012) identified as the most common creativity measurement approaches.

Thus, our third research question is: How do different methods of measuring creativity interact with the design of the online learning environments to affect learning experiences?

Method

Design

The above research questions were addressed through a randomized online experiment, in which the visual design of the environment (high structure vs. low-structure) served as independent variable. This research design was chosen to provide evidence for causal effects that are the results of the visual structure of the learning environment. Students were randomly assigned to one of two conditions, after filling out a questionnaire with demographic information and measures of trait creativity. Trait creativity was measured prior to the intervention to avoid possible treatment effects. Upon exiting the environment, student experiences were elicited based on a second questionnaire, which measured the dependent variables. Preferences for this design feature, situational motivation, as well as impulse for activation served as dependent variables. This final questionnaire also debriefed students and provided opportunity for open-ended feedback (Fig. 2). Preferences for this design feature, situational motivation, as well as impulse for learning served as dependent variables.

Fig. 2

Experimental procedure

Material

Students learned about nutrition in an online learning environment consisting of a total of fifteen topics, e.g., nutritional value, carbohydrates, fibers, metabolism, digestion, vitamins, etc. The learning environment was fully self-paced, meaning that there was no instructor that accompanied their learning experiences. This approach was chosen to allow students a flexible and free-roaming learning experience in order to avoid stinting potential engagement with the material and its features (Davies et al., 2013). As this study attempted to sample a broad variety of students in and outside of the creative domain, the course was also not associated with a higher education curriculum, but served as a stand-alone experience. The topic of nutrition was chosen to be of broad interest to most people while also ensuring it is not too complex for a diversity of participants. Moreover, it is likely that participants have not had experienced formal education about the topic, as it is outside of German school curricula. Existing appropriate materials like articles and videos were identified and embedded within the learning environment. In addition to these lessons, a set of single-choice quiz questions were implemented as well as a notepad for taking notes and an online calculator for calculating, for example, calories and nutrients associated with different meals. Students were free to leave the environment at any time, thus concluding the learning experience and automatically forwarding them to the post-questionnaire to report on their experiences.

Both versions of the learning environment were identical in content, scope, and functionality. The experimental variable being visual structure, the only difference between the experimental and control version of the environment was said structure. The experimental environment was highly visually unstructured, whereas the control environment was highly structured. Specifically, the control version displayed the contents and topics of the course in a 4 × 4 grid with the rows displaying thematically related topics. The experimental version displayed topics according to no recognizable visual structure, instead appearing cloud-like and random. Within each topic, the control version displayed the materials and texts in a straightforward vertical manner. The experimental version differed from this, each topic having a different visual layout such that elements appeared unordered (Fig. 3).

Fig. 3

Schematic visualization highlighting the design differences between experimental conditions

Recruitment

Participants for this study were recruited from the student pool of the FernUniversität in Hagen, Germany. In two student-led virtual hangout spaces, an announcement for the upcoming study was posted, asking for participation. Additionally, participants were recruited through mailing lists of the FernUniversität in Hagen, Hochschule der Bildenden Künste, as well as Hochschule für Technik und Wirtschaft. No financial reward or course credit was associated with participation and participants were informed about the general aim and structure of the experiment as well as the estimated time to complete the survey. Students provided informed consent to participate in the study and the usage of their pseudonymized data for research purposes. Minimum required age for participation was sixteen (mainly due to the age-referenced self-report scales for creativity). Assignment to experimental and control group was done though the integrated randomize function of “SoSci Survey” (https://www.soscisurvey.de), a widely used web application for developing questionnaires and collecting data in Germany. As a result of the randomize function, participants received one of two URL links, which lead them either to the experimental or control group. Participants engaged with the learning environment and reported their data from their personal computers. Because participation was generally not mandatory and the survey was quite comprehensive, there was no time limit associated with participation. Also, we expected that being free of time constraints may facilitate creative behavior (Davies et al., 2013). Upon exiting the learning environment and entering the posttest questionnaire, participants were asked to indicate the amount of time they spent in the environment. Mean time spent in the learning environment was reported to be 7 min, although with a large variation, SD = 6.8 (minimum = 1 min, maximum = 45 min).

Sample

This sample consists of a final number of participants of N = 187, of which 116 (62%) identified as female, 68 (36.4%) as male, and one missing value. Mean age is 41.3 (SD = 16). Most participants (71.7%, n = 134) indicated they are not vocationally or educationally involved in the creative domain, a minority indicated being enrolled in a creative program (3.2%, n = 6), whereas one-fourth indicated working in a creative profession (25.1%, n = 47). Examples of creative vocation or creative study program include, e.g., graphical design, software development, and fiction writing, whereas relatively non-creative domains include IT-Security, social work, and political science. Fisher’s exact test yielded no significant association between gender and vocational or educational enrollment choice (odds ratio: 1.6, p = 0.17). Most participants used their personal computer to take part in the experiment (78.6%, n = 147), while one-fifth used a mobile device (21.4%, n = 40).

Measures

Variables of interest for this study were measured either directly by verbatim utilizing validated (sub-)scales in their German version or, as in the case for “preference for visual structure” and “impulse for learning” where not adequate items were identified, by constructing a set of face-valid items based on the construct of interest. Particularly for our focal construct, creativity, we focused on using measures that are well-validated in German, to ensure their psychometric quality for our target group. In addition, we searched for measures that could be completed in a reasonable timeframe, so as to not put off our participants, as we did not offer a financial incentive. All Likert scales were measured on a five-point-basis ranging from “strongly disagree” to “strongly agree” and demonstrated sufficient internal consistency above the threshold of 0.7 (Table 1).

Table 1 Measures used for this study, with number of items, example items, and internal consistency

Openness as an individual disposition for curiosity und orientation toward the environment is a construct frequently associated with creative personalities (Barron, 1964; Guilford, 1975; Schachtel, 1959; Silvia et al., 2014). A well-established measure of personality is the Revised Personality Inventory (NEO-PI-R), a questionnaire tapping the Big Five personality model (Costa & McCrae, 1995). Openness to experience as a dimension of the Big Five model has been found to predict creative behavior (Silvia et al., 2014) and is constituted by several facets like openness for fantasy, for aesthetics, for emotions, and for ideas, that are comprehensively measured by 48 items. For survey-economic purposes, we focused on the subscales fantasy, aesthetics, and ideas. As creative behavior does not necessarily follow from creative personality (see literature review), we conceptualized creative behavior as separate dimension of creativity and measured this via openness for behavior, also using the NEO-PI-R. Additional items closely associated with the educational domain were added from a scale used in Kim and Hull (2012). These items referred to, for example, unconventional behavior in learning settings.

Creative production was measured via three production tasks, derived from the Berliner Intelligenzstruktur-Test (BIS), a diagnostic tool for estimating intelligence (Jäger et al., 1997). This is a well-validated test that is used frequently in practice, for example, in work-aptitude tests. Because this comprehensive test also includes established measures ingenuity via creative production, we chose three subtasks that appeared most relevant for our focal construct of creativity, (1) continuation and completion of a presented figure, (2) layouting a diverse set of forms and figures based on a prompted term, and (3) configuring and combining a given form into different figures (Table 1). Important for these tasks was spontaneous and non-deliberate production. Creativity operationalized creative production was then estimated by counting the amount of produced elements. Although not a psychometric scale in the literal sense, the three tasks yielded an internal consistency of 0.68.

Deriving from work reporting the relevance of the design of the learning environments (Goldschmidt & Smolkov, 2006; Hu, 2008; Yang & Chang, 2009), we used three dependent variables. The first one captures students’ preference for visual structure via four items explicitly tapping how participants liked the design of learning environment in terms of its visual structure. As our search did not uncover a validated German scale covering this specific content, we constructed four face-valid items (Appendix 1). The second instrument was designed to capture the impulse for activation that students perceived within the learning environment and was measured with three items specifically tapping the participants’ tendency to engage with the learning materials of the environment. Here, too, we found no scale fitting our requirements, leading us so generate three items to tap this variable. Finally, as a third dependent variable, we chose situational motivation, which we expect to be the downstream variable of preference for visual structure and impulse for activation (Fig. 1). This was measured using four items of the German validated Fragebogen zur aktuellen Motivation (FAM) questionnaire (Rheinberg et al., 2001), which we adapted to fit the context of this study. The FAM has also been employed and discussed in subsequent work (Rheinberg et al., 2002; Vollmeyer & Rheinberg, 2006). All three scales show sufficient internal consistency, as measured by Cronbach’s alpha (see Table 1).

Analyses

To account for the possibility that self-reported creative personality and creative behavior are not predictive as singular dimensions (“dimensional approach,” see Schnabel et al., 2002), instead, creativity configurations (i.e., typologies) are probed as moderators. These configurations will be detected via a two-step clustering algorithm, with log-likelihood as distance measure and Schwarz Bayesian criterion as clustering criterion. To answer our research questions, we compared judgements and experiences between two experimental group, making this a between-subject design. In cases where our creativity indicator is dichotomous (cluster membership and vocational choice), we thus analyze simple main effect and interaction with analysis of variance (ANOVA). We use analysis of covariance (ANCOVA) for analyses that included creative production, as this was measured continuously. Tests of homogeneity of variances as measured by Levene’s test and Bartlett’s test were nonsignificant for all analyses. Normality test via Kolmogorov–Smirnov tests were also nonsignificant, so that assumptions for ANOVA and ANCOVA are met.

To calculate statistical sensitivity for our most complex analysis, the interaction effect in ANCOVA, we used G*Power (Faul et al., 2007). Setting an alpha error probability at 5%, and Power of 0.8 (beta error of 20%), one covariate per model, two groups, and a total sample of 187 yields a lowest detectable effect of f = 0.23, which equals η² = 0.016.

Results

Descriptives

Descriptive information of independent variables is presented in Table 2. As all Likert-based variables are measured on a 5-point scale, mean self-reported creative personality is well above the scale midpoint, whereas self-reported creative behavior is only slightly so. Notably, creative production tasks are right-skewed with many participants producing zero task-conforming figures. However, some participants produced very many figures, up to the maximum of twenty. Table 2 also reports on correlation coefficients between self-reported creativity and creative production. Of six possible combinations, four are statistically significant and positive, suggesting construct validity for these measures. Correlations between creative production tasks are all significantly positive. Descriptive data for dependent variables are presented in Table 3. Here, means are around the scale midpoints, with preference for visual structure yielding the highest mean and impulse for activation yielding the lowest mean. Welch’s t test yielded no gender differences for creative production tasks (Continuation: t(140.471) = 0.07, p = 0.94; Layout: t(160.065) = 0.11, p = 0.92; Combination: t(165.220) = 0.62, p = 0.54). However, self-reported creative personality, t(133.689) = 3.78, p < 0.001, as well as self-reported creative behavior was significantly higher for women, t(149.379) = 2.14, p = 0.034. There was no significant difference in creative personality and creative behavior between experimental conditions, t(169.110) = 0.84, p = 0.40, t(182.660) = 0.67, p = 0.50, respectively. The experimental group was slightly larger (n = 96) than the control group (n = 90) and consisted of 57 women, whereas the control group consisted of 59 women. The experimental group had a slightly higher age (M = 41.9, SD = 13.2) compared to the control group (M = 40.3, SD = 12.8). However, none of these differences were statistically significant.

Table 2 Descriptives for creativity variables and bivariate correlation coefficients

Table 3 Descriptives for dependent variables

Cluster analysis

In order to detect distinct personality configurations, a cluster analysis was performed on the variables self-reported creative personality and self-reported creative behavior. Cluster analysis is an approach of statistically identifying meaningful subgroups in data. Broadly speaking, it can be seen as similar to factor analyses with the main difference in practice being that cases (i.e., respondents) instead of items are clustered into meaningful groups. A two-step clustering algorithm was chosen, which in a first step pre-clusters the data and then goes on to confirm the resulting structure in hierarchical clustering (Bacher et al., 2004). This approach has been shown to reliably reproduce subgroups of varying complexity (Kent et al., 2014) and overcomes limitations of other clustering techniques, like hierarchical or k-means clustering (Bacher et al., 2004; Everitt et al., 2011). Log-likelihood was chosen as distance measure and Schwarz Bayesian criterion was chosen as clustering criterion. The algorithm proposed a two-cluster solution with a silhouette coefficient above 0.5, indicating “good” cluster quality. Silhouette coefficients can range from − 1 to 1, with higher positive value indicating greater compactness and separation of the proposed cluster solution (Han et al., 2011). The two clusters are different in size, with a ratio of large cluster to small cluster of 1.53. Inspecting the relative importance of both predictors, we find creative personality to be the stronger predictor, with creative behavior contributing 20% less in terms of prediction. The clusters are summarized along their main features in Table 4. Women were significantly more likely to be members of the first cluster than men (odds ratio: 2.65, p = 0.003). An independent sample t test (Welch’s) using cluster membership as grouping variable and summed creative production as dependent variable suggests a significant difference, t(185) = 3.01, p = 0.003, d = 0.45, providing evidence for predictive ability of creativity clusters.

Table 4 Cluster descriptive for self-reported creativity

Preference for visual structure

A 2 × 2 analysis of variance (ANOVA) with experimental condition and cluster membership revealed a simple main effect for experimental condition, such that the high-structure design (M = 3.44, SD = 0.86) was preferred to the low-structure design (M = 3.04, SD = 0.90), F(1, 182) = 10.73, p = 0.001, d = 0.49. There was no such effect for cluster membership, as students in the high creativity cluster (M = 3.25, SD = 0.89) did not indicate significant higher preference than the low creativity cluster (M = 3.25, SD = 0.92), F(1, 182) = 0.07, p = 0.8, d = 0.04. Finally, there was no significant interaction between experimental condition and cluster membership, F(1, 182) = 2.2, p = 0.14.

A 2 × 2 ANOVA with experimental condition and vocational choice revealed no simple main effect for experimental condition, such that the high-structure design (M = 3.32, SD = 0.80) was not rated significantly different from the low-structure design (M = 3.05, SD = 0.87), F(1, 182) = 3.90, p = 0.050, d = 0.32. There was also no effect for vocational choice, as students in creative jobs or study programs (M = 3.06, SD = 0.85) did not indicate significant higher preference than those in non-creative jobs or study programs (M = 3.31, SD = 0.81), F(1, 182) = 3.38, p = 0.068, d = 0.3. Finally, there was no significant interaction between experimental condition and vocational choice, F(1, 182) = 2.5, p = 0.12.

Because creative production is a continuous predictor, an analysis of covariance (ANCOVA) was carried out with experimental condition as fixed factor and creative production as covariate. Results suggest a simple main effect for experimental condition, such that the high-structure design (M = 3.43, SD = 0.83) was rated significantly different from the low-structure design (M = 3.06, SD = 0.81), F(1, 182) = 4.69, p = 0.032, d = 0.44. There was no such effect for creative production F(1, 182) = 0.16, p = 0.69, as well as no significant interaction between experimental condition and creative production, F(1, 182) = 0.133, p = 0.72.

Situational motivation

A 2 × 2 ANOVA with experimental condition and cluster membership revealed no simple main effect for experimental condition, such that the high-structure design (M = 3.01, SD = 1.07) was not rated significantly different with regards to situational motivation than the low-structure design (M = 2.78, SD = 1.15), F(1, 182) = 2.44, p = 0.16, d = 0.21. Further, there was no simple main effect for cluster membership, as students in the high creativity cluster (M = 2.91, SD = 1.09) did not indicate significant higher situational motivation than the low creativity cluster (M = 2.88, SD = 1.13), F(1, 182) = 0.03, p = 0.86, d = 0.027. Finally, there was a significant interaction between experimental condition and cluster membership, F(1, 182) = 4.37, p = 0.038 (Fig. 4).

Fig. 4

Interaction effect of creativity_cluster*group for situational motivation

A 2 × 2 ANOVA with experimental condition and vocational choice revealed no simple main effect for experimental condition, such that the high-structure design (M = 2.86, SD = 1.12) was not rated significantly different from the low-structure design in terms of situational motivation (M = 2.90, SD = 1.05), F(1, 182) = 0.05, p = 0.83, d = 0.036. There was also no effect for vocational choice, as students in creative jobs or study programs (M = 2.85, SD = 1.04) did not indicate significant higher situational motivation than those in non-creative jobs or study programs (M = 2.91, SD = 0.95), F(1, 182) = 1.138, p = 0.71, d = 0.061. Finally, there was a significant interaction between experimental condition and vocational choice, F(1, 182) = 6.38, p = 0.012. (Fig. 5)

Fig. 5

Interaction effect of vocation_study_choice*group for situational motivation

Because creative production is a continuous predictor, an ANCOVA was carried out with experimental condition as fixed factor and creative production as covariate. Results suggest no simple main effect for experimental condition, such that the high-structure design (M = 2.99, SD = 1.01) was not rated significantly different from the low-structure design in terms of situational motivation (M = 2.82, SD = 0.98), F(1, 182) = 0.23, p = 0.63, d = 0.15. There was no such effect for creative production F(1, 182) = 0.76, p = 0.39, as well as no significant interaction between experimental condition and creative production, F(1, 182) = 0.03, p = 0.85.

Impulse for activation

A 2 × 2 ANOVA with experimental condition and cluster membership revealed no simple main effect for experimental condition, such that the high-structure design (M = 2.90, SD = 1.23) did not differ in terms of impulse for activation from the low-structure design (M = 2.73, SD = 1.19), F(1, 182) = 0.84, p = 0.36, d = 0.14. Also, there was no effect for cluster membership, as students in the high creativity cluster (M = 2.83, SD = 1.16) did not indicate significant higher impulse for activation than the low creativity cluster (M = 2.80, SD = 1.04), F(1, 182) = 0.03, p = 0.86, d = 0.03. Finally, there was no significant interaction between experimental condition and cluster membership, F(1, 182) = 2.25, p = 0.14.

A 2 × 2 ANOVA with experimental condition and vocational choice revealed no simple main effect for experimental condition, such that the high-structure design (M = 2.77, SD = 1.12) was not rated significantly different from the low-structure design in terms of impulse for activation (M = 2.77, SD = 1.13), F(1, 182) = 0.00, p = 0.1, d = 0.0. There was also no simple main effect for vocational choice, as students in creative jobs or study programs (M = 2.64, SD = 1.07) did not indicate significant higher impulse for activation than those in non-creative jobs or study programs (M = 2.89, SD = 1.03), F(1, 182) = 1.64, p = 0.20, d = 0.021. Finally, there was no significant interaction between experimental condition and vocational choice, F(1, 182) = 1.34, p = 0.25.

Because creative production is a continuous predictor, an ANCOVA was carried out with experimental condition as fixed factor and creative production as covariate. Results suggest no simple main effect for experimental condition, such that the high-structure design (M = 2.88, SD = 1.21) was not rated significantly different from the low-structure design in terms of impulse for activation (M = 2.77, SD = 1.19), F(1, 182) = 0.43, p = 0.52, d = 0.09. There was no effect for creative production F(1, 182) = 1.10, p = 0.31, as well as no significant interaction between experimental condition and creative production, F(1, 182) = 0.11, p = 0.74.

Discussion

Despite the increasing recognition of creativity as an important variable for learning design as well as technology-enhanced learning more specifically, the body of research is still emerging and relatively scattered. A largely uncharted area of this topic is creativity as an individual difference variable that may be a powerful predictor of differential learning experiences. In order to provide a starting point for exploring this, the present study investigated the effects of the visual design of the learning environments interacting with the creativity of the students, assessed through three different measurement approaches.

How predictive is vocational and study choice (creative vs. non-creative) for understanding variation in learning experiences in self-paced environments?

As our review of the literature yielded only very little research with a similar question as ours, it seemed necessary to replicate and extend the study of Rosar et al. (2018) in order to build a robust and informative research line. For this reason, we included vocational and study choice as one indicator for creativity. In line with the results of Rosar et al. (2018) we, too, found an interaction between the design of the learning environment and this indicator, such that students working or studying in the creative domain demonstrated preferences for the unstructured learning environment, compared to their peers in working or studying in comparably non-creative domains. However, our findings slightly deviate from those of Rosar et al. (2018) as our analysis suggests a crossover interaction, where the effects of design features become fully reversed. It is also important to note the differences in research questions. Whereas our experimental manipulation was the visual design, Rosar et al. (2018) manipulated the pedagogical design by treating participants with either a fully guided or an inquiry-based environment. Our research goes beyond the findings of Rosar et al. (2018) by not only investigating differential preferences but also the results of such preferences, like situational motivation for learning, an outcome variable especially important for self-paced online learning. Thus, alongside Rosar et al. (2018), our findings provide further evidence for the notion of interaction effects associated with the creativity of students.

How relevant is the visual structure for eliciting differential learning experiences in relatively creative versus relatively non-creative students?

In line with our understanding of creativity and the importance of visual design features, our results suggest an interaction between levels of creativity and the visual design of the learning environment, with respect to outcome variables. Importantly, we do not find evidence that one of our experimental conditions is per se superior to the others. Although the high-structure environment was preferred by some students, this did not affect situational motivation or impulse for activation robustly, as indicated by the mostly nonsignificant main effects of condition. Naively, this could be interpreted to mean that the visual design does not matter for learning. However, this would be a hard-to-defend claim, given the substantial literature on the design of multimedia learning material (Mayer and Moreno 2003; Mayer, 2017; Scheiter & Eitel, 2015; Richter et al., 2016) and online learning environments (Cho & Heron, 2015; Hu, 2008). Instead, strengthening the interactional assumption of our investigation, the consideration of creativity of students changes this picture dramatically. Interaction effects suggest that situational motivation is affected by the design of the environment conditional on the extent to which students are creative or not.

In general, these findings align with propositions of classic literature of Arnheim (1947), Guilford (1975), and Barron (1953) suggesting a preference of rich an unorganized stimuli of creative people. More recent empirical findings have suggested similar relationships in the processing of visual stimuli (Chamberlain & Wagemans, 2015; McCoy & Evans, 2002). Importantly, our findings add to the literature by providing a proof-of-concept of similar mechanisms for creative students in self-paced online learning environments.

Due to the unique characteristics of online learning, especially when it is self-paced, motivationally well-designed learning experiences are an imperative (Cho & Heron, 2015; Hu, 2008). Our study shows that design guidelines may differ, depending on the characteristics of the target group, i.e., what is good for relatively non-creative students may not serve very creative students in the same way. As increasing nuance in our design guidelines are an important step for intentional instructional design, this study adds one piece to the puzzle of recognizing creativity as an important auxiliary variable. Crucially, designers of self-paced online learning environments should keep the creativity of their target group in mind, in order to design appropriately motivating and effective learning experiences. For example, if an online course in higher education is developed for students in a creative study program, designers may consider a relatively unstructured design in order to foster motivation in these students. On the other hand, if the target group is outside of traditionally creative programs, designers should emphasize a highly structured and streamlined visual structure.

How do different methods of measuring creativity interact with the design of the online learning environments to affect learning experiences?

In terms of predictive abilities of the different creativity assessments, we found that self-reported creative personality and creative behavior (summarized through cluster membership) as well as vocational/study choice yielded an interaction effect, whereas the three creative production tasks did not. This can be taken to mean that self-reported creativity as well as vocational/study choice may reflect enduring and strong individual creativity beliefs and choices, respectively, whereas the creative production tasks in the experimental procedure may also be strongly influenced by situational or even mundane factors, like the ability and willingness to physically draw figures with their current technological interface (personal computer, mobile phone).

Interestingly, the interaction between creativity and visual design had relatively consistent effects on situational motivation, whereas preference for visual structure and impulse for activation were not affected. This is surprising as the preference for visual structure was expected to be the more foundational variable, from which firstly impulse for activation and secondly situational motivation would emerge. Also, it was expected that the more highly structured design would outperform the control group across groups in all dependent variables, but this difference would be diminished by accounting for creativity. Instead, our findings paint a different picture, in which (1) the high visual structure is generally preferred but not when controlling for creativity, (2) impulse for activation does not vary by group nor by group*creativity interaction, while (3) situational motivation is consistent between groups but is differentially affected by the interaction group*creativity.

Limitations

This study is not without limitations. For one, the inconclusive predictive ability of creative production tasks may be due to two limitations. Firstly, as mentioned above, creative production was mediated through digital interfaces, some of which may not be conducive to spontaneous production, like touch-screen displays for drawing figures. Indeed, answers to open-ended feedback questions indicated some participants found it cumbersome to attempt drawing with the mouse, so that this may have been a confounding factor, possibly limiting the validity of the measure. For this reason, creative potential may not have been fully tapped or there may be bias. Second, creative production was evaluated quantitatively through the amount of figures produced. Of course, the quality and characteristics of such productions may be similarly indicative, if not more, of creative potential. However, this type of evaluation was beyond the reach of this study. Future research may attempt to elicit creative production in more standardized and controlled settings with a stronger focus on qualitative evaluation of productions. One example of this would be, for example, the widely used Torrance Test of Creative Thinking (Kim, 2006; Torrance, 1966).

One explanation of the discrepancy between the effects on impulse for activation and situational motivation may be the lack of rearrangement possibilities in the presented learning environments. Visually unstructured stimuli may be perceived negatively in general, as evidenced by the main differences in preference, while creative students may perceive an activating impulse for engaging with this type of stimuli, for example, to rearrange and reconfigure. Thus, visually unstructured stimuli may afford creative engagement. However, these demand characteristics may have been stifled by the inability to actually interact with this visual structure, as the design of the experimental environment did not include this feature. As a result, creative students impulse for activation may have been thwarted, possibly explaining the lack of an effect on this variable.

While we investigated important variables like situational motivation, we did not assess actual student learning. This is mainly due to the fact that we attempted to choose an interesting domain in order to solicit as many participants as possible while remaining cognizant of the fact that participants self-selected into the study and may be differentially knowledgeable about the topic. Without controlling these factors, an assessment of learning may be biased. Due to practical constraints hindering such controlled experimental design, we decided to focus on antecedents of actual learning, hypothesizing differential effects for these variable will translate to learning. However, to formulate robust design recommendations, future research may make sure to confirm this hypothesis may assessing student learning in more controlled online settings while avoiding self-selecting samples.

Another important limitation is that only one of our three dependent variables was measured via validated scale in German language. Due to this, we cannot rule out the possibility that we sacrificed validity in measuring preference for visual structure and impulse for activation. Although our measures showed sufficient internal consistency, the results of our analyses should be interpreted accordingly.

Finally, the relatively short duration that our participants remained in the learning environment may have biased our results, as this means that our experimental conditions were only experienced briefly. It is possible that this may have attenuated effects as we expect that longer exposure to the experimental conditions will make their psychological impacts more salient to the individual, in line with psychological research findings that longer presentation of stimuli was associated with more pronounced psychological effects (Reber et al., 1998). Future research may want to incorporate this notion by planning research designs that ensure longer self-paced learning times in the environment.

Conclusions and implications

In this experimental exploration, we provided evidence for an interaction between creativity of students and the visual design of an online self-paced learning environment. Thus, creativity as an individual difference variable calls our attention to the necessity of providing more nuanced and context-specific design guidelines. From a methodological standpoint, this study provides novel experimental evidence that even relatively small design features in the learning environment can (a) affect student experiences and (b) affect them differentially according to individual differences. The fact that our experimental assignment was randomized lends credence to interpreting this as a causal effects of the environment, instead of a confounding situation, where, for example, students self-select into learning experiences based on their preferences or preconditions. From a theoretical perspective, the results of this study are valuable in integrating two conflicting lines of research, one suggesting well-structured and highly ordered learning environments for most students, the other pointing to the benefits of unorganized stimuli and environments for creative individuals. We resolve this conflict by reporting on an interaction effect that is the result of a hitherto relatively under-researched individual difference in education, trait creativity. Practically, researchers may use this proof-of-concept as a starting point for further investigations, for example, by exploring different pedagogical approaches or different design aspects with respect to creative students. Mapping out these potential interactions is beneficial for arriving at design recommendations that take student differences in creativity into account. This may be particularly relevant for designing learning experiences in creative domains like graphic design, visual arts, product design, and others. Educational practitioners may make use of these findings by designing intentional and context-specific online learning experiences for their students. Given these preliminary results, practitioners in creative domains may improve student experiences by providing learning environments that are rich in stimuli and non-highly structured.

Appendices

Appendix 1

Items for dependent variables

Preference for visual structure

• I liked the visual structure of this environment.

• The visual structure of the environment bothered me. (r)

• I would like to change the design of the learning environment, if I could. (r)

• I liked how I was could easily orient myself in this environment.

Impulse for activation

• The learning environment stimulated me to become active and explore it.

• I wanted to exit the learning environment quickly. (r)

• The learning environment stimulated me to engage with the material.

Situational motivation

• I don’t enjoy learning experiences like these. (r)

• This learning environment and its materials interested me a lot.

• I would not like to learn in an environment like this in my free time. (r)

• With learning tasks like this, I need no reward, it is enjoyable on its own.

Appendix 2

Screenshots of experimental conditions in the learning environment (top: control group/high visual structure, bottom: experimental group/low visual structure)