Peerj 289
Peerj 289
Peerj 289
ABSTRACT
For more than thirty years, it has been claimed that a way to improve software de-
velopers’ productivity and software quality is to focus on people and to provide
incentives to make developers satisfied and happy. This claim has rarely been verified
in software engineering research, which faces an additional challenge in compari- son
to more traditional engineering fields: software development is an intellectual activity
and is dominated by often-neglected human factors (called human aspects in
software engineering research). Among the many skills required for software
development, developers must possess high analytical problem-solving skills and
creativity for the software construction process. According to psychology research,
affective states—emotions and moods—deeply influence the cognitive processing
abilities and performance of workers, including creativity and analytical problem
solving. Nonetheless, little research has investigated the correlation between the
affective states, creativity, and analytical problem-solving performance of program-
mers. This article echoes the call to employ psychological measurements in software
engineering research. We report a study with 42 participants to investigate the rela-
tionship between the affective states, creativity, and analytical problem-solving skills
of software developers. The results offer support for the claim that happy developers
are indeed better problem solvers in terms of their analytical abilities. The following
Submitted 25 July 2013 contributions are made by this study: (1) providing a better understanding of the
Accepted 4 February 2014 impact of affective states on the creativity and analytical problem-solving capacities of
Published 11 March 2014 developers, (2) introducing and validating psychological measurements, theories, and
Corresponding author Daniel concepts of affective states, creativity, and analytical-problem-solving skills in empirical
Graziotin, software engineering, and (3) raising the need for studying the human factors of
[email protected]
software engineering by employing a multidisciplinary viewpoint.
Academic editor
Shane Mueller
Additional Information and
Declarations can be found on Subjects Psychiatry and Psychology, Human–Computer Interaction, Statistics
page 18 Keywords Emotion, Affective state, Software development, Analytical problem-solving, Feeling,
DOI 10.7717/peerj.289 Creativity, Mood, Human factors, Human aspects, Affect
Copyright
2014 Graziotin et al. INTRODUCTION
Distributed under For more than thirty years, it has been claimed that a way to improve software developers’
Creative Commons CC-BY 3.0
productivity and software quality is to focus on people (Boehm & Papaccio, 1988). Some
OPEN ACCESS strategies to achieve low-cost but high-quality software involve assigning developers
How to cite this article Graziotin et al. (2014), Happy software developers solve problems better: psychological measurements in
empirical software engineering. PeerJ 2:e289; DOI 10.7717/peerj.289
private offices, creating a working environment to support creativity, and providing
incentives (Boehm & Papaccio, 1988), in short, making software developers satisfied and
happy. Several Silicon Valley companies and software startups seem to follow this advice
by providing incentives and perks to make their developers happy (Drell, 2011; Google Inc.,
2014; Stangel, 2013) and, allegedly, more productive (Marino & Zabojnik, 2008).
Human factors (called human aspects in software engineering) play an important role
in the execution of software processes and the resulting products (Colomo-Palacios et al.,
2010; Feldt et al., 2010; Sommerville & Rodden, 1996). This perception of the importance of
human aspects in software development, e.g., “Individuals and interactions over processes
and tools”, led to the publication of the Agile manifesto (Beck et al., 2001). As noted by
Cockburn & Highsmith (2001), “If the people on the project are good enough, they can
use almost any process and accomplish their assignment. If they are not good enough,
no process will repair their inadequacy—‘people trump process’ is one way to say this.”
(p. 131). This claim has received significant attention; however, little evidence has been
offered to verify this claim in empirical software engineering research.
The software engineering field faces an additional challenge compared with more
traditional engineering fields; software development is substantially more complex than
industrial processes. The environment of software development is all but simple and
predictable (Dyba˚, 2000). Much change occurs while software is being developed, and
agility is required to adapt and respond to such changes (Williams & Cockburn, 2003).
Software development activities are perceived as creative and autonomous (Knobelsdorf
& Romeike, 2008). Environmental turbulence requires creativity to make sense of the
changing environment, especially in small software organizations (Dyba˚, 2000). The
ability to creatively develop software solutions has been labelled as critical for software
firms (Ciborra, 1996; Dyba˚, 2000) but has been neglected in research.
The software construction process is mainly intellectual (Darcy & Ma, 2005; Glass,
Vessey & Conger, 1992). Recently, the discipline of software engineering has begun to adopt
a multidisciplinary view and has embraced theories from more established disciplines,
such as psychology, organizational research, and human–computer interaction. For
example, Feldt et al. (2008) proposed that the human factors of software engineering could
be studied empirically by “collecting psychometrics”.1 Although this proposal has begun
1
The software engineering literature has to gain traction, limited research has been conducted on the role of emotion and mood on
sometimes used the term
psychometrics to describe general software developers’ skills and productivity.
psychological measures that might be As human beings, we encounter the world through affects; affects enable what matters
used along with other software
development metrics. However, in our experiences by “indelibly coloring our being in the situation” (Ciborra, 2002, p.
psychometrics has 161). Diener et al. (1999) and Lyubomirsky, King & Diener (2005) reported that
a specific meaning within psychological
research and involves establishing the numerous studies have shown that the happiness of an individual is related to
reliability and validity of a achievement in various life domains, including work achievements. Indeed, emotions
psychological measurement. In this
article, we use the more appropriate play a role in daily jobs; emotions pervade organizations, relationships between
term of psychological measurement to workers, deadlines, work motivation, sense-making and human-resource processes
refer to this concept.
(Barsade & Gibson, 2007). Although emotions have been historically neglected in
studies of industrial and organizational psychology (Muchinsky, 2000), an interest in the
role of affect on job
All of the students participated in the affective states measurement sessions and the two
experimental tasks. However, the results of one participant from the creativity task and
another from the analytical problem-solving task have been excluded; the two participants
did not follow the instructions and submitted incomplete data. Therefore, the sample
size for the two experiment tasks was N = 41. None of the participants reported previous
experience with the tasks.
Materials
For the two affective states measurement sessions, the participants completed the SPANE
questionnaire through a Web-based form, which included the related instructions. The
SPANE questionnaire instructions that were provided to the participants are available in
the article by Diener et al. (2009a) and are currently freely accessible on one of the author’s
academic website (Diener et al., 2009b).
Six color photographs with ambiguous meanings were required for the creativity
task. Figure 1 displays one of the six photographs. For legal reasons, the
photographs are available from the authors upon request only.
For the analytical problem-solving task, a version of the Tower of London task
implemented in the open source Psychology Experiment Building Language (PEBL;
Mueller & Piper, 2014; Mueller, 2012) that has been used previously to examine age-related
changes in planning and executive function (Piper et al., 2011) was used to assess analytic
problem solving. The PEBL instructed the participants, provided the task, and collected
several metrics, including those of interest for our study. One computer per participant was
required.
Procedure
The experimental procedure was composed of four activities: (1) the affective
states measurement (SPANE), (2) the creativity task, (3) the affective states
measurement
(SPANE), and (4) the analytical problem-solving task. The second affective states
measurement session was conducted to limit the threats to validity because the first task
may provoke a change in the affective states of the participants.
The participants arrived for the study knowing only that they would be participating in
an experiment. As soon as they sat at their workstation, they read a reference sheet, which is
included in Article S1. The sheet provided a summary of all of the steps of the experiment.
The researchers also assisted the participants during each stage of the experiment. The
participants were not allowed to interact with each other.
During the creativity task, the participants received two random photographs from the
set of the six available photographs, one at a time. The participants imagined participating
in the Best Caption of the Year contest and tried to win the contest by writing the best
captions possible for the two photographs. They wrote as many captions as they wanted
for the pictures. The creativity task instructions are available as an appendix in the study by
Forgeard (2011).
During the analytical problem-solving task, the participants opened the PEBL software.
The software was set up to automatically display the Tower of London game, namely the
Shallice test ([1, 2, 3] pile heights, 3 disks, and Shallice’s 12 problems). The PEBL software
displayed the instructions before the start of the task. The instructions stated how the game
works and that the participants had to think about the solution before starting the task,
i.e., making the first mouse click. Figure 2 provides a screenshot of the first level of the
game. Because PEBL is open-source software, the reader is advised to obtain the PEBL
software to read the instructions.
Although the participants did not have strict time restrictions for completing the tasks,
they were advised of the time usually required to complete each task and that the second
task would begin only after every participant finished the first task.
Measures
To measure creativity according to the Consensual Assessment Technique (Amabile, 1982),
independent judges who are experts in the field of creativity scored the captions using a
Likert-item related to the creativity of the artifact to be evaluated. The judges had to use
their own definition of creativity (Amabile, 1982; Kaufman et al., 2007). The Likert-item
is represented by the following sentence: This caption is creative. The value associated to
the item ranges from one (I strongly disagree) to seven (I strongly agree). The judges were
blind to the design and the scope of the experiment. That is, they received the six pictures
with all of the participants’ captions grouped per picture. The judges were not aware of
the presence of other judges and rated the captions independently. Ten independent
judges
were contacted to rate the captions produced in the creativity task. Seven judges responded,
and five of the judges completed the evaluation of the captions. These five judges included
two professors of Design & Arts, two professors of humanistic studies, and one professor of
creative writing.
The present study adopted measurements of quality and quantity for the assessment of
creativity. The quality dimension of creativity was measured by two scores. The first quality
score was the average of the scores assigned to all of the generated ideas of a participant
(ACR). The second quality score was the best score obtained by each participant (BCR), as
suggested by Forgeard (2011) because creators are often judged by their best work rather
than the average of all of their works (Kaufman et al., 2007). The quantity dimension was
represented by the number of generated ideas (NCR), as suggested by Sowden & Dawson
(2011).
Measuring analytical problem-solving skills is less problematic than measuring creativ-
ity. There is only one solution to a given problem (Cropley, 2006). The common
approach in research has been to assign points to the solution of analytical tasks (Abele-
Brehm, 1992; Melton, 1995). This study employed this approach to combine measures
of quality and quantity by assigning points to the achievements of analytical tasks and by
measuring the time spent on planning the solution. The Tower of London game (a.k.a.
Shallice’s test) is a game aimed to determine impairments in planning and executing
solutions to analytical problems (Shallice, 1982). It is similar to the more famous Tower
of Hanoi game in its
RESULTS
The data were aggregated and analyzed using the open-source R software (R Core Team,
2013). The SPANE-B value obtained from this measurement session allowed us to estimate
the SPANE-B population mean for software developers, µSPANE-B-DEV= 7.58, 95% CI
[5.29, 9.85]. The median value for the SPANE-B was nine. This result has consequences in
the discussion of our results which we offer in the next section.
The multiple linear and polynomial regression analyses on the continuous values for
the various SPANE scores and the task scores did not yield significant results.
Therefore, the data analysis was performed by forming two groups via a median split of
the SPANE-B score. The two groups were called N-POS (for non-positive) and POS (for
positive). Before the creativity task, 20 students were classified as N-POS and 21 students
were classified as POS.
The histograms related to the affective state distributions and the group compositions
have been included as supplemental files of this article (Figs. S1 and S7). These data
are not crucial for the purposes of this investigation. However, they have been attached to
this article for the sake of completeness. The same holds for the boxplots and the
scatterplots representing non-significant data.
Table 1 summarizes the task scores of the two groups for the two tasks. The two
creativity scores of ACR and BCR showed many commonalities. Visual inspections of
the scatterplots of the ACR (Fig. S5) and BCR (Fig. S6) scores versus the SPANE-B score
suggested a weak trend of higher creativity when the SPANE-B value tended to its extreme
values (−24 and +24). The median for the number of generated captions (NCR) was
four for the N-POS group and six for the POS group. However, the lower quartiles of
the two groups were almost the same, and there was a tiny difference between the two
upper quartiles (Fig. S4).
We hypothesized that affective states would impact the creative work produced by
software developers, without a direction of such impact. The hypothesis was tested using
unpaired, two-tailed t-tests. There was no significant difference between the N-POS and
POS groups on the BCR score (t(39) = 0.20, p > .05, d = 0.07, 95% CI [−0.43, 0.53]) or
the ACR score (t(39) = 0.31, p > .05, d = 0.10, 95% CI [−0.28, 0.38]). The third test, which
regarded the quantity of generated creative ideas (NCR), required a Mann–Whitney U test
because the assumptions of normality were not met (Shapiro–Wilk test for normality,
N-POS POS
Variable M (SD) 95% CI M (SD) 95% CI
ACR 3.13 (0.45) [2.92, 3.35] 3.08 (0.58) [2.81, 3.35]
BCR 4.02 (0.76) [3.67, 4.38] 3.98 (0.76) [3.63, 4.32]
NCR 4.70 (2.34) [3.60, 5.50] 5.90 (3.46) [4.00, 7.50]
APS 0.14 (0.04) [0.12, 0.17] 0.20 (0.08) [0.17 0.25]
Notes.
ACR, the average of the scores assigned to all of the generated ideas of a participant; BCR, the best score obtained
by each participant; NCR, the number of generated ideas; APS, the analytical problem-solving score; N-POS, non-
positive group; POS, positive group.
W = 0.89, p = 0.02 for N-POS and W = 0.87, p = 0.01 for POS). There was no significant
difference between the N-POS and POS groups on the NCR score (W = 167.50, p > .05,
d = −0.41, 95% CI [−2.00, 1.00]).
The second SPANE questionnaire session was performed immediately after the
participants finished the creativity task. The average value of the SPANE-B was M
=
8.70 (SD = 6.68), and the median value was 10. There was a significant increase in the
SPANE-B value of 1.02 (t(39) = 3.00, p < 0.01, d = 0.96, 95% CI [0.34, 1.71]). Therefore,
a slight change in the group composition occurred, with 19 students comprising the
N-POS group and 22 students comprising the POS group. Cronbach (1951) developed
the α as a coefficient of internal consistency and interrelatedness especially designed for
psychological tests. The value of Cronbach’s α ranges from 0.00 to 1.00, where values near
1.00 indicate excellent consistency (Cortina, 1993; Cronbach, 1951). The Cronbach’s α
reliability measurement for the two SPANE questionnaire sessions was α = 0.97 (95% CI
[0.96, 0.98]), which indicates excellent consistency. We discuss the consequences of these
results in the next section.
We hypothesized that affective states would impact the analytic problem-solving skills of
The color scheme for the graphs of software developers. The boxplots for the APS score in Fig. 33 suggest a difference between
3
DISCUSSION
Our first SPANE measurement session offered the estimation µSPANE-B-DEV = 7.58
(95% CI [5.29, 9.85]) for the population’s true mean. That is, it might be that the
central value for the SPANE-B for software developers is above seven and significantly
different from the central value of the measurement instrument, which is zero. While we
Figure 4 Scatterplot for the analytical problem-solving (APS) vs. the affect balance (SPANE-B)
between the N-POS and POS groups.
reflect on this in the Limitations section, the reader should note that our discussion of the
results takes this into account, especially when we compare our results with related work.
The empirical data did not support a difference in creativity with respect to the affective
states of software developers in terms of any of the creativity measures we used. The
results of this study agree with those of Sowden & Dawson (2011), who did not find a
difference in the creativity of the generated ideas with respect to the affective states of the
participants. We found no significant difference in the number of creative ideas generated,
which is in contrast to Sowden & Dawson (2011), who found that participants in the
Limitations
The primary limitation of this study lies in the sample; the participants were all Computer
Science students. Although there is diversity in the nationality and experience in years of
study of the participants, they have limited software development experience compared
CONCLUSIONS
For decades, it has been claimed that a way to improve software developers’ productivity
and software quality is to focus on people and to make software developers satisfied and
happy. Several Silicon Valley companies and software startups are following this advice, by
providing incentives and perks, to make developers happy. However, limited research has
supported such claim.
A proposal to study human factors in empirical software engineering research has
been to adopt psychological measurements. By observing the reference fields—primarily
psychology and organizational research—we understood that software developers solve
problems in creative and analytic ways through cognitive processing abilities. Cognitive
processing abilities are linked deeply with the affective states of individuals, i.e., emotions
and moods.
This paper reported a study—built on the acquired multidisciplinary knowledge—
on the importance of affective states on crucial software development skills and
capacities, namely analytical problem-solving skills (convergent thinking) and
creativity (divergent thinking). It has been shown that happiest software developers
are significantly better analytical problem solvers. Although the same could not be
shown for creativity, more research on this matter is needed.
The understanding provided by this study should be part of basic science—
i.e., essential—in software engineering research, rather than leading to direct, applicable
results. This work (1) provides a better understanding of the impact of the affective states
on the creativity and analytical problem-solving capacities of developers, (2) introduces
and validates psychological measurements, theories, and concepts of affective states,
creativity and analytical-problem-solving skills in empirical software engineering and
(3) raises the need to study human factors in software engineering by employing a
multidisciplinary viewpoint.
ACKNOWLEDGEMENTS
The authors would like to thank the students who participated in the experiment. The
authors would also like to acknowledge Elena Borgogno, Cristiano Cumer, Federica
Cumer, Kyriaki Kalimeri, Paolo Massa, Matteo Moretti, Maurizio Napolitano,
Nattakarn Phaphoom, and Juha Rikkila¨ for their kind help during this study. Last but
not least, the authors are grateful for the insightful comments and understanding
offered by the Academic Editor Shane T. Mueller and two anonymous reviewers.
Funding
This work was not covered by any funding mechanism.
Competing Interests
The authors declare that they have no competing interests.
Author Contributions
• Daniel Graziotin conceived and designed the experiments, performed the experiments,
analyzed the data, contributed reagents/materials/analysis tools, wrote the paper,
prepared figures and/or tables, reviewed drafts of the paper.
• Xiaofeng Wang and Pekka Abrahamsson conceived and designed the experiments,
performed the experiments, analyzed the data, contributed reagents/materials/analysis
tools, wrote the paper, reviewed drafts of the paper.
Human Ethics
The following information was supplied relating to ethical approvals (i.e., approving body
and any reference numbers):
Institutional review board approval for conducting empirical studies on human
participants was not required by the institution. However, written consent was obtained
from all of the subjects. The participants were advised, both informally and on the consent
form, about the data retained and that anonymity was fully ensured. No sensitive data were
collected in this study. The participants were assigned a random participant code to link
the gathered data. The code was in no way linked to any information that would reveal a
participant’s identity.
REFERENCES
Abele-Brehm A. 1992. Positive versus negative mood influences on problem solving: a review.
Polish Psychological Bulletin 23:187–202.
Amabile T. 1982. Social psychology of creativity: a consensual assessment technique. Journal of
Personality and Social Psychology 43:997–1013 DOI 10.1037//0022-3514.43.5.997.
Amabile T. 1996. Creativity and innovation in organizations. Harvard Business Review 396–239.
Available at http://hbr.org/search/396239.
Amabile T, Barsade SG, Mueller JS, Staw BM. 2005. Affect and creativity at work. Administrative
Science Quarterly 50:367–403 DOI 10.2189/asqu.2005.50.3.367.
Baas M, De Dreu CKW, Nijstad BA. 2008. A meta-analysis of 25 years of mood-creativity
research: hedonic tone, activation, or regulatory focus? Psychological Bulletin 134:779–806
DOI 10.1037/a0012815.
Barsade SG, Gibson DE. 2007. Why does affect matter in organizations? Academy of
Management Perspectives 21:36–59 DOI 10.5465/AMP.2007.24286163.
Beck K, Beedle M, van Bennekum A, Cockburn A, Cunningham W, Fowler M, Grenning J,
Highsmith J, Hunt A, Jeffries R, Kern J, Marick B, Robert CM, Mellor S, Schwaber K,
Sutherland J, Thomas D. 2001. Manifesto for Agile Software Development. The Agile Alliance.
Available at http://agilemanifesto.org.
Berander P. 2004. Using students as subjects in requirements prioritization. In: Proceedings of
the 2004 international symposium on empirical software engineering (ISESE 04). IEEE, 167–176
DOI 10.1109/ISESE.2004.1334904.
Berna C, Leknes S, Holmes EA, Edwards RR, Goodwin GM, Tracey I. 2010. Induction of
depressed mood disrupts emotion regulation neurocircuitry and enhances pain unpleasantness.
Biological psychiatry 67:1083–1090 DOI 10.1016/j.biopsych.2010.01.014.
Boehm BW, Papaccio PN. 1988. Understanding and controlling software costs. IEEE Transactions
on Software Engineering 14:1462–1477 DOI 10.1109/32.6191.
Brand S, Reimer T, Opwis K. 2007. How do we learn in a negative mood? Effects of a negative
mood on transfer and learning. Learning and Instruction 17:1–16
DOI 10.1016/j.learninstruc.2006.11.002.
Ciborra C. 1996. Improvisation and information technology in organizations. In: ICIS 1996
proceedings. Association for Information Systems Paper 26, 369–380. Available at http://aisel.
aisnet.org/icis1996/26/.
Ciborra C. 2002. The labyrinths of information: challenging the wisdom of systems, 1st ed. New
York: Oxford University Press.
Cockburn A, Highsmith J. 2001. Agile software development, the people factor. Computer, IEEE
34:131–133 DOI 10.1109/2.963450.
Colomo-Palacios R, Herna´ndez-Lo´ pez A, Garc´ıa-Crespo A´ , Soto-Acosta P. 2010. A study of
emotions in requirements engineering. In: Proceedings II of the third world summit on the
knowledge society (WSKS 2010). Communications in Computer and Information Science 112:1–7
DOI 10.1007/978-3-642-16324-1 1.