Adaptivity Challenges in Games and Simulations A Survey

IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES, VOL. 3, NO.
2, JUNE 2011 85
Adaptivity Challenges in Games and

Simulations: A Survey
Ricardo Lopes and Rafael Bidarra
Abstract—In computer games and simulations, content is often In an attempt to account for player individuality, games
rather static and rigid. As a result, its prescripted nature can lead often include minor variations that depend on players profiling
to predictable and impersonal gameplay, while alienating uncon- themselves. For example, by customizing the difficulty level or
ventional players. Adaptivity in games has therefore been recently
proposed to overcome these shortcomings and make games more choosing time constraints, players are classifying themselves
challenging and appealing. In this paper, we survey present re- as one of the available predefined low-resolution stereotypes,
search on game adaptivity, identifying, and discussing the main e.g., beginners or experts. However, this discrete approach
challenges, and pointing out some of the most promising directions implies that such games might fail in appealing to players who
ahead. We first survey the purposes of adaptivity, as the principles do not know how to profile themselves or who do not identify
that could steer an adaptation and generation engine. From this
perspective, we proceed to thoroughly discuss adaptivity’s targets themselves with any of the available classifications.
and methods. Current advances and successes in this emerging Static game content and its predefined variations, based
field point to many yet unexplored research opportunities. Among on low-resolution profiles, all lead to games and simulations
them, we discuss the use of gameplay expectations, learning pref- that can be played in an impersonal, predictable and inflexible
erences, and assessment data in the integrated adaptation of game fashion and that can fail to appeal to broader audiences.
worlds, scenarios, and quests. We conclude that, among other
methods, procedural content generation and semantic modeling For games with purposes other than entertainment, such as
can powerfully combine to create offline customized content and serious games and simulations, these problems can become
online adjustments to game worlds, scenarios, and quests. These more acute. Players who need to capture or practice a certain
and other promising methods, deserving ample research efforts, skill, all have different learning abilities and training needs.
can therefore, be expected to significantly contribute towards However, serious games and simulations typically do not
making games and simulations even more unpredictable, effective,
and fun. take such a high-resolution player individuality into account.
Current ad-hoc and stereotyped training conditions can induce
Index Terms—Adaptive games, declarative modeling, online players to mostly perform the same exercises in the same
adaptivity, player assessment, player modeling, procedural con-
tent generation, semantic modeling. conditions, adding little value to the learning process. This lack
of player individuality can also affect the replay value of such
games, since nothing new or different can be experienced in
I. INTRODUCTION consecutive game sessions.
To solve the above shortcomings, many researchers agree
T YPICALLY, when most commercial games are shipped,

their gameplay has been prescripted. The same happens
with simulations, which generally use game technology to emu-
that serious games and simulations have to become more chal-
lenging, unpredictable and player-centric, to be fully embraced
as an effective way of knowledge transfer [1], [2]. Several other
late reality and training conditions. In both cases, game content, researchers claim that entertainment games should also address
rules, narratives, and environments are created during develop- these issues, by catering the gaming experience to the indi-
ment, mostly as static elements with which a dynamic player vidual user, being more responsive to different player types and
will interact. Designing such predefined content is standard be- their individual needs and adapting themselves to better fit the
cause it allows games and simulations to remain robust, testable players [3]–[5].
and controllable. As a result of such rigidity, game outcomes Player-centered game adaptivity can help accomplishing the
can be more easily anticipated by players, since all possible in- above goals. Dynamically adjusting game elements, according
teractions are bounded by such static elements. Even worse, if to the individual performance of the player (i.e., personal game-
players can predict certain outcomes, their progress can be often play), can contribute to make the game experience more unique
achieved by repeatedly exploiting a successful strategy. and personal. Consider the example of a driving simulation
where a player is monitored as speeding more than desired. An
Manuscript received October 05, 2010; revised February 28, 2011; accepted adaptive game could adjust the city environment to discourage
May 02, 2011. Date of publication May 12, 2011; date of current version June this behavior. Examples could be either increasing the number
15, 2011. This work was supported by the Portuguese Foundation for Science
of speed bumps, traffic radars or police patrols or generating
and Technology under Grant SFRH/BD/62463/2009.
R. Lopes and R. Bidarra are with the Computer Graphics Group, Delft more curved roads, stoplights, or crosswalks, depending on the
University of Technology, 2628 CD Delft, The Netherlands (e-mail: player’s experience and personality.
[email protected]; [email protected]).
In this paper, we survey the present state of adaptivity in
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org. games and simulations, identify the main challenges ahead and
Digital Object Identifier 10.1109/TCIAIG.2011.2152841 discuss possible research directions to tackle them. Fig. 1 lays
1943-068X/$26.00 © 2011 IEEE

Authorized licensed use limited to: Somaiya University. Downloaded on January 17,2024 at 09:38:35 UTC from IEEE Xplore. Restrictions apply.
86 IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES, VOL. 3, NO. 2, JUNE 2011
Fig. 1. Overview of game adaptivity architectural principles: player and experience modeling steer adaptation and generation of personalized game components.
out the architectural principles that drive research on adaptive player, the size of a team or the physical environment in which
games. These principles were already latent in the preliminary the game is played. As highlighted in Section I, this article fo-
proposals of Houlette [6], Charles [3] and Magerko [5], as well cuses on player-centered adaptivity, i.e., adjustments which im-
as in the vast majority of the research that followed. In essence, prove the individual player experience. For adaptivity to achieve
game logs, recording the players’ performance, are used to this goal, it needs to be steered by some purpose that game de-
create models of players’ actions, preferences, or personality. velopers can identify, measure and influence.
Given a game state, these models assess and predict the players Knowledge on this steering purpose will determine how
desired experience for the next game state. Models for the adaptation algorithms decide what, when and how to adjust.
player experience and performance are then used to steer an For taking this decision, algorithms should identify: 1) what
adaptation and generation engine, which adjusts the appropriate triggers the need for adjustments; and 2) what should be ad-
game components to better fit both. justed. For example, if difficulty adjustment is the steering
This survey discusses game adaptivity research from an purpose, an adaptive game needs to recognize that consecutive
adaptation and generation perspective. We strongly focus on failures may be a sign of high difficulty. It also needs to know
what (targets) and how (methods) adaptation and generation concrete in-game ways of affecting the difficulty level. Under-
engines can or could adapt. In contrast, the purposes of adap- standing and choosing what to use to steer adaptivity is both
tivity are surveyed from a generic perspective, independent an essential step and a major challenge, required to ensure that
of player modeling and player experience prediction. We game adjustments induce the personalized player experience,
discuss the principles that could, ultimately, serve as input on the desired way (e.g., adjusting difficulty in the previous
and steer adaptation and generation. An in-depth analysis of example).
their implementation through player modeling and experience In this section, we survey adaptivity’s purposes, i.e., the
prediction techniques will therefore not be considered here. generic principles that support player modeling and experience
Player modeling principles have been already discussed by prediction and steer game adaptation methods.
several researchers [6]–[8], and experience prediction has been
recently surveyed by Yannakakis and Togelius [9]. A. Entertainment Games
This article is structured as follows: in Section II we look at For entertainment games, fun is the fundamental purpose.
the purposes for adapting, by analyzing what is being presently There are many different theories for explaining how to achieve
done in steering adaptivity in games and simulations. In this largely subjective emotion but, so far, adaptive games have
Section III we focus on adaptivity targets, surveying standard typically only been considering one dimension to engage fun:
adaptive game components (e.g., nonplaying characters). In challenge. This usually means that the difficulty of performing
Sections IV and V, we survey and discuss, respectively, offline game tasks must be in balance with the skills of the player,
and online methods which can be used to adapt game content. avoiding undesirable “too easy” or “too hard” situations. The
methods, algorithms and games analyzed in Sections III and V
II. STEERING ADAPTIVITY are mainly steered by this purpose.
In games and simulations, adaptivity can be used to better However, some promising work has already been done in
suit the game to a dynamic element, for example, the skills of a different directions. Yannakakis and Hallam [10] propose a
LOPES AND BIDARRA: ADAPTIVITY CHALLENGES IN GAMES AND SIMULATIONS: A SURVEY 87
(e.g., electrocardiograms, galvanic skin response, electroen-

cephalograms, palmar temperature sensors). A more in-depth
discussion of the relation between affective computing, physi-
ology detection technology, and adaptive games is out of scope
here and can be found in [9].
B. Serious Games and Simulations

Serious games and simulations can have more purposes, other
than fun. For example, they may aim at providing educational
or training experiences, where players are required to achieve
learning goals in supervised (and sometimes collaborative) en-
vironments. In this context, the motivation for steering adap-
tivity becomes clearer: improve the effectiveness of the knowl-
edge transfer between the game and its players.
Traditionally, to steer adaptivity, research in serious games
and simulations has been using a similar approach as in enter-
tainment games: finding a balance between the player’s skills
Fig. 2. Pedersen et al. [11]: predicted player emotions from a gameplay session. and the game challenge level. Reaching this balance remains rel-
evant for serious games and simulations, since it is a straightfor-
ward way of simplifying all types of learning goals and styles.
methodology for adapting games on the Playware physical In serious games, the learning component strongly influences
interactive platform. The authors explore control of user satis-
many design decisions. For example, the design philosophy of
faction rather than game difficulty, and their testbed is a “bug”
serious games needs to constantly balance play (or entertain-
(tile) stepping game for children. To model player satisfaction,
ment) with meaning (knowledge transfer) and a strong sense of
the authors identify curiosity (the spatial diversity of bugs) and
reality [15]. Therefore, in serious games, adapting to specific
challenge (pace with which bugs appear and disappear) as the
skills is more important than to the global notion of difficulty
main factors. Furthermore, Pedersen et al. [11] built quanti-
or challenge. The learning goals to achieve are usually strongly
tative models that predict the player experience in a platform
coupled with the gradual personal improvement of a skill set,
game, to be used in generating levels that are adjusted to those
most of the time, one skill at a time. As so, in this domain, adap-
predictions. These models can predict gameplay as being: fun,
tive games have specialized (and usually ad hoc) approaches,
challenging, boring, frustrating, predictable, or anxious. Fig. 2
where game components are adjusted to encourage training a
illustrates an example where, after a gameplay session, the
specific skill. Adaptivity is steered by a specific skill players
system predicts what emotions were experienced by the player.
These approaches show that there is room for going beyond need to learn in a particular moment, and influenced by their
challenge as a motivation for steering adaptivity. Magerko [5] personal proficiency.
argues that players have widely different reasons for playing Westra et al. [16], Peirce et al. [17], Magerko et al. [18] (all
and that adaptive games should capture and use them, focusing further analyzed in Section V) and Niehaus and Riedl [19] (dis-
on the players main interests and adapting to match their mo- cussed in Section III), all propose personal skill proficiency as
tivations for playing. Both methods above show promising re- the steering purpose for their adaptivity mechanisms. Another
sults in capturing, as Magerko proposed, various dimensions, skill-oriented adaptive simulation was proposed by Johnson et
beyond challenge, as useful indicators of players’ motivations al. [20], where individual language skills are modeled, deter-
for playing. Curiosity, boredom, frustration, predictability, or mining how a virtual tutor offers guidance to the player. Lane et
anxiety are powerful features that extend beyond fun or chal- al. [21] also use a virtual tutor which, constrained by the player’s
lenge. They can allow for more detailed and flexible mecha- past actions, gives feedback towards a set of skill-based training
nisms of steering adaptation and generation. goals. Martin et al. [22] automatically generate scenarios for se-
Affective computing and advances in facial, motion and rious games using training objectives as the main requirements
physiology detection can have an important role in steering for generation. Although players are not modeled, these training
adaptivity as well. When applied to games, these technologies objectives are also a list of specific tasks (or skills), appropriate
have the potential to identify the affective states players ex- for the domain of the game, e.g., hit a distant target using an ar-
perience. A better understanding of these can allow for more tillery unit.
effectiveness and higher resolution in choosing and designing Some interesting research has been done beyond pure skill
adaptivity purposes. For example, if a game can recognize and modeling and considering other aspects of the learning process.
detect laughter, heart beats or body motion, fun can be further Research on the Crystal Island narrative-centred learning game
analyzed down to humor, excitement or physical interaction. demonstrates that supervised machine learning can be used to
Recent research has been done in this direction, through the recognize players’ affective states [23] or model their knowl-
recognition of steering purposes as challenge [12], boredom, edge [24]. However, future work stills needs to be addressed to
engagement and anxiety [13] and enjoyment [14] in adap- apply the recognized data to the adaptation of game content. On
tive games, using player physiology detection technology a different direction, Magerko et al. [25] identify learning styles
(e.g., explorer, achiever) in users of an educational game; they mechanism. A major challenge still lies in exploring even fur-
then adapt the game to better fit players who have those learning ther and materializing Magerko’s [5] vision. To adapt better and
styles, to better acquire the desired knowledge. This research more, there is a stronger need to capture and be guided by the
shows that steering adaptivity in serious games and simulations real reasons why people play. These reasons can be captured
can extend further beyond specific skill modeling, to focus on by the characteristics and affective states of the gameplay that
other important features of the player’s learning mental process. players expect to experience and be immersed in. For example, a
player whose motivations for engaging in a first person shooter
C. Assessment in Serious Games and Simulations (FPS) game are to engage in a specific level of a stressful, scary,
Apart from their purposes, serious games have another dif- but low pace experience.
ferentiating aspect: assessment. Measuring, discussing and rea- For steering adaptivity with this kind of high-resolution
soning on the gameplay effectiveness is specially important in gameplay expectations, a number of challenges still need to
the simulations domain, since it can lead to reflection and there- be addressed, including: 1) capturing player expectations; 2)
fore improved learning. However, in this context, assessment quantifying these to appropriate measurable levels; and 3)
has seldom been considered in academic research. In particular, processing them to the correct game adjustments. Both 1) and
there is no work on combining game adaptivity with assessment. 2) are player modeling and experience prediction challenges,
Chen and Michael [26] have already identified the main chal- while 3) is an adaptation and generation problem. Another
lenges that assessment in serious games is facing, namely af- major challenge in this direction lies in supporting these mech-
fecting and improving player experience. The authors suggest anisms in a game domain independent fashion, so they can be
that log information and teachers/instructors knowledge should reused and consolidated.
be fully explored and, in some way, incorporated back in the Serious games and simulations are a different case. Due
to their specific learning/training aim, many specialized ap-
game, to guide its course.
proaches can adapt the game to provide opportunities to develop
So far, research in assessment for serious games has been
the most needed skills, at the appropriate proficiency level.
mainly centered on after action review (AAR) methods. Still,
However, some research shows that there is a need to better
some results already demonstrate that the direction identified
account for player individuality. Besides the case of learning
by Chen contains a lot of potential. Lampton et al. [27] pro-
styles based adaptivity [25], Rowe et al. [31] also evidence this.
pose an AAR system for military simulations where trainees and
The authors investigate individual differences in gameplay and
trainers assess exercises together. An interesting result was that
learning during student interactions with an educational game.
participants developed innovative ways to use AAR, not only
They conclude that learning preferences (student’s background
for assessing past behavior, but also for planning new future
knowledge and interests) are strongly coupled to the gameplay
training exercises. Raybourn [28] proposes a design method for
style (e.g., objects used, content read) and need to be considered
creating training simulations that promote player communica- in game design.
tion, in-game performance feedback, and sharing of strategies. The challenge in steering adaptive serious games and simu-
The author focuses on using in-game and AAR assessment in- lations still remains in reaching further beyond skill modeling.
formation to create an emergent domain culture that could allow Magerko et al. ’s learning styles approach points in a promising
the cocreation of future game scenarios. direction where a broader array of learning preferences can
Some recent research is already incorporating performance guide adaptivity. Besides styles, this scope of learning prefer-
logged data to control virtual participants in AAR sessions. Lane ences can include, among other, player’s learning background,
et al. [21] proposed a virtual reflective tutor that, given the his- interests (as highlighted by Rowe et al.), gameplay [25] or
tory of player actions, is able to automatically assess their per- even instructors/teachers assessment about players. The same
formance and even conduct an interactive deep reasoning AAR challenges discussed before for gameplay expectations still
with the player. Core et al. [29] and Gomboc et al. [30] pro- apply for future work on modeling these learning preferences.
posed explainable AI, a game log based system where AAR Assessment of past performances can also play a role in adap-
participants can directly question virtual characters about their tive simulations. In this domain, there is typically plenty of valu-
in-game actions, goals and even motivations behind those. able information emerging from game logs and AAR sessions.
All these results show that assessment information can be This information is currently far from being fully explored by
better explored, and that assessment game data could be used the game itself. Game logs usually offer an enormous amount of
in future scenarios (as suggested by Chen). This logged data is unstructured game data that is therefore difficult to interpret and
even being already incorporated to steer postgame virtual enti- use. Moreover, while AAR information emerges from trainees
ties (in this case, in AAR sessions). and their instructors, it is typically not incorporated back in the
game. Using this information as a source to steer adaptivity, as it
D. Discussion currently steers virtual entities, seems a promising, unexplored
With respect to our initial definition of adaptivity’s steering area. Interesting research opportunities exist in using assess-
purposes, entertainment games and serious games/simulations ment information to, for example, regenerate “try again” game
still form two cases far apart. Both entail valid research chal- missions, adapted and focused on what the players failed on
lenges that are now discussed. the previous session. So, offering an adapted regenerated “game
In entertainment games, some approaches are already being session” could simultaneously allow a better understanding of
explored beyond the traditional dynamic difficulty adjustment what went wrong, and better opportunities to succeed.
TABLE I recognizes the player actions and responds by adjusting NPC

CLASSIFICATION OF SURVEYED WORK ACCORDING TO ADAPTIVE behavior. Also, academic research on AI adaptation focuses
COMPONENTS AND INDUSTRY/ACADEMIA DOMAINS
strongly on the pedagogical serious games domain, due to
the extensive use of NPCs in learning contexts (e.g., tutors).
Several of the techniques surveyed in Section V are applied to
AI adaptation, both in entertainment and serious games.
Adaptivity has also been applied to game narratives, both
in the commercial and academic domains. Games can become
more personal when the progressing narrative builds up in a
unique fashion, fitting players’ behavior. Valve’s Left 4 Dead se-
ries [36], [37] introduced procedural narrative as a technique to
generate sequences of events, adapted to the pace and behavior
of the player. An AI Director analyzes players behavior (e.g.,
if they were particularly challenged by one kind of enemy) and
adds subsequent events (e.g., spawning that enemy). According
to Valve [38], this mechanism serves as a story-telling device (at
least, in simple narrative domains as most FPS games are) be-
cause players can experience some notion of intentionality on
the opponents’ side. Heavy Rain [39] is an interactive drama
game that focuses on personal gameplay, where all the specific
decisions each player takes are analyzed, in a more complex
way than before, to determine the narrative and outcomes of the
game.
In academia, there is a strong interest in interactive narratives,
story-based experiences which typically use game technology,
both for entertainment or pedagogical purposes. Roberts and
Isbell [40] have recently surveyed interactive narratives and
drama management systems, identifying, among other aspects,
Fig. 3. Scene from Max Payne, by Remedy Entertainment [33]. their adaptive capabilities. The following paragraph presents
an overview of these systems. For a more detailed discussion,
e.g., on concerns on the use of centralized manager agents
versus multiagent networks, Roberts and Isbell’s survey is
III. ADAPTIVE GAME COMPONENTS
recommended.
After discussing adaptivity’s purposes in the previous sec- Barber and Kudenko [41] researched the generation of
tion, we now turn our attention to its target recipients. Poten- dilemma-based interactive narratives. A model of player be-
tially, all components that are considered at game development havior under specific dilemmas is used to estimate and select
can become adaptive. In fact, dynamically adjusting: 1) game difficult dilemmas, which a planner weaves together to form
worlds and its objects; 2) gameplay mechanics; 3) nonplaying a story. Mott and Lester [42] use a dynamic decision network
characters (NPC) and AI; 4) game narratives; and 5) game sce- as a planner for creating interactive narratives. The decision
narios and quests, all can contribute to offer an individualized network contains nodes for the players goals, experiences and
gameplay experience. Table I illustrates how surveyed work is relationships, thus influencing decision making. In Sharma et
distributed according to game components and domain. al. drama management system [43], player preferences are de-
Gameplay mechanics, i.e., how game elements can work, in- termined by an explicit case-based player model, derived from
cluding actions like running or shooting [32], have already been the behavior of earlier players. This model guides generation
made adaptive, in commercial games. In Max Payne [33] (illus- towards stories that fit those preferences. Fairclough [44] also
trated in Fig. 3), a mechanism unknown to players altered the uses a case-based approach, but to synthesize stories from a
level of mechanics like player aim assistance, according to in- knowledge base, constrained by the player’s evolving relation-
dividual skills (thus adjusting shooting difficulty). ship with NPCs. Finally, Thue et al. [8] present an interactive
Traditionally, adaptivity has been mostly researched and narrative generation system which models the player’s style
applied within the AI domain, specifically towards NPCs, since according to five predetermined player types. Story events
behavioral adaptation is a strong means of displaying intelligent are annotated with their appeal for each player type and are
behavior. In Mario Kart Wii [34], rubber band AI techniques selected accordingly for inclusion in the narrative.
are used to increase the opponent NPC abilities when the player Game scenarios and quests only recently started to become
performs too well. Pro Evolution Soccer 08[35] introduced a target of adaptivity research. Game scenarios and quests both
Teamvision, an adaptive AI opponent system that changes its describe the flow of events and actions within a game but they
tactics and strategy to suit the player style and explore his are primarily used, respectively, in simulations and entertain-
weaknesses. In academia, and as identified in Table I, several ment games. Generation of personalized game quests is already
techniques have been proposed to support adaptive AI that being researched and is discussed in detail in Section IV-B.
As for game scenarios, they highlight the importance of adap- adapting game mechanics, NPC or narratives. The example in
tivity in the simulations domain. Niehaus and Riedl have re- Section I illustrates some of this unexplored potential, by influ-
cently proposed a methodology for adapting game scenarios to encing both the game world (generation of more speed bumps)
suit players’ learning goals [19]. A scenario adaptor adds, re- and the game scenario (instantiation of more police patrols). In-
moves or replaces abstract game events, guided by a mapping tegrated approaches can go even further by having game world
between a world domain knowledge base (i.e., the dynamics and scenario adaptation influence each other, in simple or more
of the simulation world events) and a lifelong learner model, complex ways (e.g., if police patrols are sufficient, no speed
which tracks a learner and chooses the next training objectives bumps needed).
that will help him advance. Earlier research from Magerko et Currently, game worlds are created during the design stage,
al. [18] also adapts game scenarios, and it will be discussed in prior to game release. In that process, games and simulations
detail in Section V. occasionally use procedural generation algorithms to automati-
As for results on adaptive game worlds, they are very scarce. cally create some of the game world elements, with techniques
The only example we found of game world adaptivity is in the widely researched in academia (like the ones surveyed in
commercial game Left 4 Dead 2 [37]. According to the de- Section IV). As for game scenarios, they are typically created
velopers, the layout of certain sections of levels is dependent during the programming stage, when scripts and code define
on the player’s performance [45] (a graveyard with a simpler the flow of events for the game. A major challenge in auto-
layout for underachieving players is presented as an example). matically authoring game worlds and scenarios, as in fact with
Being a recent commercial release, the game’s publishers have all game content, lies in delaying its generation until the game
not yet disclosed any technical details and the reach of this is running. This challenge is essential for adaptivity, since the
adaptivity mechanism, in terms of accounting for how much of creation of content that is adjusted to players relies on analyzing
the content is static or dynamic. It is therefore still unknown their in-game performance. There are two main methods to
which player modeling or procedural content generation tech- tackle the challenge of supporting adaptive game worlds and
niques are used, if at all. As for academic research, several scenarios, through delayed authoring: 1) offline (pregame)
projects, mainly aimed at player modeling and difficulty adjust- customized generation; and 2) online (i.e., in-game) adaptivity.
ment techniques, are focusing on adapting 2-D game level struc- In the next two sections, we will survey the present state of
tures, as discussed in Section V. These simple game levels are research on each of these topics, and how they confirm that
still far from being compared to the complexity and richness adaptive game worlds and game scenarios raise very promising
of modern game worlds. Although not fully adaptive, Charbitat and challenging research questions.
[46] (further analyzed in Section IV) is the only example we are
aware of where procedural generation of complex game worlds IV. OFFLINE ADAPTIVITY: CUSTOMIZED CONTENT
is somehow influenced by the player performance. GENERATION
Offline adaptivity implies that adjustments are made con-
A. Discussion sidering player-dependent data, but prior to initiating any
Compared to all the above game components, adaptive game gameplay. The typical example of its application would be the
worlds and scenarios (or quests) are still lacking in broad, con- processing of player data and game adjustments during the
solidated and integrated research focus. Modern game worlds loading stage of a game session. Therefore, offline adaptivity
have been made partially adaptive in the commercial domain. involves mainly a generation challenge, i.e., an emerging
Left 4 Dead 2 shows the potential in game world adaptivity but, creation process.
in the absence of more detailed information regarding the extent Automatic content generation can therefore play a significant
of what is adapted, should be regarded, in academia, as an ex- role in offline adaptivity. Research results in this field are par-
ample to improve on. As for game scenarios, research has been ticularly promising towards customized content generation, a
more fruitful. Niehaus and Riedl [19], and Magerko et al. [18] method for the automatic creation of virtual game worlds, ad-
are good examples of the advances achieved so far. However, justed to better suit players. We believe the same principles can
they still evidence research opportunities to: 1) reach beyond be extended and applied to what occurs within these worlds, i.e.,
skill-driven adaptivity (as discussed in Section II); and 2) inte- to game scenarios, even though their offline generation has been
grate scenario with world adaptation/generation. less researched than that of game worlds.
The importance of these two components, particularly if they
are integrated, can be explained through their definitions. Game A. Game Worlds
worlds are the virtual environments within which gameplay oc- Previous work in automatic content generation has tradition-
curs, its geometry, geography, layout, and objects. Game sce- ally relied on procedural methods and has often succeeded in
narios are the framework for the global progression within a creating visually convincing game environments. For the public
game level, its initial settings and the logical flow of events and eye, procedural generation has recently been successfully asso-
actions that follow [47]. As such, the fulfillment of a game sce- ciated with games, due to Spore [48]. This game extensively
nario within a game world defines and characterizes the ma- uses procedural generation for player-created creatures anima-
jority of the player experience. Integrated world and scenario tions and planet textures.
adaptivity seems therefore more likely to solve the shortcom- Regarding game worlds, many different procedures have
ings identified in Section I, certainly offering more possibilities been proposed to automatically create content such as terrain,
for affecting player experience than the current proposals for trees, plants and urban environments. Procedural methods were
Fig. 4. (a) Urban environment generated by CityEngine [53]; (b) virtual world generated by SketchaWorld [68]; (c) road network and corresponding 3-D city
geometry, generated by [56]; (d) height-map, generated by [54]; (e) complex terrain, with arches, created by [55]; (f) top view of town, generated by [60].
recently surveyed and discussed by Smelik et al. [49], who more and more detail. Users can control the generation of a city
conclude that a common shortcoming in traditional methods using their CityEngine system, allowing them to create and edit
is the lack of control over the generated output. Therefore, grammar rules, in a similar way to using scripting languages.
researchers are now aiming at more controllable procedural Fig. 4(a) shows a model for the ancient city Pompeii, as gener-
methods, seeking to allow designers to intuitively steer content ated by CityEngine.
generation. Recent research has been focused on creating new methods
In this direction, interesting work has been done in the genera- for designers to control game world generation, more intuitive
tion of 2-D platform game levels. Compton and Mateas [50] use than shape grammars. Doran and Parberry [54] propose an ap-
context-free grammars to generate platform levels, organized in proach where terrain elevation heightmaps are generated by in-
patterns and branch structures. The generated level is controlled dependent software agents, with different roles for coastlines,
by a hill-climbing algorithm that adjusts patterns to suit a target beaches, mountains, hills, and rivers. Designers are responsible
controllable difficulty. Smith et al. [51] further developed these for defining terrain features that constrain the amount of agents,
concepts, allowing designers to directly constrain properties in their lifetime, and actions and, thus, the way the terrain is gen-
the generated platform levels (e.g., level path, jump rhythm and erated. Peytavie et al. [55] present a framework for generating
frequency, etc). Sorenson and Pasquier [52] propose another ap- complex terrains that include overhangs, arches, caves and dif-
proach where genetic algorithms are used to evolve 2-D game ferent materials such as sand and rocks. Designers can control
levels towards satisfying designers constraints. An interesting the terrain generation by sculpting bedrocks, modeling cracks,
result lies in how they evaluate generated levels: they are sub- fractures, and tunnels, adding granular material, and controlling
jected to a fitness function that rewards levels based on how erosion tools. In Chen et al. [56], tensor fields are used to guide
fun (in this case, challenging) they are. These results show that the generation of street networks. Users can control the generated
generation of 2-D level structures has been succeeding in con- street network by placing basis tensor fields, using tensor field
sidering important adaptivity concepts as difficulty, challenge patterns, smoothing fields to reduce its complexity, brushing the
or fun. field to orient streets, or applying noise to make the road network
The generation of modern 3-D game worlds is facing other is- less regular. Fig. 4(c)–4(e) shows, respectively, a road network
sues, more related with intuitive and interactive control. In this created by Chen et al., a height-map generated by Doran and
domain, Müller et al. [53] proposed the use of shape grammars Parberry, and a complex terrain modeled by Peytavie et al.
to generate highly detailed cities. The grammar uses context Even more interactive and user-centric methods have been
sensitive rules to iteratively evolve building design, by creating proposed to control automatic content generation by: sketching
the silhouette and bounds of a mountain in a 3-D interface [57], role-playing games (MMORPG). Quest generation creates valid
brushing and sculpting outdoor terrains [58] and sketching game goals, which are unique for each player and game. Quest
roads, which are automatically generated to fit with the sur- uniqueness is ensured by three player-centric features that con-
rounding environment [59]. strain the generation process to produce relevant quest paths:
Some recent research results have already shown that con- the memories (past quests) of the player, his relationship to
trol over the generation process can extend beyond this type the character assigning the quest, and player attributes (needed
of interactive modeling of geometric world features. Bielikova to complete quests). Ashmore and Nitsche [63] also investi-
et al. [60] propose a system for generating educational game gate player-centric quest generation. They propose a new quest
content: quests, NPC, virtual worlds [see example in Fig. 4(f)] generator to include in the previously discussed Charbitat [46]
and narratives. In this case, domain experts, i.e., teachers, and system. Quests consist of key and lock puzzles (a key must be
not designers, control content generation. Teachers can select found to unlock an obstacle) and the generation process places
precreated game objects, add new learning content to them and within the game world, the obstacle, its key, and the challenges
create relationships between objects. Knowledge about objects along the way to obtain it. Quest generation occurs during the
and their relationships is the basis for solving and generating generation of a new world tile: possible locations for keys and
all the appropriate content. These results offer another valuable locks are scored by evaluators that are highly dependent on the
contribution: control on the generated content is applied at a procedurally generated tile. In the Charbitat case, quests become
higher level than geometric features, by using knowledge on ob- unique for each player because they are influenced by the game
jects and their relationships. world which was itself generated in such a customized fashion.
Nitsche et al. [46] introduce a case study for the procedural These results evidence some of the potential in integrating
generation of game worlds based on the gaming style of its and influencing game quest generation with its surrounding
players. In Charbitat, players steer the generation of an infinite game worlds. As stated in Section III, integration with the
world through their in-game actions. The game world is split game space is an important aspect to be considered in quest
into individual tiles and each new tile is generated using noise generation. In Pita’s case, quests are generated in a game
functions and filters, where the underlying seed value is calcu- world that was manually designed, before-hand. In Ashmore’s
lated based on player-dependent character data, i.e., his actions. approach, the game world is first procedurally generated and
Players are involved and conscious of this process: they can vol- is then evaluated for placement of quest elements. Though not
untarily influence the world generation in different directions as adaptive in any way, Dormans work [64] is a good example of
they please. Although this is an online method, this guided gen- a different approach, a constructive integrated one. The gener-
eration nature relates better with the methods and requirements ation of 2-D action–adventure game levels is broken down into
for offline adaptivity. two steps: a graph grammar generates mission structures that
Both approaches above show that automatic generation of are used in an extended shape grammar, which grows a space
game worlds can be controlled on a higher level (when com- that accommodates the generated game mission.
pared to geometric features), and can be made dependent on Offline generation of game scenarios, as defined earlier, is
player data. Both results seem successful advances towards cus- still far behind these concepts of customized quests or missions.
tomized content generation. Research in offline scenario generation is still more focused
on the methods, i.e., on how, to generate and less on its pur-
B. Game Scenarios and Quests poses, i.e., on what for, e.g., steering them to be player-cen-
Offline automatic generation of game scenarios and quests tric. As mentioned in Section II, Martin et al. [22] generate
has not been a subject of much research, specially when com- game scenarios for serious games. They use functional L-sys-
pared with online scenario adaptivity (Section V). The term tems, a variant of formal grammars, to write generation rules
game scenario, i.e., the global progression within a game level, which can expand training objectives into generated scenario
including its initial settings and the logical flow of events and elements, i.e., the initial settings and the progression of game
actions that follow, is mainly used in serious games and simu- events. Hullet and Mateas [65] also generate game scenarios
lations. Its entertainment games equivalent, game missions or from pedagogical goals, but using a planning system that de-
quests, also structures a sequence of events and actions, nor- composes pedagogical goals into tasks, subtasks, and methods,
mally associated to a game task that must be completed. which encode knowledge to achieve that goal state.
Research on this field shows that there is a growing interest in Both approaches generate game scenarios from goals that
creating player-centric quests that provide personalized game- capture which skills the players should apply during the game.
play. Sullivan et al.’s Grail framework [61] is aimed at pro- However, in both methods, these declarable goals are simply a
viding customized quests, through online player-centered ad- low-level and domain-dependent set of features that are implied
justments (analyzed in Section V), but it also includes an au- by the higher level desired gameplay skills. For example, Hullet
thoring tool for designers to control quest generation. explicitly declares the goal “a room should be blocked” to im-
Although the following two methods are in essence also on- plicitly capture the skill of breaching walls to rescue victims.
line based, their simple definition of quest avoids the usual de-
sign requirements of online methods (e.g., performance or con- C. Discussion
sistency concerns). Therefore, they relate closely to possible of- In this section we surveyed research related to customized
fline techniques. Pita et al. [62] propose a system to dynami- content generation, namely procedural generation of 2-D game
cally generate quests in persistent massively multiplayer online levels, 3-D game worlds, quests, and game scenarios.
The surveyed methods show that the generation of 2-D levels advantage of player-dependent data, as discussed with quest
is already capable of being controlled, or at least evaluated, by generation.
the same kind of criteria currently used to steer adaptivity: dif- To adapt game worlds and scenarios, using controllable
ficulty, challenge, and fun. These results highly encourage the content generation, we identified the two challenges men-
further use of player data, i.e., their preferences or performance, tioned above. Overcoming these will make it possible to create
for controlling the procedural generation of game levels. Even customized content, in the sense that game worlds and sce-
though level generation for the platform game genre is less com- narios will be generated before starting the game, according to
plex than the generation of modern game worlds, the same con- knowledge about individual players. This knowledge should
clusions could still hold for the latter. go beyond player preferences (e.g., favorite colors) and focus
However, research shows that this is still far away, since the on what affects playing motivations or purposes. For example,
generation of complex 3-D game worlds is facing other issues. consider a world domination strategy game and a player who
The main challenge is to enable designers to control the gener- aims to be an economic leader and use trade strategies. A
ation process. Controllable content generation is enabling pro- matching virtual world and scenario could be generated so that,
cedural methods to become more flexible and accurate. While for example, the player homeland is located close to natural
maintaining its automatic nature, these methods are allowing resources and trader routes in an intermediate position between
game designers to steer automatic content generation by means not so aggressive opponent cities (all of which encourage the
of a better expression of their intent. Although these results are fast development of trade strategies).
aimed at the design stage and through geometric features mod- Current research is already tackling some of the challenges
eling, they seem promising steps towards customized content identified above, and its methods could be valuable to future
generation, as they allow procedural methods to be interacted work in customized content creation. Semantic and declarative
with and controlled. modeling techniques are already capable of controlling pro-
Moreover, Bielikova et al. and Nitsche et al. show that au- cedural methods by embedding and interpreting higher level
tomatic generation of game worlds can be controlled from a knowledge in virtual objects. Tutenel et al. [66] define object
higher level (when compared to geometric features), and can be semantics as all information, beyond its 3-D model, related
made dependent on player data. The offline generation of game to a particular object within the game world (e.g., functional
worlds, adapted to fit its players’ purposes, could benefit from information like how to interact with it, possible relationships
further research on these topics. The challenges ahead mainly with other objects, etc). With semantic modeling, object re-
relate to the integration and extension of these two topics by in- lationships, features and other semantic information can be
terpreting player data to reason about high level gameplay ex- used to guide the layout generation of a game world, whether
pectations (as discussed on Section II), and use this reasoning designing it manually or creating it procedurally.
to steer content generation. Bidarra et al. [67] introduce declarative modeling of virtual
Regarding quest generation, results showed that customized worlds, explaining how semantics can help designers to create
quest generation is becoming more relevant, and it can be a virtual worlds by declaring what they want to create, instead of
successful mechanism to engage players in more enhanced, in- how to model it. Such declarative modeling enables designers
teractive and personal experiences. However, the methods sur- to control and constrain virtual worlds, through semantic spec-
veyed are still somehow rudimentary, due to their ad-hoc nature. ifications that describe what the virtual world and its objects
Quests are defined in an elementary manner, and generation is should be. Fig. 5 illustrates how this semantic level, presented
constrained to only one aspect of what a quest can include: goals to designers, is used to control the procedural level. This scheme
to accomplish and locations for objects, in the cases analyzed. differs from conventional procedurally based modeling, sporad-
Open challenges remain for: 1) affecting all sorts of events and ically used by designers and technical artists, in that it incorpo-
actions that can occur through the settings of a game world; and rates a semantics layer, between the designer and the procedural
2) seamlessly integrating quest and game world adjustments. techniques. This semantic level provides designers with a pow-
As stated in Section III, we think that fully integrating world erful front-end that generates and steers the underlying proce-
and quest/scenario generation is a potentially important mile- dural level, while encapsulating the complexity of the latter.
stone. Multiple alternatives can be devised for this: creating Many of these methods have been integrated in SketchaWorld
worlds to better accommodate the scenario generation process, [68], a prototype system for declarative modeling of virtual
creating scenarios that fit the generated worlds or, in the best worlds. In this declarative approach, designers state their intent
case, parallel or mixed approaches. In any case, the coherence by specifying which high-level features a virtual world should
and robustness of the gameplay would benefit from this integra- have, e.g., the layout of the landscape or the population size
tion, since generated events would take place in a space that is of a city. Designer’s intent is used to generate a matching
known and influenced by them, thus decreasing the risk for gen- 3-D virtual world, where each specification is procedurally
eration of related errors. expanded to a visually convincing terrain feature. Within this
As for current research in offline scenario generation, and declarative approach, interactions between terrain features are
regardless of being based on declarable goals, these approaches automatically solved using virtual world consistency mainte-
show that the challenges ahead in researching customized sce- nance, which consists of a combination of semantic definitions
nario generation are the same as with game worlds: 1) con- of the geometric and functional relationships between terrain
sidering higher level skills or goals (or the learning prefer- features, and a set of generic resolution rules. A virtual world
ences discussed in Section II) in an explicit way; and 2) taking created with SketchaWorld is shown in Fig. 4(b).
authoring. Online, or in-game, adaptivity is the term many re-

searchers use to describe the ability of a game to adjust to its
players, in real time, as they play. Although this kind of adap-
tivity is still a recent research area, there are some significant
results worth discussing here.
Most adaptivity research focuses on a low level, i.e., adapting
specific game elements through nonintegrated approaches.
However, Charles et al. [3] proposed a high-level framework
to explain how online adaptivity should be supported in every
domain, in an integrated manner. This framework captures
the main abstract ideas and approaches that are currently
adopted throughout this research area. A model of the player
is used to capture the player habits and skills, and the player
performance is monitored and compared with the model, while
playing. Whenever an adaptation of the game is identified and
performed, the framework measures its effectiveness, which
can lead to either a new adaptation or an update of the player
model.
Currently, online adaptivity mostly acts as a sandbox for re-
searching new artificial intelligence concepts and methods. As
such, most work in this field focuses on adjusting NPCs or other
intelligent game agents to better suit players or even offer a more
Fig. 5. Generic approach of declarative modeling of virtual worlds [67]. challenging game.
Peirce et al. [17] propose the ALIGN system as an approach
for noninvasively adapting NPCs behavior to enable a person-
Semantic and declarative modeling can already help in tack- alized learning experience. ALIGN’s architecture separates the
ling some of the challenges we identified throughout this sec- logic of generic adaptation from game specificities, so that game
tion. The previously explained semantic layer deals with all logic and adaptation are independently authored and operated.
high-level information relating to virtual world objects at the se- Adaptive Elements (AEs) are the basic components that support
mantics level. This information helps convey the meaning and possible adaptations. AEs are precreated and annotated with
the role of an object in the virtual world, and consists of generic metadata describing both the game settings in which they can
descriptions of classes of features, including attributes, proper- be used, and the abstract outcome of their use. Separate infer-
ties, roles, relations, etc. This encourages the incorporation of ence engines translate game events to AEs and create specific
further semantic information about player dependent gameplay in-game interventions to match the selected AE. Each in-game
purposes, and how these can be used to control object gener- intervention is influenced by a set of rules that examine a player
ation. For example, if a player needs obstacles in a race track, model and determine the desired adaptation outcome.
the semantic layer could indicate what and how obstacles can be Westra et al. [16] use agent organizations to adapt (in-game)
used with that player. Finally, declarative modeling already in- the behavior of game elements in serious games. Uncentral-
cludes semantics-based mechanisms to check and solve proce- ized and independent (learning) agents choose the tasks to be
dural conflicts in generated worlds. This shows that these tech- performed by individual game elements, e.g., a burning fire or
niques can be flexible enough to handle conflicting contexts, like NPCs. Possible behavior variations for all agents are imple-
those which would likely arise when integrating player-centric mented a priori, using domain experts knowledge. During the
with designer-centric purposes, and virtual worlds with game game, each agent infers and proposes possible actions, based on
scenarios. its own in-game goals. The agent organization framework medi-
The current state of research in semantic and declarative ates this autonomy, by controlling which behavioral adaptation
modeling, however, does not answer all of the issues discussed occurs for each agent. The agent organization framework coor-
in this section and many challenges still remain open. Among dinates individual adaptations into a combined one that adjusts
them, supporting the generation of game scenarios in a similar the global behavior of game elements to fit the player skill level
fashion (enriching them with an analogous semantic scheme), and a coherent storyline. For this coordination, Westra uses a
integrating these with virtual world generation, and measuring player model that estimates the skill levels of each player. This
player data into valid semantic knowledge, are some of the model is continuously updated to accurately steer the agent or-
issues that need to be addressed to consider semantics as a ganization framework.
possible relevant technique to customized content generation. Bakkes et al. [69] focus on adapting game AI, specifically an
AI-controlled opponent in a particular real-time strategy game.
In this case, online adaptivity takes place at the opponent AI, so
V. ONLINE ADAPTIVITY
that it can learn from its mistakes and act more effectively. The
As mentioned in Section III, offline customized generation authors propose an approach where domain knowledge is gath-
is not the only method to support adaptivity through delayed ered automatically by the game AI to form a case base (i.e., a
compilation of solutions of similar past problems), which is ex- Kazmi and Palmer [76] also direct their research to adapting
ploited immediately to evoke effective behavior. The case base game environments. They present a case study for a prototype
is compared with observations from previous games to allow of an adaptive FPS gaming environment. Player actions are
improvement on past behaviors. Preliminary research [70] has recognized through a finite state machine approach, by which
been done to incorporate opponent modeling, in this case used discrete actions reveal the skill level of players. Adaptation
for the AI to gain competitive advantage. Opponent models mechanisms try to make the game harder for players identified
are established automatically, through clustering of strategic as experts and easier for beginners. In this research, finite
feature data in game observations. Past game observations are state machines have also been used to implement all possible
classified with such models, allowing a better matching with adaptations, i.e., adjustments on NPC behavior and movement,
the case base. weapon mechanics, and game level geometry. Although this
On a similar direction, Hartley and Mehdi [71] also use a approach was mainly centered on adapting NPC behavior,
case-based approach that allows NPCs to learn, while playing, the authors successfully explore other alternative ideas. They
from the player actions, adapting the challenge level in the implemented a simple “modify geometry” mechanism that
game. Game observations are gathered in cases that take the dynamically changes the game environment so that it becomes
form of player state and action pairs. Matching these observa- more difficult to navigate safely. They conclude that the
tions with previously registered cases, can be used to predict the “modify geometry” mechanism provides the most significant
next state-action pair and, therefore, enhance the NPC decision impact in terms of effectiveness.
making. Results show that this approach succeeds in predicting In a different direction, some research has been done towards
player movement and actions in a FPS game. Therefore, despite online adaptivity in quests and game scenarios. Magerko et
its adaptivity focus on NPC (instead of player) goals, this case al. [18] ISAT project uses an intelligent director agent for
matching algorithm provides a method for adjusting the game customizing simulation training scenarios to suit individual
according to player actions. trainees. A skill model captures player proficiency levels in
Dynamic scripting [72] is another technique proposed for domain-specific skills, by monitoring and rating the trainee’s
adapting game AI, adopted for dynamic difficulty adjustment actions. Scenarios are identified as sequences of plot points, i.e.,
to the player skills. This learning technique is able to generate actions, events and skills involved in them, which are selected,
scripts (sets of behavioral rules), from rulebases associated with at run-time, for inclusion in the simulation. The director selects
NPC classes, in order to control NPC behavior. Each rulebase plot points by matching the list of tested skills with the current
comprises a set of manually designed rules and the probability state of the skill model.
that a rule is selected for a script is influenced by an attached Sullivan et al. [61] also proposed a centralized approach, the
weight value. Weights are updated according to their success Grail Game Manager, a run-time manager which dynamically
rate in the game, which includes maintaining an even and chal- generates quest structures using the player’s history and current
lenging game for players. world state. This rule-based system is able to decompose quests
Some promising work has already been done in directions (from a quest library) into separate entities (goals, actions, re-
other than AI and NPCs. Adaptive (simple) game worlds and wards, NPCs, dialog options) that can be dynamically recom-
levels are starting to be researched. Togelius et al. [73] propose bined upon generation. This process filters possible quest enti-
an approach for generating, online, tracks for a racing game. ties through preconditions based on player history and current
Their goal is to augment player satisfaction, by creating a track world state, thus creating a personalized experience.
that evolves with the player’s characteristics. A player model is
implemented by a neural network-based controller which infers A. Discussion
and simulates the behavior of the real player. This player model As surveyed above, current research results show that on-
is used to predict entertainment levels of specific players and de- line adaptivity is mainly concerned with adjusting challenge
cide how to evolve the track. Tracks are initialized as b-splines levels of NPCs and game AI. In these approaches, online adap-
with 30 segments and they evolve through a mutation done by tivity is still characterized by a certain degree of predictability,
perturbing the positions of their control points. Although the since some of the analyzed methods require all possible varia-
focus is on 2-D racing tracks, this work shows promising re- tions to be created a priori. Performance and control over the
sults regarding the use of player modeling to generate adapted game are the reasons why this balance, between static and dy-
game content. Search-based procedural content generation is namic techniques, is a recurring and important challenge in on-
also used similarly for creating personalized Super Mario Bros line adaptivity.
platform levels [74], even though an exhaustive search approach The current scope (AI and NPCs), purpose (challenge level
is preferred over evolutionary algorithms. balance) and techniques (combination of predefined content)
Jennings-Teats et al. [75] also focus on dynamically con- in these approaches show that online adaptivity is in its first
structing 2-D game environments that are adapted to players. steps. Integrated approaches, embracing online adaptive game
In this case, the Polymorph algorithm generates 2-D platform worlds and scenarios, are still far away. Confirmation of this
levels, as you play, driven by the dynamic difficulty adjustment is that player modeling and monitoring is still considered on
to the player skills. A statistical model of difficulty and a model an individual case, without sound and common theoretical
of the player’s current skill level are used, through mass data foundations.
collection and machine learning techniques, to select the appro- However, recent work has broadened the focus to adapting
priate level segments to generate for each player. game environments and other game elements. Although, for
example, Kazmi’s “modify geometry” mechanism was simple in not only adapting towards an optimal challenge level (both in
and applied to only one type of situation, one can easily foresee industry and academia), but also towards other affective states
the potential of adapting more than just the behavior of intel- like fun, frustration, predictability, anxiety or boredom. With
ligent agents. Procedural content generation (as surveyed in simulations, research is already successful in adapting to fit spe-
Section IV) is becoming more powerful and can have a role to cific skill levels and incorporate learning styles. In this domain,
play in online adaptivity as well. An example of such potential assessment methods are becoming better at creating and using
is evidenced by Kenneth Stanley’s Galactic Arms Race [77], a personal performance data.
multiplayer game where players control a spaceship and collect Current advances also show that work is already being laid
weapons throughout a game world. Weapons are procedurally out to approach some still open issues. For both entertainment
generated, at run-time, based on which weapons have been games and simulations, the main goal lies in materializing
selected and used before by players. Magerko’s vision of adapting to match the player’s real motiva-
Results both in 2-D game worlds and quest/scenarios confirm tions for playing [5]. To accomplish this, the challenge is to go
this observation: procedural content generation is becoming further beyond current results, by capturing and incorporating
more and more online efficient and player-centric. This in- higher resolution player data, including gameplay expectations,
dicates a promising research direction on extending these broad learning preferences and assessment data. There are
procedural methods and online adaptivity to the integration of many open and intriguing research questions in recognizing,
modern game worlds with the events and actions that comprise capturing, and defining these steering purposes in a detailed
game scenarios. and measurable way.
In the previous section, we already discussed the benefits Concerning the targets of adaptivity, we have concluded that
of embedding semantics, atop geometry, in a virtual world. a large community both in industry and academia has already
A promising direction could be to integrate these methods in been focusing on game mechanics, AI, NPC, and narratives.
online adaptivity, and use such knowledge to adjust properties Fewer research groups are already focusing, with success, on
of events or objects to match the player performance. Such adaptive game scenarios or quests. On the other side, concerning
work could build up on Kessing et al. [78] who introduce the adaptivity in modern and complex game worlds, many research
notion of services in game worlds. Services are defined as the questions are still unanswered.
capacity of an object to perform an action, possibly subject We think that there is a lot of potential not only in adaptive
to some requirement. For example, an apple (or any food for game worlds, but particularly in their integration with adaptive
that matter) “provides the service” of satisfying someone’s scenarios/quests. The fulfillment of a game scenario within a
hunger. These services are added to a semantic layer on an virtual world is an essential aspect of gameplay, determining
object level, which means that virtual objects know how they the sequence of events within an immersive environment, in
can affect other entities or how others can interact with them. which the player participates. Acting upon these, in an inte-
Semantic services are specified as generic characteristics of grated manner, can create plenty of (yet unexplored) possibil-
classes of objects and, therefore, object instances can acquire ities for improving gameplay.
their own independent behavior, instead of a rigidly predefined Regarding the methods which can support adaptivity, some
one. Including player data as a possible service requirement important advances have already been achieved with offline and
could act as a starting point for a new form of online adaptive online techniques. One of the most promising methods is the
worlds and scenarios. In this way, different services could act procedural generation of off-line (i.e., pregame) content that is
differently according to the player, therefore steering available customized to fit each player. Procedural content generation is
events in the game world. becoming more controllable, although mainly through control
Finally, modifying landscapes and topography of virtual over geometric features of that content. Some preliminary work
worlds has already been suggested as a valid direction for has been done to: 1) incorporate player data; 2) control genera-
online adaptivity [3]. However, future methods for dynamically tion using high level information (e.g., object metadata, seman-
changing game environments, on a world scale, must tackle tics); and 3) integrate game level and event generation. Further
important challenges to be successful. Among them, main- advances are being achieved in generating personalized basic
taining coherence (e.g., mountains cannot magically change quests, for entertainment games, and generating scenarios from
shape) and ensuring performance and scalability are important declarable learning goals, for serious games. For offline adap-
aspects, so that this type of adaptivity does not undermine the tivity, customized content generation seems the most promising
player experience. method in terms of future research. In this field, the main issues
to address relate to the interpretation of player data to, for ex-
ample, generate insight on gameplay expectations and learning
VI. CONCLUSION preferences. Such “intelligence” could then control what and
In this paper, we surveyed the present state of adaptivity in how to generate next, particularly in integrated game worlds
games and simulations. We focused on the purposes, targets, and scenarios. This high level knowledge fits well with the con-
and methods that have been proposed so far to support adaptive cepts used in present semantic modeling research. We therefore
game technology, from both academia and industry. conclude that virtual world and object semantics can be instru-
Our first conclusion is that adaptivity is already establishing mental to customized content generation.
itself as a rapidly maturing field regarding its purposes. Current Online (i.e., in-game) adaptivity is also an essential method
advances, both in industry and academia, indicate good results to consider. Current research is succeeding in using player
[7] C. Beal, J. Beck, D. Westbrook, and P. Cohen, “Intelligent modeling

of the user in interactive entertainment,” in Proc. AAAI Spring Symp.
Artif. Intell. Interact. Entertain., Stanford, CA, 2002, pp. 8–12.
[8] D. Thue, V. Bulitko, M. Spetch, and E. Wasylishen, “Interactive sto-
rytelling: A player modelling approach,” in Proc. 3rd Artif. Intell. In-
teract. Dig. Entertain. Conf., Stanford, CA, Jun. 2007, pp. 43–48.
[9] G. N. Yannakakis and J. Togelius, “Experience-driven procedural con-
tent generation,” IEEE Trans. Affect. Comput., vol. 99, 2011.
[10] G. Yannakakis and J. Hallam, “Real-time game adaptation for opti-
mizing player satisfaction,” IEEE Trans. Comput. Intell. AI Game., vol.
1, no. 2, pp. 121–133, Jun. 2009.
[11] C. Pedersen, J. Togelius, and G. N. Yannakakis, “Modeling player ex-
perience for content creation,” IEEE Trans. Comput. Intell. AI Game.,
vol. 2, no. 1, pp. 54–67, Mar. 2010.
[12] P. Rani, N. Sarkar, and C. Liu, “Maintaining optimal challenge in
computer games through real-time physiological feedback,” in Proc.
Fig. 6. Open challenges for adaptivity in games and simulations.
11th Int. Conf. Human Comput. Interact., Las Vegas, NV, 2005, pp.
184–192.
[13] G. Chanel, C. Rebetez, M. Bétrancourt, and T. Pun, “Boredom, engage-
ment and anxiety as indicators for adaptation to difficulty in games,” in
models to control the adaptation of NPC run-time behavior.
Proc. 12th Int. Conf. Entertain. Media Ubiquitous Era, Tampere, Fin-
Moreover, promising work is being done on dynamically con- land, 2008, pp. 13–17.
structing game scenarios, and even of game environments, that [14] S. Tognetti, M. Garbarino, A. Bonarini, and M. Matteucci, “Modeling
are adapted to the player in-game performance. For adaptivity’s enjoyment preference from physiological responses in a car racing
game,” in Proc. IEEE Conf. Comput. Intell. Games, Copenhagen,
purposes, online adjustment of game worlds and scenarios is Denmark, Aug. 2010, pp. 321–328.
likely to achieve better results than with NPCs. However, new [15] C. Harteveld, R. Guimarães, I. Mayer, and R. Bidarra, “Balancing play,
methods will need a paradigm shift, already encouraged by meaning and reality: The design philosophy of LEVEE PATROLLER,”
the recent advances in procedural content generation: from Simulation and Gaming, vol. 3, no. 43, pp. 316–340, 2010.
[16] J. Westra, F. Dignum, and V. Dignum, “Keeping the trainee on track,”
searching and selecting among predefined solutions to gener- in IEEE Conf. Comput. Intell. Game., Copenhagen, Denmark, Aug.
ating dynamic emerging ones. 2010, pp. 450–457.
Fig. 6 summarizes the challenges we discussed in this ar- [17] N. Peirce, O. Conlan, and V. Wade, “Adaptive educational games: Pro-
viding non-invasive personalised learning experiences,” in Proc. 2nd
ticle and how they relate to each other. Gameplay expectations, IEEE Int. Conf. Digital Game. Intell. Toys Based Edu. , Banff, AB,
learning preferences or assessment data can be used to guide Canada, Nov. 2008, pp. 28–35.
both offline customized generation and online adaptivity of in- [18] B. Magerko, B. Stensrud, and L. Holt, “Bringing the schoolhouse inside
tegrated game worlds and scenarios. the box—a tool for engaging, individualized training,” in Proc. 25th
Army Sci. Conf., Orlando, FL, Nov. 2006.
These challenges open up a variety of promising research di- [19] J. Niehaus and M. O. Riedl, “Scenario adaptation: An approach
rections. Pursuing them will likely lead to the development of to customizing computer-based training games and simulations,” in
new methods and techniques, which in turn will improve present Proc. AIED Workshop Intell. Edu. Game., Brighton, U.K., 2009, pp.
89–98.
adaptive game technology. As a result, games and simulations [20] W. L. Johnson, C. Beal, A. Fowles-Winkler, U. Lauper, S. Marsella,
can become more flexible, agile, and complete in the way they S. S. Narayanan, D. Papachristou, and H. Vilhjalmsson, “Tactical lan-
adapt to the player. Ultimately, a better adaptivity will foster guage training system: An interim report,” in Proc. Conf. Intell. Tu-
the potential to make games and simulations even more unpre- toring Syst., Berlin, Germany, Jun. 2004, pp. 336–345.
[21] H. C. Lane, M. Core, D. Gomboc, A. Karnavat, and M. Rosenberg,
dictable, effective and fun. “Intelligent tutoring for interpersonal and intercultural skills,” in Proc.
Interservice/Industry Train., Simul., Edu. Conf., 2007.
[22] G. A. Martin, C. E. Hughes, S. Schatz, and D. Nicholson, “The use
ACKNOWLEDGMENT of functional L-systems for scenario generation in serious games,” in
The authors would like to thank the anonymous reviewers for Proc. ACM Workshop Procedural Content Generation Games, Mon-
terey, CA, 2010, pp. 6:1–6:5.
their insightful comments. [23] J. Rowe, B. Mott, S. McQuiggan, J. Robison, S. Lee, and J. Lester,
“Crystal island: A narrative-centered learning environment for eighth
grade microbiology,” in Proc. Workshop Intell. Edu. Game., Brighton,
REFERENCES
U.K., 2009, pp. 11–20.
[1] C. Aldrich, “A field guide to educational simulations,” Learn. Circuits, [24] J. Rowe and J. Lester, “Modeling user knowledge with dynamic
2002. Bayesian networks in interactive narrative environments,” in Proc.
[2] S. Blackman, “Serious games … and less!,” ACM SIGGRAPH Comput. 6th Annu. AI Interact. Digital Entertain. Conf., Stanford, CA, 2010,
Graphics, vol. 39, no. 1, pp. 12–16, 2005. pp. 57–62.
[3] D. Charles, A. Kerr, M. McNeill, M. McAlister, M. Black, J. Kücklich, [25] B. Magerko, C. Heeter, J. Fitzgerald, and B. Medler, “Intelligent adap-
A. Moore, and K. Stringer, “Player-centred game design: Player mod- tation of digital game-based learning,” in Proc. ACM Conf. Future
elling and adaptive digital games,” in Proc. DiGRA Conf. Changing Play, Toronto, ON, Canada, 2008, pp. 200–203.
Views—Worlds in Play, Vancouver, BC, Canada, 2005, pp. 285–298. [26] S. Chen and D. Michael, “Proof of learning: Assessment in serious
[4] K. M. Gilleade and A. Dix, “Using frustration in the design of adaptive games,” 2005 [Online]. Available: http://www.gamasutra.com/fea-
videogames,” in Proc. ACM SIGCHI Int. Conf. Adv. Comput. Enter- tures/20051019/chen_01.shtml
tainment Technol., Singapore, 2004, pp. 228–232. [27] D. R. Lampton, J. Bliss, and G. A. Martin, “Performance measurement
[5] B. Magerko, “Adaptation in digital games,” IEEE Comput. Mag., vol. and training feedback in a military collaborative virtual environment,”
41, no. 6, pp. 87–89, Jun. 2008. in Proc. 1st Int. Conf. Virtual Reality, Las Vegas, NV, Jul. 2005.
[6] R. Houlette, “Player modeling for adaptive games,” in AI Game Pro- [28] E. M. Raybourn, “Applying simulation experience design methods
gramming Wisdom II. Boston, MA: Charles River Media, 2004, pp. to creating serious game-based adaptive training systems,” Interact.
557–566. Comput., vol. 19, no. 2, pp. 206–214, 2007.
[29] M. G. Core, H. C. Lane, M. van Lent, D. Gomboc, S. Solomon, and [54] J. Doran and I. Parberry, “Controlled procedural terrain generation
M. Rosenberg, “Building explainable artificial intelligence systems,” using software agents,” IEEE Trans. Comput. Intell. AI Game., vol. 2,
in Proc. 18th Conf. Innovative Appli. Artif. Intell., Boston, MA, 2006, no. 2, pp. 111–119, Jun. 2010.
vol. 2, pp. 1766–1773. [55] A. Peytavie, E. Galin, S. Merillou, and J. Grosjean, “Arches: A Frame-
[30] D. Gomboc, S. Solomon, M. G. Core, H. C. Lane, and M. van Lent, work for modeling complex terrains,” Comput. Graphics Forum (Proc.
“Design recommendations to support automated explanation and tu- Eurographics), vol. 28, no. 2, pp. 457–467, 2009.
toring,” in Proc. 14th Conf. Behav. Representation Modeling Simul., [56] G. Chen, G. Esch, P. Wonka, P. Müller, and E. Zhang, “Interactive
Universal City, CA, 2005. procedural street modeling,” ACM Trans. Graphics, vol. 27, no. 3, pp.
[31] J. P. Rowe, L. R. Shores, B. W. Mott, and J. C. Lester, “Individual dif- 1–10, 2008.
ferences in gameplay and learning: a narrative-centered learning per- [57] J. Gain, P. Marais, and W. Strasser, “Terrain sketching,” in Proc. Symp.
spective,” in Proc. 5th Inter. Conf. Found. Digital Game., Monterey, Interact. 3D Graphics Game., Boston, MA, 2009, pp. 31–38.
CA, 2010, pp. 171–178. [58] G. J. P. de Carpentier and R. Bidarra, “Interactive GPU-based pro-
[32] B. Brathwaite and I. Schreiber, Challenges for Game Designers. cedural heightfield brushes,” in Proc. 4th Int. Conf. Found. Digital
Boston, MA: Charles River Media, 2009. Game., Orlando, FL, 2009, pp. 55–62.
[33] Remedy Entertainment, Max Payne, Gathering of Developers, 2001 [59] J. McCrae and K. Singh, “Sketch-based path design,” in Proc. Graphics
[Online]. Available: http://www.rockstargames.com/maxpayne/ Interface Canadian Inform. Process. Soc., Toronto, ON, Canada, 2009,
splash.html pp. 95–102.
[60] M. Bielikova, M. Diveky, P. Jurnecka, R. Kajan, and L. Omelina, “Au-
[34] Nintendo EAD, Mario Kart Wii, 2008 [Online]. Available: http://www.
tomatic generation of adaptive, educational and multimedia computer
mariokart.com/wii/launch/
games,” Signal Image Video Process., vol. 2, no. 4, pp. 371–384, Dec.
[35] Konami, Pro Evolution Soccer 2008, 2007 [Online]. Available: http://
2008.
www.games.konami-europe.com/game.do?idGame=149
[61] A. Sullivan, M. Mateas, and N. Wardrip-Fruin, “Rules of engagement:
[36] Valve Corporation, Left 4 Dead, 2008 [Online]. Available: http://www.
Moving beyond combat-based quests,” in Proc. Intell. Narrative
l4d.com/game.html Technol. III Workshop, Monterey, CA, 2010, pp. 11:1–11:8.
[37] Valve Corporation, Left 4 Dead 2 2009 [Online]. Available: http:// [62] J. Pita, B. Magerko, and S. Brodie, “True story: Dynamically generated,
www.l4d.com/game.html contextually linked quests in persistent systems,” in Proc. Conf. Future
[38] G. Newell, Gabe Newell Writes for Edge 2008 [Online]. Available: Play, Toronto, ON, Canada, 2007, pp. 145–151.
http://www.next-gen.biz/blogs/gabe-newell-writes-edge [63] C. Ashmore and M. Nitsche, “The quest in a generated world,” in
[39] Quantic Dream, Heavy Rain, Sony Computer Entertainment, 2010 Proc. Digital Games Res. Assoc. (DiGRA) Conf.: Situated Play, Tokyo,
[Online]. Available: http://www.heavyrainps3.com/ Japan, Sep. 2007, pp. 503–509.
[40] D. L. Roberts and C. L. Isbell, “A survey and qualitative analysis of [64] J. Dormans, “Adventures in level design: generating missions and
recent advances in drama management,” Int. Trans. Syst. Sci. Appl., spaces for action adventure games,” in Proc. Workshop Procedural
vol. 4, no. 2, pp. 61–75, 2008. Content Generation Game., Monterey, CA, 2010, pp. 1:1–1:8.
[41] H. Barber and D. Kudenko, “Dynamic generation of dilemma-based [65] K. Hullett and M. Mateas, “Scenario generation for emergency rescue
interactive narratives,” in Proc. 3rd Artif. Intell. Interact. Digital En- training games,” in Proc. 4th Int. Conf. Found. Digital Games, Or-
tertain. Conf., Stanford, CA, 2007, pp. 2–7. lando, FL, 2009, pp. 99–106.
[42] B. W. Mott and J. C. Lester, “U-director: A decision-theoretic narrative [66] T. Tutenel, R. Bidarra, R. M. Smelik, and K. J. de Kraker, “The role of
planning architecture for storytelling environments,” in Proc. 5th Int. semantics in games and simulations,” Comput. Entertain., vol. 6, no.
Joint Conf. Autonom. Agents Multiagent Syst., Hakodate, Japan, 2006, 4, pp. 1–35, 2008.
pp. 977–984. [67] R. Bidarra, K. J. de Kraker, R. Smelik, and T. Tutenel, “Integrating se-
[43] M. Sharma, S. Ontan, C. Strong, M. Mehta, and A. Ram, “Towards mantics and procedural generation: Key enabling factors for declara-
player preference modeling for drama management in interactive tive modeling of virtual worlds,” in Proc. FOCUS K3D Conf. Semantic
stories,” in Proc. 20th Int. FLAIRS Conf., Key West, FL, 2007, pp. 3D Media Cont., Sophia Antipolis, France, Feb. 2010, pp. 51–55.
571–576. [68] R. Smelik, T. Tutenel, K. de Kraker, and R. Bidarra, “A declarative
[44] C. Fairclough, “Story games and the OPIATE system,” Ph.D. disser- approach to procedural modeling of virtual worlds,” Compu. Graphics,
tation, Dept. Comput. Sci., Univ. Dublin–Trinity College, Dublin, Ire- vol. 35, no. 2, pp. 352–363, 2011.
land, 2006. [69] S. Bakkes, P. Spronck, and J. van den Herik, “Rapid and reliable adap-
[45] Walker, J. for Rock, Paper, Shotgun Webzine, Left 4 Dead 2: Exclusive tation of video game AI,” IEEE Trans. Comput. Intell. and AI in Games,
RPS Hands-On Preview, 2009 [Online]. Available: http://www.rockpa- vol. 1, no. 2, pp. 93–104, 2009.
pershotgun.com/2009/06/01/left-4-dead-2-exclusive-rps-preview [70] S. Bakkes, P. Spronck, and J. van den Herik, “Opponent modelling for
[46] M. Nitsche, C. Ashmore, W. Hankinson, R. Fitzpatrick, J. Kelly, and case-based adaptive game AI,” Entertain. Comput., vol. 1, no. 1, pp.
K. Margenau, “Designing procedural game spaces: A case study,” in 27–37, 2009.
Proc. FuturePlay, London, ON, Canada, Oct. 2006. [71] T. Hartley and Q. Mehdi, “Online action adaptation in interactive com-
[47] C. van Est and R. Bidarra, “High-level scenario editing for simulation puter games,” Comput. Entertain., vol. 7, no. 2, pp. 28:1–28:31, 2009.
games,” in Proc. 6th Int. Conf. Comput. Graphics Theory Appl., Al- [72] P. Spronck, M. Ponsen, I. Sprinkhuizen-Kuyper, and E. Postma,
“Adaptive game AI with dynamic scripting,” Mach. Learn., vol. 63,
garve, Portugal, 2011.
pp. 217–248, Jun. 2006.
[48] Maxis, Spore, Electronic Arts, 2008 [Online]. Available: http://www.
[73] J. Togelius, R. De Nardi, and S. Lucas, “Towards automatic person-
spore.com
alised content creation for racing games,” in Proc. IEEE Symp. Comp.
[49] R. Smelik, K. J. de Kraker, T. Tutenel, R. BidarraS, and G. Groe-
Intell. Game., Apr. 2007, pp. 252–259.
newegen, “A survey of procedural methods for terrain modelling,” in
[74] N. Shaker, J. Togelius, and G. N. Yannakakis, “Towards automatic per-
Proc. CASA Workshop 3D Adv. Media Gaming Simul., Amsterdam, sonalized content generation for platform games,” in Proc. 6th AAAI
The Netherlands, 2009, pp. 25–34. Conf. Artif. Intell. Interact. Digital Entertain., Stanford, CA, Oct. 2010,
[50] K. Compton and M. Mateas, “Procedural level design for platform pp. 63–68.
games,” in Proc. 2nd Artif. Intell. Interact. Digital Entertain. Conf., [75] M. Jennings-Teats, G. Smith, and N. Wardrip-Fruin, “Polymorph: Dy-
Marina del Rey, CA, 2006, pp. 109–111. namic difficulty adjustment through level generation,” in Proc. Work-
[51] G. Smith, M. Treanor, J. Whitehead, and M. Mateas, “Rhythm-based shop Procedural Content Generation Game., Monterey, CA, 2010, pp.
level generation for 2D platformers,” in Proc. 4th Int. Conf. Found. 11:1–11:4.
Dig. Game., Orlando, FL, Apr. 2009, pp. 175–182. [76] S. Kazmi and I. Palmer, “Action recognition for support of adaptive
[52] N. Sorenson and P. Pasquier, “Towards a generic framework for au- gameplay: A case study of a first person shooter,” Int. J. Comput. Game.
tomated video game level creation,” in Applications of Evolutionary Technol., 2010.
Computation, EvoApplicatons 2010, Proceedings, ser. Lecture Notes [77] E. Hastings, R. Guha, and K. Stanley, “Automatic content generation
in Computer Science. Berlin, Germany: Springer-Verlag, 2010, vol. in the galactic arms race video game,” IEEE Trans. Comput. Intell. AI
6024, pp. 131–140. Game., vol. 1, no. 4, pp. 245–263, 2009.
[53] P. Müller, P. Wonka, S. Haegler, and A. U. Van Gool, “Procedural mod- [78] J. Kessing, T. Tutenel, and R. Bidarra, “Services in game worlds: A se-
eling of buildings,” ACM Trans. Graphics, vol. 25, no. 3, pp. 614–623, mantic approach to improve object interaction,” in Proc. 8th Int. Conf.
2006. Entertain. Comput., 2009, vol. 5709, pp. 276–281.
Ricardo Lopes received the B.Sc. and M.Sc. degrees Rafael Bidarra graduated in 1987 in electronics en-
in information systems and computer engineering gineering at the University of Coimbra, Portugal, and
from the Technical University of Lisbon, Lisbon, received the Ph.D. degree in computer science from
Portugal, in 2007 and 2009, respectively. His Ph.D. Delft University of Technology, Delft, The Nether-
research subject is the “Generation of adaptive game lands, in 1999.
worlds.” He is currently an Associate Professor of Game
His current research interests include adaptivity in Technology at the Faculty of Electrical Engineering,
games, player modeling, interpretation mechanisms Mathematics and Computer Science, Delft Univer-
for in-game data, and (online) procedural generation sity of Technology. He leads the research line on
techniques. game technology at the Computer Graphics Group.
His current research interests include procedural
and semantic modeling techniques for the specification and generation of both
virtual worlds and game play; semantics of navigation; serious gaming; game
adaptivity and interpretation mechanisms for in-game data. He has published
numerous papers in international journals, books, and conference proceedings.
He integrates the editorial board of several journals, and has served in many
conference program committees.

Adaptivity Challenges in Games and Simulations A Survey

Uploaded by

Copyright:

Available Formats

Adaptivity Challenges in Games and Simulations A Survey

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Adaptivity Challenges in Games and Simulations A Survey

Uploaded by

Copyright:

Available Formats

IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES, VOL. 3, NO.

Adaptivity Challenges in Games and

T YPICALLY, when most commercial games are shipped,

1943-068X/$26.00 © 2011 IEEE

(e.g., electrocardiograms, galvanic skin response, electroen-

B. Serious Games and Simulations

TABLE I recognizes the player actions and responds by adjusting NPC

authoring. Online, or in-game, adaptivity is the term many re-

[7] C. Beal, J. Beck, D. Westbrook, and P. Cohen, “Intelligent modeling

You might also like