Concept Art by AI Art Generators:

Analyzing the Benefits, Limitations, and

Potential of Using AI Art Generators to Create
Concept Art for Film and TV Animation
Production

Stephanie Ngoei
ID: 1385985
The University of Melbourne
Friday, 4 November 2022
Table of Contents

Executive Summary ............................................................................................................................... 3

Introduction ........................................................................................................................................... 4
1. Concept Art: What is Involved .......................................................................................................... 4
1.1. Concept Art and Concept Artists ................................................................................................. 4
1.2. Skills Needed to Create Concept Art ............................................................................................ 6
1.3. Criteria of Good Concept Art ....................................................................................................... 6
2. AI Art Generators ............................................................................................................................... 7
2.1. How AI Art Generators Work ...................................................................................................... 7
2.2. Examples of AI Art Generators .................................................................................................... 8
3. Discussion ......................................................................................................................................... 10
4. Conclusion ........................................................................................................................................ 12
Bibliography ......................................................................................................................................... 14

2
Executive Summary

This research report discusses the benefits, limitations, and potential that AI art generators have
when used to create concept art for film and tv (FTV) animation production. Laying out what is in
store and what is at stake when using AI art generators to create concept art will allow concept
artists, as well as studios and other animation industry professionals, to weigh on their prospects
and assess how they can adapt their career with the continuous development of machine learning.

Analyzing theories and case studies helped to pinpoint a major issue in AI art generators: although
they could create concept art much faster and more novel than artists, they do not understand the
context and meaning behind the patterns they are trained with, and therefore have difficulty
generating artwork that has functionality/plausibility in the audience’s eyes. Functionality is an
important criterion of good concept art, but many art generators are not able to fulfill this despite
showing good performance in the other two criteria: visual attractiveness and adherence to written
concept.

To conclude, concept artists and AI art generators should not be pitted against each other but
complement each other’s strengths and limitations.

3
Introduction

AI algorithms used to make art started developing during the 1970s to 1990s. Computer-aided
design (CAD) software, powered by AI algorithms, initially generated abstract art but learned to
make more complex art, such as humans, plants, and rocks (Kundu 2022).

It was not until recently that AI-generated art and AI art generators, namely DALL-E and
Midjourney became mainstream and widely recognized by internet users, fuelled by viral news
such as when an AI-generated artwork won first place at a state fair fine arts competition (Gault
2022).

But since its presence is still very new, its roles and purposes in various sectors and industries
have not been cemented yet. For digital artists, AI-generated art poses risks and provides
opportunities at the same time. This research report focuses specifically on the benefits and
limitations, and potential that concept artists (who are part of the digital artist demography)
have when using AI art generators to create concept art for film and tv (FTV) animation
production.

The objectives of this report in detail are as follows:

a. To understand concept art and what is involved in its creation

b. To explore how AI art generators work and are used in the present moment
c. To explore how AI art generators can be used to create concept art for FTV animation
production
d. To discuss the benefits, limitations, and future potential of using AI art generators to
create concept art for FTV animation.

By discussing the objectives above, this report can help concept artists, as well as other
professionals in animation production, decide whether these technologies are more beneficial or
disadvantageous to them and what next steps they can take to respond to its development in
the future.

Finally, due to time constraints, this report cannot account for all possible benefits, limitations,
and potential that AI art generators have when used in the concept art industry. This report is
also based on the current conditions and abilities of AI art generators, and as the technology
becomes more advanced, future situations regarding AI and the animation industry may be
different than predicted in this report. This report cannot account for every possible future
scenario and is therefore open to any future confounding.

1. Concept Art: What is Involved

1.1. Concept Art and Concept Artists

4
 The production of animated films and TV shows usually follows a pipeline, divided into three
major categories: pre-production, production, and post-production. Concept art falls into the
category of pre-production because, as its name suggests, it involves creating the overall
concept of the show/movie in visual form. In other words, concept art is the ‘worldbuilding’
stage of the production process.

A concept artist’s job involves empathizing with

the producer’s vision, studying the
script/written concept, researching materials
and references, ideating and creating
illustrations and designs, focusing on specific
ideas and visual style, and finally revising them
with the help of the director’s feedback. This
shows how they implement design thinking
(Figure 1), which makes the job more associated
to design compared to art.
Figure 1. https://www.interaction-
Because concept art is a broad field, artists tend design.org/literature/article/5-stages-in-the-design-
thinking-process
to further specialize in more specific areas
under the concept art umbrella, such as
character design (Figure 2), environment design (Figure 3), hard-surface design (Figure 4), color
script (Figure 5) and others in-between (Engler and Trnka 2021).

Figure 3. http://www.matthiaslechner.com/zootopia.html

Figure 2. https://blog.herzing.ca/the-2d-animation-
pipeline#scene-setup

5
 Figure 5.
https://twitter.com/disneyanimation/status/150522798560
Figure 4. https://www.artstation.com/artwork/oOwdgq 7004163?lang=ha

1.2. Skills Needed to Create Concept Art

A concept artist must come up with original design solutions that are plausible for the audience.
Therefore, the creation of concept art should allow for both creativity and design thinking, while
adhering to a strict timeframe. To keep up with the animation industry’s quality-driven and fast
production environment, concept artists need to acquire and master a set of skills, software, and
techniques.

Creating concept art involves a traditional mastery of drawing, painting, and sculpting, which
help the artists understand and use digital software. Software or apps used for concept art allow
artists to work with a mixture of 2D and 3D techniques. Some common examples are Adobe
Photoshop, Substance Painter, Autodesk Maya, Blender, and Zbrush. New techniques developed
from these apps to help concept artists work more efficiently, such as sketching, thumbnailing,
speedpainting, photobashing, kitbashing, 3D overpainting, sculpting, and 3D/VR sketching
(Engler & Trnka 2021).

1.3. Criteria of Good Concept Art

Concept art must look functional, visually attractive, and must be built on its narrative purpose
(Urschel, 2017). Firstly, functional in this context means that the new, imaginative concepts
created should be based on visual and theoretical analyses of reality. For example, the animals,
vehicles, and cities in the Disney animated feature Zootopia are entirely fictional but are still
fundamentally derived from animals, vehicles, and cities in real life, which aligns with the
audience’s logic on how things should look and function. (Figures 6, 7, 8).

6
 Figure 7.
https://disneyconceptsandstuff.tumblr.com/post/141454
275255/prop-designs-from-zootopia-by-matthias-lechner

Figure 6.
https://jsdchamp99.tumblr.com/post/140882112452/i-
finally-managed-to-get-the-art-of-zootopia

Figure 8. http://www.matthiaslechner.com/zootopia.html

Visually attractive means that just like any kind of art, good concept art makes good use of the
fundamental elements and principles of art, which include perspective, values of light, color
theory, shape and line design, as well as knowledge of the form and anatomy (Engler & Trnka
2021).

Finally, concept art, being part of design, should serve as a visual solution for a written narrative.
Concept artists work closely with fellow artists, the scriptwriter, and the art director, who
oversees the overall visual identity of the project to ensure that it adheres strongly to the
narrative concept.

2. AI Art Generators

2.1. How AI Art Generators Work

AI art generators are a more general terminology for Generative Adversarial Networks (GAN). (In
this report, however, we will be using the term ‘AI art generators’ instead of GAN). GANs are
models of deep learning algorithms, a subfield of machine learning that is comprised of complex

7
 neural networks (Figure 9) (Kavlakoglu 2020). The hierarchy between artificial intelligence,
machine learning, neural networks, and deep learning is visualized in Figure 10.

Figure 9. https://www.ibm.com/cloud/blog/ai-vs-machine-learning-
vs-deep-learning-vs-neural-networks

Figure 10. https://www.ibm.com/cloud/blog/ai-vs-machine-learning-vs-

deep-learning-vs-neural-networks

GANs are composed of two neural networks, the generator, and the discriminator, which are
pitted against one another to train each other. The generator is trained to produce fake data
(output images), and the discriminator is trained to distinguish the generator’s fake data from
actual examples (input images). Once the training is complete, the generator is able to create
perfectly mimicked images (that go undetected by the discriminator), while the discriminator
learns to accurately classify the real images from the fake ones.

Some commonly used GAN variants include Progressive GAN, Conditional Gan (CGAN), Deep
Convolutional GAN (DCGAN), CycleGAN, Generative Adversarial Text to Image Synthesis, Style
GAN, and Super Resolution GAN (SRGAN) (Brownlee 2019; Shah 2022).

GANs, or AI art generators, are the prime examples of AI capable of ‘fully autonomous creation’,
part of the AI taxonomy mentioned in Manovich’s theory of aesthetic AI (2019). They are able to
generate images based on a set of parameters (e.g. Midjourney and DALL-E), create new images
by combining and altering existing images (e.g. Deep Dream), and generate images that mimic
the style of a particular artist or are similar to a particular type of art (e.g. Stable Diffusion).

2.2. Examples of AI Art Generators

8
Most AI art generators generate images based on the Generative Adversarial Text to Image
Synthesis model, but they are marketed with additional tools and features that distinguish them
from each other. One notable example is DALL-E 2, which creates images based on a text
description/prompt in natural language. The uses of DALL-E 2 range from generating 3D designs
to creating illustrations (Figures 11 and 12).

Figure 11. https://cdn.openai.com/papers/dall-e-2.pdf

Figure 12. https://cdn.openai.com/papers/dall-e-2.pdf

Meanwhile, Midjourney is a simpler generator compared to DALL-E 2 but is popular among users
because it is entirely free of charge. It uses text prompts which are then translated into four
pictures that correlate to the words. The user can then ‘upscale’ (add more detail, upgrade its
resolution, or develop even further) the image(s) of their choice (Yalalov 2022). Midjourney is
particularly skilled in making landscape and background art, exemplified by John Walther’s
creations below (Figures 13 and 14).

9
Figure 13. Figure 14.
https://cdna.artstation.com/p/assets/images/images/052/ https://cdnb.artstation.com/p/assets/images/images/052/
177/240/large/john-walther-4.jpg?1659122239 312/857/large/john-walther-5.jpg?1659475879

3. Discussion

3.1. AI art generators for concept art: Technical discussions with Benefits, Limitations, and Potential

As mentioned before, concept art is divided into specializations such as character design,
environment design, hard-surface design, and color script. Theoretically, if an AI art generator
can create sketches, illustrations, vector designs, and 3D models, they should be able to work in
all concept art specializations, but to assess the performance of these generators, we need to
look at the plausibility of and how the users (other artists in the pipeline and the directors)
receive the generated work (Manovich 2019). How then, do AI art generators fare in each of
these specializations?

For character design, we can refer to a demonstration of DALL-E 2 by Bakz T. Future on YouTube
(2022). He showed that it was possible to create variants of a character using the same prompt.
However, character design is also about exploring the gestures, facial expressions, and
personality of the character, as opposed to just creating character models. But this may become
a problem when using art generators like Midjourney, which usually ends up distorting faces and
anatomy (Nielsen 2022) although they are skilled at creating environment designs. But why
specifically faces and anatomy? This may partly be because the generator is trained to “extract
patterns and formal structures” but is not trained to consider how the patterns will affect the
audience’s “subjective interpretation and reaction” (Arielli 2022). Concept artists are trained by
observation and education of human mannerisms (that audiences can associate with) that they
translate into their work, but AI generators, not understanding the human meaning behind the
datasets they are given, may not be able to create the faces and anatomy that the audience
perceives as accurate to real life.

AI art generators seem to perform the best when making environment/background designs, but
that also depends on which generator is being used. We can compare two environment designs,
one made with Artbreeder and one with Dream, below (Figures 15 and 16). Artbreeder and
DALL-E’s popularity can be attributed to their skills for making visually aesthetic environments
that apply proper use of art elements and principles such as depth, composition, and
perspective to elevate the environment’s plausibility. Plausibility in the eyes of the audience,
referred to by Engler and Trnka (2021) as functionality, is an important criterion of success in
both concept art and aesthetic AI. The environment created by Dream may look visually

10
 appealing, but it lacks plausibility, evidenced by the way some of the elements look distorted, do
not connect with each other, and do not adhere to the rules of perspective. At best, Dream can
be used to generate concept prompts for environment designers but cannot be used as a
finalized design.

Figure 15. Environment design made by Manu T. in Dream by Figure 16. Environment Design made by John Walther in
WOMBO. Artbreeder.
https://cdnb.artstation.com/p/assets/images/images/049/886/ https://cdnb.artstation.com/p/assets/images/images/0
541/large/manu-t-cyberpunk.jpg?1653551293 43/311/629/large/john-walther-3.jpg?1636922327

Hard-surface design often involves creating 3D models of complex objects such as vehicles and
robots (which are time-consuming to draw), which will then be textured or painted over.
Google’s DreamFusion translates text prompts into 3D models which can be used in any 3D
environment (Heikkilä 2022). Using 3D generators would save a lot of artists’ time from having
to start modeling from scratch, but similar to the cases above with character and environment
design, the artist needs to take the role of quality checker to ensure that the hard-surface
designs are functional, visually attractive, and adhere to the written concept.

Finally, for color scripts, the AI art generator must be able to create simple illustrations that
highlight the use of color and show color cohesion between one frame and another. AI art
generators may be skilled in creating singular illustrations and sketches, but they fall short on
color and style consistency if a recognizable art style is not included in the text prompt, or if they
are not trained to replicate art style of the concept artists working on the project.

From here, we will be discussing in a more general sense about benefits, limitations, and
potential of AI art generators in concept art.

Concept artists are constantly adopting new skills, software, and techniques to keep up with the
fast-paced and quality-driven environment of the animation industry. In this context, one of the
major benefits of using AI art generators is that it saves time creating what are usually very time-
consuming processes. By automating parts of these processes, concept artists have more time to
explore different ideas and visual languages that they otherwise would not be able to because of
time constraints. Doing this can help studios try new visual styles for new films and shows.

AI art generators may not be able to create finalized concept art currently due to limitations in
their system, but this means that concept artists can use AI-generated images for prompts,
references, or inspiration for their work instead. According to Arielli (2021), each individual has
their own aesthetic and creative capacity, similar to how we have limited physical strength and
cognitive skills. AI such as art generators can therefore become a tool to augment or overcome
our limited aesthetic capabilities. In the context of this report, AI art generators may be a

11
 solution to a concept artist’s “art blocks”, which are periods of inability to conjure ideas and
concepts.

However, one of the limitations of AI art generators is that they are bound to repeat patterns
after some time if training processes are not updated with new data. This can lead to what
Manovich (2019) refers to as a “decrease in aesthetic diversity”. Therefore, one way to ensure
high aesthetic diversity in art generators is to implement Self-Supervised Machine Learning that
can “train existing models with newly available data without retraining the model from scratch”
(Kundu 2022).

Another way to increase aesthetic diversity in art generators is to develop more models in the
future that enable users to input more data to increase control over the network’s output.
Additionally, making generating processes more transparent and assessable helps counter the
‘black box’ idea (Manovich 2019) that is often associated with neural networks.

Arielli (2021) also argues that the current AI art generators cannot replicate the process of
creation that concept artists go through because they are only mimicking existing art styles that
are recognizable. This is why it is more difficult for them to reproduce concept art by artists or
studios who do not follow a defined style. Varying styles are “ill-defined tasks” that are more
difficult to automate compared to defined styles, the “well-defined problems” suitable for
automation. The art generator needs to be trained with the art style of a particular studio/artist
before they can generate concept art, otherwise, the generated concept art would lack cohesion
with each other—one image would have a different style from the other and thus ruin the
overall visual identity of the project. Lack of visual cohesion was a major issue when an artist
named Marcel Saleta (2022) attempted to recreate the concept art for his animated short using
Midjourney.

Finally, there are ethical problems to consider. It is unclear whether AI art generators are
authorized to use the data, artworks, and art styles that they are trained with to generate
artworks, leading to copyright issues that we might not have solutions for at this point in time.

4. Conclusion

At this point in time, using AI art generators to create concept art seems to have bigger
limitations compared to benefits. After analyzing how they would perform in concept art
specializations such as character, environment, and hard surface design, a concept artist would
still need to take the responsibility of quality checker to ensure that the designs are functional,
visually attractive, and adhere to the written concept. This is because the modern-day AI art
generators struggle with extracting the meaning and context behind the datasets they are given
and thus fail to generate art that considers the audience’s reaction and interpretation; struggle
to follow a uniform visual identity; as well as struggle to fulfil one of the three criteria that
indicate a good concept art: functionality (or plausibility) in the eyes of the audience. This is
speaking generally since every art generator faces different limitations.

However, AI art generators’ ability to automate time-consuming processes, resulting in faster

work time, as well as its role to augment or overcome our limited aesthetic capabilities are very

12
valuable qualities and can be built on further to overcome the struggles that they face today in
the concept art realm.

Finally, here are some suggestions that industry professionals can consider when dealing with AI
art generators in the future:

- Engineers can train generators to “use a sequence of actions”, similar to the thinking and
working process of a concept artist, instead of training it on a large dataset of existing
artworks that teach it how to mimic patterns.

- AI art generators have the potential to restructure the animation pre-production pipeline as
studios and companies will view this as an opportunity to cut costs. Instead of
hiring/outsourcing fewer concept artists to be replaced by engineers to train art generators,
companies and studios should instead support concept artists by giving them resources and
opportunities to grow their aesthetic capabilities and creative processes.

- Concept artists are recommended to unionize to protect their jobs and prevent layoffs due
to being replaced AI art generators and other future machine learning technologies.

13
Bibliography:

Brownlee, Jason. “A Tour of Generative Adversarial Network Models,” Machine Learning Mastery,
July 10, 2019. https://machinelearningmastery.com/tour-of-generative-adversarial-
network-models/

Engler, Martin, and Andrej Trnka. “Concept Art: The Essential Part of Visual Pre-production in the
Entertainment Industry.” Acta Ludologica 4, no. 1 (2021): 112-124

Engler, Martin, and Andrej Trnka. “Fundamental Knowledge behind Creation of Concept Art.”
European Journal of Media, Art & Photography 9, no. 2 (2021): 106-115

Future, Bakz T. “Concept Art Design Using DALL-E 2 AI (Characters, Environment, Assets, and
more...)” YouTube, April 28, 2022.
https://www.youtube.com/watch?v=zzKqtxeX2zc&t=183s

Gault, Matthew. “An AI-Generated Artwork Won First Place at a State Fair Fine Arts Competition,
and Artists Are Pissed,” VICE, September 1, 2022.
https://www.vice.com/en/article/bvmvqm/an-ai-generated-artwork-won-first-
place-at-a-state-fair-fine-arts-competition-and-artists-are-pissed

Heikkilä, Melissa. “Get ready for the next generation of AI,” MIT Technology Review, October 3,
2022. https://www.technologyreview.com/2022/10/03/1060561/get-ready-for-the-
next-generation-of-ai/

Kavlakoglu, Eda. “AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What’s the
Difference?” IBM, May 27, 2020. https://www.ibm.com/cloud/blog/ai-vs-machine-
learning-vs-deep-learning-vs-neural-networks

Kundu, Rohit. “AI-Generated Art: From Text to Images & Beyond,” V7 Labs, October 25, 2022.
https://www.v7labs.com/blog/ai-generated-art

Manovich, Lev. “AI and production of culture.” In AI Aesthetics. Moscow: Strelka Press, 2019.

Manovich, Lev, and Emanuele Arielli. Artificial Aesthetics: A Critical Guide to AI, Media and Design.

Nielsen, Lars. “Taking generated Midjourney (MJ) images to the next level,” Medium, September 7,
2022. https://medium.com/mlearning-ai/taking-generated-midjourney-mj-images-to-the-
next-level-b83bfc63410d

“The 2D Animation Pipeline.” Herzing Blog, August 24, 2022. https://blog.herzing.ca/the-2d-

animation-pipeline

Saleta, Marcel. “Can you use AI to make an animated film?! | SCARY!!!” Youtube, July 19, 2022.
https://www.youtube.com/watch?v=sRqFaq0osyo

Shah, Deval. “The Complete Guide to Generative Adversarial Networks [GANs],” V7 Labs, October 3,
2022. https://www.v7labs.com/blog/generative-adversarial-networks-guide

Urschel, J. “What Is Concept Art?” In The Ultimate Concept Art Career Guide. Worcester: 3Dtotal
Publishing, 2017.

14