International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022

International Journal of Research Publication and Reviews

Journal homepage: www.ijrpr.com ISSN 2582-7421

Cartoonifying an Image Using Machine Learning

Madugula Sai Joshitha

Department of Information Technology, GMR Institute of Technology

ABSTRACT

Machine learning is a branch of artificial intelligence (AI) which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving
its accuracy. Machine learning allows the user to feed a computer algorithm an immense amount of data and have the computer analyze and make data-driven
recommendations and decisions based on only the input data.In this study,the discussion is all about the Cartoonifying an Image.Cartoonify uses a neural network
to turn your uploaded photo into a unique a cartoon.There are several techniques for image to cartoon conversion such as using photoshop, adobe illustrator,
windows MAC, paint.net and much more. Python is the pool of libraries. Rather than using web applications such as photoshop , use libraries in python which
gives us better output .One such library is OpenCV. OpenCV is a cross-platform library used for Computer Vision. It includes applications like video and image
capturing and processing. It is majorly used in image transformation, object detection, face recognition, and many other stunning applications. We will use some
of the standard libraries such as numpy, matplotlib. You can use a photo of your own in a profile image, create an amusing avatar or turn your photo into a
cartoon.Using Machine learning,we can design such projects. Thus, we will build a python application that will transform an image into its cartoon using
OpenCV.Python programming is very much important in machine learning projects.The outcomes of the study is “how to cartoonify a image using machine
learning”.

Keywords: Cartoonifying , open cv , machine learning

1. INTRODUCTION

Cartoons are commonly used in various kinds of applications. As we know cartoons are made artistically it requires elegant and fine human artistic skills.
While portraying Cartoons in humongous numbers for any animated movies it gets time-consuming for the artist as they need to define the sketch of the
cartoon properly to get a good result.Cartoons are not only done in film industry but also done in different fields like people try to convert their own
image into cartoons to know their appearance in Cartoon effect.But this is all time taking process and requires more resources which might be difficult
to collect all resources and execute accordingly.This technique of cartoonifying image reduces the time consumption for Artists in Films.They can get
the same output as before with this technique.But compared to sketching an image,cartoonifying an image is efficient and quick.Python programming
language is used for writing a code to this technique.And different python libraries are used to get most accurate output.Mainly OpenCV,One of the
library in python,is used in this method of cartoonifying.

1.1 Open CV:

OpenCV is an open-source library in python that is used mainly for computer vision tasks in the areas of machine learning and artificial intelligence.
Nowadays, openCV is playing a major role in the field of technology. Using OpenCV we can process images and videos for some tasks like object
detection, face detection, object tracking, and all.

OpenCV has c, c++, java, and python interfaces and it supports all kinds of systems such as Windows, Linux, Android, Mac OS, IoS, and all…

1.2 Requirements:

Python: We use python as a programming language for building the application.

cv2: We use cv2 for image processing.

Numpy: Mainly NumPy is used for dealing with arrays. Here the images that we use are stored in the form of arrays. So for that, we use NumPy.

Matplotlib: Matplotlib is used for visualization purposes. Here we plot the images using matplotlib.

OS: Here in our application os is used for dealing with paths like reading images from the path and saving the image to the path
 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1082

1.3 Steps To Develop Image To Cartoon

1. Importing the required modules

2. Building a File Box to choose a particular file

3. How is an image stored?

4. Transforming an image to grayscale

5. Smoothening a grayscale image

6. Retrieving the edges of an image

7. Preparing a Mask Image

8. Giving a Cartoon Effect

9. Plotting all the transitions together

10. Functionally of save or download button

2. LITERATURE SURVEY

Following research papers are studied in details to understands the proposed recommendation technique and experimental result for predicting the output
.

2.1 MD.Salar Mohammad, Bollepalli Pranitha,Shivani Goud Pandula, Pulakanti Teja Sree.(2021). “Cartoonizing the Image” an
International Journal of Advanced Trends in Computer Science and Engineering, Volume 10,No.4, July – August 2021.

In this paper , Processing technique is used to cartoonize an image.Image Processing-In the field of the research processing of an image consisting of
identifying an object in an image identify the dimensions , no of objects , changing the images to blur effect and such effects are highly appreciated in
the modern era of media and communication.

There are multiple properties in the Image Processing.Each of the property estimates the image to be produced more with essence and sharper image.Each
image is examined to various grid.

Each picture element together is viewed as a 2-D Matrix.With each of the cell store different pixel values corresponding to the each of the picture element

2.2 S.Rajatha, Anusha Shrikant Makkigadde, Neha L. Kanchan, Sapna, K. Janardhana Bhat,”Cartoonizer: Convert Images and Videos to
Cartoon-Style Images and Videos” an International Journal of Research in Engineering, Science and Management Volume 4, Issue 7, July

In this paper , Cartoon GAN , where GAN stands for Generative Adversarial Network is used to transform images (snapshots) to the finest cartooned
image(animated image).

With the help of the loss function and of two types named as Adversarial loss and content loss , we got a flexible as well as a clear edge defined images.

Also with the help of CV2 which is Computer Vision , we have transformed video into animation(cartoonized video).

2.3 Zengchang Qin, Zhenbo Luo, Hua Wang, " Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative
Adversarial Networks”, International Conference on Image Processing, 2017.

Recently, realistic image generation using deep neural network has became a hot topic in machine learning and computer vision.

In this paper , the sketch-to-image synthesis problem is investigated by using Conditional Generative Adversarial Networks(CGAN).The new model is
not only capable of painting hand-draw sketch with proper colors , but also allowing users to indicate preferred colors .Experimental results on two sketch
datasets show that the auto-painter performs better that existing image-to-image methods.

Auto-painter is a supervised learning model, given a black-and-white sketch , the model can generate a painted colorful image based on given sketch-
image pairs in the training data.

2.4 Harshitha R,Kavya S Muttur,Prof.Jyothi Shetty Dept. Computer Science,RV College of Engineering,Bangalore . “Cartooniation Using
White-box Technique in Machine Learning”(2020).

In this paper , White-box Technique is used to cartoonize the image.By observing the cartoon painiting behaviour and consulting artists , this paper
propose to separately identify three white-box representations from images.
 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1083

The surface representation that contains a smooth surface of cartoon images.The structure representation that refers to the sparse colorblocks and flatten
global content is the celluloid style workflow.

And the texture representation that reflects high frequency texture , contours , and details in cartoon images.

2.5 Akanksha Apte, Ashwathy Unnikrishnan, Navjeevan Bomble, Prof. Sachin Gavhane, “Transformation of Realistic Images and Videos
into Cartoon Images and Video using GAN” an International Journal of Research in Engineering, Science and Management Volume 4,
Issue 7, July2021.

In this paper , a method is proposed that is white-box controllable image cartoonization framework with the help of GAN , which can generate high-
quality cartoonized images and videos from real-world photos and videos.

Images are decomposed under three cartoon representations surface , structure and texture representation.

Image processing models such as OpenCV , PIL , etc are used to extract three representations for network training , and result styles could be controlled
by adjusting the weight of each representation.

Extensive qualitative and quantitative experiments, as well as user studies, have been conducted to validate the performance of the method.

3. METHODOLOGY

3.1 Cartoonizing the Image:

• The process to create a cartoon effect image can be initially branched into 2 divisions –To detect, blur and bold the edges of the actual RGB
color image. To smooth,quantize and the conversion of the RGB image to grayscale.The results involved in combining the image and help
achieve the desired result.

• In order to convert the image given by the user to cartoon ,first it is converted to grayscale. Here, our first step is to convert the image into
grayscale. Thus, we use the BGR2GRAY flag. This returns the image in grayscale. A grayscale image is stored as grayScaleImage.

• To smoothen an image, we simply apply a blur effect. This is done using medianBlur() function. We use bilateral Filter which removes the
noise. It can be taken as smoothening of an image to an extent. Here, we will try to retrieve the edges and highlight them. This is attained by
the adaptive thresholding technique. We perform bitwise and on two images to mask them. This finally CARTOONIFY our image.

3.2 Cartoonizer: Convert Images to Cartoon-Style Image:

 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1084

Fig.3.2.1.Flowchart for Workflow of Webapp

• Above Figure represents Flowchart for cartoonizer webapp. Firstly, the user clicks on the button select the image or video. After the system
opens window for local storage. System checks for image/video formats allowed. Then the selected image or video will be uploaded to convert
into cartoon image/video. After the successful upload of image/video the system processes input and produces the output. After the cartoonized
output gets displayed on screen, the user may download the image/video by clicking on download button. Downloaded result will be stored
into the local storage. Finally, the user can come out of the system.

• After the cartoonized output gets displayed on screen, the user may download the image/video by clicking on download button. Downloaded
result will be stored into the local storage. Finally, the user can come out of the system.

• For videos, the video being uploaded must have size less than or equal to 30mb. If the video is longer than 15sec, then it is trimmed to 15sec
and converted to cartoon-style video. Audio will be added to the cartoonized video. First 15sec of the video will be considered.

3.3 Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks:

Fig:3.3.1 Generative Adversarial Networks

 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1085

Fig.3.3.2 Discriminator

Support As we can see from Figure , in order to decoding out low-level information of sketch, we concatenate encoder layer A to the decoding layer A’
for generating the final colorful cartoon image, where A contains the sketch edge information and A’ mainly contains trained color painting information.

• In first figure, the yellow blocks represent layers in encoder, and blue blocks are decoder layers. In each layer of decoding, the corresponding
layers of encoder are concatenated to current layer to decode the next layer. E.g., B and B’ are concatenated to obtain A’ through deconvolution.
The right-hand side figure is the patchGAN discriminator: input of discriminator is either the pair of sketch (yellow block) and real target
image(red block), or the pair of sketch and generated image (blue block).

• The discriminator only has the encoder units comparing to the generator, it aims to classify whether the input sketch-image pair is `real' or
`fake'. The network is trained to maximize the classification accuracy. As we can see from Figure, the output of the discriminator is a matrix
of probability, in which each element gives the probability of being real for a pair of correspond patches sampled using Markov random field
or PatchGAN. In our research, we take the patch size of 70*70 and output a probability matrix of 30*30.The convolutional layers between the
input and the output extract the high-level features of the input pairs.

3.4 Cartoonization Using White-box Technique:

Fig.3.4.1 White Box Technique

• It is an image synthesizer that helps to generate new data using joint probability. To generate new images it uses Generator and
Discriminator.The generator makes images and Discriminator checks images to be real or fake and then sends feedback to the generator thus
asking him to generate better data. The more both networks are trained, the better images we get.

• The input image is dissolved in three parts wiz Surface Representation, Structural Representation,and TextureRepresentation.A GAN model
with a generator G and two discriminators Ds and Dt are introduced. The goal of Ds is to characterize surface features extracted from model
outputs and cartoons,whereas Dt is responsible for separate textural information from model outputs and cartoons.To pluck high-level features
and to impose a spatial constraint on global content between outputs and provided paired cartoons we use pre-trained VGGNetwork.

• GAN method is implemented with TensorFlow.A discriminator Ds is proposed to verify whether result and paired cartoon images have
similar surfaces, and regulate the generator G to learn the information stored in the extracted surface representation.we implement
Felzenszwalb algorithm to segment the areas.To impose a spatial constraint on global content between outputs and provided paired cartoons
we use pretrained VGGNetwork.
 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1086

3.5 Transformartion of Realistic Images into Cartoon Images using GAN:

Fig.3.5.1 Generator and Discriminator

• GAN i.e. “Generative Adversarial Networks” algorithm is used for the implementation. GAN is a combination of a generative model and a
discriminative model. The generative model creates new instances of data that resemble the training data.

• Conversion of Video is basically similar to getting an animated video out of the normal one. The video is first divided into frames and saved
into an array, then passed through the generator and discriminator with the help of OpenCV to get the animated(cartoonized video).Similarly
an Image can be converted into Cartoon Image in its own method using different techniques.

Fig.3.5.2 Flowchart for GAN

The above activity diagram shows the working of the proposed method. Where it gives an overall brief idea on how the images and video taken from
the camera are converted into cartoon style (Animation).

4. RESULTS & DISCUSSION

4.1 Cartoonizing the Image:

Fig.4.1.1 command for uploading the image

 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1087

Fig.4.1.2 image window with the detected objects

Fig.4.1.3 Cartoonified Image

4.2 Cartoonizer: Convert Images to Cartoon-Style Image:

Fig.4.2.1 Real and Cartoonized image of a cats

4.3 Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks:
 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1088

Fig.4.3.1 Cartoonization using GAN

Given an input image of resolution 512 x 512, the auto-painter can generate a painted colorful output image within 1 second, which enables instant
feedback to design an interactive image editing tools. The auto-painter trained on the Minions provides a tool for users to design virtual images of `minions
style'. As it shown in the left-hand side of Figure 5, based on the given initial sketch, one can modify the original sketch, the glass or the gesture, etc. It
is interesting that even on the simple hand draw sketches, the auto-painter can result in `minions style' images.

4.4 Cartooniation Using White-box Technique:

This model is the fastest among four methods on all devices and all resolutions, and has the smallest model size. Especially, this model can process a
720*1280 image on GPU within only 17.23ms, which enables it for real-time High-Resolution video processing tasks. This model applied on diverse
real-world scenes,including natural landscape, city views, people, animals, and plants.

Fig.4.4.1 Cartoonization using White Box

4.5 Transformartion of Realistic Images into Cartoon Images using GAN:

In the below results, the first image consists of the real world which is taken from a video clip and later after using Cartoon GAN with the help of
OpenCV we get the cartooned image of the video as you can see below:-
 International Journal of Research Publication and Reviews, Vol 3, no 11, pp 1081-1089 November 2022 1089

Fig.4.5.1 Cartoonization using GAN

5.CONCLUSION

• The Main theme of this paper is to get high quality cartoonized image from the normal image. And this can be done using different techniques
and methods as discussed in above papers. Generally to convert any general image into cartoon, it requires more skills,technique and also
consumes much time to get a proper cartoon but now a days with developed technology it became easy.The time required in this method is
comparatively very less.Hence the latest methods became popular for getting cartoon images.

• There are different techniques for converting an image to cartoon.Based on User requirements, suitable technique is used to get an efficient
output. And with developing technology many new methods/techniques are introduced for better implementation and best results.Even Videos
can be animated using these techniques.So with this different algorithms are implemented.

• Finally it can be concluded that an image can be converted into a cartoonized image with suitable techniques which gives efficient output and
less time consumption.Though many applications are developed for Cartoonizing, implementation using Python or any other language gives
better understanding and better outputs.

6. REFERENCES

1. MD. Salar Mohammad, Bollepalli Pranitha,Shivani Goud Pandula, Pulakanti Teja Sree.(2021). “Cartoonizing the Image” an International Journal of
Advanced Trends in Computer Science and Engineering, Volume 10,No.4, July – August 2021.

2.S. Rajatha, Anusha Shrikant Makkigadde, Neha L. Kanchan, Sapna, K. Janardhana Bhat,”Cartoonizer: Convert Images and Videos to Cartoon-Style
Images and Videos” an International Journal of Research in Engineering, Science and Management Volume 4, Issue 7, July 2021.

3. Zengchang Qin, Zhenbo Luo, Hua Wang, " Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial
Networks”, International Journal on Image Processing, 2017.

4. Harshitha R,Kavya S Muttur,Prof.Jyothi Shetty Dept. Computer Science,RV College of Engineering,Bangalore . “Cartooniation Using White-box
Technique in Machine Learning”(2020).

5. Akanksha Apte, Ashwathy Unnikrishnan, Navjeevan Bomble, Prof. Sachin Gavhane, “Transformation of Realistic Images and Videos into Cartoon
Imagesand Video using GAN” an International Journal of Research in Engineering, Science and Management Volume 4, Issue 7, July2021.

6. Silviya D’monte, Aakash Varma, Ricky Mhatre, Rahul Vanmali, Yukta sharma

7. Chinmay Joshi, Devendra Jaiswal, Akshata Patil Department Name of Organization: Information Technology Name of Organization: Kc College of
Engineering Management Studies and Research City: Kopari Thane (East) Country: India