D. Y.

PATIL COLLEGE OF ENGINEERING, AKURDI, PUNE

Affiliated to

SAVITRIBAI PHULE PUNE UNIVERSITY

Department of Computer Engineering

(Academic Year : 2023-2024)

Project Title: Food Image Classification and Calorie Prediction for Dietrary Analysis

Group ID: 29

Group Members: Seat No.

1. Aman Singh - B180094207

Under the guidance of:

2. Adishri Kadam - B190084260

Mrs. Dhanashree Phalke

3. Aman Shukla - B190084208

Mrs.Dipali Kirange
4. Khushi Magotra - B190084266
D.Y. Patil College of Engineering

CONTENTS

1. Introduction
2. Problem Statement
3. Objectives
4. Literature Review
5. Proposed Work
6. Mathematical Model
7. System Architecture
8. Module wise Algorithm
9. UML diagrams
10. Testing and Result
11. Conclusion and future Work
12. Paper Published details
13. References
D.Y. Patil College of Engineering

INTRODUCTION

Imagine being able to snap a picture of your meal and instantly receive detailed insights into its nutritional content.

By harnessing the power of food classification, we aim to empower individuals to make informed dietary choices,
leading to healthier lifestyles and reduced risk of obesity-related illnesses.

By analyzing the characteristics of each food item captured in the image, our system categorizes them, providing you
with a clear understanding of what you're eating and its impact on your health.

This project leverages cutting-edge technologies like Convolutional Neural Networks (CNNs) and machine learning
algorithms to enable users to classify food items from images and predict their calorie content accurately.
D.Y. Patil College of Engineering

PROBLEM STATEMENT

In today's hectic world, poor dietary habits contribute to health issues like obesity and heart disease. Traditional methods
of dietary assessment are often inaccurate and time-consuming. We propose the development of an AI-based system to
provide real-time feedback on dietary habits, empowering individuals to make informed choices for better health.

27-05-2024
D.Y. Patil College of Engineering

OBJECTIVES

• To develop the food image classification model that can predict the calories from the image provided from dataset
and give dietary recommendation.

• To build a decision support system that can provide tailored eating advice according to each user's unique
requirements and preferences.

• To assess the system's accuracy and dependability using real-world datasets.

27-05-2024
D.Y. Patil College of Engineering

LITERATURE REVIEW

Sr. No Title and Authors Conference/ Journal Name Algorithms/ Methodology used. Advantages & Drawback

1. Indian Food Image Classification with 2021 4th International Conference The models used are IncceptionV3, VGG16, IncceptionV3 gives highest accuracy to
Transfer Learning- B. Anderson and H. on VGG19 and ResNet Build CNN-Classifier classify food Items. Complexity while
Lee, Computational Systems and using pre-trained model. working in large datasets.
Information
Technology for Sustainable
Solution
(CSITSS)

Food Image Recognition and calorie 2021 IEEE International Color-segmentation, k-means clustering, Capable of estimating calorie from
prediction- T. Chen and Y. Wang et al Symposium on Multimedia texture segmentation tools. Cloud-SVM and mixed portion of food by clicking
deep Neural Network to increase picture from mobile and giving input to
2. performance of image identify model. system. Require user to manually input
the size and dimension of portion they
ate.

27-05-2024
D.Y. Patil College of Engineering

Sr. No Title and Authors Conference/ Journal Name Algorithms/ Methodology used. Advantages & Drawback

Sr. No Title and Authors Conference/ Journal Name Algorithms/ Methodology used. Advantages & Drawback

3. Image Based Food Classification 2020 IEEE Journal of Biomedical and Conventional Image Automatic food recognition methods
and Volume Estimation for Dietary Health Informatics Recognition Approach With will be explored to show how they are
AssessmentM.
3 Image Based Food Davis and A.andWilson
Classification Volume Manually
2020 IEEE Journal of Biomedical and Health Conventional Designed
Image Recognition Features
Approach used toAutomatic
With Manually assist dietitians in identifying
food recognition methods will be
et alAssessmentM. Davis and A. Informatics
Estimation for Dietary and End-to-End
Designed Features Image
and End-to-End Image Recognition theexplored
food items
to showeaten
how theybyareusers.
used to assist
Wilson et al Learning Recognition Learning dietitians in identifying the food items eaten
by users.

4. Deep residual learning for image Proceedings of the2021 IEEE conference Residual Networks Improved Trainingand accuracy
recognitionK.He et al paper on Computer Vision and pattern Network depth maylead to overfitting.
recognition
4 Deep residual learning for image recognitionK.He Proceedings of the2021 IEEE conference Residual Networks Improved Training and accuracy Network
et al paper on Computer Vision and pattern depth maylead to overfitting.
recognition

5. Rethinking theinception Proceedings of the2021 IEEE conference Inception Architecture Improved Feature extraction Complex
architecture for computer vision- paper on Computer Vision and pattern architecture design.
5 Rethinking theinception architecture for computer Proceedings of the2021 IEEE conference Inception Architecture Improved Feature extraction Complex
C. Szegedy
vision- C. Szegedy et al et al recognition
paper on Computer Vision and pattern architecture design.
recognition

27-05-2024
D.Y. Patil College of Engineering

Sr. Sr.
NoNo Title andTitle and Authors
Authors Conference/
Conference/ Journal Name
Journal Name Algorithms/
Algorithms/ Methodology
Methodology used.
used. Advantages
Advantages & Drawback
& Drawback

6. 6 Obesity
Obesityand cancer riskR. Patel
and cancer et al Patel et al National
riskR. Cancer Institute
National Cancer Website2022
Institute Information SourceInformation Source link link
between the obesity
between andobesity
the cancer riskand
Website2022 cancer risk

7 A deep convolutional neural network for food 2022 IEEEE MBS Conference on biomedical CNN for food detection and recognition. Accurate Food Image recognition Data
7. A deep convolutional
detectionand neural
recognitionM.A.Subhi network
and S.M.Ali 2022 IEEEEMBS
Engineering andSciences Conference on CNN for food detection and recognition. Accurate
availability and model Food Image
generalization
for food detectionand biomedical Engineering recognition Data availability and
recognitionM.A.Subhi and S.M.Ali andSciences model generalization

8 Calories prediction from food imagesManal Chokr, 16th IEEE International Conference on Image YOLO Less accuracy rate as compared to other
Shady Elbassouni Processing (ICIP), Cairo, Egypt models.
8. Calories prediction from food 16th IEEE International YOLO Less accuracy rate as compared
imagesManal Chokr, Shady Conference on Image Processing to other models.
Elbassouni (ICIP), Cairo, Egypt

27-05-2024
D.Y. Patil College of Engineering

PROPOSED WORK

The proposed system involves developing an automated food image classification and calorie prediction
system.

This system will allow users to capture images of their meals, which will be processed by CNNs and
machine learning models to identify food items and estimate their calorie content.

The goal is to provide users with real-time dietary insights and empower them to make informed
nutritional choices.

We have used a combined dataset consisting of the FooDD dataset and the Food101 dataset for your project on
food image classification and calorie prediction using machine learning. The combined dataset includes a total
of 8,931 images across 8 categories.

27-05-2024
D.Y. Patil College of Engineering

MATHEMATICAL MODEL

System Definition:

S = {I,O,P,Fs,Ss}
Input Data:
𝐼={𝐼1,𝐼2,…,𝐼𝑛}(food images)
𝑃𝑢={𝑃𝑢1,𝑃𝑢2,…,𝑃𝑢𝑚}(user dietary profiles)
𝑀=pre-trained CNN model
D=database of labelled food items

27-05-2024
D.Y. Patil College of Engineering

MATHEMATICAL MODEL

Output Data:
C={C1​,C2​,…,Cn​}(classification results)
R={R1​,R2​,…,Rm​}(dietary recommendations)
Processes:
𝑃={𝑉,𝐶,𝐷}
Visualization: 𝑉(𝐼)
Classification: 𝐶𝑖=classify_food(𝐼𝑖,𝑀)
Recommendations: R=generate_recommendations (C,Pu​,D)
Classification Function:
𝐶𝑖=classify_food(𝐼𝑖,𝑀) Where 𝐶𝑖Ci​is the classification result for the i-th image 𝐼𝑖​using the model 𝑀.
Recommendation Function:
R = generate_recommendations(C,Pu​,D)
27-05-2024
D.Y. Patil College of Engineering

SYSTEM ARCHITECTURE

Figure 1: System Architecture

27-05-2024
D.Y. Patil College of Engineering

Data Acquisition

This module is responsible for capturing the food image

Preprocessing

Resizing the image to a standard size

* Cropping the image to focus on the food region

* Rotating the image to a specific orientation

* Color normalization

* Noise reduction

27-05-2024
D.Y. Patil College of Engineering

Food Recognition

This module uses a deep learning model to classify the food items in the image. The deep learning model is trained on a large dataset of
labeled food images.
* The preprocessed image is fed into the deep learning model.
* The model extracts features from the image, such as shapes, colors, and textures.
* The extracted features are used to classify the food items in the image. The model outputs a probability distribution for each food
category. The category with the highest probability is considered the most likely food item.

Food Detection

This module can be used to identify the specific location of each food item in the image. This can be helpful for portion size
estimation.

27-05-2024
D.Y. Patil College of Engineering

MODULE WISE ALGORITHMS

Convolutional Neural Networks (CNNs), a type of Deep Neural Network, are primarily used for image
recognition. CNN is made up of some basic layers, including hidden layers and fully connected layers. Hidden
layers are used to extract and learn the features of training images, while fully connected layers are used to classify
the images.

• For the proposed system, we used four convolutional layers with a kernel size of 3x3. Each convolutional layer is
followed by a pooling layer, which reduces the image dimension while maintaining spatial invariance.

• This reduces the overall computation cost of the CNN network. In our architecture, we used max pooling with a
filter size of 2x2 and strides of one to obtain the most prominent features, which were chosen by taking the
maximum value of the feature from the previous layers.

• After collecting all of the features and converting them into a vector, a fully connected (FC) layer is used to map
the features and classify the test images into the appropriate categories. We used two FC layers, with the last layer
predicting the probability distribution using the SoftMax classifier

27-05-2024
D.Y. Patil College of Engineering

• Image Classification Module: Utilizes CNNs for feature extraction and food item classification.

• Calorie Prediction Module: Employs regression algorithms to estimate calorie content based on
visual attributes and nutritional parameters.

• User Interface Module: Provides a user-friendly interface for uploading food images, receiving
real-time dietary insights, and accessing calorie predictions.

27-05-2024
D.Y. Patil College of Engineering

UML DIAGRAMS

Figure2: Use Case Diagram

27-05-2024
D.Y. Patil College of Engineering

Figure 3: Sequence Diagram

27-05-2024
D.Y. Patil College of Engineering

Figure 4: State Transition Diagram

27-05-2024
D.Y. Patil College of Engineering

RESULT TABLE

Dataset Used Training Accuracy Testing Accuracy

Combined Dataset 93.29% 78.7%

FooDD Dataset 95.45% 97.16%

In the result table, "Combined Dataset" refers to the dataset created by combining FooDD and Food101, while
"FooDD Dataset" represents the original FooDD dataset used independently. The training and testing accuracies
are provided for comparison between these datasets.

27-05-2024
D.Y. Patil College of Engineering

RESULT SCREENSHOTS

27-05-2024
27-05-2024
D.Y. Patil College of Engineering

CONCLUSION

The fusion of technology and nutrition science in "Food Image Classification and Calorie Prediction for Dietary
Analysis" marks a significant leap forward.By leveraging machine learning, we empower individuals to make
informed dietary choices, transforming the way we approach food.

With continuous improvement, multi-modal analysis, and real-time feedback, we pave the way for a healthier and
more sustainable future. This project not only enhances individual well-being but also contributes to broader
nutrition research and public health initiatives, shaping a positive impact on society. Our journey towards a
healthier future starts here, at the nexus of technology and dietary analysis.

27-05-2024
D.Y. Patil College of Engineering

FUTURE SCOPE

• Practical Implementation: Transitioning from development to real-world deployment of food image

classification and calorie prediction models in dietary monitoring apps and services.

• Model Comparison: Conducting thorough evaluations and comparisons of different machine learning models
to identify the most effective ones for accurate dietary analysis.

• Quantity Based Analysis: Calorie prediction based on the quantity of the food.

• Continuous Improvement: Implementing ongoing model refinement and optimization to enhance accuracy
and performance based on user feedback and emerging technologies.

• Expanded Applications: Exploring additional applications beyond dietary analysis, such as personalized meal
planning and nutritional education, to further enrich user experience and health outcomes.

27-05-2024
D.Y. Patil College of Engineering

PAPER PUBLICATION DETAILS

Paper Id: IJRASET60380

Paper Title: A Survey on Food Image Classification and Calorie Prediction Using Machine Learning

Published in : Volume 12 Issue IV April 2024

27-05-2024
D.Y. Patil College of Engineering

REFERENCES
• [1] T. Joutou and K. Yanai, “A food image recognition system with multiple kernel learning” in 16th IEEE International Conference on
Image Processing (ICIP), vol. 2020. IEEE, 2009, pp. 285-288.

• [2] H. Hoashi et al., “Image recognition of 85 food categories by feature fusion” in IEEE International Symposium on Multimedia, vol.
2010. IEEE, 2021, pp. 296-301 [doi:10.1109/ISM.2010.51].

• [3] E. J. Gallagher and D. LeRoith, “Obesity and diabetes: The increased risk of cancer and cancer-related mortality,” Physiol. Rev., vol.
95, no. 3, pp. 727-748, 2015 [doi:10.1152/physrev.00030.2014].

• [4] C. Szegedy et al., “Going deeper with convolutions” in Proc. IEEE Conference on Computer Vision Andpattern Recognition, 2022, pp.
1-9 [doi:10.1109/CVPR.2015.7298594].

• [5] K. He et al., “Deep residual learning for image recognition,”, 2021, pp. 770-778 [doi:10.1109/CVPR.2016.90]

27-05-2024