Orientation

TO
Computing II
Big Data and Data Science
Data Science & Big Data
• Data Science and its need
• Applications of data Science /Big Data
• Data Science life-cycle with use case,
• Big data and its 3V’s
• Challenges of Big data
• Skills needed for Big data
• Tools usage like Apache Hadoop, Tableau, R-language Excel , Big data
on cloud
• Use of Big data in different areas
• Job roles and Skillset for Data Science and Big data
WHAT IS DATA SCIENCE?
• Data science is a multidisciplinary field that involves the use of
scientific methods, processes, algorithms, and systems to extract
insights and knowledge from structured and unstructured data.

• It combines elements from statistics, mathematics, computer science,

information theory, and domain-specific expertise to analyze and
interpret complex data sets.
Key components of Data Science
1.Data Collection: Gathering relevant data from various sources, which can
include databases, sensors, social media, and more.
2.Data Cleaning and Preprocessing: Ensuring data quality by cleaning and
preprocessing it, dealing with missing values, and converting data into a suitable
format for analysis.
3.Exploratory Data Analysis (EDA): Investigating and visualizing data to
discover patterns, trends, and relationships that can inform further analysis.
4.Modeling: Developing statistical or machine learning models to make
predictions or uncover patterns within the data.
5.Evaluation: Assessing the performance of models and refining them as needed.
6.Interpretation and Communication: Interpreting the results of analyses and
communicating findings to non-technical stakeholders.
Need for Data Science Depends on these Factors
1.Data Explosion: The digital age has led to an explosion of data generated from various
sources, such as social media, sensors, and transactions. Organizations have access to
large volumes of data that can be leveraged for insights.
2.Decision-Making: Data science helps organizations make informed decisions by
providing insights and predictions based on data analysis. This is crucial for strategic
planning and business operations.
3.Competitive Advantage: Businesses can gain a competitive edge by using data science to
identify market trends, customer preferences, and areas for improvement.
4.Innovation: Data science plays a crucial role in innovation, enabling the development of
new products, services, and business models.
5.Personalization: Data science allows for personalized recommendations and experiences,
such as personalized marketing, content recommendations, and user experiences.
6.Risk Management: Data science is used for risk assessment and management in various
industries, including finance, insurance, and healthcare.
7.Scientific Research: In scientific research, data science techniques are employed to
analyze experimental data, simulate scenarios, and make predictions.
Applications of Data Science/ Big data
1.Healthcare:
1. Predictive Analytics: Forecasting disease outbreaks and patient admission
rates.
2. Personalized Medicine: Analyzing genetic data for customized treatment
plans.
3. Fraud Detection: Identifying healthcare fraud through anomaly detection.
2.Retail:
1. Recommender Systems: Offering personalized product recommendations to
customers.
2. Demand Forecasting: Predicting product demand to optimize inventory
management.
4. Telecommunications:
• Network Optimization: Improving network performance through data analysis.
• Fraud Detection: Detecting fraudulent activities, such as SIM card cloning.
5.Marketing:
• Customer Analytics: Understanding customer behavior and preferences.
• Social Media Analytics: Analyzing social media data for sentiment analysis and
campaign evaluation.
6. Education:
• Student Performance Analytics: Analyzing student data for personalized learning.
• Predictive Modeling: Identifying students at risk of dropping out.
• Learning Analytics: Analyzing data to improve the effectiveness of educational
programs.
7. Government:
• Predictive Policing: Forecasting areas with a high likelihood of criminal activity.
• Fraud Detection: Identifying fraudulent activities in government programs.
• Traffic Management: Optimizing traffic flow through data analysis.
Data science Lifecycle with use case
• The data science lifecycle consists of several stages, each with its own set of tasks
and goals. The typical data science lifecycle along with a use case for each stage:
1.Problem Definition:
• Use Case: Predictive Maintenance in Manufacturing
• Description: Identify the problem or business challenge. In this case, the goal is
to predict equipment failures in a manufacturing plant to schedule maintenance
proactively and minimize downtime.
2.Data Collection:
• Use Case: E-commerce Customer Segmentation
• Description: Gather relevant data from various sources, such as customer
transactions, website interactions, and demographic information, to understand
customer behavior.
3.Data Cleaning and Exploration:
• Use Case: Housing Price Prediction
• Description: Clean and preprocess the housing dataset, handle missing values,
and explore the data to understand the distribution of features, identify outliers,
and check for correlations.
4. Feature Engineering:
• Use Case: Credit Scoring
• Description: Create new features or transform existing ones to improve the
predictive power of the model. For credit scoring, features like credit
utilization ratios and payment history might be engineered.
5. Model Development:
• Use Case: Spam Email Classification
• Description: Build and train a machine learning model to classify emails as
spam or not spam based on features such as email content, sender, and other
relevant attributes.
6. Model Evaluation:
• Use Case: Disease Prediction
• Description: Evaluate the performance of the disease prediction model using
metrics such as accuracy, precision, recall, and F1 score to ensure it meets the
desired level of accuracy and reliability.
7. Model Deployment:
• Use Case: Stock Price Prediction
• Description: Deploy the stock price prediction model to a production environment,
allowing it to make real-time predictions based on the latest market data.
8. Monitoring and Maintenance:
• Use Case: Network Anomaly Detection
• Description: Implement monitoring mechanisms to track the model's performance over
time. In the case of network anomaly detection, the system continuously monitors network
traffic for unusual patterns.
9. Feedback and Iteration:
• Use Case: Customer Churn Prevention
• Description: Gather feedback from the deployed model's predictions and use it to
iteratively improve the model. For example, if the customer churn model misclassifies
certain cases, retrain the model with additional data to enhance its accuracy.
10. Communication of Results:
• Use Case: Social Media Sentiment Analysis
• Description: Communicate the results of sentiment analysis on social media data to
stakeholders. Share insights about public sentiment toward a product or brand based on the
analysis of social media posts and comments.
Big data and its 3Vs
Big data is a term used to describe large and complex datasets that cannot be effectively
managed, processed, and analyzed using traditional data processing tools and methods. The
concept of big data is often characterized by the three Vs: Volume, Velocity, and Variety.
• Volume:
• Definition: Refers to the sheer size of the data generated or collected.
• Example: Social media posts, sensor data, and transaction records can produce massive volumes
of data. For instance, the increasing use of IoT devices and sensors generates large volumes of data
in real-time.
• Velocity:
• Definition: Relates to the speed at which data is generated, collected, and processed.
• Example: Social media updates, financial transactions, and sensor data from devices stream in
real-time. High-velocity data requires rapid processing and analysis to derive meaningful insights
or make timely decisions.
• Variety:
• Definition: Involves the diverse types of data that can be structured, semi-structured, or
unstructured.
• Example: Data comes in various formats, such as text, images, videos, and sensor data. For
instance, a single dataset may include structured data from a database, unstructured text from social
media, and semi-structured data from XML or JSON files.
Challenges of Big data
Volume Management:
• Challenge: The sheer volume of data generated can be overwhelming. Storing, managing,
and processing large datasets require robust infrastructure and storage solutions.
Velocity Handling:
• Challenge: The high speed at which data is generated and needs to be processed in real-
time can be challenging. Traditional data processing systems may struggle to keep up with
the pace of data streaming in.
Variety of Data:
• Challenge: Big data comes in various formats, including structured, semi-structured, and
unstructured data. Integrating and analyzing data from diverse sources can be complex.
Data Quality:
• Challenge: Ensuring the accuracy, consistency, and reliability of big data can be difficult.
Poor data quality can lead to inaccurate analyses and unreliable insights.
Data Security and Privacy:
• Challenge: Managing the security and privacy of sensitive data is a major concern. With
large volumes of data, the risk of data breaches and unauthorized access increases.
Scalability Issues:
• Challenge: As data volumes grow, systems and infrastructure need to scale accordingly.
Ensuring scalability without sacrificing performance can be a complex task.
Complexity in Integration:
• Challenge: Integrating big data technologies with existing IT infrastructure can be
challenging. Compatibility issues and the need for seamless integration pose hurdles for
organizations.
Lack of Skilled Professionals:
• Challenge: There is a shortage of skilled professionals who possess the expertise to work
with big data technologies and tools. This scarcity can hinder the effective implementation
of big data projects.
Cost Management:
• Challenge: The cost associated with storing, processing, and analyzing large volumes of
data can be significant. Organizations need to carefully manage costs to ensure a positive
return on investment.
Limited Awareness and Understanding:
• Challenge: Some organizations may lack a clear understanding of the potential benefits
and use cases of big data. Limited awareness can hinder the adoption of big data
technologies.
Skill needed for Big data
Programming Skills:
• Languages: Proficiency in languages such as Python, Java, or Scala is essential for working with
big data frameworks and libraries.
Data Management and Storage:
• Hadoop Ecosystem: Understanding Hadoop components like HDFS, MapReduce, and Hive is
crucial.
• NoSQL Databases: Familiarity with NoSQL databases like MongoDB, Cassandra, or HBase for
handling unstructured and semi-structured data.
Data Processing and Analysis:
• Apache Spark: Knowledge of Spark for distributed data processing and analytics.
• DataFrames and SQL: Competence in working with data using SQL and DataFrame APIs.
Data Warehousing:
• SQL: Proficiency in SQL for querying and managing large datasets in traditional relational
databases.
Machine Learning:
• Algorithms and Models: Understanding machine learning algorithms and models for data analysis
and prediction.
• Frameworks: Familiarity with machine learning frameworks like TensorFlow or scikit-learn.
Data Visualization:
• Visualization Tools: Proficiency in tools like Tableau, Excel, R- programming, Power
BI(business Intelligent), or Matplotlib for creating meaningful visualizations from data.
Statistical Analysis:
• Statistical Knowledge: Understanding statistical concepts for data analysis and
interpretation.
Distributed Computing:
• Parallel Computing: Understanding principles of parallel and distributed computing
for scalable data processing.
Cloud Computing:
• Cloud Platforms: Familiarity with cloud platforms like AWS, Azure, or Google Cloud
for scalable and cost-effective storage and processing.
Data Cleaning and Preprocessing:
• Data Wrangling: Skills in cleaning and preprocessing data for analysis.
Problem-Solving and Critical Thinking:
• Analytical Skills: The ability to analyze complex problems and derive meaningful
insights from data.
Big Data on the Cloud
• some key aspects of implementing big data on the cloud:
1.Scalability:
• Cloud platforms provide on-demand scalability, allowing organizations to easily scale
up or down based on the volume of data and processing requirements.
2.Storage Services:
• Cloud providers offer scalable and cost-effective storage services, such as Amazon S3,
Google Cloud Storage, and Azure Blob Storage, for storing large volumes of data.
3.Compute Resources:
• Cloud platforms provide powerful and scalable compute resources, such as virtual
machines (VMs) and containerized services, to process and analyze big data.
4.Serverless Computing:
• Serverless computing options, like AWS Lambda or Azure Functions, allow for event-
driven, cost-effective processing without the need to provision or manage servers.
5. Managed Big Data Services:
• Cloud providers offer managed big data services like Amazon EMR, Google Dataproc, and
Azure HDInsight, which simplify the deployment and management of big data frameworks
like Apache Hadoop and Apache Spark.
6. Data Warehousing:
• Cloud data warehouses, such as Amazon Redshift, Google BigQuery, and Azure Synapse
Analytics, offer high-performance analytics for large datasets, enabling organizations to run
complex queries efficiently.
7. Streaming Data Processing:
• Cloud platforms support real-time data processing and analytics with services like Amazon
Kinesis, Google Cloud Dataflow, and Azure Stream Analytics.
8. Integration with Big Data Tools:
• Cloud providers integrate with popular big data tools and frameworks, making it easier for
organizations to use tools like Apache Flink, Apache Kafka, and Apache NiFi in a cloud
environment.
9. Data Security and Compliance:
• Cloud providers implement robust security measures and compliance certifications, helping
organizations meet data security and regulatory requirements.
10. Cost Management:
• Cloud platforms offer cost-effective pricing models, allowing organizations to
pay for the resources they consume. This is particularly beneficial for
managing costs associated with variable workloads.
11. Global Reach:
• Cloud providers have data centers distributed globally, enabling organizations
to deploy big data solutions close to their target audience, reducing latency
and improving performance.
12. Collaboration and Integration:
• Cloud platforms facilitate collaboration among teams and departments by
providing tools and services that integrate with various data science and
analytics tools.
Use of Big Data in different areas
• Healthcare
• Finance
• Retail
• Telecommunications
• Manufacturing
• Marketing
• Transportation and Logistics
• Energy
• Education
• Government
• Entertainment and Media
• Sports
• Agriculture
Job roles and skillset for Data science
Job Role Skillset
Statistical analysis, machine learning, programming (Python, R), data visualization,
Data Scientist
domain expertise

Data querying and analysis, statistical analysis, data visualization, database

Data Analyst
management, business intelligence tools

Machine learning algorithms, model development and deployment, programming

Machine Learning Engineer
(Python, Java), deep learning, natural language processing

Database design, ETL (Extract, Transform, Load) processes, data warehousing,

Data Engineer
programming (SQL, Python), cloud platforms

Business Intelligence (BI) Analyst BI tools (Tableau, Power BI), data visualization, data querying, business analysis

Statistician Statistical modeling, hypothesis testing, experimental design, data analysis

Quantitative Analyst Financial modeling, risk analysis, statistical analysis, programming (e.g., Python, R)

Operations Analyst Process optimization, data analysis, business process modeling, programming
 Job roles and skillset for Big data
Job Role Skillset
Hadoop ecosystem (Hive, Pig, HBase), Apache Spark, NoSQL databases,
Big Data Engineer
ETL processes, programming (Java, Scala)
System architecture design, data modeling, distributed computing, cloud
Big Data Architect
platforms, Hadoop ecosystem
Data Warehouse Architect Data warehousing, database design, ETL processes, SQL, cloud platforms
Cloud platforms (AWS, Azure, Google Cloud), data migration, data
Cloud Data Engineer
integration, programming
Real-time data processing, stream processing frameworks (e.g., Apache
Streaming Data Engineer
Flink), messaging systems (e.g., Apache Kafka)
DevOps practices, version control, automated testing, continuous
DataOps Engineer
integration/continuous deployment (CI/CD), data pipeline management
Analytical and problem-solving skills, domain expertise, communication
Big Data Consultant
skills, proficiency in big data technologies
Data governance frameworks, compliance knowledge, data quality
Data Governance Analyst
management, metadata management
Data security measures, encryption techniques, access controls,
Data Security Analyst
compliance knowledge
Statistical analysis, machine learning, big data technologies (Hadoop,
Data Scientist (with Big Data Focus)
Spark), programming (e.g., Python, R)
 UNIT II
Artificial Intelligence
&
Machine Learning
Topic to be Cover
• Introduction to AI, ML and Deep Learning
• Expert systems
• Fuzzy systems
• Augmented Reality
• Use of AI in different fields - NLP Healthcare, Agriculture, Social media
monitoring
• Tools and techniques for implementing AI
• Application of AI and ML
• Job roles and skillset for AI and ML
Artificial Intelligence (AI)
• Artificial Intelligence (AI) refers to the simulation of human
intelligence in machines that are programmed to think, learn, and
perform tasks that traditionally required human intelligence.

• AI includes a broad range of techniques and approaches aimed at

creating intelligent agents capable of reasoning, problem-solving,
understanding natural language, and adapting to new situations.

• It includes both narrow AI, which is designed for specific tasks, and
general AI, which would possess human-like intelligence across a
wide range of domains.
Machine Learning (ML)
• Machine Learning is a subset of AI that focuses on the development of
algorithms and statistical models that enable computers to learn from data
and improve their performance on a specific task without explicit
programming. ML algorithms use patterns and insights from data to make
predictions or decisions. There are three main types of machine learning:
1.Supervised Learning: The algorithm is trained on a labeled dataset,
where the input data is paired with corresponding output labels. It learns
to map inputs to outputs.
2.Unsupervised Learning: The algorithm is given unlabeled data and must
find patterns or relationships within the data without explicit guidance.
3.Reinforcement Learning: The algorithm learns by interacting with an
environment and receiving feedback in the form of rewards or penalties.
It aims to learn the optimal actions to take in different situations
 Deep Learning
• Deep Learning is a specialized subfield of machine learning that involves
neural networks with multiple layers (deep neural networks).
• These networks, often referred to as artificial neural networks, are inspired
by the structure and function of the human brain.
• Deep Learning excels in automatically learning hierarchical representations
from data, allowing it to capture complex patterns and features.
• Convolutional Neural Networks (CNNs) are commonly used for image
recognition, while Recurrent Neural Networks (RNNs) are effective for
sequence data like language.
Expert systems
• An Expert System (ES) is a computer program or software designed to
emulate the decision-making ability and problem-solving skills of a
human expert in a specific domain.
• Expert systems leverage knowledge, reasoning processes, and
decision-making rules to provide expert-level advice or solutions in a
particular field.
• These systems are part of the broader field of artificial intelligence
(AI) and have been used in various applications to address complex
problems.
Components of Expert Systems
1. Knowledge Base (KB):
The knowledge base is a repository that stores information, facts, and rules relevant to a specific domain. It
represents the expertise of human specialists and is a critical component of an expert system.

2. Inference Engine:
The inference engine is the reasoning component of the expert system. It processes the information stored in
the knowledge base, applies rules, and makes logical inferences to arrive at conclusions or recommendations.

3. User Interface:
The user interface allows users to interact with the expert system. It may include natural language interfaces,
graphical interfaces, or other input/output mechanisms to facilitate communication between the system and
users.

4. Explanation Facility:
Expert systems often include an explanation facility to provide users with a clear understanding of the
system's reasoning and the basis for its recommendations. This enhances transparency and user trust.

5. Knowledge Acquisition System:

Knowledge acquisition is the process of capturing and entering human expertise into the system's knowledge
base. Knowledge acquisition tools assist experts in transferring their knowledge to the expert system.
Key Characteristics of Expert Systems
• Domain Specificity:
Expert systems are designed for specific domains or narrow areas of expertise. They
excel in well-defined and limited problem-solving tasks.
• Knowledge-Intensive:
These systems rely on a substantial amount of explicit knowledge, often captured from
human experts, to make decisions and solve problems.
• Rule-Based Reasoning:
Expert systems use a set of rules and logical reasoning mechanisms to process
information and draw conclusions. These rules represent the expertise encoded in the
knowledge base.
• Decision Support:
The primary purpose of expert systems is to provide decision support by assisting users
in making informed decisions or solving complex problems within a specific domain.
• Learning and Adaptation (in some cases):
While traditional expert systems are rule-based and static, some advanced systems
incorporate learning mechanisms to adapt and improve over time based on feedback and
new data.
Applications of Expert Systems
• Medicine: Diagnosing medical conditions and recommending
treatment plans.
• Finance: Assessing investment strategies and financial planning.
• Engineering: Troubleshooting and decision support in engineering
design.
• Customer Support: Providing expert assistance in troubleshooting
and problem resolution.
• Education: Tutoring systems to support learning in specific subjects.
Fuzzy systems
• Fuzzy systems, or fuzzy logic systems, are computational models
based on fuzzy logic
• Fuzzy logic allows for the representation of uncertainty and
inaccuracy in reasoning, making it suitable for applications where
traditional binary logic may not capture the complexity of real-world
decision-making.
• Developed by Lotfi Zadeh in the 1960s, fuzzy logic has found
applications in various fields, ranging from control systems to
artificial intelligence.
Applications of Fuzzy Systems
• Control Systems:
Fuzzy logic is widely used in control systems, especially when dealing with systems that
are difficult to model precisely, such as in automotive applications (e.g., anti-lock braking
systems).
• Consumer Electronics:
Fuzzy logic is used in appliances like washing machines for efficient control of washing
cycles based on imprecise input information.
• Pattern Recognition:
Fuzzy systems can be applied in pattern recognition tasks where the boundaries between
different classes are not well-defined.
• Decision Support Systems:
In decision support systems, fuzzy logic can handle imprecise information and assist in
decision-making.
• Artificial Intelligence:
Fuzzy logic is used in AI applications for reasoning under uncertainty and capturing
human-like decision-making processes.
Augmented Reality
• Augmented Reality (AR) is a technology that overlays computer-
generated content, such as images, videos, or 3D models, onto the
real-world environment.
• Unlike virtual reality, which creates a completely immersive digital
experience, AR enhances the real world by adding digital elements to
it.
• AR systems use devices like smartphones, tablets, smart glasses, or
AR headsets to blend virtual and physical elements seamlessly.
Key Components of Augmented Reality
Sensors:
AR devices are equipped with various sensors such as cameras, accelerometers,
gyroscopes, and GPS to understand the user's surroundings and movements.
Display:
The display component presents the augmented content to the user. This can be
achieved through the device's screen (as in smartphones) or through AR glasses
and headsets that provide a more immersive experience.
Processing Unit:
AR devices have a powerful processing unit responsible for analyzing sensor
data, tracking the user's position and orientation, and rendering digital content in
real-time.
Applications of Augmented Reality
• Gaming:
Popularized by games like Pokémon GO, AR gaming involves integrating
virtual elements into the real world for an interactive gaming experience.
• Healthcare:
AR is employed in medical training, surgery planning, and patient education by
overlaying 3D models on anatomical structures.
• Navigation:
AR is used in navigation apps to overlay directions and points of interest onto
the real-world view seen through a smartphone camera.
Use of AI in different fields - NLP
• Chatbots and Virtual Assistants:
• Description: AI-driven chatbots and virtual assistants use NLP to understand and respond to
user queries in natural language. They are employed in customer support, providing
information, and automating tasks.
• Sentiment Analysis:
• Description: NLP is used to analyze text data, such as social media posts, reviews, and
comments, to determine sentiment. This helps businesses understand public opinion,
customer feedback, and brand perception.
• Language Translation:
• Description: AI-based language translation systems use NLP to translate text from one
language to another. They analyze the structure and context of sentences to produce
accurate translations.
• Text Summarization:
• Description: NLP is applied to automatically summarize long texts or articles. Summarization
algorithms analyze the content and extract key information, providing concise summaries.
• Speech Recognition:
• Description: NLP is used in speech recognition systems to convert spoken language into
written text. This technology is applied in voice assistants, transcription services, and
accessibility tools.
• Named Entity Recognition (NER):
• Description: NER in NLP involves identifying and classifying entities such as names of people,
organizations, locations, and other specific terms in text data. It is used in information
extraction and data categorization.
• Question-Answering Systems:
• Description: AI-powered question-answering systems use NLP to understand and respond to
user questions. These systems can provide information from structured or unstructured data
sources.
• Text Classification:
• Description: NLP is employed in text classification tasks, such as spam detection, sentiment
analysis, and topic categorization. Machine learning models analyze textual content and assign
predefined labels or categories.
• Content Generation:
• Description: NLP models are used to generate human-like text content. This is applied in
content creation, creative writing, and even in the generation of news articles.
• Language Understanding in Virtual Reality (VR):
• Description: NLP enhances user interactions in virtual reality environments by enabling natural
language communication with virtual characters or systems within the VR space.
• Medical Text Mining:
Description: In healthcare, NLP is used for mining information from medical records, research
articles, and clinical notes. It aids in information retrieval, disease mapping, and clinical decision
Use of AI in different fields - healthcare
Medical Imaging:
Application: Radiology and Pathology
Description: AI is used in the interpretation of medical images, such as X-rays, CT scans, and pathology slides.
Machine learning algorithms assist in early detection of diseases, including cancer, by analyzing complex image
data.
Disease Diagnosis:
Application: Clinical Decision Support Systems
Description: AI systems analyze patient data, including medical history, symptoms, and test results, to assist
healthcare professionals in diagnosing diseases. These systems provide recommendations and insights for more
accurate diagnoses.
Drug Discovery:
Application: Drug Design and Development
Description: AI accelerates the drug discovery process by analyzing biological data, identifying potential drug
candidates, and predicting their efficacy. This helps in reducing the time and cost associated with drug
development.
Personalized Medicine:
Application: Genomic Data Analysis
Description: AI analyzes genomic data to identify individual variations and genetic factors that influence disease
susceptibility and treatment response. This enables the development of personalized treatment plans.
Remote Patient Monitoring:
Application: Wearable Devices and IoT
Healthcare Chatbots:
Application: Virtual Health Assistants
Description: Chatbots powered by AI and natural language processing (NLP) provide instant responses
to patient queries, offer health-related information, and assist in appointment scheduling.
Predictive Analytics:
Application: Patient Outcome Prediction
Description: AI analyzes patient data to predict disease progression, potential complications, and patient
outcomes. This information aids healthcare providers in making informed decisions about treatment
plans.
Robotic Surgery:
Application: Robot-Assisted Surgery
Description: AI-powered robotic systems assist surgeons in performing minimally invasive surgeries
with precision. These systems can enhance surgical capabilities and improve patient outcomes.
Epidemiological Research:
Application: Disease Surveillance
Description: AI analyzes vast amounts of data to track and predict the spread of diseases, contributing to
epidemiological research and public health planning.
Mental Health Support:
Application: Mental Health Chatbots and Monitoring
Description: AI-based applications offer mental health support by providing resources, monitoring mood
Use of AI in different fields - agriculture
Precision Farming:
Application: Crop Monitoring and Management
Description: AI-powered sensors, drones, and satellite imagery analyze data related to soil health, crop conditions,
and weather patterns. Farmers use this information to optimize irrigation, fertilization, and pest control, leading to
more efficient resource use.
Crop Prediction and Yield Optimization:
Application: Predictive Analytics
Description: AI models analyze historical and real-time data to predict crop yields. This helps farmers plan for
harvests, optimize planting schedules, and make informed decisions regarding crop rotation.
Weed and Pest Control:
Application: Automated Monitoring and Decision Support
Description: AI-driven systems use computer vision to identify and classify weeds and pests in the field. This
information helps in targeted application of herbicides and pesticides, reducing the need for widespread chemical use.
Autonomous Farming Equipment:
Application: Autonomous Tractors and Harvesters
Description: AI-powered autonomous vehicles equipped with sensors and GPS technology perform various tasks,
such as plowing, seeding, and harvesting, without human intervention. This enhances efficiency and reduces labor
requirements.
Livestock Monitoring:
Application: Health Monitoring and Behavior Analysis
Description: AI-based systems monitor the health and behavior of livestock using sensors and cameras. This helps
farmers detect early signs of illness, optimize feeding schedules, and enhance overall animal welfare.
Weather Prediction and Climate Modeling:
Application: Climate-Resilient Agriculture
Description: AI analyzes large datasets, including historical weather patterns, to predict future
climate conditions. Farmers can use this information to make climate-resilient decisions, such as
adjusting planting times and choosing suitable crops.
Disease Detection and Diagnosis:
Application: Plant Disease Identification
Description: AI, including computer vision and machine learning, is used to identify plant diseases
based on images of leaves and crops. Early detection allows for timely intervention and prevention.
Farm Management Software:
Application: Decision Support Systems
Description: AI-driven farm management platforms integrate data from various sources to provide
insights and recommendations. Farmers can use these systems for planning, monitoring, and
optimizing their operations.
Water Management:
Application: Smart Irrigation Systems
Description: AI helps in optimizing water usage by analyzing soil moisture levels and weather
conditions. Smart irrigation systems adjust watering schedules based on real-time data, promoting
efficient water management.
Use of AI in different fields -social media
Monitoring
Sentiment Analysis:
Description: AI-driven sentiment analysis evaluates social media content to determine the sentiment behind posts,
comments, and mentions. It helps businesses understand public opinion, customer sentiment, and brand perception.
Automated Content Curation:
Description: AI algorithms curate and categorize content based on predefined criteria. This includes sorting posts, articles,
and media to provide users with relevant and personalized content.
Social Listening:
Description: AI tools perform social listening by monitoring mentions of specific keywords, brands, or topics across social
media platforms. This helps organizations stay informed about discussions related to their brand and industry.
Influencer Identification:
Description: AI analyzes social media data to identify influencers—individuals with significant impact and reach in specific
niches. This information is valuable for influencer marketing campaigns.
Fake News Detection:
Description: AI algorithms can detect patterns associated with misinformation and fake news. This helps in identifying and
mitigating the spread of false information on social media platforms.
Social Media Analytics:
Description: AI-driven analytics tools provide insights into user engagement, audience demographics, and performance
metrics. These insights assist businesses in refining their social media strategies.
Tools and techniques for implementing AI
• 1. Programming Languages:
• Python: Widely used for AI development due to its extensive libraries and frameworks,
including TensorFlow, PyTorch, and scikit-learn.
• R: Particularly used for statistical analysis and data visualization in AI applications.
• 2. Machine Learning Frameworks:
• TensorFlow: Developed by Google, it's an open-source machine learning library widely
used for building and training deep learning models.
• PyTorch: Developed by Facebook, it's known for its dynamic computational graph and is
popular for research in deep learning.
• Scikit-learn: A versatile machine learning library for classical machine learning
algorithms.
• 3. Deep Learning Frameworks:
• Keras: High-level neural networks API, often used with TensorFlow as a backend.
• Caffe: A deep learning framework developed for speed and efficiency.
• MXNet: A flexible and efficient deep learning framework, particularly popular in certain
industries.
4. Natural Language Processing (NLP) Tools:
NLTK (Natural Language Toolkit): A comprehensive library for natural
language processing.
Spacy: An open-source library for advanced natural language processing.
Gensim: Particularly used for topic modeling and document similarity analysis.
5. Data Preparation and Cleaning:
Pandas: A powerful data manipulation and analysis library for cleaning and
preprocessing data.
NumPy: Essential for numerical operations and working with arrays.
Scrapy: A framework for extracting data from websites.
6. Data Visualization:
Matplotlib: A popular plotting library for creating static, interactive, and
animated visualizations in Python.
Seaborn: Built on top of Matplotlib, it provides a high-level interface for
drawing attractive statistical graphics.
Plotly: A versatile library for creating interactive visualizations.
7. Cloud Platforms for AI:
AWS (Amazon Web Services): Offers various AI services, including
SageMaker for machine learning and AI development.
Google Cloud AI Platform: Provides tools and services for building,
training, and deploying machine learning models.
Microsoft Azure AI: Offers a range of AI services, including Azure
Machine Learning for model development.
8. Automated Machine Learning (AutoML) Tools:
Google AutoML: Enables the development of custom machine learning
models with minimal effort.
H2O.ai: Provides AutoML capabilities for building and deploying
machine learning models.
DataRobot: An enterprise AI platform that automates the end-to-end
process of building, deploying, and managing machine learning models.
9. Version Control:
Git: Essential for version control and collaborative development.
10. Model Deployment and Serving:
Docker: Used for containerization to package and deploy AI applications.
Kubernetes: A container orchestration platform for managing containerized
applications.
TensorFlow Serving: A flexible, high-performance serving system for machine
learning models.
11. Reinforcement Learning Frameworks:
OpenAI Gym: A toolkit for developing and comparing reinforcement learning
algorithms.
Stable Baselines: A set of high-quality implementations of reinforcement
learning algorithms.
12. Model Monitoring and Management:
MLflow: An open-source platform for managing the end-to-end machine
learning lifecycle.
TensorBoard: A web-based tool for visualizing machine learning experiments.
Present famous Examples of AI
• Google Translator
• Driverless Car
• ALEXA
• Siri
• ChatGPT
Current trends and opportunities for AI and Machine Learning
1. Explainable AI (XAI):
• Trend: There is an increasing emphasis on making AI systems more transparent and interpretable.
Explainable AI focuses on developing models that provide clear explanations for their decisions,
addressing concerns about the "black-box" nature of complex algorithms.
• Opportunity: Developing XAI solutions is crucial for industries with regulatory requirements and
for building trust among users and stakeholders.
2. Edge AI:
• Trend: The deployment of AI models directly on edge devices (e.g., smartphones, IoT devices) is
gaining traction. Edge AI reduces latency, enhances privacy by processing data locally, and
improves efficiency in applications such as autonomous vehicles and smart sensors.
• Opportunity: Opportunities lie in developing lightweight and efficient models for edge devices,
enabling real-time inference in resource-constrained environments.
3. AI in Healthcare:
• Trend: AI is increasingly used in healthcare for medical image analysis, drug discovery,
personalized medicine, and patient monitoring. The COVID-19 pandemic has accelerated the
adoption of AI for diagnostics and epidemiological research.
• Opportunity: Opportunities abound for developing AI solutions that improve healthcare
outcomes, enhance diagnostics, and contribute to advancements in medical research.
4. AI in Cybersecurity:
Trend: AI is increasingly utilized for threat detection, anomaly detection, and
cybersecurity automation. Machine learning models can analyze large datasets to
identify and respond to security threats in real time.
Opportunity: Opportunities exist in developing AI-driven cybersecurity solutions that
enhance the resilience of organizations against evolving cyber threats.
5. AI for Sustainability:
Trend: AI is being leveraged to address environmental and sustainability challenges.
Applications include optimizing energy consumption, monitoring and managing
resources, and mitigating the impact of climate change.
Opportunity: Developing AI solutions that contribute to sustainable practices, such as
precision agriculture, renewable energy optimization, and climate modeling.
6. AI Ethics and Responsible AI:
Trend: There is growing awareness of the ethical implications of AI, including issues
related to bias, fairness, and accountability. Organizations are placing greater emphasis
on responsible AI practices.
Opportunity: Opportunities lie in developing tools and frameworks for ethical AI,
conducting bias audits, and ensuring that AI systems adhere to ethical guidelines.
7. Natural Language Processing (NLP) Advancements:
Trend: Advances in NLP, including transformer architectures, have led to significant
improvements in language understanding, generation, and translation. Large pre-trained
language models are driving breakthroughs in various NLP applications.
Opportunity: Opportunities lie in leveraging state-of-the-art NLP models for applications
such as chatbots, language translation, sentiment analysis, and content generation.
8. AI in Finance:
Trend: The financial industry continues to adopt AI for fraud detection, risk management,
algorithmic trading, and customer service. AI models are used to analyze financial data,
identify patterns, and make data-driven decisions.
Opportunity: Opportunities exist for developing AI solutions that enhance the efficiency and
accuracy of financial services, including personalized financial advice and improved risk
assessment.
9. Continuous Learning and Self-Supervised Learning: -
Trend: The focus is shifting toward continuous learning, where models can adapt and learn
incrementally from new data without retraining from scratch. Self-supervised learning, where
models learn from unlabeled data, is gaining attention.
Opportunity: Opportunities lie in developing algorithms and systems that enable continuous
learning in dynamic environments and leveraging self-supervised learning for unsupervised
Job roles and skillset for AI and ML
Job Role Key Skills
Statistical analysis, programming (Python, R), machine learning algorithms, data wrangling, data visualization,
Data Scientist
SQL/database knowledge.
Programming (Python, Java), machine learning algorithms, model deployment, TensorFlow, PyTorch, software
Machine Learning Engineer
engineering.
Strong mathematics, theoretical computer science, research and development, neural network architectures,
AI Research Scientist
advanced programming (C++).
Linguistics, text processing, NLP libraries (NLTK, spaCy), pre-trained language models (BERT, GPT), programming
NLP Engineer
(Python).
Image processing, computer vision algorithms, deep learning for image recognition, vision libraries (OpenCV),
Computer Vision Engineer
CNN expertise, programming (Python, C++).
Business understanding, AI solution design, cloud platforms (AWS, Azure, GCP), integration, communication,
AI Solutions Architect
presentation.
Data pipeline development, database management, ETL processes, big data technologies (Hadoop, Spark),
Data Engineer
programming (Python, SQL).
Mechanical engineering, robotics programming (ROS, Python), control systems, machine learning for robotics,
Robotics Engineer
sensor integration.
Knowledge of ethical considerations, understanding bias and fairness, ability to assess and mitigate ethical risks,
AI Ethicist
communication, advocacy.
Business strategy, market analysis, AI technology understanding, product development lifecycle, communication,
AI Product Manager
decision-making.
 Unit III
Introduction to Cyber Security
&
Secure web-browsing
 Topic to be Cover
• Introduction to Cyber Security
• Information security concepts
• Threats, Types of malware
• Types of attacks
• Use of cyber security in different industries
• Tools for cyber security assessment
• Cyber security opportunities in market and skillset
• Recognize a secure connection
• Recognize suspicious links
• Update Browsers and plugins
• Recognize untrusted source warnings
Introduction to Cyber Security

• Day by day increasing reliance on digital technologies, cybersecurity

has become a critical aspect of safeguarding sensitive information and
ensuring the smooth operation of businesses, governments, and
individuals.

• Cybersecurity is the practice of protecting computer systems,

networks, and data from unauthorized access, attacks, and damage to
ensure confidentiality, integrity, and availability.
Information security concepts
Confidentiality:
Protecting sensitive information from unauthorized access or disclosure.
Integrity:
Ensuring the accuracy and reliability of data by preventing unauthorized modification.
Availability:
Ensuring that systems and data are accessible and operational when needed.
Authentication:
Verifying the identity of users or systems attempting to access resources.
Authorization:
Granting or restricting access to resources based on authenticated users' permissions.
Non-Repudiation:
Ensuring that users cannot deny their actions or transactions.
Threats and Types of attack
1. Malware:
Definition: Malicious software designed to harm or exploit computer systems.
Types: Viruses, worms, Trojans, ransomware, spyware, adware.
2. Phishing:
Definition: Deceptive attempts to obtain sensitive information by posing as a trustworthy entity.
Methods: Email phishing, spear phishing, vishing (voice phishing), and smishing (SMS phishing).
3. Denial-of-Service (DoS) Attacks:
Definition: Overloading a system or network to disrupt normal functioning and deny service to legitimate
users.
Types: Flooding attacks, SYN/ACK attacks, distributed denial-of-service (DDoS) attacks.
4. Man-in-the-Middle (MitM) Attacks:
Definition: Intercepting and potentially altering communication between two parties without their
knowledge.
Methods: Eavesdropping, session hijacking, DNS spoofing.
5. Data Breaches:
Definition: Unauthorized access leading to exposure or theft of sensitive data.
Causes: Weak authentication, poor access controls, hacking, insider threats.
6. Insider Threats:
Definition: Security risks posed by individuals within an organization who may intentionally or
unintentionally compromise security.
Examples: Malicious employees, negligent behavior, unintentional data leaks.
7. Ransomware:
Definition: Malware that encrypts files or systems, demanding payment for their release.
Examples: WannaCry, CryptoLocker, Ryuk.
8. Zero-Day Exploits:
Definition: Attacks exploiting vulnerabilities in software before developers release patches or solutions.
Risk: Limited time for organizations to respond before a fix is available.
9. SQL Injection:
Definition: Injecting malicious SQL code into input fields to manipulate databases.
Target: Websites and applications with vulnerable database queries.
10. Social Engineering: -
Definition: Manipulating individuals to divulge confidential information or perform actions that may
compromise security. –
Methods: Impersonation, pretexting, quid pro quo.
Types of Malware:
1. Viruses:
Characteristics: Infects executable files, spreads through user actions.
Function: Replicates and attaches to host files, often causing damage.
2. Worms:
Characteristics: Self-replicating, spreads over networks without user interaction.
Function: Exploits vulnerabilities to propagate and deliver payloads.
3. Trojans:
Characteristics: Disguised as legitimate software, tricks users into installing.
Function: Opens backdoors, steals data, or delivers other malware.
4. Ransomware:
Characteristics: Encrypts files or systems, demands payment for decryption.
Function: Blocks access to files until ransom is paid.
5. Spyware:
Characteristics: Monitors user activity, collects sensitive information.
Function: Stealthily gathers data without user consent.
6. Adware:
Characteristics: Displays unwanted advertisements to users.
Function: Generates revenue for attackers through ad clicks.
7. Keyloggers:
Characteristics: Records keystrokes to capture sensitive information.
Function: Logs usernames, passwords, and other input.
8. Botnets:
Characteristics: Network of compromised computers controlled by a single entity.
Function: Performs coordinated actions, such as DDoS attacks or spam distribution.
9. Rootkits:
Characteristics: Conceals malicious activities by subverting system controls.
Function: Provides persistent access for attackers.
10. Logic Bombs:
Characteristics: Malicious code triggered by specific conditions or events.
Function: Activates under predefined circumstances, causing damage.
11. Brute Force Attacks:
Definition: Attempting to gain unauthorized access by systematically trying all possible
combinations of passwords or keys.
Target: Login credentials, encryption keys.
12. Cross-Site Scripting (XSS) Attacks:
Definition: Injecting malicious scripts into websites viewed by other users.
Outcome: Allows attackers to steal information or manipulate content viewed by users.
13. Cross-Site Request Forgery (CSRF) Attacks:
Definition: Forcing users to perform actions on a website without their knowledge or consent.
Outcome: Unauthorized actions performed on behalf of the user.
14. DNS Spoofing/Cache Poisoning:
Definition: Manipulating the domain name system (DNS) to redirect traffic or deceive users.
Outcome: Redirects users to malicious websites.
15. Eavesdropping/Sniffing Attacks:
Definition: Intercepting and monitoring communication between parties without their
knowledge.
Methods: Packet sniffing, network eavesdropping
16. Drive-By Downloads:
Definition: Automatically downloading malicious software onto a user's device without their
knowledge.
Method: Exploiting vulnerabilities in web browsers or plugins.
17. Watering Hole Attacks:
Definition: Compromising websites that are likely to be visited by a specific target group.
Target: Users with common interests or affiliations.
18. Malvertising:
Definition: Spreading malware through online advertisements, often on legitimate websites.
Outcome: Users unknowingly download malware by clicking on infected ads.
19. IoT (Internet of Things) Exploitation:
Definition: Exploiting vulnerabilities in IoT devices to gain unauthorized access or control.
Outcome: Compromised IoT devices used in attacks.
20. Fileless Attacks:
Definition: Executing malicious code directly in memory without leaving traces on the file
system.
Challenge: Difficult to detect using traditional antivirus tools.