Orientation To Computing
Orientation To Computing
Orientation To Computing
TO
Computing II
Big Data and Data Science
Data Science & Big Data
• Data Science and its need
• Applications of data Science /Big Data
• Data Science life-cycle with use case,
• Big data and its 3V’s
• Challenges of Big data
• Skills needed for Big data
• Tools usage like Apache Hadoop, Tableau, R-language Excel , Big data
on cloud
• Use of Big data in different areas
• Job roles and Skillset for Data Science and Big data
WHAT IS DATA SCIENCE?
• Data science is a multidisciplinary field that involves the use of
scientific methods, processes, algorithms, and systems to extract
insights and knowledge from structured and unstructured data.
Business Intelligence (BI) Analyst BI tools (Tableau, Power BI), data visualization, data querying, business analysis
Quantitative Analyst Financial modeling, risk analysis, statistical analysis, programming (e.g., Python, R)
Operations Analyst Process optimization, data analysis, business process modeling, programming
Job roles and skillset for Big data
Job Role Skillset
Hadoop ecosystem (Hive, Pig, HBase), Apache Spark, NoSQL databases,
Big Data Engineer
ETL processes, programming (Java, Scala)
System architecture design, data modeling, distributed computing, cloud
Big Data Architect
platforms, Hadoop ecosystem
Data Warehouse Architect Data warehousing, database design, ETL processes, SQL, cloud platforms
Cloud platforms (AWS, Azure, Google Cloud), data migration, data
Cloud Data Engineer
integration, programming
Real-time data processing, stream processing frameworks (e.g., Apache
Streaming Data Engineer
Flink), messaging systems (e.g., Apache Kafka)
DevOps practices, version control, automated testing, continuous
DataOps Engineer
integration/continuous deployment (CI/CD), data pipeline management
Analytical and problem-solving skills, domain expertise, communication
Big Data Consultant
skills, proficiency in big data technologies
Data governance frameworks, compliance knowledge, data quality
Data Governance Analyst
management, metadata management
Data security measures, encryption techniques, access controls,
Data Security Analyst
compliance knowledge
Statistical analysis, machine learning, big data technologies (Hadoop,
Data Scientist (with Big Data Focus)
Spark), programming (e.g., Python, R)
UNIT II
Artificial Intelligence
&
Machine Learning
Topic to be Cover
• Introduction to AI, ML and Deep Learning
• Expert systems
• Fuzzy systems
• Augmented Reality
• Use of AI in different fields - NLP Healthcare, Agriculture, Social media
monitoring
• Tools and techniques for implementing AI
• Application of AI and ML
• Job roles and skillset for AI and ML
Artificial Intelligence (AI)
• Artificial Intelligence (AI) refers to the simulation of human
intelligence in machines that are programmed to think, learn, and
perform tasks that traditionally required human intelligence.
• It includes both narrow AI, which is designed for specific tasks, and
general AI, which would possess human-like intelligence across a
wide range of domains.
Machine Learning (ML)
• Machine Learning is a subset of AI that focuses on the development of
algorithms and statistical models that enable computers to learn from data
and improve their performance on a specific task without explicit
programming. ML algorithms use patterns and insights from data to make
predictions or decisions. There are three main types of machine learning:
1.Supervised Learning: The algorithm is trained on a labeled dataset,
where the input data is paired with corresponding output labels. It learns
to map inputs to outputs.
2.Unsupervised Learning: The algorithm is given unlabeled data and must
find patterns or relationships within the data without explicit guidance.
3.Reinforcement Learning: The algorithm learns by interacting with an
environment and receiving feedback in the form of rewards or penalties.
It aims to learn the optimal actions to take in different situations
Deep Learning
• Deep Learning is a specialized subfield of machine learning that involves
neural networks with multiple layers (deep neural networks).
• These networks, often referred to as artificial neural networks, are inspired
by the structure and function of the human brain.
• Deep Learning excels in automatically learning hierarchical representations
from data, allowing it to capture complex patterns and features.
• Convolutional Neural Networks (CNNs) are commonly used for image
recognition, while Recurrent Neural Networks (RNNs) are effective for
sequence data like language.
Expert systems
• An Expert System (ES) is a computer program or software designed to
emulate the decision-making ability and problem-solving skills of a
human expert in a specific domain.
• Expert systems leverage knowledge, reasoning processes, and
decision-making rules to provide expert-level advice or solutions in a
particular field.
• These systems are part of the broader field of artificial intelligence
(AI) and have been used in various applications to address complex
problems.
Components of Expert Systems
1. Knowledge Base (KB):
The knowledge base is a repository that stores information, facts, and rules relevant to a specific domain. It
represents the expertise of human specialists and is a critical component of an expert system.
2. Inference Engine:
The inference engine is the reasoning component of the expert system. It processes the information stored in
the knowledge base, applies rules, and makes logical inferences to arrive at conclusions or recommendations.
3. User Interface:
The user interface allows users to interact with the expert system. It may include natural language interfaces,
graphical interfaces, or other input/output mechanisms to facilitate communication between the system and
users.
4. Explanation Facility:
Expert systems often include an explanation facility to provide users with a clear understanding of the
system's reasoning and the basis for its recommendations. This enhances transparency and user trust.