Data Science with Python

Natural Language Processing (NLP) with

SciKit Learn
 Learning Objectives

By the end of this lesson, you will be able to:

Define natural language processing

Explain the importance of natural language processing

List the applications using natural language processing

Outline the modules to load content and category

Apply feature extraction techniques

Implement the approaches of natural language processing

Introduction to Natural Language
Processing
 Natural Language Processing (NLP)

Natural language processing is an automated way to understand and analyze natural human languages and extract
information from such data by applying machine algorithms.

Extract information

Analyze human languages

Machine algorithms and

translations
(mathematics and statistics)
Data from various sources
 Natural Language Processing

It is also referred to as, the field of computer science or AI to extract the

linguistics information from the underlying data.

Extract the linguistics information

 Why Natural Language Processing

The world is now connected globally due to the advancement of technology and devices.

Analyzing tons of data

Identifying various languages

Applying quantitative analysis

Handling ambiguities
 Why Natural Language Processing

NLP can achieve full automation by using modern software libraries, modules, and packages.

Full Intelligent
automation processing

Knowledge about Modern Machine

languages and world software models
libraries
 NLP Terminology

Determines where one word Word Splits words, phrases, and idioms
ends and the other begins boundaries Tokenization

Discovers topics in a collection Stemming Maps to the valid root word

Topic
of documents NLP
models

Disambig-
Determines meaning and uation Tf-idf
sense of words (context vs. Semantic Represents term frequency and
intent) analytics inverse document frequency

Compares words, phrases, and

idioms in a set of documents to
extract meaning
 NLP Approach for Text Data

Let us look at the Natural Language Processing approaches to analyze text data.

Conduct basic text

processing

Analyze the meaning

Categorize and tag words

Build feature-based NLP Classify text

structure

Analyze sentence
Extract information
structure
NLP Environmental Setup

Problem Statement: Demonstrate the installation of NLP environment

Access: Click on the Practice Labs tab on the left side panel of the LMS. Copy or note the
username and password that is generated. Click on the Launch Lab button. On the page that
appears, enter the username and password in the respective fields, and click Login.
Sentence Analysis

Problem Statement: Demonstrate how to perform the sentence analysis

Access: Click on the Practice Labs tab on the left side panel of the LMS. Copy or note the
username and password that is generated. Click on the Launch Lab button. On the page that
appears, enter the username and password in the respective fields, and click Login.
Applications of NLP
 Applications of NLP

Machine translation is used to translate one language into another. Google Translate
Machine Translation is an example. It uses NLP to translate the input data from one language to another.

Speech Recognition

Sentiment Analysis
 Applications of NLP

The speech recognition application understands human speech and uses it as input
Machine Translation information. It is useful for applications like Siri, Google Now, and Microsoft Cortana.

Speech Recognition

Sentiment Analysis
 Applications of NLP

Sentiment analysis is achieved by processing tons of data received from different

Machine Translation interfaces and sources. For example, NLP uses all social media activities to find out
the popular topic of discussion or importance.

Speech Recognition

Sentiment Analysis
 Major NLP Libraries

NLTK

Scikit-learn

NLP libraries
TextBlob

spaCy
The Scikit-Learn Approach
 The Scikit-Learn Approach

It is a very powerful library with a set of modules to process and analyze natural language data, such as text and
images, and extract information using machine learning algorithms.

Built-in module Feature extraction Model training

Contains built-in A way to extract Analyzes the content

modules to load information from based on particular
the dataset’s data which can be categories and then
content and text or images. trains them according
categories. to a specific model.
 The Scikit-Learn Approach

It is a very powerful library with a set of modules to process and analyze natural language data, such as texts and
images, and extract information using machine learning algorithms.

Pipeline building
mechanism

A technique to Various stages of pipeline

streamline the learning 1. Vectorization
NLP process into 2. Transformation
stages. 3. Model training and application
 The Scikit-Learn Approach

It is a very powerful library with a set of modules to process and analyze natural language data, such as texts and
images, and extract information using machine learning algorithms.

Pipeline building Performance Grid search for finding

mechanism optimization good parameters

A technique in In this stage It’s a powerful way

Scikit-learn we train the to search
approach to models to parameters
streamline the optimize the affecting the
NLP process into overall outcome for
stages. process. model training
purposes.
Modules to Load Content and Category
 Modules to Load Content and Category

Scikit-learn has many built-in datasets. There are several methods to load these datasets with the help of a data
load object.

Container
folder

Category 1

Data load object

Category 2

Data load object

 Modules to Load Content and Category

The text files are loaded with categories as subfolder names.

Container Extract features

folder

Category 1

NumPy array
SciPy matrix

Category 2
 Modules to Load Content and Category

In [ ] : #Build a feature extraction transformer

From sklearn.feature_extraction.text import <appropriate transformer>
 Modules to Load Content and Category

The attributes of a data load object are:

Contains fields and can be accessed

Bunch
as dict keys or an object
Attributes

Data load object Target names Has the list of requested categories

Data Refers to an attribute in the memory

 Modules to Load Content and Category

The example shows how a dataset can be loaded using Scikit-learn:

Import the dataset

Load dataset

Describe the dataset

 Modules to Load Content and Category

Let us see how functions like type, .data, and .target help in analyzing a dataset.

View type of dataset

View data

View target
 Feature Extraction

Feature extraction is a technique to convert the content into the numerical vectors to perform machine learning.

For example: Large datasets or documents

Text feature extraction

For example: Patch extraction, hierarchical

clustering

Image feature extraction

Bag of Words
 Bag of Words

Bag of words is used to convert text data into numerical feature vectors with a fixed size.

Storing

Counting Store as the

value feature
Tokenizing Number of
occurrences of
each word
Assign a fixed
integer id to each
word

Token 1 Token 2 Token 3 Token 4

Document 1 42 32 119 3
Corpus of document Document 2 1118 0 0 89
Document 3 0 0 0 55
 CountVectorizer Class Signature

Specifies number of
components to keep
Class class
sklearn.feature_extraction.text.CountVectorizer
Encoding used to
File name or (input='content', encoding='utf-8',
decode the input
sequence of strings decode_error='strict', strip_accents=None,
Removes accents
lowercase=True, preprocessor=None,
Overrides string
tokenizer tokenizer=None, stop_words=None, Built-in stop words list

token_pattern='(?u)\b\w\w+\b', ngram_range=(1, 1), Min Threshold

analyzer='word', max_df=1.0, min_df=1,
Max Threshold
max_features=None, vocabulary=None,
binary=False, dtype=<class 'numpy.int64'>)
Bags of Words

Problem Statement: Demonstrate the Bag of Words technique

Access: Click on the Practice Labs tab on the left side panel of the LMS. Copy or note the
username and password that is generated. Click on the Launch Lab button. On the page that
appears, enter the username and password in the respective fields, and click Login.
Text Feature Extraction Considerations
 Text Feature Extraction Considerations

This utility deals with sparse matrix while storing them in memory. Sparse data
Sparse is commonly noticed when it comes to extracting feature values, especially for
large document datasets.

It implements tokenization and occurrence. Words with minimum two letters

Vectorizer
get tokenized. We can use the analyzer function to vectorize the text data.

It is a term weighing utility for term frequency and inverse document

frequency. Term frequency indicates the frequency of a particular term in the
Tf-idf
document. Inverse document frequency is a factor which diminishes the
weight of terms that occur frequently.

Decoding This utility can decode text files if their encoding is specified.
 Model Training

An important task in model training is to identify the right model for the given dataset. The choice of model
completely depends on the type of dataset.

Models predict the outcome of new observations and datasets, and classify
documents based on the features and response of a given dataset.
Supervised
Example: Naïve Bayes, SVM, linear regression, K-NN neighbors

Models identify patterns in the data and extract its structure. They are also used to
Unsupervised group documents using clustering algorithms.

Example: K-means
 Naïve Bayes Classifier

It is the most basic technique for classification of text.

Advantages: Uses:
• It is efficient as it uses limited CPU and memory. • Naïve Bayes is used for sentiment analysis,
• It is fast as the model training takes less time. email spam detection, categorization of
documents, and language detection.
• Multinomial Naïve Bayes is used when multiple
occurrences of the words matter.
 Naïve Bayes Classifier

Let us take a look at the signature of the multinomial Naïve Bayes classifier:

Learn Class prior probabilities

Class

class sklearn.naive_bayes.MultinomialNB(alpha=1.0, fit_prior=True, class_prior=None)

Smoothing parameter
(0 for no smoothing) Prior probabilities of the
classes
 Grid Search and Multiple Parameters

Document classifiers can have many parameters. A Grid approach helps to search the best parameters for
model training and predicting the outcome accurately.

Category 1

Extract features of a document

Document classifier

Category 2
 Grid Search and Multiple Parameters

Document classifier Parameter

Parameter Grid searcher

Parameter
Best parameter
 Grid Search and Multiple Parameters

In grid search mechanism, the whole dataset can be divided into multiple grids and a search can be run on the
entire grid or a combination of grids.

Grid searcher
Best parameter Parameter 1

Parameter 2

Parameter 3
 Pipeline

A pipeline is a combination of vectorizers, transformers, and model training.

Extracts features
around the word of
interest

Transformer Model Training

Vectorizer
(tf-idf) (document classifiers)

Converts a collection
Helps the
of text documents into
model predict
a numerical feature
vector accurately
Pipeline and Grid Search

Problem Statement: Demonstrate the Pipeline and Grid Search technique.

Access: Click on the Practice Labs tab on the left side panel of the LMS. Copy or note the
username and password that is generated. Click on the Launch Lab button. On the page that
appears, enter the username and password in the respective fields, and click Login.
Analyzing the Spam Collection Dataset

Problem Statement:

Analyze the given Spam Collection dataset to:

1.View information on the spam data
2.View the length of messages,
3. Define a function to eliminate stop words
4. Apply Bag of Words
5. Apply tf-idf transformer
6. Detect Spam with Naïve Bayes model
Analyzing the Spam Collection Dataset

Instructions on performing the assignment:

•Download the Spam Collection dataset from the “Resource” tab. Upload it using the right
syntax to use and analyze it.

Common instructions:
•If you are new to Python, download the “Anaconda Installation Instructions” document
from the “Resources” tab to view the steps for installing Anaconda and the Jupyter
notebook.
•Download the “Assignment 01” notebook and upload it on the Jupyter notebook to access
it.
•Follow the provided cues to complete the assignment.
Analyzing the Sentiment Dataset using NLP

Problem Statement:

Analyze the Sentiment dataset using NLP to:

1. View the observations
2. Verify the length of the messages and add it as a new column
3. Apply a transformer and fit the data in the bag of words
4. Print the shape for the transformer
5. Check the model for predicted and expected values
Analyzing the Sentiment Dataset using NLP

Instructions on performing the assignment:

• Download the Sentiment dataset from the “Resource” tab. Upload it to your Jupyter
notebook to work on it.

Common instructions:
• If you are new to Python, download the “Anaconda Installation Instructions” document
from the “Resources” tab to view the steps for installing Anaconda and the Jupyter
notebook.
• Download the “Assignment 02” notebook and upload it on the Jupyter notebook to
access it.
• Follow the provided cues to complete the assignment.
 Key Takeaways

You are now able to:

Define natural language processing

Explain the importance of natural language processing

List the applications using natural language processing

Outline the modules to load content and category

Apply feature extraction techniques

Implement the approaches of natural language processing

Knowledge Check
Knowledge
Check
In NLP, tokenization is a way to _______________________.
1

a. Find the grammar of the text

b. Analyze the sentence structure

c. Find ambiguities

d. Split text data into words, phrases, and idioms

 Knowledge
Check
In NLP, tokenization is a way to _______________________.
1

a. Find the grammar of the text

b. Analyze the sentence structure

c. Find ambiguities

d. Split text data into words, phrases, and idioms

The correct answer is d

Splitting text data into words, phrases, and idioms is known as tokenization and each individual word is
known as token.
Knowledge
Check
What is the tf-idf value in a document?
2

a. Directly proportional to the number of times a word appears

b. Inversely proportional to the number of times a word appears

c. Offset by frequency of the words in corpus

d. Increase with frequency of the words in corpus

 Knowledge
Check
What is the tf-idf value in a document?
2

a. Directly proportional to the number of times a word appears

b. Inversely proportional to the number of times a word appears

c. Offset by frequency of the words in corpus

d. Increase with frequency of the words in corpus

The correct answer is a,c

td-idf value reflects how important a word is to a document. It is directly proportional to the number of
times a word appears and is offset by frequency of the words in corpus.
Knowledge
Check
In grid search, if n_jobs = -1, then which of the following is correct?
3

a. Uses only 1 CPU core

b. Detects all installed cores and uses them all

c. Searches for only one parameter

d. All parameters will be searched on a given grid

 Knowledge
Check
In grid search, if n_jobs = -1, then which of the following is correct?
3

a. Uses only 1 CPU core

b. Detects all installed cores and uses them all

c. Searches for only one parameter

d. All parameters will be searched on a given grid

The correct answer is b

Detects all installed cores on the machine and uses all of them.
Knowledge
Check
Identify the correct example of Topic Modeling from the following options:
4

a. Machine translation

b. Speech recognition

c. News aggregators

d. Sentiment analysis
 Knowledge
Check
Identify the correct example of Topic Modeling from the following options:
4

a. Machine translation

b. Speech recognition

c. News aggregators

d. Sentiment analysis

The correct answer is c

‘Topic model’ is statistical modeling and used to find latent groupings in the documents based upon the
words. For example, news aggregators.
Knowledge
Check How do we save memory while operating on Bag of Words which typically contain high-
dimensional sparse datasets?
5

a. Distribute datasets in several blocks or chunks

b. Store only non-zero parts of the feature vectors

c. Flatten the dataset

d. Decode them
 Knowledge
Check How do we save memory while operating on Bag of Words which typically contain high-
dimensional sparse datasets?
5

a. Distribute datasets in several blocks or chunks

b. Store only non-zero parts of the feature vectors

c. Flatten the dataset

d. Decode them

The correct answer is b

In features vector, there will be several values with zeros. The best way to save memory is to store only non-
zero parts of the feature vectors.
Knowledge
Check
What is the function of the sub-module feature_extraction.text.CountVectorizer?
6

a. Convert a collection of text documents to a matrix of token counts

b. Convert a collection of text documents to a matrix of token occurrences

c. Transform a count matrix to a normalized form

d. Convert a collection of raw documents to a matrix of TF-IDF features

 Knowledge
Check
What is the function of the sub-module feature_extraction.text.CountVectorizer?
6

a. Convert a collection of text documents to a matrix of token counts

b. Convert a collection of text documents to a matrix of token occurrences

c. Transform a count matrix to a normalized form

d. Convert a collection of raw documents to a matrix of TF-IDF features

The correct answer is a

The function of the sub-module feature_extraction.text.CountVectorizer is to convert a collection of text

documents to a matrix of token counts.
Thank You