CC306/BIC3233

Intelligent Systems
Lecture 9
More Bio-Inspired Approaches
and Natural Language
Processing
 Objectives
On completion of this module, students should be able to:

• understand both the appeals and limitations of natural

systems as the inspiration for artificially intelligent systems;

• understand of some of the many artificial intelligence

techniques that have been inspired by natural systems (eg.
artificial neural networks, evolutionary computing, ant-like
systems);

• implement one or more of these systems to solve a

particular problem (eg., using an artificial ant colony to solve
TSP
TSP).
 Bio-Inspired Computing
Finding the best solution
increasingly becomes very difficult
to identify, if not impossible, due to
the very large and dynamic scope of
solutions and complexity of
images from EVALife site
computations. Often, the optimal
solution for such a NP hard problem A. Schools, Flocks, & Herds
is a point in the n-dimensional • “and the thousands of fishes moved as a huge
hyperspace and identifying the beast, piercing the water.
They appeared united, inexorably bound to a
solution is computationally very common fate.
expensive or even not feasible in How comes this unity?”
limited time. — anon., 17th cent.
 Bio-Inspired Computing
• The computing inspired from biology is a field of study
based on the social behavior of animals, insects and other
living organisms, including also connectionism and
emergence.
• Bio-inspired computing uses computers to model nature
and the study of nature to improve the usage of computers.

Bio-inspired
computing

Biological Artificial
computation Intelligence
Bio-Inspired Computing
 Bio-Inspired Computing

Motivation
 Dealing too complex problems
Incapable to solve by human proposed
solution
Absence of complete mathematical
model
 Existing of similar problem in nature
Adaptation
Self-organization
Communication
Optimization
 Bio-Inspired Computing
Some areas of bio-inspired computing are:
• genetic algorithm
• particle swarm
• ant colony optimization
• artificial bee colony
• bacterial foraging
• cuckoo search
• Firefly
• leaping frog
• bat algorithm
• flower pollination
• artificial plant optimization
• neural networks
 Swarm
Intelligence
 Swarm Intelligence
• The SI-based algorithms belong to a wider class of the algorithms, called
the bio-inspired algorithms.
• we can observe that SI-based ⊂ bio-inspired ⊂ nature-inspired.
• Population of simple
agents
• Decentralized
• Self-Organized
• No or local
communication
• Example
 Ant/Bee colonies
 Bird flocking
 Fish schooling
 Coordinated Collective Movement

• Groups of animals can behave almost like a single organism

• Can execute swift maneuvers
• for predation or to avoid predation
• Individuals rarely collide hit, even in state of attack or escape
• Shape is characteristic of species, but flexible

TIGER Algorithm 2020

predation
 Coordinated Collective Movement

Adaptive Significance

• Prey avoiding predation

• More efficient predation by predators
• Other efficiencies
Avoiding Predation
• More compact aggregation
• predator risks injury by attacking
• Confusing predator by:
• united erratic maneuvers (e.g. zigzagging)
• separation into subgroups (e.g., flash expansion & fountain effect)

03/25/2024 14
Flash Expansion

Fig. from Camazine & al., Self-Org. Biol. Sys.

Flash Expansion

Fig. from Camazine & al., Self-Org. Biol. Sys.

03/25/2024 16
Fountain Effect

Fig. from Camazine & al., Self-Org. Biol. Sys.

Fountain Effect

Fig. from Camazine & al., Self-Org. Biol. Sys.

Fountain Effect

Fig. from Camazine & al., Self-Org. Biol. Sys.

Fountain Effect

Fig. from Camazine & al., Self-Org. Biol. Sys.

Better Predation
• Coordinated movements to trap prey
• e.g. parabolic formation of tuna
• More efficient predation
• e.g., killer whales encircle dolphins
• take turns eating
Other Efficiencies
• Fish schooling may increase hydrodynamic
efficiency
• endurance may be increased up to 6
• school acts like “group-level vehicle”
• V-formation increases efficiency of geese
• range 70% greater than that of individual
• Lobsters line up single file by touch
• move 40% faster than when isolated
• decreased hydrodynamic drag
 Characteristic Arrangement of School

• Shape is characteristic of species

• Fish have preferred distance, elevation & bearing relative to neighbors
• Fish avoid coming within a certain minimum distance
• closer in larger schools
• closer in faster moving schools
Ant Colony
Optimization
 Ant Colony Optimization

• mimic the foraging behavior of

social ants.
• Ants primarily use pheromone as
a chemical messenger.
• pheromone concentration can be
considered as the indicator of
quality solutions to a problem of
interest.
• The movement of an ant is
controlled by pheromone, which
will evaporate over time.
• the probability of ants at a
particular node i to choose the
route from node i to node j is
given by
 Ant Colony Optimization
Proposed by Marco Dorigo in 1991

Inspired in the behavior of real ants

Multi-agent approach for solving complex combinatorial

optimization problems

Applications:

Traveling Salesman Problem

Scheduling
Network Model Problem
Vehicle routing
 Ant Behavior

Nest Obstacle Food

Pavlova picture from: https://www.pngwing.com/en/free-png-zfxdc

 Ant Behavior

Nest Obstacle Food

Pavlova picture from: https://www.pngwing.com/en/free-png-zfxdc

 Ant Behavior

Nest Obstacle Food

Pheromones
 Ant Behavior

Quest for food

Start

Walk randomly Follow Trail

Found food Continue

Lay pheromone Trail

 Ant Behavior

Ant behavior is stochastic

The behavior is induced by indirect communication

(pheromone paths) - Stigmergy

Ants explore the search space

Limited ability to sense local environment

Act concurrently and independently

High quality solutions emerge via global

cooperation
 Stigmergy

Term coined by French biologist Pierre-Paul Grasse, means

interaction through the environment

Indirect communication via interaction with environment

Agents respond to changes in the environment

Allows simpler agents

Decreases direct communication
 Pheromones

Ants lay pheromone trails while traveling

Pheromones accumulate with multiple ants using a path

This behavior leads to the appearance of shortest

paths

Pheromones = long-term memory of an ant colony

Pheromones evaporate
Avoids being trapped in local optima

ρ small ⇒ low evaporation ⇒ slow

adaptation

ρ large ⇒ high evaporation ⇒ fast

 ACO Algorithm
Construct solutions
Explore the search space
Choose next step probabilistically according to the
pheromone model
Apply local search to constructed solutions (Optional)
Update pheromones (add new + evaporate)
Example: TSP
Example: TSP
 Advantages and Disadvantages
Advantages
Can be used in dynamic applications
Positive Feedback leads to rapid discovery of good
solutions
Distributed computation avoids
premature convergence

Disadvantages
Convergence is guaranteed, but
time to convergence
uncertain
Coding is not straightforward
Summary
 Human and Language

• Understanding language requires inferences about

speaker’s goals, knowledge and expectations of domain
and reason effectively about them.
• Humans are commonly identified by the languages that
they produce (“speak your mind”).

One reason for studying language — and for me personally the

most compelling reason — is that it is tempting to regard
language, in the traditional phrase, as a “mirror of mind”.
Chomsky,
1975
40
MIT
 Human and Language
• There needs to be an ability to characterize and explain the
multitude of linguistic observations around us in
conversations, writing, media, etc...
• Understanding and processing language concerns:
i. the cognitive side (information processing in the brain)
of how human acquire, produce and understands
language
ii. understanding the relationship between linguistic
utterances and the world
iii. understanding the linguistic structures by which
language communicates
41
 Why is Language Hard?

• Ambiguities in Language
• “School dropouts cut in half this year”
• “Hospitals Are Sued by 7 Foot Doctors”
• “Include Your Children When Baking Cookies”
• “Kids Make Nutritious Snacks”

• Complexity in Language
• “There is a boy, who lost his toy, who jumped with joy, who
drank the soy, who made everyone annoyed
42
 What is NLP?
The process of building computational models for understanding
natural language.

A coherent study of the human language from the point of views of

several disciplines : Linguistics, Psychology, Cognitive Science,
Computer Science, Statistics and Mathematics.

A theoretically motivated range of computational techniques for

analyzing and representing naturally occurring texts at one or more
levels of linguistic analysis for the purpose of achieving human-like
language processing for a range of tasks or applications

Also known as Computational Linguistics 43

 Multidisciplinary NLP

Computer Science
Linguistics

Artificial
Intelligence Computational
Linguistics
Cognitive
Psychology Information
Theory
Psychology
Statistics

44
Where does NLP fit in CS?

45
 Common NLP Tasks

Part-of-Speech (POS) Tagging

Assigning a part-of-speech to each word in a sentence
“Malaysia/N has/V 23/NUM million/N people/N”.
Computational Morphology
Processing of words and word forms, in both their graphemic
(written form) and their phonemic (spoken form)
Example: finite state morphology
a r s
c 1 2 3 4
0 t g
d g
5 6 7 8 9
i e r 46
 Common NLP Tasks

Sentence Parsing
Analysing a sentence into its component categories and functions
(refer Slide 6)
Machine Translation (MT)
An automated system that analyzes text from source language
and produces “equivalent” text in the target language

Source Target
Language Language
MT
algorithm
47
 Common NLP Tasks
Sentiment Analysis
Identify, analyze and classify opinions in text into categories such
as "positive" , "negative" or “neutral”
“I love Macintosh.” (Positive)
“I hate Blackberry! ” (Negative)
“The food is not bad at all” (Neutral)
“What a great car, it did not start the first day” (Ambiguous)

Co-reference Resolution
Two textual entities that refers to the same object in the “real world”
(Mitkov)
Saha Hisham Ismail1, 45, said poor drainage2 in the village3 was the main cause of
the problem4. “We1,3 have reported it2 to the authorities5 and they5 have promised to
look into it2, but nothing has been done to rectify the problem4.” 48
 Common NLP Tasks
Text Summarization
Automatically reducing a text document to create a summary that preserves the most
important points of the original document.
Example : Given a single document, produce abstract, outline and headline.
Topic Categorization
Classifies documents according to their topics
“Serena and Nadal relieved after surviving tough opponents in Madrid”
[Sports]
“Facebook eyes $1billion deal for GPS app Waze” [Technology]
“Property, constructions to lead stock market” [Business]
“All eyes on cabinet lineups” [Politics]

49
 Levels of Language Processing and Analysis

Discourse Analysis
Morphological and Lexical
Analysis Pragmatics Analysis
Syntactic Analysis
Semantic Analysis
Semantic Analysis
Pragmatics Analysis
Discourse Analysis Syntactic Analysis

Morphological and
Lexical Analysis
50
 Syntactic Analysis- Phrases
Noun Phrase, NP
He is a(ART) policeman(N).
My(N) house(N) is in Gombak.
Verb Phrase, VP
They play(V) street(N) soccer(N)
Perform(V) your(N) prayers(N) 5 times a day
Adjectival Phrase-AP
My grandfather is a slow(ADJ) driver(N)
The delicious(ADJ) meal(N) is gone.
Prepositional Phrase, PP
The astronaut travels to(P) the(DET) moon(N)
The food on(P) the(DET) table(N) is contaminated.

51
 Syntactic Analysis- Grammar

sentence(S) -> noun_phrase (NP) verb_phrase (VP)

noun_phrase(NP) -> proper_noun (N)
noun_phrase (NP) -> determiner (DET) noun (N)
verb_phrase (VP)-> verb (V) noun_phrase (NP)
prepositional_phrase (PP) -> preposition(P)noun phrase(NP)
proper_noun(N) -> [Luqman]
noun(N) -> [apple]
verb (V) -> [ate]
determiner(DET) -> [the]

52
Syntactic Analysis- Parsing (Example 1)

S
Grammar rules used:
NP VP
S -> NP VP
VP – > V NP
N V NP
NP -> DET N
NP -> N
DET N

Luqman ate the apple

53
Syntactic Analysis- Parsing (Example 2)

S Grammar rules used:

NP VP S -> NP VP
VP – > V NP
DET N V NP NP -> DET N
NP -> N
DET N

The boy kicks the ball

54
 Syntactic Analysis- Parsing (Example 3)

S
Grammar rules used:
NP VP
S -> NP VP
VP – > V NP
N V NP
NP -> ART (ADJ) N
NP -> N
ART ADJ N *Note : rules in ( )
are optional

Islam is a beautiful
religion
noun verb article adjective noun
55
 Syntactic Analysis- Parsing (Example 4)
S
Grammar rules used:
NP VP
S -> NP VP
VP – > V NP
N V NP
NP -> NP PP
NP -> DET N
NP PP
NP -> ART (ADJ) N
PP -> P NP
DET N P NP NP -> N
*Note : rules in ( )
DET N are optional
He chases the thief with a bicycle
56
 Syntactic Analysis- Parsing (Example 5)

S
Grammar rules used:
VP CC VP
S -> VP CC VP
V NP V NP S- > NP VP
VP – > V NP
NP PP NP  NP PP
PP -> P NP
P NP NP -> N
N N N

Give food to others and speak good

57
 Issues in Syntactic Analysis

• Handling ambiguity
• Syntactic ambiguity:
• “Fruit flies like a banana”
• “Fruit flies like a banana”

• Having to parse syntactically incorrect sentences

• “(The boy) (the ball) kicks”
(NP) (NP) (VP)

58
Examples of NLP Applications

Google Translate

59
Examples of NLP Applications

Arabic WordNet

60
 Examples of NLP Applications

Gmail Spell Checker

61
 Examples of NLP Applications

Spam Classifier

62
 Examples of NLP Applications

Mango Speech Recognition

63
 References

Dorigo M., Blum C., Ant colony optimization theory: A survey,

Theoretical Computer Science, Volume 344, Issues 2-3,
November 2005
Blum C., Ant colony optimization: Introduction and recent
trends, Physics of Life Reviews, Volume 2, Issue 4, December
2005
Dorigo M., Stutzle T., Ant Colony Optimization, Ant Colony
Optimization, MIT Press 2004
NATURAL LANGUAGE PROCESSING (NLP)

END OF LECTURE 9
What have you learned?
NATURAL LANGUAGE PROCESSING (NLP)

Questions ?