International Scientific Conference Computer Science’2008

Knowledge Representation and Reasoning in Natural

Language Processing Systems

Diana Grigorova, Nikolay Nikolov

TU of Sofia, Bulgaria

Abstract: This paper presents a short analysis of the basic methods for knowledge repre-
sentation in the systems with artificial intelligence. The pros and cons of every method have
been reviewed. Reasoning capabilities of the methods are discussed, too. Based on the analy-
sis made in the paper, a frame knowledge representation for the application has been chosen.
The application itself is an analyzer of natural language sentences. An example illustrates the
capabilities of the created analyzer, which is both problem- and language- independent.
Keywords: knowledge representation, reasoning, first order predicate logic, frames, de-
scription logics.

1. INTRODUCTION
Knowledge representation is a key element in the systems with artificial intelligence.
The choice of the method for knowledge representation is extremely important since it
impacts the capabilities of the system. The natural language processing systems need
knowledge for words, linguistic knowledge and knowledge about the specific domain.
Further, we will use the word “knowledge” in the sense of knowledge from the latter
type. In order to be successful, the knowledge representation method must possess
several characteristics. Those characteristics are often contradictory and can hardly be
present at the same time in the single method. These characteristics are [8]:
1. Verifiability.
The system must be capable to compare and juxtapose the meaning of the input
sentence against the content of the knowledge base.
2. Unambiguity.
Although natural languages are ambiguous, the meaning representation of input
sentence must be unambiguous.
3. Canonical form.
The input sentences with identical content must have identical representation.
4. Reasoning capability.
The system must be able to draw conclusions about the truth of propositions which
are not present explicitly in the knowledge base, but are logically derivable from the
available propositions.
5. Expressiveness.
The method for knowledge representation must be sufficiently expressive, so that it
can easily represent knowledge with different scope and granularity. The idea of a
universal language, which adequately and easily represents the meaning of any
sensible natural language utterance, is lucrative, but we must note that there is al-
ways a trade-off between the good expressiveness and the ease of inference [6].
The relation between the last two characteristics is reciprocal.

641
 International Scientific Conference Computer Science’2008

2. KNOWLEDGE REPRESENTATION METHODS

2.1. First order predicate logic

Historically, the first order predicate logic is the first method for knowledge representa-
tion in artificial intelligence systems. It meets the requirements for verifiability, reasoning
capability and relatively good expressiveness. Its main advantage is that it combines a
language for expressing of knowledge propositions and rules for deriving of new facts
from those that are already known, i.e. there is an embedded reasoning mechanism
named resolution. The conclusions made through resolution are correct and complete.
The resolution is applicable only for first order logic. If an argument of a predicate is an-
other predicate, the resolution process may be endless. Even when the resolution proc-
ess is not endless, the time it takes increases exponentially with the increase of number
of clauses. The risk of combinatorial explosion is real [12]. The safe approach to prevent
combinatorial explosion is to transfer the control of the reasoning process to the user,
who, based on his experience can avoid a redundant and endless search. Another ap-
proach for avoiding a combinatorial explosion is the introduction of heuristics through
which the reasoning process can become problem-dependent. We can conclude that
the correctness, completeness and problem-independent method of reasoning can
change from advantages into disadvantages when the subject area is wide. Another
disadvantage of this method is that it does not support structured knowledge presenta-
tion, which implies that semantically connected information has to be grouped together
syntactically [4]. Formulating the proper facts, which are always true and logical, is a se-
rious problem. This problem is more a subject of ontology than subject of logic, but it
shouldn’t be underestimated [15]. A disadvantage of the method is the completely re-
stricted capability for drawing conclusions with incomplete and imprecise information. In
spite of the mentioned disadvantages, the first order logic and its modifications are be-
ing used in modern natural language processing systems. It is the preferred when the
sentences are limited both grammatically and lexically [14] or when the application is fo-
cused on a certain type of reasoning (default reasoning) within a narrow subject area
[20]. [13] presents a semantic analyzer based on predicate logic extended with addi-
tional quantifiers. The usage of conceptual graphs [15], as the graphical interpretation of
the predicate logic, is described in [18]. The formalism of relaxed conceptual graphs is
used in [9]. Two further modifications of the predicate logic are described in [3]. One of
them is an extension of the FOL with generalized quantifiers and constructs for describ-
ing the context. The other modification is used for representing the model of knowledge
domain. This same knowledge domain, albeit wide, is fixed and unchangeable. The
connection with the database is always achieved through the so-called conceptual
predicates, which according to the authors [3] have very much in common with the idea
of frames.

2.2 Frames
The search for possibilities for clear knowledge structuring led the researchers to the
idea of the semantics nets. The main critique to them stems from the fact that the
meaning of a given semantic net is left to the intuition of the developers who implement
programs for processing of the nets. Their advantages – knowledge structuring and
property inheritance – have been built into the idea for the next method of knowledge
representation: the frames. The term frame as a data structure for presentation of a
stereotypical situation was defined by M. Minsky [12]. As is known, the frames have a
common part and slots which define different characteristics of the objects or relations

642
 International Scientific Conference Computer Science’2008

through other objects. An important part of every frame is the pointer to a more general
frame. The slots are filled with fillers which can be either atomic values or names of
other frames. The slots of the generic frames can have procedures attached to them.
The reasoning in frame-based system starts by identifying of a given object as an in-
stance of a generic frame. After that all slot fillers which have not been set explicitly but
can be inherited, are inherited. Where available, the procedures for frame recognition
and inheritance are invoked. They can cause changes in the knowledge base which re-
sult in cascade invocation of other procedures. A characteristic feature of inheritance
with the frame systems is that it is defeasible, i.e. the inherited value is used only when
the corresponding characteristic can not be filled in a different way.
The frame systems have much in common with conventional object-oriented systems.
The main difference is that frame systems have centralized control regime and work in
the following cycle: frame instantiating, value inheritance, and procedure invocation.
The object-oriented systems are less decentralized and less patterned.
A disadvantage of a frame-based knowledge base is that it cannot process objects
which characteristics are not known in advance and it cannot process non typical situa-
tions. Another disadvantage is related to the procedural knowledge which is present in
frame systems. The procedural knowledge is not represented by a frame, but by pro-
gram called. The system can perform reasoning with that knowledge, but not about it
[19]. The frames cover all five characteristics mentioned in the introduction while the
reasoning capability is least developed.
Minsky’s idea about frames has met support among the researchers dealing with arti-
ficial intelligence. As a result, the frames have become widely used in knowledge based
system. Frame-like structures in combination with rules are used in expert systems [2].
Examples of works on toolkit for processing of knowledge represented by frames can be
found in [11, 17]. [1] analyses the applicability of frames for modeling of the knowledge
domain in intelligent systems for planning. A combination of frame structure and formal-
ism based on description logic is presented in [10].
2.3 Description logic
The language, developed by Ronald Brachman - Knowledge Language One (KL-
ONE) is the first of the class of languages with generic name “description logic”. The
basic notions of the description logic are concepts (analogue of frames), roles (ana-
logue of slots) and constants. The description logic uses concepts and an extensive set
of logical operators for describing the knowledge domain. Consequently, the languages
from this group are both class-based and logically based [7]. Unlike the slots used with
the frames, the roles may have more than one filler at any given time. A distinctive fea-
ture of the description logic is the existence of non-atomic concepts which meanings are
completely defined by the meanings of their parts. The strictly hierarchy taxonomy of the
concepts allows for two types of reasoning: 1) To determine if a constant satisfies a
given concept and 2) To determine if a concept is a sub-concept of another one, i.e. if it
is subsumed by the other. The inheritance is simpler than the one used by frames. As it
is strict and does not allow exceptions. The difference between the taxonomy of a frame
system and a system based on description logic is that with frame systems the designer
can create hierarchy in any chosen line, while with description logic the logic of the con-
cepts determines the position of every concept in the resulting taxonomy. The descrip-
tion logic covers best all five characteristics of the knowledge representation method. A
study of the tradeoff between expressiveness and tractability of the reasoning is pre-
sented in [6]. The addition of a single operator, improving the expressiveness, results in

643
 International Scientific Conference Computer Science’2008

significant complexity of reasoning. One of the successful systems, based on descrip-

tion logic having proven effectiveness in real - life applications is CLASSIC [5]. The lan-
guage LOOM used in [16] also belongs to the family of languages, based on description
logic.
The conclusion drawn from the analyses of the knowledge representation methods is
as follows: The choice of the knowledge representation method depends on the goal of
the application. Depending on the set goal, some of the methods may be more appro-
priate and preferred. When the focus of the application system is on the useful and easy
knowledge representation, then frames should be preferred. Their object orientation and
inherent capability to create hierarchies helps people to better comprehend the knowl-
edge. When the focus is on the reasoning capability of the system, then the predicate
logic is preferable but within a narrow knowledge domain. The languages based on de-
scription logic were created in pursuit of balance between expressiveness and tractabil-
ity without implying that they are universally applicable. The use of such language is not
recommended for applications where the focus is on mathematical objects and rela-
tions. If this were the case for knowledge representation, applications which do not have
to draw complex conclusions, would unnecessarily consume resources. Description
logic is not the best choice if the application is targeted to default reasoning. It is also
not suitable for applications for time and space reasoning.

3. THE APPLICATION
The task the authors of the article have defined is to create an analyzer of context
free natural language sentences. The main requirements are the easy creation of the
model of the knowledge domain and the easy grammar definition. The analyzer must be
problem and language independent and the switching between different knowledge do-
mains and different languages to be achieved easily. With the above formulated goals
we believe that the most suitable method for knowledge representation is the frame
structure.
The adjustment of the analyzer to a specific knowledge domain takes place by en-
tering new words into the dictionary and by changing the model of the knowledge do-
main. The adjustment to a specific language takes place by changing the grammar
The dictionary consists of all words and all their forms, i.e. no morphological analy-
sis is performed. Looking up the dictionary is a sequential process and is performed
once before the analysis begins. If a word cannot be found in the dictionary, it is inter-
preted as a proper name.
The analysis algorithm uses the class of extended context-free grammars. The ex-
tension allows a meta-symbol, which can be linked to the right side of every symbol in
every production, to define the number of possible occurrences of the original symbol.
The possible meta-symbols are: „?” – the symbol can exist zero or one time; „*” – the
symbol can exist zero or more times; „+” – the symbol can be repeated one or more
times. If there is no meta-symbol, linked to the symbol, it must exist exactly once.
The syntactic and semantic analyses take place simultaneously in close interac-
tion. The process begins with bottom-up syntactic analysis, which tries to identify some
structure in the sequence of lexical units using the syntactic rules. When a syntactic
group is found, the syntax tree is built and it is passed to the semantic analyzer. It tries
to construct a semantic tree with the elements of passed structure using the model of
the knowledge domain. If the semantic analysis is successful, the group of lexical units
is confirmed as a whole entity with the structure, defined by the analysis. The syntax

644
 International Scientific Conference Computer Science’2008

analyzer continues to work looking for the next structure in the input stream of lexemes.
It uses the syntax tree, which has already been built. The analysis is successful if there
are no more words in the input sentence and the groups have been reduced to one non-
terminal symbol – the start non-terminal symbol of the grammar. If at a given stage, the
analyzer cannot continue with the syntax or semantic analysis as a result of contradic-
tion or lack of a certain rule or choice alternative, it returns to the point of the last ambi-
guity and takes up the next alternative. The analysis goes on until all alternatives have
been exhausted, i.e. it does not stop at the first successful variant. The result of analysis
is a set of syntactic trees and semantic representations. The syntactic trees can be dif-
ferent, but the semantic representations must be equivalent. If they are not equivalent,
this means that the knowledge base needs corrections.

Fig. 1
Fig. 1 shows semantic trees built during the analysis of the sentences “Outside the
temperature decreased rapidly” and “The temperature outside dropped quickly”. The
semantic trees are identical, because the meaning of the sentences is identical.

4. CONCLUSIONS AND FUTURE WORK

The basic functionality of the application is achieved. The results are satisfactory.
Improvements can be looked for in the organization of the dictionary. The words can be
sorted, so that a binary search can take place. When an unknown word is found, before
it is interpreted as a proper name, a search for close words can be provided as an algo-
rithm computes the proximity between those words. The words, close to the unknown
can be displayed to the user to choose the right one. This approach will find a casual
spelling mistake in a word, which is not a proper name.
Improvements can be sought in the direction of limiting the checks of unsuitable
variants during the process of analysis. The syntax rules can be set to determine which
pre-terminal symbol is the basic, leading word in the group, to which the root-frame of
the group corresponds. On the other side, for every slot of the frame, it can be indicated

645
 International Scientific Conference Computer Science’2008

what kind of syntax structure is expected to fill it. Further development of the analyzer
will go on, trying to find a way of processing of contextual connected sentences.

REFERENCES
[1] Маринов М., Анализ на техниките на планиране, използвани в интелигентните системи, сп. “Ав-
томатика и информатика”, No 5-6, 25-33 (1997).
[2] Aikins J.S. Prototypical Knowledge for Expert Systems: a Retrospective Analysis, Artificial Intelligence
59 (1993), Elsevier, pp.207 – 211
[3] Alshawi H., D. Carter, R. Crouch, S. Pulman, M. Rayner A. Smith CLARE – A contextual reasoning
and cooperative response framework for the core languge engine. Final report, SRI International,
Cambridge UK, 1992. Available from: http://citeseer.ist.psu.edu/alshawi92clare.html.
[4] Baader F. Logic based knowledge representation, Artificial intelligence today, Recent trends and
developments, Springer-Verlag, 1999. Available from: http://citeseer.ist.psu.edu/244181.html.
[5] Brachman R., McGuiness D., Patel-Schneider P., Resnik L., Borgida A., Living with CLASSIC: when
and how to use a KL-ONE-language, Principal of semantic networks: explorations in the representa-
tion of knowledge, Morgan-Kaufman publishers, San Mateo, CA, 1991, pp. 401-456
6] Brachman R. J., H. J. Levesque, Knowledge representation and reasoning, Morgan Kaufmann pub-
lishers, 2004.
[7] Calvanese D., De Jacomo G., Lenzerini M., Description logics: foundation for class-based knowledge
representation, Proceedings of the 17th annual IEEE symposium on logic in computer science
(LICS’02). Available from: http://csdl2.computer.org/dl/proceedings/lics/2002/1483/00/14830359.pdf
[8] Jurafsky D., Martin J., Speech and language processing, Prentice-Hall, 2000
[9] Kang I., Bae J., Lee J., Database semantics representation for natural language access, Proceedings
of the first international symposium on cyber worlds (CW’02). Available from:
http://csdl2.computer.org/dl/proceedings/cw/2002/1862/00/18620127.pdf
[10] Kemke Ch., Action representation for natural language interfaces to agent systems, 2006 Interna-
tional conference on hybrid information technology (ICHIT'06) IEEE. Available from:
http://csdl2.computer.org/dl/proceedings/ichit/2006/2674/02/267420001.pdf
[11] Marinov M. An interactive tool for frame representation, Information Technologies & Control, No. 1,
16-19 (2003).
[12] Minsky, M. A framework for representing knowledge, MIT – AI laboratory memo 306, June, 1974.
Available from: http://web.media.mit.edu/~minsky/papers/Frames/frames.html.
[13] Onet A., Tatar D. Semantic analysis in dialogue interfaces, Studia univ. Babes-Bolyai, Informatica,
Volume XLV, Number 1, 2000, pp.79 – 88. Available from: http://citeseer.ist.psu.edu/503979.html.
[14] Schwitter R., Dynamic semantics for a controlled natural language, Proceedings of the 15th interna-
tional workshop on database and expert systems applications (DEXA’04). Available from:
http://csdl2.computer.org/dl/proceedings/dexa/2004/2195/00/21950043.pdf
[15] Sowa J. F., Knowledge representation. Logical, philosophical and computational foundations,
Brooks/Cole 2000.
[16] Stede M. Lexical semantics and knowledge representation in multilingual sentence generation, PhD
thesis University of Toronto, 1996. Available from: ftp://ftp.csri.toronto.edu/csri-technical-
reports/347/Stede-thesis.pdf
[17] Valkovska I., Grundspenkis J. Development of frame systems shell for learning of knowledge
representation issues, Proceedings of conference CompSysTech’05, pp.IV.11-1 – IV.11-6
[18] Velardi P., Pazienza M. T., Giovanetti M. D., Conceptual graphs for the analysis and generation of
sentences, IBM journal of research & development, 1988, 32:2, 251-267
[19] Welty C. A. An integrated Representation for Software Development and Discovery, PhD thesis,
Rensselaer Polytechnic Institute, 1996. Available from:
http://www.cs.vassar.edu/faculty/welty/papers/phd/
[20] Zowghi D., Gervasi V., McRae A., Using default reasoning to discover inconsistencies in natural lan-
guage requirements, Proceedings of the eighth Asia-pacific software engineering conference
(APSEC’01). Available from:
http://csdl2.computer.org/dl/proceedings/apsec/2001/1408/00/14080133.pdf

646