l Using NaWmMan uage

~ Processing f m Proe?uce
Eli Goldberg and Norbert Driedger, Atmospheric Environment Service, Environment Canada
1 Richard 1. Kittredge, University of Montreal

and the United Kingdom operate supercom-

puters that produce daily simulations of
USING RULES AND NATURAL-LANGUAGE
global weather pattems. Weather forecasters GENERATION, FOG CONVERTS WEATHER M I P S
take the output from these simulation mod-
els, interpret it, and translate it into a “prod- INTO FORECAST TEXT. THE NATURAL-LANGUAGE
uct line” of weather forecasts. Users of these ZSSUES W O L V E D ARE RELEVANT TO ANYOhE
forecasts include not only the general pub-
lic, but also specialized interests in agricul- DESlGNlNG A SlMlLAR SYSTEM.
ture. industry, transportation, and the mili-
tary. The content and structure of these
forecasts depend on the type of forecast, but vices to examine how and where human fore- that lets them develop graphical forecasts,
most are text messages that use a compressed casters add value to forecast production. which then drive a variety of applications,
(“telegraphic”) language. There are conflicting views about the virtue including automatically generating graphi-
One of the problems of operating a fore- of replacing manually produced forecasts cal and textual forecasts. The FPA uses so-
cast service is that forecasting is labor- with computer-generated products. At stake phisticated graphical display and manipula-
intensive, requiring advanced education and is the national forecast system’s account- tion techniques to let forecasters concentrate
training. Also, advances in meteorology and ability for public health and safety in haz- on the day’s meteorological problems, and
communications technology have conspired ardous weather. Nevertheless, forecasters expert system techniques to transform the
to put an ever-growing amount of raw infor- would like to automate the job’s more rou- graphical representations into the conceptual
mation at the forecaster’s fingertips. End- tine aspects so they can concentrate on sci- content of text forecasts. Natural-language-
users also need more sophisticated weather entific questions. generation techniques convert these meteo-
information, so weather services have faced The Forecast Production Assistant was de- rological concepts to the final text.
increasingly complex demands for more spe- veloped with just these issues in mind.’ The The Forecast Generator (FOG) was the
cialized products. The meteorological com- basic assumption guiding its development is first application developed on the FPA. FOG
munity has already increased productivity by that forecasters add value to the forecast is a bilingual (English and French) report
automating the gathering and distribution of process, as a result of their human pattern- generator that produces routine and special-
information, so further gains must come recognition skills, coupled with human judg- purpose forecasts directly from the FPAs
mostly from automating forecast production. ment and decision-making skills in the face graphical weather depictions. It uses a pow-
Improved numerical weather prediction of uncertainty. The FPA capitalizes on these erful and extensive language model, and al-
(NWP) simulations have led weather ser- strengths by giving forecasters a workstation lows a variety of text-planning mechanisms.
-
APRIL 1994 0885-9000/94/$4.00 0 1994 IEEE 45
 LAKE SIMCOE.
WINDS LIGHT GENERALLY SOUTH. FAIR.
NORTH CHANNEL.
WINDS SOUTHWEST 10 KNOTS BECOMING WEST 15 NEAR NOON. FAIR EXCEPT CHANCE
OF AN EVENING THUNDERSTORM. WAVES NEAR 0.5 METRE. current during normal operations. As soon as
GEORGIAN BAY. the FPA produces the time-interpolated
TOBERMORY TO MEAFORD.. WINDS VARIABLE 10 KNOTS. FAIR EXCEPT PATCHES OF
charts, FOG can generate forecasts.
HAZE AND MIST. WAVES LESS THAN 0.5 METRE.
MEAFORD TO KILLARNEY.. WINDS SOUTH 1 0 KNOTS BECOMING SOUTHWEST THIS One to four IatitudeAongitude pairs repre-
AFTERNOON. FAIR EXCEPT PATCHES OF HAZE AN0 MIST. WAVES LESS THAN 0.5 METRE. sent each forecast area (such as Belle Isle in
Figure 1 c). Local forecasters preassign these
sample points when the program is set up for
WINDS NORTHWEST 15 DIMINISHING TO LIGHT MONDAY AFTERNOON. CLOUDY WITH their area. FOG invokes a Sampler program
OCCASIONAL LIGHT SNOW. FOG PATCHES. VISIBILITIES 2 TO 5 NM IN SNOW. at each point to obtain the values of the re-
quired meteorological fields. The Sampler
accepts a request for a given parameter (such
as surface air temperature) at the sample
BELLE ISLE
NORTHEAST GULF point and scans the interpolated charts for the
NORTHEAST COAST. field value at that point. It then returns the
GALE WARNING IN BELLE ISLE AND NORTHEAST GULF ISSUED. time series of the values to FOG.
GALE WARNING IN NORTHEAST COAST CONTINUED.
FREEZING SPRAY WARNING CONTINUED.
WINDS SOUTHWEST 15 TO 20 KNOTS INCREASING TO WEST GALES 35 NEAR NOON Conceptual (mctcorological) processing.
FRIDAY. SNOW BEGINNING OVERNIGHTTHEN ENDING FRIDAY AFTERNOON. One marine forccast product uses 76 differ-
VISIBILITY FAIR IN SNOW. OCCASIONAL FREEZING SPRAY OVER OPEN WATER. ent sample points. With interpolated charts
TEMPERATURES MINUS 1 4 TO MINUS IO. available hourly, and a forecast lasting 48
OUTLOOK FOR SATURDAY. .. GALE FORCE WEST WINDS BECOMING STRONG TO GALE hours, each time series will contain at least 40
FORCE EASTERLIES.
data values. The forecast requires sampling
five differerit fields (wind, temperature, vis-
ibility, weather, and freezing spray), so h e
BLIZZARD WARNING ENDED. raw forecast may contain more than 15,000
TONIGHT.. SNOW AND BLOWING SNOW. WINDS SOUTHWESTERLY 35 KM/H OCCASIONALLY
GUSTING TO 50. LOW NEAR MINUS 30. data values (76 points, each having 5 time
THURSDAY.. MAINLY CLOUDY. WINDS SOUTHEASTERLY 30. VERY HIGH WINDCHILLS. series o f 4 0 points).
TEMPERATURE NEAR MINUS 30. The first part of this processing stage re-
FRIDAY.. CLOUDY PERIODS. WINDS WESTERLY 25. LOW NEAR MINUS 32. HIGH NEAR duces this large amount of information to a
MINUS 28. small number of significant events. Objec-
PROBABILITY OF PRECIPITATION IN PERCENT 100 TONIGHT. 3 0 THURSDAY AND 20 FRIDAY.
tive guidelines exist for this, but forecasters
exercise considerable professional judgment
over which information they should mention
Figure 1. Forecast text messages: (a) Great lakes marine; (b) Arctic marine; ( 0 Atlantic marine; (d) public. and which i s insignificant. The number of
events finally realized depends on the
weather situation, but forecasters typically
aim for five to seven paragraphs, with ap-
proximately six events per paragraph
It is used daily in routine forecast operations, Extracting data from weather charts. Be- (roughly 35 to 45 events per forecast issue).
producing forecast texts that are indistin- fore invoking FOG, the forecaster must de- The Atlantic marine forecast in Figure 1 c, for
guishable from texts written by human fore- velop a time series of weather depiction example, contains eight events (if we include
casters (see Figure 1; we have omitted the charts on the FPA. This series normally ex- descriptions of static conditions). The thresh-
corresponding French texts). tends for 48 to 60 hours, with charts avail- old for what constitutes a significant event
When developing FOG, we dealt with able hourly. (The local weather center con- shifts according to the variety of weather in-
such challenges as describing an output lan- figures time spans and increments according formation available to the forecaster. On
guage, coherently ordering information, to its operational requirements.) quiet days, relatively minor weather varia-
choosing exact terminology, generating text Of course, it would not be practical to re- tions are treated distinctly, while on a “bad
in several languages, and selecting a lan- quire forecasters to continually monitor and weather” day, such fluctuations are less sig-
guage model. Our experience should help maintain so many charts. Instead, forecast- nificant and can be safely discarded
anyone developing a natural-language report ers maintain a sequence of charts at longer In FOG we refer to this process as rime
generator for a practical application. intervals on the FPA, and the FPA uses com- and space merging. Much of the data ma-
puter animation to produce intermediate nipulation during this process is govemed by
charts at whatever interval is required. For rules that vary according to forecast type
FOG’S components instance, a 48-hour sequence in 12-hour in- (marine or public). This stage also handles
crements involves only five charts. Nor- most of the regional variations.
FOG converts data to text in three stages: mally, forecasters prepare a new 48-hour The first step is to determine which fore-
data extraction, conceptual processing, and prognosis when they receive new NWPguid- cast areas to group together by comparing
linguistic processing (see Figure 2 ) . ance (twice daily). The earlier charts are kept the time series from the various sample

46
 --_I_-- - ._
!
points. For marine forecasts, the primary cri-
!
teria are wind speeds and directions; for pub- I
lic forecasts, cloud cover and precipitation
are more significant. If the time series are I I
sufficiently similar, then FOG merges the
areas represented by the sample points. We
consider the merged area to have multiple
I I
sample points, and at the end of space merg- I
ing, we obtain representative time series I
from the multiple samples. FOG then classi-
fies this data and time-merges it to identify
significant changes in weather conditions.
Since this event data captures the fore-
cast’s physical content without the wording, English text French text
it can serve other purposes. Quality control ........._ _..._
. ....... ......._.
..._..
. ._...

schemes (what meteorologists refer to as

“forecast verification”) can use it to auto- Figure 2. Schematic diagram of FOG.
matically monitor the performance of the
forecast production system.
Table 1 illustrates the process for a marine
forecast. From the graphical weather depic- Table 1. Time-mergingdata to concepts.
tions, FOG samples hourly wind speeds and
directions for a point. The data represent sev- SAMPLEDATA CONCEPTS
eral areas, depending on the results of space TIME WINO DIRECTION WINO SPEED WINO OlRECTlON WINO SPEED
merging. FOG classifies the data into wind
direction and speed, and then time-merges 6 a.m. 223 13 southwest 15-20
them (as indicated by the arrows in the table). 7 a.m. 235 17 U U
9 a.m. 231 21 U U
Sampled data may fall into several cate-
gories, so time merging incorporates a vari-
ety of rules to determine the most economi-
cal transitions. Forecasters try to minimize 9 p.m. 280 12 (west) light
the length of a forecast text, without unduly
10 p.m. 307 11 (northwest) U
11 p.m. 182 8 (south) U
compromising the information. For exam- Midnight 246 10 (southwest) U
-~ - ~- -~
ple, a wind direction of 235 degrees could be ~ ~ ~ ~ ~ ~~ ~~

classified as southwest, westerly, or Text Winds southwest 15 to 20 knots diminishing to light late this evening
southerly. The choice should lead to the
fewest transitions, and thus the fewest con-
cepts, in the final results.
The forecast text must give wind speeds ucts. The expert system will therefore return ily control a text product’s content without
to the nearest 5 knots, so a 13-knot wind is different conceptual results for different tar- recourse to the weather maps. In Table 1,
expressed as 15 knots, and a 21-knot wind as get products, even when the input sample the input sample data consists of 19 wind
20 knots. Wind speed ranges can be used, but data is the same. directions and 19 wind speeds (hourly val-
the practice is restricted. For instance, a 5- ues from 6 a.m. to midnight). The expert
knot wind-speed range is permissible only Graphical bulletin editor. Although the system reduced these to two wind events,
during the forecast’s first 18 to 24 hours. Be- weather maps represent the forecasters’ best which the forecaster can adjust easily
yond that time, the unpredictability of efforts, they must occasionally overrule that through the editor.
weather mandates that wind speeds be re- objective data. For example, when forecast- But this approach presents the meteoro-
garded as accurate only to k5 knots. Conse- ers are uncertain about issuing or canceling logical events out of context. To properly as-
quently, a statement of “15 to 20 knots” for weather warnings, they often must wait for sess the combinations of events that go into
the forecast’s second day would imply a pos- confirmation before changing the warning a forecast, the forecaster uses geographical
sible range of 10 to 25 knots. That much vari- status. They can do this using FOG’Sgraph- displays and applies pattem recognition skills
ability is not permitted, so the 15- to 20-knot ical bulletin editor; if necessary, they can also and knowledge of atmospheric physics. The
range in Table 1 could occur only during the use the editor to overrule the expert system FPA’s long-term goal is to refine the con-
early part of the forecast. Once the winds fall that performs time and space merging. ceptual expert system’s expertise so the edi-
below 13 knots, the wind speed is classified The graphical bulletin editor displays the tor is rarely (if ever) required. Until then, it
as light, making direction insignificant (“di- events produced by the conceptual expert is not always practical to change weather
minishing to light late this evening”). This system as interactive histograms. The fore- maps when a rapid response is required (as
rule does not apply to all forecast text prod- caster can adjust these with a mouse to eas- for forecast amendments).

APRIL 1994 ~
47
 The alternative to the editor would be to We first performed a distributional analy- the grammatical and conceptual statements
manually edit the forecast text, but we want sis of word use in a large corpus of forecast for each textual forecast product and check
to avoid this. Since the text products must be texts to group words into similarity classes. them against weather service standards,
issued in French and English, manual text We then used the sublanguage word classes to which document some of the accepted norms
editing would require editing in both lan- state the sentence patterns and relationships and preferred usages for forecasts. At this
guages. It would also interrupt the flow of needed for a sentence grammar. A secondary stage we also simplified the task by engi-
meteorological information and reintroduce analysis of the sentence types in whole fore- neering some rare events out of the design;
the forecast consistency problems that the casts led to a preliminary text grammar. this also promoted consistent and under-
FPA is designed to eliminate. The editor, We based the initial description of marine standable final texts.
however, can control consistency when the forecasts on an analysis of more than a mil- There was no corresponding corpus of
data is adjusted. lion words of archived (English) marine fore- French texts. In Canada, most weather fore-
casts. As part of this analysis, linguists rep- casts are produced in English, and a ma-
Linguistic processing. Once FOG has de- resented the syntactic structures of forecasts chine-translation system produces the French
termined the meteorological concepts, its lin- with Backus Naur Form (BNF) grammars text. Research and development for that sys-
guistic processor takes the unordered and un- used for programming language syntax. tem in the 1970s found that the information
structured concept set and outputs fluent They did not intend the BNF grammars to content of the forecasts in both languages
English and French texts according to pro- capture the context sensitivity of forecasts, was close enough to be considered identical
fessional style. There are two major process- for most purposes. Moreover, human trans-
ing stages: text planning and text realization. lations typically break the text into sentences
Text planning enriches the unstructured at the same points in both languages. We de-
concept set with additional derived concepts, cided to exploit these findings by deriving
and then organizes it into a sequence of sen- TO MEET THE NATURAL- texts in both languages from the same con-
tence-sized chunks. This determines the final LANGUAGE REQUlREMENTS tent representation, and by identically scop-
sentence order, but provides only an abstract ing English and French sentences. This
specification of each sentence’s content. The OF FOG’S LINGUISTIC helped reduce the requirement for an exten-
specification is an interlingual structure, sive corpus of French text, although we used
PROCESSOR, WE FIRST HAD
which is input for text realization. some professional French forecasts to clar-
During text realization, grammatical and TO DETERMhW THE TYPE ify specific issues.
lexical modules operate on each sentence
specification to derive a final written text for
OF TEXT THAT OUR SYSTEM Designing the text planner. While devel-
each language. The English and French re- WOULD OUTPUT. oping FOG for different types of forecasts,
alizers use the same general grammatical for- we have implemented two different text plan-
malism and processing substages, but act in- but only to delimit the set of permissible ner designs. The text planner for marine fore-
dependently. structures so forecasters could verify hy- casts relies on data salience to control the
potheses made by linguists about what is or order of sentences and help determine sen-
is not “sayable” in the forecasting domain. tence boundaries. Data salience is the rela-
Designing the linguistic (As a formal language, forecasts are not con- tive significance of different information
prwessor text-free. For example, in French sentences, types to the intended user. In Figure IC,the
nouns and their adjectives-which can eas- content follows a schema whose data
To meet the natural language requirements ily be noncontiguous-must agree in num- salience (starting with the most significant)
of FOG’S linguistic processor, we first had ber and gender.) is: forecast areas, waming status, wind fore-
to determine the type of text that our system We repeated the linguistic analysis for cast, weather forecast (for example, sky
would output. To generate texts that meet op- each forecast product to identify the lexical cover and precipitation), visibility, freezing
erational standards in a domain such as stock and inventory of syntactic structures spray, air temperature, and outlook winds. A
weather forecasting, we must describe the used naturally in human-composed texts. It text schema specifies the text’s global struc-
sublanguage (of English or another natural was easy to identify differences between ture, yet it allows variability in the syntactic
language) that experts use spontaneously to products and between regional variants of structures of the output sentences. It also al-
communicate in that domain.2 As Figure 1 the same product, and to state distinct gram- lows context sensitivity between the
shows, weather forecasts use a distinctive matical and lexical descriptions of those dif- schema’s linguistic structures. For example,
telegraphic style that is quite different from ferences. The analysis also provided an em- forecast sentences often have long-distance
standard English and French. The sentences pirical basis for building a conceptual syntactic and semantic dependencies: When
do not use tensed verbs, articles, or many apparatus that could mediate between raw the end of a paragraph mentions freezing
other words needed to communicate the meteorological facts and texts. This was due spray, as in Figure IC, the beginning must
same information in a more conversational to the fortunate fact that, in a situationally re- have a warning statement. Also, marine vis-
style. Consequently, existing English gram- stricted sublanguage, word classes and word- ibility and the conditions that reduce it are
mars are of little practical use, and we had to class syntax more clearly reflect domain se- normally mentioned in different sentences.
build a specialized sublanguage grammar for mantics than the whole language does. Data salience works well for planning the
forecasts in each language. Expert forecasters still needed to validate text of marine forecasts, but temporal order
~ ~

48 IEEE EXPERT
 change-in-precipitation ::=

(1) <precipitation> changing to < <simple-precip> I <mixed-precip> > <time> then

to<simple-precip> < <time> I <approximate-time> >
governs the public forecast. A public fore- (2) <precipitation> changing to < <simple-precip> I <mixed-precip> > <time> then
cast’s main body may be divided into time {possibly) to <simple-precip> <time>
periods, each organized according to salience (3) <precipitation> <initial-change> (<subsequent_changes>)
for the general public (see Figure Id). We de-
signed the public forecast text planner to re-
flect this alternative organization.
The marine and public forecast planners Choose BNF (1):
also differ in how they determine sentence
If the precipitation changes character then type, or type then character, and if the final state
scope and the specific conjunctions between is “certain”,
sentence clauses. The marine forecast plan- (If there is uncertainty with respect to the time of occurrence, then use
ner applies general principles to derive the <approximate-time> otherwise, use <time>.)
correct scoping and conjunctions. The pub-
Choose BNF (2):
lic forecast planner directly implements rules
of thumb provided by forecasters. These If the precipitation changes character then type, or type then character, and if the final state
rules, which may be slightly different for dif- is “uncertain”,
ferent weather offices, have evolved from an
Choose BNF (3) in all other cases.
effort at the Quebec Weather Center to im-
prove communication between computa- (b)
tional linguists and forecasters.
This work uses the Backus Naur Form
Figure 3. Text-planning rule for public forecast: [a) BNF for changing precipitation; (b) Correspondingobjective criteria.
grammars for public forecasts, coupling sen-
tence pattem choices with objective criteria
(or O C s , specialized knowledge about weather
forecast text composition, which is not repre-
sented el~ewhere).~ The criteria link specific ciples to the text planner. But for now, we ponent) serve both languages. Because of
BNFs to input data values (see Figure 3). need broader experience in planning well- local syntactic differences in the two lan-
We redesigned the text planner to exploit entrenched varieties of text. guages, however, we designed an abstract in-
this extended specification, so that meteo- terlingua to represent the lexical and deep
rologists can use BNFs to transcribe their Lexical choice and multilingualconcerns. structural similarities between correspond-
text-composition rules into a form easily ac- When designing FOG, we faced two major ing English and French texts. Text-planning
cessible to FOG’s developers. lexical choice issues: choosing profession- produces a string of interlingual sentence
The public forecast planner’s Prolog code ally acceptable terms that express all the re- structures, using interlingual lexical items,
now closely corresponds with the BNF-OC quired components of meaning in a language which FOG can then map directly onto the
rules. (However, the system rules recast (such as English), and generating texts in two deep structures (with appropriate lexemes)
BNFs as interlingual dependency structures.) separate languages from the same input data. of the corresponding sentences in either lan-
This has accelerated planner development by In English, many terms describe changes in guage. FOG applies separate language gram-
encouraging the involvement of meteorolo- marine winds: shifting, veering, backing, mars to those structures to derive the final
gists. Besides offering a clear and fast de- strengthening, decreasing, becoming, and so form texts. Technically speaking, the lexical
velopment path, the correspondence rules let on. FOG uses rules suggested by meteorolo- item changing to in Figure 3a is interlingual
regional forecast staff with little or no train- gists to select the correct term for the fore- and does not become English (or French)
ing in natural-language processing specify cast text. until the interlingual structure is mapped to
changes in forecast content and structure. Some weather situations require specific a specific language structure.
One potential disadvantage of this ap- technical terms to communicate or empha- A deep syntactic interlingua, while satis-
proach, however, is that by “short-circuiting” size hazardous conditions. In other situations, factory for English and French, would be in-
more general text-planning mechanisms, the forecasters may choose from several accept- appropriate for generating additional lan-
BNF-OC rules might make it harder to in- able terms to produce text that contains nat- guages whose semantic system or
troduce stylistic variation or adapt the plan- ural variations in terminology and is not communication style is different. Instead, a
ner for significantly different needs. Most stilted or redundant. FOG does not currently “deeper” semantic net or conceptual inter-
differences between current forecast text va- support this variability, but we could incor- lingua might be needed. FOG’s Meaning-
rieties originate in the text planner, and the porate it into FOG by introducing limited Text framework (which we’ll discuss later)
planner’s capacity and flexibility will be- randomized selection. provides for a semantic representation level,
come even more important as new sources As we’ve mentioned, English and French which could serve as the basis for a seman-
and types of meteorological information forecasts are similarly segmented into sen- tic interlingua, if required in the future.
(such as Doppler radar) stimulate demand for tences and clauses. Moreover, for each Eng-
new types of weather messages. One of this lish weather term there is usually a single ap- Selecting a language model for realization.
project’s long-term goals is to tailor forecasts propriate term in Canadian French, and vice Because the text realizer transforms the in-
to specific user needs, which will probably versa. Therefore, the lexical selection for terlingua into English and French text, each
require adding more general rhetorical prin- English can (with a parallel realization com- output language requires a separate grammar

-
APRIL 1994 49
 “southwest‘ A practical example of
linguistic processing
Let’s now look at how the planner and re-
“late”
1 0 ‘this“ alizer transform the conceptual content in
“light“ Table 1 into the output sentence “Winds
southwest 15to 20 knots diminishing to light
II late this evening.”

I Text planner. We’ll assume that the text

20“
planner has already determined the marine
ATTR forecast’s global structure (using data
I salience), and focus on planning individual
I sentences. The determination of conceptual
‘knot’
content has identified meaningful categories
~~ ~ ~

for discrete weather states (for example,

FigGe4.lnterlingG for concepts in Table 1. “light” winds). The planner builds concepts
to describe transitions between these states.
It then groups and structures the enriched
and dictionary (together constituting a lan- technology rather than “reinvent the wheel” conceptual content into sentence-sized
guage model). These must be built accord- (this is particularly important). It should also chunks, and outputs the interlingua. Table 1’s
ing to general principles (a theory), to im- provide easy long-term maintenance. content thus becomes the interlingua in Fig-
prove the chances of coherently meeting Finally, the model should support speech ure 4. Two wind events have been cast into a
diverse needs. When selecting a language synthesis. Many weather forecast texts are dependency tree that represents one sentence.
model, we considered several fact0rs.4.~ read into automatic telephone-answering However, under other conditions, such as a
First, the linguistic knowledge encoded in systems as an economical way to manage greater number of wind events, sentence
the grammar and dictionary should be as de- routine inquiries. This is tedious and time- scoping rules might produce several depen-
clarative as possible, easily separated from consuming for the forecast office staff, and dency trees (sentences).
the algorithms that process the rules and en- simple text-to-speech systems did not meet The interlingua’s nodes are labeled with
tries. Intertwining linguistic knowledge with operational requirements when we initially the names of abstract concepts that are point-
procedures for applying it to linguistic rep- developed FOG. ers to English and French lexical entries.
resentations produces a complex and het- After reviewing several well-known lan- Arcs connecting the nodes indicate the de-
erogeneous program that is difficult to main- guage theories that have been tested in com- pendency types. I and I1 represent a predi-
tain, use, or expand. putational models of English and French, we cate’s first and second arguments, and ATTR
Second, the language model should incor- decided that the Meaning e-> Text Theory is the attributive relation.
porate several grammatical representation (MTT) best met our requirements. Aside One node in Figure 4 is labeled “dimin-
levels or “strata” (for example, semantics, from MTT’s overall approach (see the side- ish.” The transition from “southwest 15 to
syntax, morphology, and phonology). Such bar), there were practical reasons for choos- 20” to “light winds” led the planner to choose
models can have simpler rules for relating ing it: There were already well-developed this concept. At the interlingual level, “di-
representations on one stratum with repre- descriptions of French and English in M7T; minish” denotes decreasing values of any
sentations on neighboring strata. As the num- MTT had already been applied in a real ap- measurable parameter. Similarly, the planner
ber of phenomena to be covered on each level plication;6and there was a ready pool of local created the subtree describing the transition
grows (with increasing linguistic complexity expertise in implementing MTT models in time (“late this evening”).
and grammar size), this greater modularity real-world applications. We developed a
keeps the complexity of individual rules man- complete representation of the weather fore- Text realizer. FOGStext realizer is a 5tream-
ageable, even though each pair of adjacent cast sublanguage using MTT-style depen- lined version of MTT. It uses five represen-
strata requires a separate set of mapping rules. dencies. Also, the text planner produces an tation levels: interlingua, two syntactic lev-
Third, the model should be sufficiently interlingua whose structure is closely related els, a morphological level, and the final text.
flexible (rich enough) to accommodate a va- to the text realizer’s MTT model. FOG obtains the deep syntactic represen-
riety of forecast types, and regional needs We could have used other language mod- tations (see Figures 5 and 6) by replacing the
and tastes. els, including simpler ones, to generate the abstract interlingual concepts (the node la-
Fourth, the underlying theory should have weather forecasts that FOG has produced so bels such as “wind” and “diminish” in Figure
been tested over a variety of languages, and far. However, we expect M?T’s advantages 4)with language-specific equivalents from
the existing descriptions of English and to increase as FOG generates more complex its lexicon. So, “diminish” becomes DI-
French should be as complete as possible and texts and provides output in more languages. MINISH (Figure 5) and DIMINUER (Fig-
readily accessible. We also expect MTT’s ease of software de- ure 6). The lexicon stores the rules that gov-
Fifth, the model should have been applied sign and maintenance to become crucial as the ern this replacement. DIMINISH’S lexical
previously; that is, it should use available number of textual forecast products increases. entry contains a rule that states that DIMIN-

50 IEEE EXPERT
 I/*“. ATTR
‘
,\DIMINISH v, change
SOUTHWEST OIoc
ISH can replace “diminish if the interlin- cn
gual node has “wind” or “gale” as dependent
I, and “wind,” “gale,” or the null node
“omega” as dependent II. However, if the de-
adv preposed
pendency tree involved temperatures, then
the above rule would not be satisfied, and the
“diminish” node would become the verb
LOWER (for example, “Temperatures low- II ’
ering to minus lo...’’), using the rules for that
lexical entry. 1
O*’num plur
The deep syntactic trees in Figures 5 and
6 are similar to the interlingua in Figure 4, ATTR ’
but their nodes are labeled with full lexemes,
and each lexeme has its syntactic class and 1
‘”’‘meas unit
morphosyntactic features (cn for common
~- -~
noun, prep for preposition, and so on). ~~~ ~ ~~ ~

Figure 5. English deep syntactic representotion for interlinguo in Figure 4.

The transition from deep syntax to surface
syntax (not shown) may produce new nodes
(auxiliary lexemes) and new dependency re-
lations. An example is the treatment of DI-
MINISH compared to BECOMING. The
auxiliary lexeme TO augments DIMINISH:
“Winds southwest 15 to 20 knots becom-
ing southeast 15 late this evening.”
“Winds southwest 15 to 20 knots dimin-
ishing to light late this evening.”
The transformation from surface syntac-
tic representation to morphological repre-
sentation involves linearizing the surface
syntactic tree and propagating grammatical
features from head words to dependents for
gender agreement in French and number
agreement in both languages.
The realizer then produces text using in- I

formation from the lexicon. It takes each lex- ATTR ~

eme’s surface form directly from the lexicon, 1

or computes it according to the root form and ‘‘‘0“’ meas unit masc
the morphological features and description. _ _ _ _ _ _ ~ _ _ _ _ _ _ _ ~ .
~~~ _ _ _
~~ _ _ _ ~ _ ~

Figure 6. French deep syntactic representationfor interlingua in Figure 4.

The benefits of natural=

language processing
FOG has produced Atlantic marine fore- language-processing techniques. model, but more difficult with a slot-filler
casts since the winter of 1991-92.We have Is this cost justified compared to that of approach.
also implemented FOG on four other marine more traditional Fortran slot-filler tech- Natural-language processing also facili-
forecast types and one public forecast. All niques? (See the sidebar on other ap- tates extensions to other types of linguistic
share a common architecture and nearly proaches.) That depends on whether you applications. For example, researchers are
identical realization components despite want to simplify the output language (slot- homing in on ways to generate better speech
considerable structural variations in the final filler techniques require this), and what flex- from deep language representations (often
text. Natural-language processing has been ibility you need for future changes. A good referred to as concept-to-speech systems). A
essential to producing forecast text that case can be made for natural-language pro- shallow language model would not have the
meets current operational requirements, cessing, based on the maintenance and mod- “hooks” for integrating good speech synthe-
while having the flexibility to change and ification portions of the software life cycle. sis, but FOG requires only extending the lin-
grow with the forecast system. The only in- For instance, changing how statement con- guistic model through a phonological level.
cremental cost has been the investment to junctions are made in single sentences in a This would let the conceptual input and syn-
train staff to use Prolog and natural- variety of cases is straightforward using our tactic representation control variations in the

APRIL 1994 51
 Meaningedext Theory
Meaning<->Text Theory is based on a large 4,5, and 6 in the article). Aconstituent of the mappings that facilitate software design and
structured dictionary (lexicon) and a set of sentence is a word and all the words that hang maintenance. Second, the systemic lexicon
correspondence rules that provide mappings from it. Because the arc labels carry additional does not provide M l T s features for organiz-
between several representation levels. information, dependency trees require ahout ing lexical relations. In MTT, a variety of lexi-
Some linguistic theories emphasize the lexi- half the nodes of phrase structure trees. h4lT cal functions let general rules rephrase a given
con’s role as a linguistic knowledge repository, uses dependency trees whose nodes have no sentence in many ways (semantic paraphrases)
while others emphasize information in the horizontal ordering. It represents linear order when stylistic variation is desirable.
grammatical rules at the lexicon’s expense. between words only at the morphological level.
During generation, where much of the seman- This direct relational linkage between Applying MTT to another domain. The LFS
tic information partially determines lexical words favors a lexically oriented grammar; system for providing bilingual statistical
choices before determining the major syntactic that is, one in which small lexical sets or even reports on labor force surveys also uses MTT.6
choices, a lexically oriented model is more ad- single words closely control syntactic rules. LFS uses a semantic net representation of sen-
vantageous than a syntactically oriented This is particularly important for representing tences as input for realization, and theme and
model. MTT gives the lexicon enough priority restricted lexical combinations, including id- rheme constraints help optimize lexical and
to allow powerful synthesis techniques. iomatic expressions. MTT’s explanatory- syntactic choices. The LFS interlingua is much
To describe the generation of written text, combinatory dictionary has special provisions deeper than FOG’Sdeep-syntactic interlingua,
MTT provides seven representation levels: for handling word collocations. so it can introduce more subtle and natural-
one semantic, two syntactic, two morphologi- sounding semantic differences between Eng-
cal, and two phonological levels. (MTT mod- MTT and other language models. Text gen- lish and French sentences, which are required
els for English and French typically use five: eration in a single language (not to mention for the more complex LFS sublanguages.
semantic, deep syntactic, surface syntactic, two) places heavy demands on a language Eventually, we may adopt such an interlingua
morphological, and the final written form.) model. The model must account for all the as we extend FOG to cover new languages and
MTT is oriented toward text synthesis (mean- steps in the complex mapping from meaning new text varieties.
ing-to-text) rather than text analysis (text-to- representations to text. Traditional transforma-
meaning), although it was formulated to pro- tional grammars cannot do this. As of the late
vide bidirectional (reversible) models. 1980s, only three kinds of linguistic models 1. LA. Mel’cuk, “Meaning-Text Models: A
M’lT’s strata allow for expansion and gen- had been implemented extensively for genera- Recent Trend in Soviet Linguistics,” Ann.
eralization while maintaining clarity and sim- tion: Meaning-Text models,’,’ tree-adjoining Rev. ofAnthropology, Vol. 10, 1981, pp.
plicity for individual rules. Also, the two syn- grammars (TAGs) as implemented in 27-62.
tactic strata (the deep syntactic and surface Mumble,3 and systemic grammars!.’ 2. L. Iordanskaja, R. Kittredge, and A. Pol-
syntactic levels) facilitate the treatment of dif- MTT and other Meaning-Text models have gukre, “Lexical Selection and Paraphrase
ferences between French and English. We did several advantages over TAGs. MTT treats the in a Meaning-Text Generation Model,” in
not initially know if we needed a full language full multistage mapping from semantics to Natural-Language Generation in Artifi-
theory with several representation levels to phonology, whereas TAGs mostly handle a full cial Intelligence and Computational Lin-
generate the limited number of forms in fore- range of syntactic phenomena and relate them guistics, C. Paris, W. Swartout, and W.
casts. But we decided early in the project to to a morphological or textual level. MTT in- Mann, eds., Kluwer Academic Publish-
implement a well-developed theory to provide cludes a semantic layer (for more complex ers, Boston, 1991, pp. 293-312.
a solid reference model for later extensions forecasts and dissimilar languages), theme and 3. M. Meteer et al., “Mumble-86: Design
and modifications. rheme modeling (for speech synthesis), and and Implementation,” Tech. Report 87-
Linguistic models typically represent syn- lexical functions (for paraphrasing); TAGs do 87, Dept. of Computer and Information
Science, Univ. of Mass., Amherst, Mass.,
tax as phrase structure trees or dependency not. Although FOG currently uses lexical 1987.
trees. Phrase structure trees represent relations functions in relatively minor ways, future ex-
between sentence constituents. Each node is tensions (such as for the generation of weather 4. M.A.K. Halliday, An Introduction to
Functional Grammar, Edward Amold,
labeled with a constituent name (for example, synopses) will make heavier demands. London, 1985.
a noun phrase) and is linked vertically by un- Like MTT, the systemic grammar of the
labeled arcs to its immediate constituents (de- Penman text-generation system covers the full 5. C.W. Mann and C. Matthiessen, “Nigel:
A Systemic Grammar for Text
terminer, adjective, noun, and so on); lexemes transition from semantics to ph~nology.~ It Generation,” Tech. Report RR-83-105,
(dictionary words) appear at the leaves. Hori- goes beyond MTT by integrating pragmatic Information Sciences Inst., Univ. of
zontal ordering of the tree’s node labels issues such as social dimensions of language Southem Calif., Los Angeles, 1983.
reflects linear order among co-constituents. use and nonliteral meaning. However, this po-
6. L. Iordanskaja et al., “Generation of Ex-
In contrast, all dependency tree nodes and tential advantage is of little use in the weather tended Bilingual Statistical Reports,”
leaves are lexical items, linked by oriented de- domain, and systemic approaches are deficient Proc. Coling 92, 15th Int’l Con$ Compu-
pendency relations. The grammatical relations for FOG in two important areas. First, a sys- tational Linguistics, Assoc. for Computa-
between words are explicitly expressed by la- temic grammar does not lend itself to the sim- tional Linguistics, Morristown, N.J.,
bels on the arcs linking the words (see Figures ple, intuitive modularization of level-to-level 1992, pp. 1019-1023.

prosody of spoken output (the pitch, accent, Recent experiments with PHOG, an ex- additions to existing rules, lets PHOG gener-
and timing), thus allowing more economical tended phonological model for FOG, have ate enriched output strings that can be fed to
automatic telephone answering systems to shown that we may achieve good rule-driven a speech synthesizer. To fully realize this po-
handle the millions of calls received each prosody in the near future. The addition of a tential we need a synthesizer that accepts
year for weather forecast information. phonological representation level, plus small high-level prosodic specifications. Commer-

51 IEEE EXPERT
 limitations to previous approaches
More than twenty years ago, advances in computer-generated text so forecaster inter- were written in Fortran, and some are now
numerical weather prediction stimulated inter- vention would be unnecessary. However, being rewritten in C. Experience has shown
est in computer-worded weather forecasts. NWP still has difficulty predicting actual sur- that this software is difficult to maintain. This
Since then, a number of difficulties have pre- face weather conditions, so this type of com- has hampered the testing and implementation
vented widescale application of techniques for puter-worded text is used most often as guid- of the software and has made it difficult to up-
wording forecasts. ance or advice for the meteorologist instead date the programs for changing user require-
One problem has been the human of information for the public. Consequently ments. This is a critical factor. Although
interface. Initial forecast data usually comes the computer-worded forecast adds to the in- Canadian textual forecast products fall into
from central NWP atmosphere simulations. formation overload instead of solving the several common broad categories (marine
Until recently, workstations with the graphi- problem. forecasts, public forecasts, and so on), each
cal interfaces needed to conveniently visual- The complexity and brittleness of software category contains many regional variations.
ize and modify meteorological fields were developed to produce computer-worded fore- Also, content, structure, and terminology tend
not available. Consequently, to use computer- casts is a more severe impediment. Earlier to vary with time, albeit slowly. To succeed, a
worded forecast systems, meteorologists Canadian and American schemes resemble system must address variations between fore-
were forced to work with large tables of nu- “slot-filler” techniques, which closely link cast types, variations between geographical
merical data that were neither efficient nor words and phrases to data values and the rules regions in a forecast type, and gradually
effective. This motivated attempts to perfect that govem their use. Most of these programs changing requirements.

cia1 synthesizers are only now beginning to References meteorological systems from the University of
accept good prosody specifications. In the Guelph in 1981, his MSc in theoretical physics
1. R. Paterson et al., “The Forecast Production from the University of Western Ontario in 1970,
meantime, we need a more complete linguis- Assistant,” Proc. Fourth AESKMOS Work- and his BSc in physics from the University of Cal-
tic description of the prosody rules required shop on Operational Meteorology, Canadian gary. He can be reached at goldberge@aestor.
for voice output of each text forecast product. Meteorological and Oceanographic Soc., dots.doe.ca
Toronto, 1992, pp. 262-269.
2. R. Kittredge and J. Lerhberger, eds., Sublan-
T H E WORKPLACE AUTOMATION guaget Studies of Language in Restricted Se-
mantic Domains, de Gruyter, Berlin, 1982. Richard Kittredge is profes-
made possible by the FPA and FOG lets fore- sor of linguistics at the
casters exchange the task of text composi- 3. R. Kittredge, T. Korelsky, and 0. Rambow, University of Montreal,
tion for that of preparing more detailed “On the Need for Domain Communication where he previously di-
graphical weather depictions. The result is Knowledge,” Computational Intelligence, rected the TAUM ma-
Vol. 7, No. 4,Nov. 1991, pp. 305-314. chine-translation project,
that forecasters can accomplish more with
4. A. Polgukre, “Grammatical and Lexical For- and is president of CoGen-
the same effort. One site has added an addi- Tex, a business specializ-
tional forecast product to its production suite: malisms in FOG-89,” intemal report, Atmos-
pheric Environment Service, Environment ing in text generation. His
it now routinely produces three text forecasts technical interests are mul-
Canada, Downsview, Ontario, Canada, 1989.
instead of two. Another has maintained the tilingual text generation,
5. R. Kittredge and A. Polguere, “Dependency sublanguage analysis, and machine language. He
same set of products, but can extend the fore- received his PhD in formal linguistics at the Uni-
Grammars for Bilingual Text Generation: In-
cast’s period to include the 36- to 60-hour side FOG’s Stratificational Models,” Proc. versity of Pennsylvania in 1969, and his BA in
time frame. Manually produced forecasts Int’l Con$ Current Issues in Computational mathematics from Swarthmore College in 1963.
provided weather conditions for today with Linguistics, Universiti Sains Malaysia, He is a member of the Association for Computa-
an outlook for tomorrow; forecasts produced Penang, Malaysia, 1991, pp. 318-330. tional Linguistics, AAAI, CSCSI, ACM SIG Art,
and IFIP WG 12.4. He can be reached at the Dept.
on the FPA provide information on today and 6. L. Iordanskaja, R. Kittredge, and A. Polguere, of Linguistics and Translation, Univ. of Montreal,
tomorrow with an outlook for the next day. “Lexical Selection and Paraphrase in a Mean- c.p. 6128, station A, Montreal, Canada H3C 3J7.
We anticipate more significant savings as ing-Text Generation Model,” in Natural-
Language Generation in Artificial Intelli-
soon as FOG can produce other varieties of gence and Computational Linguistics, C .
text. To this end, we are making FOG’s pub- Paris, W. Swartout, and W. Mann, eds.,
lic forecast implementation operational as Kluwer Academic Publishers, Boston, 1991, Norbert Driedger is a com-
pp. 293-3 12. puter programmer with the
quickly as possible. Meteorological Research
Branch of the Atmos-
Eli Goldberg is a research pheric Environment Ser-
meteorologist with the vice of Canada. His tech-
Ac knowledgments Meteorological Research nical interests include
Branch of the Atmospheric pattern recognition, liter-
We thank Alain Polgukre for his energy and cre- Environment Service of ate programming, and AI-
ativityin implementingFOG, and Myung-Hee Kim Canada, where he devel- related languages. He re-
for her valuable technical assistance in maintain- ops computer applications ceived his BSc in applied
ing and upgradingit. We also thankTanya Korelsky, to support weather fore- physics from the University of Waterloo, Canada.
Owen Rambow, and Ehud Reiter for their substan- casting operations. His He can be reached at King Radar Facility, Envi-
tive comments on this work. Finally, we thank three primary focus is the appli- ronment Canada, 14780 Jane St., R.R. #1, King
anonymous reviewers whose comments and sug- cation of AI to weather City, Ontario, Canada LOG 1KO; Internet
gestions helped focus and clarify this article. forecasting. He obtained his PhD in bydro- ndriedger @rpn.aes.doe.ca

APRIL 1994 53