Jump to content

Natural language generation: Difference between revisions

From Wikipedia, the free encyclopedia
Browse history interactively
← Previous editNext edit →
Content deleted Content added
VisualWikitext
Ereiter (talk | contribs)
74 edits
Removed online link to pollen demo, which hasn't worked for at least a year
Ereiter (talk | contribs)
74 edits
update "recent weather forecast system" from 10-year old ARNS to 2014 UK Met system
Line 58: Line 58:


==Applications==
==Applications==
NLG systems effectively generate jokes (see [[computational humor]]), but from a commercial perspective, the most successful NLG applications
The popular media has paid the most attention to NLG systems which generate jokes (see [[computational humor]]), but from a commercial perspective, the most successful NLG applications
have been ''data-to-text'' systems which generate textual summaries of databases and data sets; these
have been ''data-to-text'' systems which generate textual summaries of databases and data sets; these
systems usually perform [[data analysis]] as well as text generation. In particular, several systems have
systems usually perform [[data analysis]] as well as text generation. In particular, several systems have
Line 69: Line 69:
,<ref>{{cite journal |author=Reiter E, Sripada S, Hunter J, Yu J, Davy I |title=Choosing Words in Computer-Generated Weather Forecasts |journal=Artificial Intelligence |volume=167 |issue= |pages=137–69 |year=2005 |doi=10.1016/j.artint.2005.06.006}}</ref> and a demonstration that statistical techniques
,<ref>{{cite journal |author=Reiter E, Sripada S, Hunter J, Yu J, Davy I |title=Choosing Words in Computer-Generated Weather Forecasts |journal=Artificial Intelligence |volume=167 |issue= |pages=137–69 |year=2005 |doi=10.1016/j.artint.2005.06.006}}</ref> and a demonstration that statistical techniques
could be used to generate high-quality weather forecasts.<ref>{{cite journal |author=Belz A |title=Automatic Generation of Weather Forecast Texts Using Comprehensive Probabilistic Generation-Space Models |journal=Natural Language Engineering |volume=14 |pages=431–55 |year=2008 |doi=10.1017/s1351324907004664}}</ref>
could be used to generate high-quality weather forecasts.<ref>{{cite journal |author=Belz A |title=Automatic Generation of Weather Forecast Texts Using Comprehensive Probabilistic Generation-Space Models |journal=Natural Language Engineering |volume=14 |pages=431–55 |year=2008 |doi=10.1017/s1351324907004664}}</ref>
Recent applications include the [[Met_Office | UK Met Office's]] [http://www.metoffice.gov.uk/public/weather/forecast-data2text text-enhanced forecast] <ref>S Sripada, N Burnett, R Turner, J Mastin, D Evans(2014). [http://www.aclweb.org/anthology/W/W14/W14-4401.pdf Generating A Case Study: NLG meeting Weather Industry Demand for Quality and Quantity of Textual Weather Forecasts.] ''Proceedings of INLG 2014''</ref>
Recent applications include the [http://www.cogentex.com/solutions/arns/index.shtml ARNS system] used to summarise
conditions in US ports.


In the 1990s there was interest in using NLG to summarise financial and business data. For example the SPOTLIGHT system developed at [[A.C. Nielsen]] automatically generated readable English text based on the analysis of large amounts of retail sales data.<ref>{{cite book |last1=Anand |first1=Tej |last2=Kahn |first2=Gary |chapter=Making Sense of Gigabytes: A System for Knowledge-Based Market Analysis |chapterurl=http://www.aaai.org/Papers/IAAI/1992/IAAI92-006.pdf |editor=Klahr, Philip; Scott, A. F. |title=Innovative applications of artificial intelligence 4: proceedings of the IAAI-92 Conference |publisher=AAAI Press |location=Menlo Park, Calif |year=1992 |pages=57–70 |isbn=0-262-69155-8 }}</ref>
In the 1990s there was interest in using NLG to summarise financial and business data. For example the SPOTLIGHT system developed at [[A.C. Nielsen]] automatically generated readable English text based on the analysis of large amounts of retail sales data.<ref>{{cite book |last1=Anand |first1=Tej |last2=Kahn |first2=Gary |chapter=Making Sense of Gigabytes: A System for Knowledge-Based Market Analysis |chapterurl=http://www.aaai.org/Papers/IAAI/1992/IAAI92-006.pdf |editor=Klahr, Philip; Scott, A. F. |title=Innovative applications of artificial intelligence 4: proceedings of the IAAI-92 Conference |publisher=AAAI Press |location=Menlo Park, Calif |year=1992 |pages=57–70 |isbn=0-262-69155-8 }}</ref>

Revision as of 15:49, 10 October 2014

Natural Language Generation (NLG) is the natural language processing task of generating natural language from a machine representation system such as a knowledge base or a logical form. Psycholinguists prefer the term language production when such formal representations are interpreted as models for mental representations.

It could be said an NLG system is like a translator that converts a computer based representation into a natural language representation. However, the methods to produce the final language are different from those of a compiler due to the inherent expressivity of natural languages.

NLG may be viewed as the opposite of natural language understanding: whereas in natural language understanding the system needs to disambiguate the input sentence to produce the machine representation language, in NLG the system needs to make decisions about how to put a concept into words.

Simple examples are systems that generate form letters. These do not typically involve grammar rules, but may generate a letter to a consumer, e.g. stating that a credit card spending limit was reached. More complex NLG systems dynamically create texts to meet a communicative goal. As in other areas of natural language processing, this can be done using either explicit models of language (e.g., grammars) and the domain, or using statistical models derived by analysing human-written texts.

Example

The Pollen Forecast for Scotland system [1] is a simple example of an NLG system. This system takes as input six numbers, which give predicted pollen levels in different parts of Scotland. From these numbers, the system generates a short textual summary of pollen levels as its output.

For example, using the historical data for 1-July-2005, the software produces

Grass pollen levels for Friday have increased from the moderate to high levels of yesterday with values of around 6 to 7 across most parts of the country. However, in Northern areas, pollen levels will be moderate with values of 4.

In contrast, the actual forecast (written by a human meteorologist) from this data was

Pollen counts are expected to remain high at level 6 over most of Scotland, and even level 7 in the south east. The only relief is in the Northern Isles and far northeast of mainland Scotland with medium levels of pollen count.

Comparing these two illustrates some of the choices that NLG systems must make; these are further discussed below.

Stages

The process to generate text can be as simple as keeping a list of canned text that is copied and pasted, possibly linked with some glue text. The results may be satisfactory in simple domains such as horoscope machines or generators of personalised business letters. However, a sophisticated NLG system needs to include stages of planning and merging of information to enable the generation of text that looks natural and does not become repetitive. The typical stages of natural language generation, as proposed by Dale and Reiter,[2] are:

Content determination: Deciding what information to mention in the text. For instance, in the pollen example above, deciding whether to explicitly mention that pollen level is 7 in the south east.

Document structuring: Overall organisation of the information to convey. For example, deciding to describe the areas with high pollen levels first, instead of the areas with low pollen levels.

Aggregation: Merging of similar sentences to improve readability and naturalness. For instance, merging the two sentences Grass pollen levels for Friday have increased from the moderate to high levels of yesterday and Grass pollen levels will be around 6 to 7 across most parts of the country into the single sentence Grass pollen levels for Friday have increased from the moderate to high levels of yesterday with values of around 6 to 7 across most parts of the country.

Lexical choice: Putting words to the concepts. For example, deciding whether medium or moderate should be used when describing a pollen level of 4.

Referring expression generation: Creating referring expressions that identify objects and regions. For example, deciding to use in the Northern Isles and far northeast of mainland Scotland to refer to a certain region in Scotland. This task also includes making decisions about pronouns and other types of anaphora.

Realisation: Creating the actual text, which should be correct according to the rules of syntax, morphology, and orthography. For example, using will be for the future tense of to be.

Applications

The popular media has paid the most attention to NLG systems which generate jokes (see computational humor), but from a commercial perspective, the most successful NLG applications have been data-to-text systems which generate textual summaries of databases and data sets; these systems usually perform data analysis as well as text generation. In particular, several systems have been built that produce textual weather forecasts from weather data. The earliest such system to be deployed was FoG,[3] which was used by Environment Canada to generate weather forecasts in French and English in the early 1990s. The success of FoG triggered other work, both research and commercial. Recent research in this area include an experiment which showed that users sometimes preferred computer-generated weather forecasts to human-written ones, in part because the computer forecasts used more consistent terminology ,[4] and a demonstration that statistical techniques could be used to generate high-quality weather forecasts.[5] Recent applications include the UK Met Office's text-enhanced forecast [6]

In the 1990s there was interest in using NLG to summarise financial and business data. For example the SPOTLIGHT system developed at A.C. Nielsen automatically generated readable English text based on the analysis of large amounts of retail sales data.[7] More recently there is interest in using NLG to summarise electronic medical records. Commercial applications in this area are appearing ,[8] and researchers have shown that NLG summaries of medical data can be effective decision-support aids for medical professionals.[9] There is also growing interest in using NLG to enhance accessibility, for example by describing graphs and data sets to blind people.

An example of an interactive use of NLG is the WYSIWYM framework. It stands for What you see is what you meant and allows users to see and manipulate the continuously rendered view (NLG output) of an underlying formal language document (NLG input), thereby editing the formal language without learning it.

Content generation systems assist human writers and makes writing process more efficient and effective. A content generation tool based on web mining using search engines APIs has been built.[10] The tool imitates the cut-and-paste writing scenario where a writer forms its content from various search results. Relevance verification is essential to filter out irrelevant search results; it is based on matching the parse tree of a query which the parse trees of candidate answers.[11] In an alternative approach, a high-level structure of human-authored text is used to automatically build a template for a new topic for automatically written Wikipedia article.[12]

Further examples of companies in the Natural Language Generation industry whose software transforms data into narrative content include Yseop, Inc, a multinational software company headquartered in Dallas, Texas,[13] Automated Insights[14]and Narrative Science[15].

Evaluation

As in other scientific fields, NLG researchers need to test how well their systems, modules, and algorithms work. This is called evaluation. There are three basic techniques for evaluating NLG systems:

  • Task-based (extrinsic) evaluation: give the generated text to a person, and assess how well it helps him perform a task (or otherwise achieves its communicative goal). For example, a system which generates summaries of medical data can be evaluated by giving these summaries to doctors, and assessing whether the summaries helps doctors make better decisions.[9]
  • Human ratings: give the generated text to a person, and ask him or her to rate the quality and usefulness of the text.
  • Metrics: compare generated texts to texts written by people from the same input data, using an automatic metric such as BLEU.

An ultimate goal is how useful NLG systems are at helping people, which is the first of the above techniques. However, task-based evaluations are time-consuming and expensive, and can be difficult to carry out (especially if they require subjects with specialised expertise, such as doctors). Hence (as in other areas of NLP) task-based evaluations are the exception, not the norm.

Recently researchers are assessing how well human-ratings and metrics correlate with (predict) task-based evaluations. Work is being conducted in the context of Generation Challenges shared-task events. Initial results suggest that human ratings are much better than metrics in this regard. In other words, human ratings usually do predict task-effectiveness at least to some degree (although there are exceptions [16]), while ratings produced by metrics often do not predict task-effectiveness well. These results are preliminary. In any case, human ratings are the most popular evaluation technique in NLG; this is contrast to machine translation, where metrics are widely used.

See also

References

  1. ^ R Turner, S Sripada, E Reiter, I Davy (2006). Generating Spatio-Temporal Descriptions in Pollen Forecasts. Proceedings of EACL06
  2. ^ Dale, Robert; Reiter, Ehud (2000). Building natural language generation systems. Cambridge, U.K.: Cambridge University Press. ISBN 0-521-02451-X.
  3. ^ Goldberg E, Driedger N, Kittredge R (1994). "Using Natural-Language Processing to Produce Weather Forecasts". IEEE Expert. 9 (2): 45–53. doi:10.1109/64.294135.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  4. ^ Reiter E, Sripada S, Hunter J, Yu J, Davy I (2005). "Choosing Words in Computer-Generated Weather Forecasts". Artificial Intelligence. 167: 137–69. doi:10.1016/j.artint.2005.06.006.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  5. ^ Belz A (2008). "Automatic Generation of Weather Forecast Texts Using Comprehensive Probabilistic Generation-Space Models". Natural Language Engineering. 14: 431–55. doi:10.1017/s1351324907004664.
  6. ^ S Sripada, N Burnett, R Turner, J Mastin, D Evans(2014). Generating A Case Study: NLG meeting Weather Industry Demand for Quality and Quantity of Textual Weather Forecasts. Proceedings of INLG 2014
  7. ^ Anand, Tej; Kahn, Gary (1992). "Making Sense of Gigabytes: A System for Knowledge-Based Market Analysis". In Klahr, Philip; Scott, A. F. (ed.). Innovative applications of artificial intelligence 4: proceedings of the IAAI-92 Conference. Menlo Park, Calif: AAAI Press. pp. 57–70. ISBN 0-262-69155-8. {{cite book}}: External link in |chapterurl= (help); Unknown parameter |chapterurl= ignored (|chapter-url= suggested) (help)CS1 maint: multiple names: editors list (link)
  8. ^ Harris MD (2008). "Building a Large-Scale Commercial NLG System for an EMR" (PDF). Proceedings of the Fifth International Natural Language Generation Conference. pp. 157–60. {{cite conference}}: Unknown parameter |booktitle= ignored (|book-title= suggested) (help)
  9. ^ a b Portet F, Reiter E, Gatt A, Hunter J, Sripada S, Freer Y, Sykes C (2009). "Automatic Generation of Textual Summaries from Neonatal Intensive Care Data". Artificial Intelligence. 173 (7–8): 789–816. doi:10.1016/j.artint.2008.12.002.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  10. ^ Galitsky, Boris (2013). "A Web Mining Tool for Assistance with Creative Writing". Advances in Information Retrieval. Lecture Notes in Computer Science. 7814: 828–831. doi:10.1007/978-3-642-36973-5_95.
  11. ^ Galitsky B, de la Rosa JL, Dobrocsi G. (2012). "Inferring the semantic properties of sentences by mining syntactic parse trees". Data & Knowledge Engineering. 81–82: 21–45. doi:10.1016/j.datak.2012.07.003.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  12. ^ Sauper, Christina and Barzilay, Regina (2009). "Automatically Generating Wikipedia Articles: A Structure-Aware Approach". Proceedings of ACL.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  13. ^ Woods, Dan. "Why You Should Hire A Robot To Write For You". http://www.forbes.com/sites/danwoods/2014/08/28/why-you-should-hire-a-robot-to-write-for-you/
  14. ^ Harrison, J.D. "Steve Case makes a spur-of-the-moment million-dollar investment in 10 start-ups". Washington Post.
  15. ^ Pletz, John. "Narrative Science gets $11.5 million to write next chapter". Crain's Chicago Business. Crain's. Retrieved 6 June 2014.
  16. ^ Law A, Freer Y, Hunter J, Logie R, McIntosh N, Quinn J (2005). "A Comparison of Graphical and Textual Presentations of Time Series Data to Support Medical Decision Making in the Neonatal Intensive Care Unit". Journal of Clinical Monitoring and Computing. 19 (3): 183–94. doi:10.1007/s10877-005-0879-3. PMID 16244840.{{cite journal}}: CS1 maint: multiple names: authors list (link)

Further reading

  • Dale, Robert; Reiter, Ehud (2000). Building natural language generation systems. Cambridge, UK: Cambridge University Press. ISBN 0-521-02451-X.{{cite book}}: CS1 maint: multiple names: authors list (link)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Natural_language_generation&oldid=629066464"