Assistant Professor Mohit Iyyer, a CDS-affiliated faculty member, was recently awarded the 2020 IBM Faculty Award for his project titled “Exploring Modeling and Evaluation for Long-form QA.” Long-form QA (question answering), a natural language generation (NLG) task in which answers are free-form paragraphs, is crucial in real-world settings such as community and conversational question answering. However, researchers are still presented with unsolved long-form QA challenges, such as maintaining logical structure, and avoiding hallucinating (producing false facts) or altering facts.
Iyyer, who received his PhD from the University of Maryland, College Park and completed a postdoc at Allen Institute for AI, is interested in using machine learning to generate text at the paragraph or document scale. This idea introduces linguistic issues -- in a four-sentence paragraph, the sentences need to be consistent with each other, and not introduce new concepts or contradict past sentences.
This project will use Facebook’s ELI5 dataset, created from the “Explain it Like I’m Five” community on Reddit. In this community, users attempt to explain complicated ideas in a simple way (so that a five year old could understand). The project will explore staged answer generation methods that combine traditional NLG planning with neural language models, augmentation of the dataset with synthetic data derived from an inverse model trained to generate a question given the answer, and style-aware decoding algorithms using a limited vocabulary, since simple words and sentences are preferable for the ELI5 dataset.
An example from the paper introducing the ELI5 dataset illustrates the problems (such as inconsistencies and contradicting past sentences) that Iyyer’s research will attempt to address.
Question: How do cruise ships get wifi in the middle of the ocean?
Generated answer: There are a couple of ways to do this. One is to turn on the radio and turn off the wifi. The other is to make sure that the wifi is turned on. If you turn the wifi on, the radio will turn on and the wifi will turn off. This is done by turning on the wifi and turning off the radio. If the WiFi is turned off, then the WiFi will turn back on, but the radio won’t turn on.
The IBM Faculty Award is intended to foster collaboration between academic researchers and IBM researchers, and to promote curriculum innovation to stimulate growth in areas that are strategic to IBM.