Entities in Context
Partner: Meta Platforms Inc.
Participants: Saeed Goodarzi, Nikhil Kagita, Dennis Minn
Description: We revisited the generalization effectiveness of LLMs by focusing on named entities. Named entities are ubiquitous in current Natural Language Understanding benchmarks, yet they have been largely ignored in order to examine the impact on models’ reasoning capabilities. We subjected models to the same evaluation data while modifying them to iterate through a large array of named entities from diverse demographics.