Allen Institute for AI logo

Editing Transformer Models with Common Sense Knowledge (EMNLP Conference, Dec. 2023)

Partner: Allen Institute for AI

Participants: Anshita Gupta, Debanjan Mondal, Akshay Krishna Sheshadri

Description: Memory editing for updating encyclopedic knowledge in transformers has received increasing attention, but it is unclear if these methods can be adapted for nuanced common sense knowledge. In this research, we proposed an adaptation of MEMIT to edit common sense mistakes in GPT-2 Large and XL. We extend editing to various token locations and employ a robust layer selection strategy. Our results suggest a promising path for improving GPT by incorporating context-specific user feedback about common sense through direct model editing as well as fixing and customizing model behaviors using human-in-the-loop-systems.