The University of Massachusetts Amherst
University of Massachusetts Amherst

Search Google Appliance


Alexander Mathis - DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Data Science Tea
Monday, October 15, 2018 - 4:00pm
Computer Science Building, Room 150/151



DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Abstract: Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. I will present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data.  I demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy. I will also discuss applications of this approach to study animal behavior.


Bio: Alexander Mathis is a postdoctoral fellow in the laboratories of Profs. Venkatesh N. Murthy at Harvard University and Matthias Bethge at the University of Tübingen. He studies motor control and deep learning methods for behavioral analysis from videos, as well as the sense of smell. Previously he did his PhD in the group of Prof. Andreas V.M. Herz at the University of Munich, where he worked on deriving tuning properties of grid cells from first principles.