The University of Massachusetts Amherst
University of Massachusetts Amherst

Search Google Appliance


What can go here?: A New Method to Automatically Populate a 3D Scene

In a paper that will be presented at the ICCV 2019 conference, CICS doctoral candidate Yang Zhou and Associate Professor Evangelos Kalogerakis describes a new approach to automatically populate a 3D scene with appropriate objects, based on context. Where existing methods identify and place objects using a probabilistic models and image-based convolutional neural networks, Zhou and Kalogerakis’s neural message passing approach (called SceneGraphNet) takes context into account, improving performance over current methods.

SceneGraphNet can be used to enhance 3D object recognition, in addition to automatically populating 3D scenes with more objects. This can be particularly useful to designers trying to interactively model a 3D scene.


The neural message passing method models the scene as a graph, where nodes represent existing objects and edges represent relationships between them.


“Our work introduces a new graph neural network that provides users with recommendations of 3D objects (e.g., furniture, decorations) that match an input indoor environment (e.g., living room, bedroom),” Kalogerakis says. “This is especially useful for AR/VR applications that allow users to visualize furniture suggestions in indoor scenes and help them choose items to buy. Our network is also useful for 3D object recognition in indoor scenes e.g., it can help robots automatically recognize the type of objects that they interact with (e.g., chairs, tables) while navigating an indoor environment.”


ICCV is the premier international computer vision expo. More than 5,000 students, academics, industry professionals, and researchers from across the world are expected to attend.