Objaverse: a virtual world for AI with 800k props

To make its way from the chat box into our living rooms, AI must improve its understanding of spaces and objects.

To advance the development of artificial intelligence, the Allen Institute for AI has produced an expansive and varied database of 3D models of commonplace objects. This database, named Objaverse, is designed to aid in the creation of simulations that more closely resemble real-world environments, allowing AI models to train and learn more effectively.

Unlike modern video games, which offer photorealistic graphics and detailed environments, training simulators often lack detail, variation, and interactivity. Objaverse aims to address these limitations with its collection of over 800,000 3D models, each with a range of metadata. The objects included in this database represent a broad spectrum of everyday items, from food to furniture to appliances.

By providing variations on common objects, Objaverse enables AI models to learn how to recognize objects despite their differences. This is a significant improvement over outdated object libraries like ShapeNet, which only contains around 50,000 less detailed models. For instance, an AI assistant that has only ever encountered a generic lamp with no distinguishing features will struggle to recognize a lamp with a unique shape or pattern. However, Objaverse includes variations on lamps, enabling the AI model to learn what defines a lamp regardless of its appearance.

While it may seem unnecessary for an AI assistant to differentiate between a medieval and a modern bookcase, it is essential for it to recognize the difference between a peeled and an unpeeled banana. Using photorealistic imagery, captured through photogrammetry, Objaverse provides a diverse and realistic collection of objects. This level of detail and diversity enables AI models to learn to recognize objects in different states, such as an unmade bed, and understand how they move or change, such as a refrigerator or laptop opening and closing.