Abstract: Various aspects of the subject technology relate to methods, systems, and machine-readable media for generating semantic scene representations using sparse three-dimensional queries. Various aspects may include receiving, at a first timestep, images captured from cameras that observe a three-dimensional scene. Aspects may also include accessing a queue of previously generated sparse three-dimensional queries propagated from preceding timesteps. Aspects may also include forming, based on the previously generated queries, current sparse three-dimensional queries representing the scene at the first timestep. Aspects may also include iteratively refining, based on the images and previously generated queries, the current queries. Aspects may also include determining, based on the current queries, three-dimensional Gaussian primitives. Aspects may also include generating, based on the Gaussian primitives, a semantic representation of the scene.

Abstract: One embodiment of the present invention sets forth a technique for identifying users. The technique includes generating a first set of image crops of users in an environment based on estimates of a first set of poses for the users in a first set of images collected by a set of tracking cameras. The technique also includes applying an embedding model to the first set of image crops to produce a first set of embeddings and aggregating the first set of embeddings into clusters representing the users. The technique further includes upon matching, to a cluster, a second set of embeddings produced by the embedding model from a second set of image crops of an interaction between a user and an item, storing a representation of the interaction in a virtual shopping cart associated with the cluster.

Abstract: One embodiment of the present invention sets forth a technique for detecting a first user in the autonomous store based on analysis of a first set of images collected by a set of tracking cameras in the autonomous store, creating a first virtual shopping cart for the first user without requiring the first user to establish an identity with the autonomous store, updating the first virtual shopping cart with a set of items in the autonomous store based on observations from a set of sensors of interactions between the first user and the set of items, and automatically performing at least one check-out operation related to the set of items in the first virtual shopping cart upon detecting that a first location of the first user is within proximity of a checkout location in the autonomous store.