Abstract: Techniques of image synthesis using a neural radiance field (NeRF) includes generating a deformation model of movement experienced by a subject in a non-rigidly deforming scene. For example, when an image synthesis system uses NeRFs, the system takes as input multiple poses of subjects for training data. In contrast to conventional NeRFs, the technical solution first expresses the positions of the subjects from various perspectives in an observation frame. The technical solution then involves deriving a deformation model, i.e., a mapping between the observation frame and a canonical frame in which the subject's movements are taken into account. This mapping is accomplished using latent deformation codes for each pose that are determined using a multilayer perceptron (MLP). A NeRF is then derived from positions and casted ray directions in the canonical frame using another MLP. New poses for the subject may then be derived using the NeRF.

Abstract: Systems and methods for three-dimensional object category modeling can utilize figure-ground neural radiance fields for unsupervised training and inference. For example, the systems and methods can include a foreground model and a background model that can generate an object output based at least in part on one or more learned embeddings. The foreground model and background model may process position data and view direction data in order to output color data and volume density data for a respective position and view direction. Moreover, the object category model may be trained to generate an object output, which may include an instance interpolation, a view synthesis, or a segmentation.

Abstract: This disclosure describes systems and techniques for detecting certain activity in image data, such as frames of video data. For example, the systems and techniques may create and utilize an activity classifier for detecting and classifying certain human activity in video data of a facility. In some instances, the classifier may be trained to identify, from the video data, certain predefined activity such as a user picking an item from a shelf, a user returning an item to a shelf, a first user passing an item to a second user, or the like. In some instances, the techniques enable activity detection using only video data, rather than in addition to data acquired by other sensors.

Abstract: Apparatuses, systems, and techniques estimate a pose of an object based on images generated from a combined image volume. In at least one embodiment, the combined image volume is obtained from a plurality of image volumes generated based on a plurality of images of an object.