Abstract: Systems and methods of the present disclosure are directed to computer-implemented method for contextual processing via inter-model between pre-trained machine-learned models. The method includes obtaining, by a computing system comprising one or more computing devices, input data. The method includes processing, by the computing system, the input data with two or more pre-trained models to generate output data, wherein processing the input comprises executing a structured inter-model communication schema for inter-model communication between the two or more pre-trained models over a communications channel. The method includes providing, by the computing system, the output data as an output.

Abstract: An example assistant system can use a multimodal multitask medical machine-learned model to perform image processing to answer natural language queries. A device can process speech data or other natural language inputs to obtain a query. The query can be processed alongside image data that provides context for the query. The example system can receive a query associated with a particular task domain; generate, based on the query, a query input that comprises query instruction data from a first modality and query context data from a second modality; generate a combined input comprising the query input and an exemplar input, wherein the exemplar input comprises exemplar instruction data from the first modality and an exemplar context placeholder in lieu of exemplar context data from the second modality; process the combined input with a multimodal machine-learned model to generate output data; and output a query response based on the output data.

Abstract: Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view. 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving one or more observations of an environment; receiving an input text sequence that describes a task to be performed by a robot in the environment; generating an encoded representation of the input text sequence in an embedding space; generating a corresponding encoded representation of each of the one or more observations in the embedding space; generating a sequence of input tokens that comprises the encoded representation of the input text sequence and the corresponding encoded representation of each observation; processing the sequence of input tokens using a language model neural network to generate an output text sequence that comprises high-level natural language instructions; and determining, from the high-level natural language instructions, one or more actions to be performed by the robot.

Abstract: Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view. 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.

Abstract: Systems and methods for probabilistic semantic sensing in a sensory network are disclosed. The system receives raw sensor data from a plurality of sensors and generates semantic data including sensed events. The system correlates the semantic data based on classifiers to generate aggregations of semantic data. Further, the system analyzes the aggregations of semantic data with a probabilistic engine to produce a corresponding plurality of derived events each of which includes a derived probability. The system generates a first derived event, including a first derived probability, that is generated based on a plurality of probabilities that respectively represent a confidence of an associated semantic datum to enable at least one application to perform a service based on the plurality of derived events.

Abstract: Systems and methods for probabilistic semantic sensing in a sensory network are disclosed. The system receives raw sensor data from a plurality of sensors and generates semantic data including sensed events. The system correlates the semantic data based on classifiers to generate aggregations of semantic data. Further, the system analyzes the aggregations of semantic data with a probabilistic engine to produce a corresponding plurality of derived events each of which includes a derived probability. The system generates a first derived event, including a first derived probability, that is generated based on a plurality of probabilities that respectively represent a confidence of an associated semantic datum to enable at least one application to perform a service based on the plurality of derived events.

Abstract: Methods, devices, systems, and non-transitory process-readable storage media for controlling lighting nodes of a lighting system associated with a lighting infrastructure based on composited lighting models. An embodiment method performed by a processor of a computing device may include operations for obtaining a plurality of lighting model outputs generated by lighting control algorithms that utilize sensor data obtained from one or more sensor nodes within the lighting infrastructure, combining the plurality of lighting model outputs in an additive fashion to generate a composited lighting model, calculating lighting parameters for a lighting node within the lighting infrastructure based on the composited lighting model and other factors, and generating a lighting control command for configuring the lighting node within the lighting infrastructure using the calculated lighting parameters.

Abstract: Systems and methods for probabilistic semantic sensing in a sensory network are disclosed. The system receives raw sensor data from a plurality of sensors and generates semantic data including sensed events. The system correlates the semantic data based on classifiers to generate aggregations of semantic data. Further, the system analyzes the aggregations of semantic data with a probabilistic engine to produce a corresponding plurality of derived events each of which includes a derived probability. The system generates a first derived event, including a first derived probability, that is generated based on a plurality of probabilities that respectively represent a confidence of an associated semantic datum to enable at least one application to perform a service based on the plurality of derived events.

Abstract: Methods, devices, systems, and non-transitory process-readable storage media for controlling lighting nodes of a lighting system associated with a lighting infrastructure based on composited lighting models. An embodiment method performed by a processor of a computing device may include operations for obtaining a plurality of lighting model outputs generated by lighting control algorithms that utilize sensor data obtained from one or more sensor nodes within the lighting infrastructure, combining the plurality of lighting model outputs in an additive fashion to generate a composited lighting model, calculating lighting parameters for a lighting node within the lighting infrastructure based on the composited lighting model and other factors, and generating a lighting control command for configuring the lighting node within the lighting infrastructure using the calculated lighting parameters.