Abstract: Embodiments of the present invention provide systems, methods, and computer storage media directed to optimizing the accuracy of local feature detection in a variety of physical environments. Homographic adaptation for facilitating personalization of local feature models to specific target environments is formulated in a bilevel optimization framework instead of relying on conventional randomization techniques. Models for extraction of local image features can be adapted according to homography transformations that are determined to be most relevant or optimal for a user's target environment.

Abstract: In various embodiments there is a method for camera localization within a scene. An image of a scene captured by the camera is input to a machine learning model, which has been trained for the particular scene to detect a plurality of 3D scene landmarks. The 3D scene landmarks are pre-specified in a pre-built map of the scene. The machine learning model outputs a plurality of predictions, each prediction comprising: either a 2D location in the image which is predicted to depict one of the 3D scene landmarks, or a 3D bearing vector, being a vector originating at the camera and pointing towards a predicted 3D location of one of the 3D scene landmarks. Using the predictions, an estimate of a position and orientation of the camera in the pre-built map of the scene is computed.

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media directed to optimizing the accuracy of local feature detection in a variety of physical environments. Homographic adaptation for facilitating personalization of local feature models to specific target environments is formulated in a bilevel optimization framework instead of relying on conventional randomization techniques. Models for extraction of local image features can be adapted according to homography transformations that are determined to be most relevant or optimal for a user's target environment.

Abstract: A robotic system is controlled. Audiovisual data representing an environment in which at least part of the robotic system is located is received via at least one camera and at least one microphone. The audiovisual data comprises a visual data component representing a visible part of the environment and an audio data component representing an audible part of the environment. A location of a sound source that emits sound that is represented in the audio data component of the audiovisual data is identified based on the audio data component of the audiovisual data. The sound source is outside the visible part of the environment and is not represented in the visual data component of the audiovisual data. Operation of a controllable element located in the environment is controlled based on the identified location of the sound source.

Abstract: A three-dimensional (3D) map inconsistency detection machine includes an input transformation layer connected to a neural network. The input transformation layer is configured to 1) receive a test 3D map including 3D map data modeling a physical entity, 2) transform the 3D map data into a set of 2D images collectively corresponding to volumes of view frustums of a plurality of virtual camera views of the physical entity modeled by the test 3D map, and 3) output the set of 2D images to the neural network. The neural network is configured to output an inconsistency value indicating a degree to which the test 3D map includes inconsistencies based on analysis of the set of 2D images collectively corresponding to the volumes of the view frustums of the plurality of virtual camera views.

Abstract: A three-dimensional (3D) map inconsistency detection machine includes an input transformation layer connected to a neural network. The input transformation layer is configured to 1) receive a test 3D map including 3D map data modeling a physical entity, 2) transform the 3D map data into a set of 2D images collectively corresponding to volumes of view frustums of a plurality of virtual camera views of the physical entity modeled by the test 3D map, and 3) output the set of 2D images to the neural network. The neural network is configured to output an inconsistency value indicating a degree to which the test 3D map includes inconsistencies based on analysis of the set of 2D images collectively corresponding to the volumes of the view frustums of the plurality of virtual camera views.

Abstract: A robotic system is controlled. Audiovisual data representing an environment in which at least part of the robotic system is located is received via at least one camera and at least one microphone. The audiovisual data comprises a visual data component representing a visible part of the environment and an audio data component representing an audible part of the environment. A location of a sound source that emits sound that is represented in the audio data component of the audiovisual data is identified based on the audio data component of the audiovisual data. The sound source is outside the visible part of the environment and is not represented in the visual data component of the audiovisual data. Operation of a controllable element located in the environment is controlled based on the identified location of the sound source.