Abstract: Apparatus and method for validation of complex perceptional systems with efficient simulation of optical variants. For example one embodiment of a method comprises: identifying foreground objects having a layered object representation in a scene, the layered object representation including surface properties, the scene comprising a sequence of images to be used in a simulation for testing an autonomous driving (AD) system; incrementally rendering one or more images in the sequence of images by performing the operations of: using a previously-rendered image with foreground objects removed as a starting point for rendering a current image; rendering the foreground objects of the current image based on the layered representations; and rendering regions outside of the foreground objects which are influenced by the foreground objects based on the layered representations.

Abstract: Methods, systems, and apparatus, including computer programs encoded on non-transitory computer storage medium(s), for efficiently executing autonomous driving (AD) simulations. The AD simulation can include segmented time events that include variations. The simulation including its variations can be implemented as a graph structure in which the simulation may be executed in a manner similar to traversing a graph.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for handling data including images with privacy-sensitive data. In one aspect, a method may include recognizing privacy-sensitive sub-image(s) in an acquired or captured image. The sub-image(s) can be included in a second image that is encrypted. The acquired image can be changes by obscuring the privacy-sensitive sub-image(s) of the acquired image so as not to reveal any personal identifiable information.

Abstract: A sensor data evaluation device may include an artificial neural network, configured to receive and process first testing data, and output a first testing data output, wherein the first testing data represent sensor data of a first sensor detecting a sensor input; and one or more processors, configured to receive sensor deficiency information representing a deficiency of a second sensor; generate second testing data, wherein the second testing data are based on the sensor deficiency information; implement the artificial neural network to receive and process the second testing data; and output from the output layer a second testing data output; determine a difference between the first testing data output and the second testing data output; and if the difference is outside of one or more predetermined ranges or does not satisfy one or more predetermined criteria, send a signal representing a result of the determined difference.

Abstract: Herein is disclosed a vehicle comprising a plurality of image sensors, configured to detect image data of a vicinity of the vehicle from a plurality of views; a receiver, configured to receive the detected image data and deliver the detected image data to one or more processors, and the one or more processors, configured to generate from the detected image data a virtual three-dimensional reconstruction of the vicinity of the vehicle; and output the three-dimensional image in real-time.

Abstract: Methods, systems, and apparatus, including computer programs encoded on non-transitory computer storage medium(s), for efficiently executing autonomous driving (AD) simulations. The AD simulation can include segmented time events that include variations. The simulation including its variations can be implemented as a graph structure in which the simulation may be executed in a manner similar to traversing a graph.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for handling data including images with privacy-sensitive data. In one aspect, a method may include recognizing privacy-sensitive sub-image(s) in an acquired or captured image. The sub-image(s) can be included in a second image that is encrypted. The acquired image can be changes by obscuring the privacy-sensitive sub-image(s) of the acquired image so as not to reveal any personal identifiable information.

Abstract: A method and apparatus for light source estimation are disclosed. In one embodiment, the apparatus comprises: an on-board tracking device to obtain a position and an orientation of a mobile device at a number of different times; one or more image sensors of the mobile device operable to capture a plurality of light probes of a scene from different positions and orientations at the number of different times; and a light source position determination processor coupled to receive the plurality of light probes and the position and orientation of the mobile device and operable to determine positions of discreet light sources of the scene using the plurality of light probes and the position and orientation of the mobile device.

Abstract: A system, article, and method of providing user facial displays in virtual or augmented reality for face occluding head mounted displays.

Abstract: Techniques related to geometric camera self-calibration quality assessment.

Abstract: Techniques related to geometric camera self-calibration quality assessment.

Abstract: A method of providing a visual representation of a scene to a participant in a studio is provided, wherein the studio is arranged to be viewed by a camera to provide a camera image and has at least a portion of a background object arranged to be keyed out of the camera image by a keying process. The method includes obtaining an estimate of the viewpoint of the participant; rendering an image of the scene based on the viewpoint of the participant; and projecting at least a portion of the rendered image onto the background object to provide a projected image portion visible to the participant. The projected image portion is projected so as to be keyed out of the camera image by the keying process.

Abstract: A method of providing a volumetric representation of a three dimensional object includes defining a plurality of voxels in a three dimensional space, categorizing voxels as foreground or background according to at least one silhouette image of the object, foreground voxels being assigned a first binary value, and background voxels being assigned a second binary value, and assigning to at least some voxels a value intermediate between the first and second binary values.

Abstract: The invention provides a method of providing a volumetric representation of a three-dimensional object. The method comprises several steps, including obtaining a line list of points of at least one set of lines having start and end points corresponding to intersection of the lines with object boundary positions and obtaining an approximate volumetric representation of the object. Then, a modified volumetric representation is produced by modifying the approximate volumetric representation based on the start and end points of the set of lines and the approximate volumetric representation. The resultant modified volumetric representation of the object is more accurate than the approximate volumetric representation of the object. Also, a resultant data set from the present invention that defines the volumetric representation of the object contains continuous data as to the shape of the object, rather than mere discrete data. Therefore, the resultant data set contains more realistic data defining the object.

Abstract: A method of providing a volumetric representation of a three dimensional object includes defining a plurality of voxels in a three dimensional space, categorizing voxels as foreground or background according to at least one silhouette image of the object, foreground voxels being assigned a first binary value, and background voxels being assigned a second binary value, and assigning to at least some voxels a value intermediate between the first and second binary values.

Abstract: A method for determining the position and orientation of a camera, which may not rely on the use of special markers. A set of reference images may be stored, together with camera pose and feature information for each image. A first estimate of camera position is determined by comparing the current camera image with the set of reference images. A refined estimate can be obtained using features from the current image matched in a subset of similar reference images, and in particular, the 3D positions of those features. A consistent 3D model of all stored feature information need not be provided.

Abstract: A method of providing a visual representation of a scene to a participant in a studio is provided, wherein the studio is arranged to be viewed by a camera to provide a camera image and has at least a portion of a background object arranged to be keyed out of the camera image by a keying process. The method includes obtaining an estimate of the viewpoint of the participant; rendering an image of the scene based on the viewpoint of the participant; and projecting at least a portion of the rendered image onto the background object to provide a projected image portion visible to the participant. The projected image portion is projected so as to be keyed out of the camera image by the keying process.

Abstract: The invention provides a method of providing a volumetric representation of a three-dimensional object. The method comprises several steps, including obtaining a line list of points of at least one set of lines having start and end points corresponding to intersection of the lines with object boundary positions and obtaining an approximate volumetric representation of the object. Then, a modified volumetric representation is produced by modifying the approximate volumetric representation based on the start and end points of the set of lines and the approximate volumetric representation. The resultant modified volumetric representation of the object is more accurate than the approximate volumetric representation of the object. Also, a resultant data set from the present invention that defines the volumetric representation of the object contains continuous data as to the shape of the object, rather than mere discrete data. Therefore, the resultant data set contains more realistic data defining the object.

Abstract: A method for generating a desired view of a real scene from a selected desired viewpoint by identifying objects in a real image, determining the positions of the identified objects, and rendering a view of the scene from a selected viewpoint using image data from the real image to render at least some of the identified objects. Other portions of the rendered view can be rendered using other source data which may be generic or historic. Identified objects may be tracked over a period of time to determine a trajectory or path. A user interface can be provided to assist in object tracking. A number of cameras can be used to provide a number of real images, and certain cameras may be controlled using the parameters of other cameras.