Abstract: Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.

Abstract: Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.

Abstract: Various embodiments of the disclosure are directed to a scene reconstruction and machine learning system. In embodiments, the system comprises a storage medium configured to store image data, one or more scene models, one or more relightable matter fields, and information related to a machine learning model. In one or more embodiments, the system comprises an input circuit configured to receive image data characterizing light in a scene. In embodiments the system includes a processor. In embodiments the processor configured to reconstruct a scene model representing the scene using the image data. In embodiments the processor is configured to extract a relightable matter field from the scene model representing the object, store the scene model and the relightable matter field representing the object in the storage medium, apply the relightable matter field as an input to the machine learning model, and generate an output from the machine learning model.

Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec comprises aplenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.

Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: Systems and methods deform the surface of the material with a probe, such as a mechanical device or a gas/liquid jet, while optically recording in detail the three-dimensional (3D) topography of the resulting surface deformation. The probe effectively applies a forcing function to the material, the attributes of which are known by performing calibrations prior to use or by direct measurement while it is applied. The topography is effectively the system output that is measured as indicative of the underlying mechanical properties of the material. In one application, systems and methods that apply a pressure in-vivo to human tissue and analyze a three-dimensional topography of the resulting surface deformation to identify localized inhomogeneities and anomalies in the human tissue.

Abstract: A visual cognition system immersed in a medium with object and the system is also an object. The system includes means for conveying energy, which include one or more dispersive elements. The system includes means for sensing energy. The means for sensing include a plurality of detectors. The system includes means for modeling sensed energy, thereby creating a sensed energy model. The sensed energy model represents the sensed energy at a plurality of frequency bands, a plurality of polarization states, a plurality of positions and a plurality of times, using the sensed data. The system includes means for modeling a scene, thereby creating a scene model. The scene model represents the scene in three-dimensional space. The means for modeling a scene uses the sensed energy model from a plurality of directions at a plurality of times.

Abstract: In accordance with one aspect, the present invention provides a real-time 3D visualization system. The system includes means for conveying electromagnetic energy emanating from one or more 3D surfaces including a scene and means for sensing spatial phase characteristics of the electromagnetic energy as the configuration of the 3D surfaces relative to the system changes. The system includes means for creating a 3D scene model utilizing the spatial phase characteristics sensed in a plurality of configurations and means for displaying the 3D scene model in real-time. The means for displaying includes means for synthesizing depth cues.

Abstract: Systems and methods deform the surface of the material with a probe, such as a mechanical device or a gas/liquid jet, while optically recording in detail the three-dimensional (3D) topography of the resulting surface deformation. The probe effectively applies a forcing function to the material, the attributes of which are known by performing calibrations prior to use or by direct measurement while it is applied. The topography is effectively the system output that is measured as indicative of the underlying mechanical properties of the material. In one application, systems and methods that apply a pressure in-vivo to human tissue and analyze a three-dimensional topography of the resulting surface deformation to identify localized inhomogeneities and anomalies in the human tissue.

Abstract: In accordance with one aspect, the present invention provides a real-time 3D visualization system. The system includes means for conveying electromagnetic energy emanating from one or more 3D surfaces including a scene and means for sensing spatial phase characteristics of the electromagnetic energy as the configuration of the 3D surfaces relative to the system changes. The system includes means for creating a 3D scene model utilizing the spatial phase characteristics sensed in a plurality of configurations and means for displaying the 3D scene model in real-time. The means for displaying includes means for synthesizing depth cues.