Abstract: A stored volumetric scene model of a real scene is generated from data defining digital images of a light field in a real scene containing different types of media. The digital images have been formed by a camera from opposingly directed poses and each digital image contains image data elements defined by stored data representing light field flux received by light sensing detectors in the camera. The digital images are processed by a scene reconstruction engine to form a digital volumetric scene model representing the real scene. The volumetric scene model (i) contains volumetric data elements defined by stored data representing one or more media characteristics and (ii) contains solid angle data elements defined by stored data representing the flux of the light field. Adjacent volumetric data elements form corridors, at least one of the volumetric data elements in at least one corridor represents media that is partially light transmissive.

Abstract: Methods and apparatus for systems using a scene codec are described, where systems are either providers or consumers of multi-way, just-in-time, only-as-needed scene data including subscenes and subscene increments. An example system using a scene codec includes a plenoptic scene database containing one or more digital models of scenes, where representations and organization of representations are distributable across multiple systems such that collectively the multiplicity of systems can represent scenes of almost unlimited detail. The system may further include highly efficient means for the processing of these representation and organizations of representation providing the just-in-time, only-as-needed subscenes and scene increments necessary for ensuring a maximally continuous user experience enabled by a minimal amount of newly provided scene information, where the highly efficient means include a spatial processing unit.

Abstract: A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified to the system using eye tracking input, and a graphical tag is displayed on the image marking the feature.

Abstract: A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified to the system using a user input handle of a surgical robotic system, and a graphical tag is displayed on the image marking the feature.

Abstract: A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified to the system using a user input handle of a surgical robotic system, and a graphical tag is displayed on the image marking the feature.

Abstract: A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified and an overlay is displayed on the image marking the feature. Computer vision is used to detect in real time changes in the image that are indicative of the feature of interest being at least partially obscured. In response to such detected changes, a quality of the overlay is altered, e.g. to make it more bright or opaque, based on the change in visibility of the feature of interest in the image.

Abstract: A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified and an overlay is displayed on the image marking the feature. Computer vision is used to detect in real time changes in the image that are indicative of the feature of interest being at least partially obscured. In response to such detected changes, a quality of the overlay is altered, e.g. to make it more bright or opaque, based on the change in visibility of the feature of interest in the image.