Abstract: A system for social interaction using a photo-realistic novel view of an event includes a multi-view reconstruction system for developing transmission data of the event a plurality of client-side rendering devices, each rendering device receiving the transmission data from the multi-view reconstruction system and rendering the transmission data as the photo-realistic novel view. A method of social interaction using a photo-realistic novel view of an event includes transmitting by a server side transmission data of the event; receiving by a first user on a first rendering device the data transmission; selecting by the first user a path for rendering on the first rendering device at least on novel view; rendering by the first rendering device the at least one novel view; and saving by the user on the first rendering device novel view date for the at least one novel view.

Abstract: A system for social interaction using a photo-realistic novel view of an event includes a multi-view reconstruction system for developing transmission data of the event a plurality of client-side rendering devices, each rendering device receiving the transmission data from the multi-view reconstruction system and rendering the transmission data as the photo-realistic novel view. A method of social interaction using a photo-realistic novel view of an event includes transmitting by a server side transmission data of the event; receiving by a first user on a first rendering device the data transmission; selecting by the first user a path for rendering on the first rendering device at least on novel view; rendering by the first rendering device the at least one novel view; and saving by the user on the first rendering device novel view date for the at least one novel view.

Abstract: A system for social interaction using a photo-realistic novel view of an event includes a multi-view reconstruction system for developing transmission data of the event a plurality of client-side rendering devices, each rendering device receiving the transmission data from the multi-view reconstruction system and rendering the transmission data as the photo-realistic novel view. A method of social interaction using a photo-realistic novel view of an event includes transmitting by a server side transmission data of the event; receiving by a first user on a first rendering device the data transmission; selecting by the first user a path for rendering on the first rendering device at least on novel view; rendering by the first rendering device the at least one novel view; and saving by the user on the first rendering device novel view date for the at least one novel view.

Abstract: A method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event includes dividing by the subdivision module the volume into sub-volumes; projecting from each camera each of the sub-volumes to create a set of sub-volumes masks relative to each camera; creating an imaging mask for each camera; comparing for each camera by the subdivision module the respective imaging mask to the respective subvolume mask and extracting at least one feature from at least one imaging mask; saving by the subdivision module the at least one feature to a subspace division mask; cropping the at least one feature from the imaging frames using the subspace division mask; and processing only the at least one feature for a 3D reconstruction. The system includes cameras for recording the event in imaging frames; and a subdivision module for dividing the volume into sub-volumes.

Abstract: A method of generating user-selectable novel views of an event on a viewing device includes reconstructing by a server system for each camera image data into at least one foreground model for the respective camera and an environment model for the respective camera; joining by the server system the foreground model for each camera to create a visual atlas of all foreground models; creating by the server system foreground mapping data for foreground image data in the visual atlas; creating by the server system environment mapping data for environment image data in each respective environment model; transmitting by the server system each compressed data in a sequence it was compressed; receiving by the viewing device all compressed data; uncompressing by the viewing device all compressed data; selecting by a user the novel view; and rendering by the viewing device each novel view.

Abstract: A method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event includes dividing by the subdivision module the volume into sub-volumes; projecting from each camera each of the sub-volumes to create a set of sub-volumes masks relative to each camera; creating an imaging mask for each camera; comparing for each camera by the subdivision module the respective imaging mask to the respective subvolume mask and extracting at least one feature from at least one imaging mask; saving by the subdivision module the at least one feature to a subspace division mask; cropping the at least one feature from the imaging frames using the subspace division mask; and processing only the at least one feature for a 3D reconstruction. The system includes cameras for recording the event in imaging frames; and a subdivision module for dividing the volume into sub-volumes.

Abstract: A system for multi-view reconstruction of a photo-realistic rendering of an event includes cameras for imaging the event with image frames; a controller having a CEM module for modeling an environment from image data of the image frames, an FES module for segmenting a foreground from the environment from image data of the image frames and constructing a 3D data representation; and a configuration engine includes a path selection module, the configuration engine for configuring and rendering the photo-realistic rendering along a path selected by a user using the path selection module, the path having at least one novel view image. The photo-realistic rendering has less than a 10% discrepancy between output pixel raster values of the novel view image and the image frames imaged by the cameras.

Abstract: A method of generating user-selectable novel views of an event on a viewing device includes reconstructing by a server system for each camera image data into at least one foreground model for the respective camera and an environment model for the respective camera; joining by the server system the foreground model for each camera to create a visual atlas of all foreground models; creating by the server system foreground mapping data for foreground image data in the visual atlas; creating by the server system environment mapping data for environment image data in each respective environment model; transmitting by the server system each compressed data in a sequence it was compressed; receiving by the viewing device all compressed data; uncompressing by the viewing device all compressed data; selecting by a user the novel view; and rendering by the viewing device each novel view.

Abstract: A method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event includes dividing by the subdivision module the volume into sub-volumes; projecting from each camera each of the sub-volumes to create a set of sub-volumes masks relative to each camera; creating an imaging mask for each camera; comparing for each camera by the subdivision module the respective imaging mask to the respective subvolume mask and extracting at least one feature from at least one imaging mask; saving by the subdivision module the at least one feature to a subspace division mask; cropping the at least one feature from the imaging frames using the subspace division mask; and processing only the at least one feature for a 3D reconstruction. The system includes cameras for recording the event in imaging frames; and a subdivision module for dividing the volume into sub-volumes.

Abstract: A method of generating user-selectable novel views of an event on a viewing device includes reconstructing by a server system for each camera image data into at least one foreground model for the respective camera and an environment model for the respective camera; joining by the server system the foreground model for each camera to create a visual atlas of all foreground models; creating by the server system foreground mapping data for foreground image data in the visual atlas; creating by the server system environment mapping data for environment image data in each respective environment model; transmitting by the server system each compressed data in a sequence it was compressed; receiving by the viewing device all compressed data; uncompressing by the viewing device all compressed data; selecting by a user the novel view; and rendering by the viewing device each novel view.

Abstract: A method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event includes dividing by the subdivision module the volume into sub-volumes; projecting from each camera each of the sub-volumes to create a set of sub-volumes masks relative to each camera; creating an imaging mask for each camera; comparing for each camera by the subdivision module the respective imaging mask to the respective sub-volume mask and extracting at least one feature from at least one imaging mask; saving by the subdivision module the at least one feature to a subspace division mask; cropping the at least one feature from the imaging frames using the subspace division mask; and processing only the at least one feature for a 3D reconstruction. The system includes cameras for recording the event in imaging frames; and a subdivision module for dividing the volume into sub-volumes.

Abstract: A method of limiting processing by a 3D reconstruction system of an environment in a 3D reconstruction of an event includes dividing by the subdivision module the volume into sub-volumes; projecting from each camera each of the sub-volumes to create a set of sub-volumes masks relative to each camera; creating an imaging mask for each camera; comparing for each camera by the subdivision module the respective imaging mask to the respective sub-volume mask and extracting at least one feature from at least one imaging mask; saving by the subdivision module the at least one feature to a subspace division mask; cropping the at least one feature from the imaging frames using the subspace division mask; and processing only the at least one feature for a 3D reconstruction. The system includes cameras for recording the event in imaging frames; and a subdivision module for dividing the volume into sub-volumes.

Abstract: A method of generating user-selectable novel views of an event on a viewing device includes reconstructing by a server system for each camera image data into at least one foreground model for the respective camera and an environment model for the respective camera; joining by the server system the foreground model for each camera to create a visual atlas of all foreground models; creating by the server system foreground mapping data for foreground image data in the visual atlas; creating by the server system environment mapping data for environment image data in each respective environment model; transmitting by the server system each compressed data in a sequence it was compressed; receiving by the viewing device all compressed data; uncompressing by the viewing device all compressed data; selecting by a user the novel view; and rendering by the viewing device each novel view.

Abstract: A system for multi-view reconstruction of a photo-realistic rendering of an event includes cameras for imaging the event with image frames; a controller having a CEM module for modeling an environment from image data of the image frames, an FES module for segmenting a foreground from the environment from image data of the image frames and constructing a 3D data representation; and a configuration engine includes a path selection module, the configuration engine for configuring and rendering the photo-realistic rendering along a path selected by a user using the path selection module, the path having at least one novel view image. The photo-realistic rendering has less than a 10% discrepancy between output pixel raster values of the novel view image and the image frames imaged by the cameras.

Abstract: A system for social interaction using a photo-realistic novel view of an event includes a multi-view reconstruction system for developing transmission data of the event a plurality of client-side rendering devices, each rendering device receiving the transmission data from the multi-view reconstruction system and rendering the transmission data as the photo-realistic novel view. A method of social interaction using a photo-realistic novel view of an event includes transmitting by a server side transmission data of the event; receiving by a first user on a first rendering device the data transmission; selecting by the first user a path for rendering on the first rendering device at least on novel view; rendering by the first rendering device the at least one novel view; and saving by the user on the first rendering device novel view date for the at least one novel view.