Abstract: In one embodiment, a method includes scanning a real-world environment with a first device associated with a first user; generating a three-dimensional model of the real-world environment, transmitting the three-dimensional model to a head-mounted device associated with the first user, determining a pose of the head-mounted device by localizing the head-mounted device within the three-dimensional model based on images captured by a second camera of the head-mounted device, displaying, on the head-mounted device, a virtual space corresponding to the scanned real-world environment generated based on the three-dimensional model as viewed from the pose, and transmitting, to a remote head-mounted device of a second user, data corresponding to the three-dimensional model and the pose of the head-mounted device, the data being configured for rendering, by the remote head-mounted device, the virtual space with a first avatar corresponding to the first user having the pose.

Abstract: In one embodiment, a method includes the steps of capturing an image from a camera viewpoint, the image depicting a physical keyboard, detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, the predetermined shape template representing visual characteristics of spaces between keyboard keys, accessing predetermined shape features of a keyboard model associated with the physical keyboard, and determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.

Abstract: In one embodiment, a method includes the steps of capturing an image from a camera viewpoint, the image depicting a physical keyboard, detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, the predetermined shape template representing visual characteristics of spaces between keyboard keys, accessing predetermined shape features of a keyboard model associated with the physical keyboard, and determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.

Abstract: Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.

Abstract: Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.

Abstract: Disclosed herein are related to a system and a method for porting a physical object in a physical space into a virtual reality. In one approach, the method includes detecting an input device in a physical space relative to a user of the input device. In one approach, the method includes presenting, by a display device to the user, a virtual model of the detected input device in a virtual space at a location and an orientation. The location and the orientation of the virtual model in the virtual space may correspond to a location and an orientation of the input device in the physical space relative to the user. In one approach, the method includes visually providing relative to the virtual model in the virtual space, through the display device, spatial feedback on the user's interaction with the input device in the physical space.

Abstract: In one embodiment, a method includes generating a geometrical arrangement of a surrounding area, the geometrical arrangement describing a location of a first set of visual features in the space of the surrounding area; determining parameters of a camera, the parameters of the camera indicating one or more of an approximate location, orientation, or optical properties of the camera; applying determined parameters of the camera to the geometrical arrangement of the surrounding area to display the first set of visual features on a feature image; superimposing a second set of visual features extracted from an image recorded with the camera on the feature image; determining a measure of concordance between the locations of the first and second sets of visual features in the feature image; and if the measure of concordance has passed a limit value, using determined parameters of the camera as actual parameters of the camera.

Abstract: A method and image processing system determine parameters of a camera. According to the method, an image of a surrounding area is captured by the camera, and camera parameters are initially determined. Furthermore, a three-dimensional geometric description of visual features of the surrounding area is provided. A feature detector is used on the captured image in order to extract visual features. The initially determined camera parameters are applied to the three-dimensional geometric description of the visual features of the surrounding area in order to display said visual features on a feature image. A quantity of the visual features extracted from the image is compared with a quantity of the visual features in the feature image to determine a degree of concordance between the two quantities. The camera parameters are changed repeatedly to determine additional feature images for which the degree of concordance is determined until said degree exceeds a threshold.

Abstract: A method and image processing system for removing a visual object from an image, e.g., a moving image, is described. An image is provided in which a visual object to be removed is selected, for which a mask is determined. Pixels outside the mask that are intended to be used for replacing pixels inside the mask are selected, based on a similarity measure, comprising an appearance similarity measure representing the visual similarity between a previously selected pixel for replacing the pixel to be replaced and the pixel to be selected, and a geometry similarity measure, representing the closeness of the pixel to be selected to the pixel among the pixels outside the mask that was previously used for replacing the pixel adjacent to the pixel to be replaced. The pixels inside the mask are replaced by copying the properties of the selected pixels to the respective pixels to be replaced.

Abstract: A method and image processing system determine parameters of a camera. According to the method, an image of a surrounding area is captured by the camera, and camera parameters are initially determined. Furthermore, a three-dimensional geometric description of visual features of the surrounding area is provided. A feature detector is used on the captured image in order to extract visual features. The initially determined camera parameters are applied to the three-dimensional geometric description of the visual features of the surrounding area in order to display said visual features on a feature image. A quantity of the visual features extracted from the image is compared with a quantity of the visual features in the feature image to determine a degree of concordance between the two quantities. The camera parameters are changed repeatedly to determine additional feature images for which the degree of concordance is determined until said degree exceeds a threshold.

Abstract: A method and image processing system for removing a visual object from an image, e.g., a moving image, is described. An image is provided in which a visual object to be removed is selected, for which a mask is determined. Pixels outside the mask that are intended to be used for replacing pixels inside the mask are selected, based on a similarity measure, comprising an appearance similarity measure representing the visual similarity between a previously selected pixel for replacing the pixel to be replaced and the pixel to be selected, and a geometry similarity measure, representing the closeness of the pixel to be selected to the pixel among the pixels outside the mask that was previously used for replacing the pixel adjacent to the pixel to be replaced. The pixels inside the mask are replaced by copying the properties of the selected pixels to the respective pixels to be replaced.

Abstract: Methods and apparatuses for removing a visual object from a visual data stream. A live visual data stream containing present and subsequent images is received. A visual object is selected by defining a contour. A binary mask based on said contour is determined. Starting at the coarsest layer, regions outside the mask for replacing a region within the mask are selected. The region within the mask is replaced by content of the selected regions. The selecting and the replacing are successively repeated at each of the finer layers of the resolution pyramid finishing at the finest layer. The repeated replacing is also based on the content replaced at the next coarser layer. The step of determining a binary mask, calculating a resolution pyramid, and repeated selecting and repeated replacing are repeated for the received subsequent image. The repeated replacing is also based on the content replaced in the preceding image.

Abstract: Methods and apparatuses for removing a visual object from a visual data stream. A live visual data stream containing present and subsequent images is received. A visual object is selected by defining a contour. A binary mask based on said contour is determined. Starting at the coarsest layer, regions outside the mask for replacing a region within the mask are selected. The region within the mask is replaced by content of the selected regions. The selecting and the replacing are successively repeated at each of the finer layers of the resolution pyramid finishing at the finest layer. The repeated replacing is also based on the content replaced at the next coarser layer. The step of determining a binary mask, calculating a resolution pyramid, and repeated selecting and repeated replacing are repeated for the received subsequent image. The repeated replacing is also based on the content replaced in the preceding image.