Abstract: A method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) generating at least one 2D synthetic image based at least on the camera parameter set by rendering the 3D model in a view range for generating training data.

Abstract: An exemplary method includes generating a first image containing a model image based on a 3D model at a pose. Second images are acquired containing an image not containing the model image. Training image patches are extracted from the first and second images, each training image patch being associated with a class representing whether the corresponding image patch contains at least a part of the model image. An algorithm model is trained with the training image patches and the respective classes to derive the model image's position relative to the first image. Parameters defining the trained algorithm model are stored. Another exemplary method includes a camera acquiring an image containing an object in a scene. An enhancement filter is applied to the acquired image. Image patches are extracted from the filtered image. The object's position is determined in the image by applying the trained algorithm model to the image patches.

Abstract: A method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) generating at least one 2D synthetic image based at least on the camera parameter set by rendering the 3D model in a view range for generating training data.

Abstract: A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object, and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) receiving a selection of data representing a view range, (D) generating at least one 2D synthetic image based on the camera parameter set by rendering the 3D model in the view range, (E) generating training data using the at least one 2D synthetic image to train an object detection algorithm, and (F) storing the generated training data in one or more memories.

Abstract: A method includes steps of: (a) causing a display device to display an image of a real object based on captured image data of a real object; (b) causing the display device to display a 2D model obtained from a projection of a 3D model corresponding to the real object based at least on a view; and (c) storing appearance data of the real object and data of the 2D model into a memory device while associating the appearance data and the data of the 2D model with each other after the real object and the 2D model are displayed so as to be aligned with or substantially aligned with each other by the display device.

Abstract: A method includes acquiring, from the camera, a camera data sequence including a first image frame of a real object in a scene, tracking a pose of the real object with respect to the camera along the camera data sequence, and displaying an XR object on the display by rendering the XR object based at least on the pose. Flag data is set, in a memory area of the at least one memory, indicative of whether or not the displayed XR object is consistent in pose with the real object. The method includes outputting, to a separate computing device having another processor, second image frames in the camera data sequence acquired when the flag data indicates that the displayed XR object is consistent in pose with the real object.

Abstract: A method includes acquiring, from the camera, a camera data sequence including a first image frame of a real object in a scene, tracking a pose of the real object with respect to the camera along the camera data sequence, and displaying an XR object on the display by rendering the XR object based at least on the pose. Flag data is set, in a memory area of the at least one memory, indicative of whether or not the displayed XR object is consistent in pose with the real object. The method includes outputting, to a separate computing device having another processor, second image frames in the camera data sequence acquired when the flag data indicates that the displayed XR object is consistent in pose with the real object.

Abstract: A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object, and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) receiving a selection of data representing a view range, (D) generating at least one 2D synthetic image based on the camera parameter set by rendering the 3D model in the view range, (E) generating training data using the at least one 2D synthetic image to train an object detection algorithm, and (F) storing the generated training data in one or more memories.

Abstract: A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object, and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) receiving a selection of data representing a view range, (D) generating at least one 2D synthetic image based on the camera parameter set by rendering the 3D model in the view range, (E) generating training data using the at least one 2D synthetic image to train an object detection algorithm, and (F) storing the generated training data in one or more memories.

Abstract: A method of detecting an object in a real scene using a computer includes specifying a 3D model corresponding to the object. The method further includes acquiring, from a capture camera, an image frame of a reference object captured from a first view angle. The method further includes generating a 2D synthetic image by rendering the 3D model in a second view angle that is different from the first view angle. The method further includes generating training data using (i) the image frame, (ii) the 2D synthetic image, (iii) the run-time camera parameter, and (iv) the capture camera parameter. The method further includes storing the generated training data in one or more memories.

Abstract: A method comprising: acquiring, from a camera, a video sequence of a real object; deriving a pose of the real object included in at least one image frame using a 3D model corresponding to the real object in the case where the at least one image frame is selected from the video sequence; tracking or deriving the pose of the real object included in image frames in the video sequence in forward and/or backward directions from the at least one image frame; and storing, for at least one of the tracked or derived pose, appearance information obtained from the corresponding image frame and data of a 2D model obtained from a projection of the 3D model using the corresponding tracked pose so that the appearance information and the data of the 2D model are associated with the corresponding tracked or derived pose.

Abstract: A method of detecting an object in a real scene using a computer includes specifying a 3D model corresponding to the object. The method further includes acquiring, from a capture camera, an image frame of a reference object captured from a first view angle. The method further includes generating a 2D synthetic image by rendering the 3D model in a second view angle that is different from the first view angle. The method further includes generating training data using (i) the image frame, (ii) the 2D synthetic image, (iii) the run-time camera parameter, and (iv) the capture camera parameter. The method further includes storing the generated training data in one or more memories.

Abstract: A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object, and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) receiving a selection of data representing a view range, (D) generating at least one 2D synthetic image based on the camera parameter set by rendering the 3D model in the view range, (E) generating training data using the at least one 2D synthetic image to train an object detection algorithm, and (F) storing the generated training data in one or more memories.

Abstract: A method comprising: acquiring, from a camera, a video sequence of a real object; deriving a pose of the real object included in at least one image frame using a 3D model corresponding to the real object in the case where the at least one image frame is selected from the video sequence; tracking or deriving the pose of the real object included in image frames in the video sequence in forward and/or backward directions from the at least one image frame; and storing, for at least one of the tracked or derived pose, appearance information obtained from the corresponding image frame and data of a 2D model obtained from a projection of the 3D model using the corresponding tracked pose so that the appearance information and the data of the 2D model are associated with the corresponding tracked or derived pose.

Abstract: A method includes steps of: (a) causing a display device to display an image of a real object based on captured image data of a real object; (b) causing the display device to display a 2D model obtained from a projection of a 3D model corresponding to the real object based at least on a view; and (c) storing appearance data of the real object and data of the 2D model into a memory device while associating the appearance data and the data of the 2D model with each other after the real object and the 2D model are displayed so as to be aligned with or substantially aligned with each other by the display device.

Abstract: Aspects of the present invention comprise holocam systems and methods that enable the capture and streaming of scenes. In embodiments, multiple image capture devices, which may be referred to as “orbs,” are used to capture images of a scene from different vantage points or frames of reference. In embodiments, each orb captures three-dimensional (3D) information, which is preferably in the form of a depth map and visible images (such as stereo image pairs and regular images). Aspects of the present invention also include mechanisms by which data captured by two or more orbs may be combined to create one composite 3D model of the scene. A viewer may then, in embodiments, use the 3D model to generate a view from a different frame of reference than was originally created by any single orb.

Abstract: Aspects of the present invention comprise holocam systems and methods that enable the capture and streaming of scenes. In embodiments, multiple image capture devices, which may be referred to as “orbs,” are used to capture images of a scene from different vantage points or frames of reference. In embodiments, each orb captures three-dimensional (3D) information, which is preferably in the form of a depth map and visible images (such as stereo image pairs and regular images). Aspects of the present invention also include mechanisms by which data captured by two or more orbs may be combined to create one composite 3D model of the scene. A viewer may then, in embodiments, use the 3D model to generate a view from a different frame of reference than was originally created by any single orb.