Abstract: The description relates to resource aware object detection for encoded video streams that can identify frames of the video stream that include an object of interest, such as a human, without decoding the frames.

Abstract: The description relates to cameras, and camera calibration for enhancing user experiences. One example can receive a first image of a user at a first location relative to a camera. The first image can include the user's upper body but does not include the user from head to toe. The example can receive a second image of the user at a second location relative to a camera. The second image can include the user's upper body but does not include the user from head to toe. The example can estimate a distance of the second location from the first location relative to the camera and calibrate a height and tilt angle of the camera from the first image, the second image, and the estimated distance and without a full body image of the user.

Abstract: In a device including a processor and a memory in communication with the processor is described, the memory includes executable instructions that, when executed by the processor, cause the processor to control the device to perform functions of: generating, based on a plurality of local 3D models, a global 3D model representing a portion of a real-world environment; determining a location of a 3D virtual object in the global 3D model; and generating augmentation data for rendering the 3D virtual object to be seen at a location of the real-world environment corresponding to the location of the 3D virtual object in the global 3D model.

Abstract: In a device including a processor and a memory in communication with the processor is described, the memory includes executable instructions that, when executed by the processor, cause the processor to control the device to perform functions of: generating, based on a plurality of local 3D models, a global 3D model representing a portion of a real-world environment; determining a location of a 3D virtual object in the global 3D model; and generating augmentation data for rendering the 3D virtual object to be seen at a location of the real-world environment corresponding to the location of the 3D virtual object in the global 3D model.

Abstract: A user device comprises a network interface, a rendering module, and a scene modification module. The network interface is configured to receive a video signal from another device via a network. The rendering module is configured to control display apparatus of the user device to display a virtual element to a user of the user device, the virtual element comprising a video image derived from the video signal. The modification module is configured to generate rendering data for displaying a modified version of the virtual element at the other device. The modified version does not include said video image. The network interface is configured to transmit the rendering data to the other device via the network. Alternatively or in addition, the rendering data can be modified at the other device to the same end.

Abstract: An augmented reality (AR) system receives a plurality of local 3D models of a part of a real-world environment, each having been generated by a different AR device when located in the real-world environment. The local 3D models are combined to generate a global 3D model, at least part of which is transmitted to a device remote from the real-world environment. The global 3D model represents a greater portion of the real-environment than any of the local 3D models individually. The AR system receives rendering data from the remote device, and transmits it to an AR device when the AR device is located in the real-world environment. Alternatively, the rendering data may be transmitted from the remote device to the AR device via a network directly. The rendering data is for use in rendering a virtual object at the AR device in the real-world environment.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data for the creation of a video channel defining the shared scene; an intrinsic/extrinsic data determiner configured to determine intrinsic/extrinsic capture device data associated with the image capture device; and a video encoder configured to encode the image data and intrinsic/extrinsic capture device data within the video channel.

Abstract: Within a communication system, packets are received for a video channel defining a shared scene and a surface reconstruction channel associated with surface reconstruction data of image data. The image data and intrinsic and extrinsic capture device data are from an image capture device and are encoded and packetized within the video channel. The surface reconstruction data is encoded and packetized within the surface reconstruction channel. The image data and the intrinsic and extrinsic capture device data are extracted from within the video channel. The surface reconstruction data is extracted from within the surface reconstruction channel. The shared scene is rendered using the image data, the intrinsic and extrinsic capture device data, and the surface reconstruction data.

Abstract: A user device for generating a scene, the user device comprising: an object determiner configured to determine an object for the scene, the object being associated with at least one audio signal; a relative position/orientation determiner configured to determine a relative position/orientation between the user device's user and the object; an audio position processor configured to spatially audio signal process the at least one audio signal to generate at least two channel audio signal based on the relative position/orientation.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data and intrinsic/extrinsic capture device data for the creation of a video channel defining a shared scene; a surface reconstruction entity configured to determine surface reconstruction data associated with the image data from the image capture device; a video channel configured to encode and packetize the image data and intrinsic/extrinsic capture device data; a surface reconstruction channel configured to encode and packetize the surface reconstruction data; a transmitter configured to transmit the video and surface reconstruction channel packets; and a bandwidth controller configured to control the bandwidth allocated to the video channel and the surface reconstruction channel.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data and intrinsic/extrinsic capture device data for the creation of a video channel defining a shared scene; a surface reconstruction entity configured to determine surface reconstruction data associated with the image data from the image capture device; a video channel configured to encode and packetize the image data and intrinsic/extrinsic capture device data; a surface reconstruction channel configured to encode and packetize the surface reconstruction data; a transmitter configured to transmit the video and surface reconstruction channel packets; and a bandwidth controller configured to control the bandwidth allocated to the video channel and the surface reconstruction channel.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data and intrinsic/extrinsic capture device data for the creation of a video channel defining a shared scene; a surface reconstruction entity configured to determine surface reconstruction data associated with the image data from the image capture device; a video channel configured to encode and packetize the image data and intrinsic/extrinsic capture device data; a surface reconstruction channel configured to encode and packetize the surface reconstruction data; a transmitter configured to transmit the video and surface reconstruction channel packets; and a bandwidth controller configured to control the bandwidth allocated to the video channel and the surface reconstruction channel.

Abstract: An augmented reality (AR) system receives a plurality of local 3D models of a part of a real-world environment, each having been generated by a different AR device when located in the real-world environment. The local 3D models are combined to generate a global 3D model, at least part of which is transmitted to a device remote from the real-world environment. The global 3D model represents a greater portion of the real-environment than any of the local 3D models individually. The AR system receives rendering data from the remote device, and transmits it to an AR device when the AR device is located in the real-world environment. Alternatively, the rendering data may be transmitted from the remote device to the AR device via a network directly. The rendering data is for use in rendering a virtual object at the AR device in the real-world environment.

Abstract: A user device within a communication architecture, the user device comprising a session viewer configured to: receive at least one annotation object associated with a mixed reality/virtual reality session; determine a field of view position; and generate an image overlay based on the determined field of view position and at least one annotation object to display an indicator to guide a user of the user device to a representation of the annotation object.

Abstract: A user device comprises a network interface, a rendering module, and a scene modification module. The network interface is configured to receive a video signal from another device via a network. The rendering module is configured to control display apparatus of the user device to display a virtual element to a user of the user device, the virtual element comprising a video image derived from the video signal. The modification module is configured to generate rendering data for displaying a modified version of the virtual element at the other device. The modified version does not include said video image. The network interface is configured to transmit the rendering data to the other device via the network. Alternatively or in addition, the rendering data can be modified at the other device to the same end.

Abstract: Augmented reality apparatus comprises stereoscopic display apparatus, a computer interface, and a rendering module. The stereoscopic display apparatus is arranged to provide to a user of the augmented reality apparatus a view of a real-world environment in which the user is located. The display apparatus is configured to generate a stereoscopic image that is visible to the user simultaneously with the real-world view. The computer interface is configured to receive from a network externally generated 3D model data of the real-world environment in which the user is located. The rendering module is configured to use the externally generated 3D model data to control the display apparatus to render a virtual element in a manner such that it is perceived by the user as a 3D element located at a desired location in the real-world environment.

Abstract: A user device within a communication architecture, the user device comprising an asynchronous session viewer configured to: receive asynchronous session data, the asynchronous session data comprising at least one image, camera pose data associated with the at least one image, and surface reconstruction data associated with the camera pose data; select a field of view position; and edit the asynchronous session data by adding/amending/deleting at least one annotation object based on the selected field of view.

Abstract: A user device within a communication architecture, the user device comprising an asynchronous session generator configured to: capture at least one image; determine camera pose data associated with the at least one image; capture surface reconstruction data, the surface reconstruction data being associated with the camera pose data; generate an asynchronous session comprising asynchronous session data, the asynchronous session data comprising the at least one image, the camera pose data, surface reconstruction data, and at least one annotation object wherein the asynchronous data is configured to be stored and retrieved at a later time.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data and intrinsic/extrinsic capture device data for the creation of a video channel defining a shared scene; a surface reconstruction entity configured to determine surface reconstruction data associated with the image data from the image capture device; a video channel configured to encode and packetize the image data and intrinsic/extrinsic capture device data; a surface reconstruction channel configured to encode and packetize the surface reconstruction data; a transmitter configured to transmit the video and surface reconstruction channel packets; and a bandwidth controller configured to control the bandwidth allocated to the video channel and the surface reconstruction channel.

Abstract: A user device within a communication architecture, the user device comprising: an image capture device configured to determine image data and intrinsic/extrinsic capture device data for the creation of a video channel defining a shared scene; a surface reconstruction entity configured to determine surface reconstruction data associated with the image data from the image capture device; a video channel configured to encode and packetize the image data and intrinsic/extrinsic capture device data; a surface reconstruction channel configured to encode and packetize the surface reconstruction data; a transmitter configured to transmit the video and surface reconstruction channel packets; and a bandwidth controller configured to control the bandwidth allocated to the video channel and the surface reconstruction channel.