Abstract: Methods and systems are presented for generating a virtual scene rendering usable in a captured scene based on a camera position of a camera in a stage environment, a mapping of a plurality of subregions of a virtual scene display in the stage environment to corresponding positions in the stage environment, and details of a virtual scene element. The details might include a subregion of the plurality of subregions for the virtual scene element where on the virtual scene display the given virtual scene element would, at least in part, appear, and stage subregion depth values. A blur factor for a corresponding subregion might be determined based at least in part on the stage subregion depth value and the virtual subregion depth value. Rendering the virtual scene might take into account the blur factor for the given virtual scene element.

Abstract: The present disclosure advantageously provides a hybrid memory artificial neural network hardware accelerator that includes a communication bus interface, a static memory, a non-refreshed dynamic memory, a controller and a computing engine. The static memory stores at least a portion of an ANN model. The ANN model includes an input layer, one or more hidden layers and an output layer, ANN basis weights, input data and output data. The non-refreshed dynamic memory is configured to store ANN custom weights for the input, hidden and output layers, and output data. For each layer or layer portion, the computing engine generates the ANN custom weights based on the ANN basis weights, stores the ANN custom weights in the non-refreshed dynamic memory, executes the layer or layer portion, based on inputs and the ANN custom weights, to generate layer output data, and stores the layer output data.

Abstract: An information processing system that acquires video data captured by an image pickup unit; detects an object from the video data; detects a condition corresponding to the image pickup unit; and controls a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

Abstract: Embodiments of the present disclosure provide a method and apparatus for processing an image. An embodiment of the method includes: acquiring a two-dimensional garment image, where the two-dimensional garment image includes a style identifier of a garment; selecting a three-dimensional garment model matching the style identifier from a pre-established set of three-dimensional garment models, wherein the three-dimensional garment model includes scatter points labeled thereon; labeling the two-dimensional garment image with scatter points based on a pre-established coordinate mapping relationship between the two-dimensional garment image and the three-dimensional garment model and the scatter points of the selected three-dimensional garment model; generating a three-dimensional garment image of the acquired two-dimensional garment image based on the selected three-dimensional garment model and a result of the labeling.

Abstract: A method includes receiving sensor data representing a first object in an environment and generating, based on the sensor data, a first state vector that represents physical properties of the first object. The method also includes generating, by a first machine learning model and based on the first state vector and a second state vector that represents physical properties of a second object previously observed in the environment, a metric indicating a likelihood that the first object is the same as the second object. The method further includes determining, based on the metric, to update the second state vector and updating, by a second machine learning model configured to maintain the second state vector over time and based on the first state vector, the second state vector to incorporate into the second state vector information concerning physical properties of the second object as represented in the first state vector.

Abstract: Systems for mapping a stage include a lighting fixture, a scanner, and a controller. The lighting fixture is configured to at least partially illuminate the stage with structured light. The scanner is configured to detect the structured light on the stage. The controller is connected to the lighting fixture and the scanner. The controller is configured to control the lighting fixture to illuminate a portion of the stage with the structured light, control the scanner to scan the structured light on the portion of the stage, receive a signal from the scanner as scan data, and translate the scan data into three-dimensional location data.

Abstract: A system, article, and method of point cloud registration using overlap regions for image processing.

Abstract: Virtual tools are used to manipulate aspects of a three-dimensional medical image volume. The virtual tools are geo-registered with the image volume. Presentation of the image volume is manipulated by an image processor in response to use of the virtual tools. The virtual tools may be used to facilitate analysis of the image volume. The virtual tools may include: a virtual focal point pen; a virtual 3D cursor; a virtual transport viewer; a virtual pedestal; a virtual knife; a virtual catheter; a virtual road sign; a virtual ablation tool; a virtual table; a virtual contrast tool; and virtual icons.

Abstract: Disclosed are various embodiments of variable transform systems for three-dimensional engines. In some aspects, transform data is identified for an object. The object is associated with a base transform class of a three-dimensional engine. A variable transform class generates global transform data using the transform data. The global transform data is expressed according to a cartesian coordinate system used by the three-dimensional engine. The variable transform class provides the global transform data to the base transform class of the three-dimensional engine to position the object in world space.

Abstract: In some implementations, a computing device can test the client-side rendering capabilities of a map module running on a computing device. For example, the map module can include a test module that sends test data to a rendering module of the browser. The rendering module can render the test data into a rendering context based on the test data. The test module can compare the pixel data in the rendering context to expected pixel data to determine whether the rendering module is capable of accurately rendering an image based on the test data. If the rendering module is capable of accurately rendering the rendering context based on the test data, the map module can be configured to perform client-side rendering of images. For example, the map module can be configured to perform client-side rendering of map data based on map data received from a map server.

Abstract: A three-dimensional reconstruction method includes: acquiring first images captured by a first camera, each of the first images being an image containing a target object; determining a shooting angle of the each of the first images, the shooting angle being configured to characterize a shooting direction relative to the target object when the first camera shoots the first images; determining an angle interval corresponding to the each of the first images from a plurality of angle intervals included in an angle range [0, 360°) based on the shooting angle, and setting the first image as a target image in the angle interval; and three-dimensionally reconstructing the target object based on the target images in the respective angle intervals to obtain a three-dimensional image of the target object.

Abstract: In implementations of correcting dust and scratch artifacts in digital images, an artifact correction system receives a digital image that depicts a scene and includes a dust or scratch artifact. The artifact correction system generates, with a generator of a generative adversarial neural network (GAN), a feature map from the digital image that represents features of the dust or scratch artifact and features of the scene. A training system can train the generator adversarially to reduce visibility of dust and scratch artifacts in digital images against a discriminator, and train the discriminator to distinguish between reconstructed digital images generated by the generator and real-world digital images. The artifact correction system generates, from the feature map and with the generator, a reconstructed digital image that depicts the scene of the digital image and reduces visibility of the dust or scratch artifact of the digital image.

Abstract: Systems, methods, and computer-readable storage media for quantifying augmented reality interaction. An exemplary system can be configured to capture, via a camera of a mobile computing device, a video feed, and extract, via a processor of the mobile computing device, an image from the video feed. The system can then determine, via the processor, that the image corresponds to a stored image in a plurality of stored images stored in an image database, resulting in a determination, and retrieve, via the processor and based on the determination, interactive content associated with the stored image. The system can then display, via a display of the mobile computing device, the interactive content as an augmented reality overlay.

Abstract: An example method includes acquiring a first image using a medical imaging modality that includes a two-dimensional field of view to include a patient and a multi-modal marker. A second image is acquired using the medical image modality. The second image includes the patient and the multimodal marker and is along a non-coincident angle with respect to the first image. Predetermined portions of the multi-modal marker are visible in the first and second images and have a known location and orientation with respect to at least one sensor detectable by a tracking system. The method also includes estimating a three-dimensional position for predetermined portions of the multi-modal marker. The method also includes determining an affine transformation for registering a three-dimensional coordinate system of the tracking system with a three-dimensional coordinate system of the medical imaging modality.

Abstract: Disclosed herein is a point cloud data transmission method including encoding the point cloud data, encapsulating the point cloud data, and transmitting point cloud data. Disclosed herein is a point cloud data reception device including a receiver configured to receive the point cloud data, a decapsulator configured to decapsulate the point cloud data, and a decoder configured to decode the point cloud data.

Abstract: A method includes: receiving three-dimensional (3D) information generated by a first 3D system, the 3D information including images of a scene and depth data about the scene; identifying, using the depth data, first image content in the images associated with a depth value that satisfies a criterion; and generating modified 3D information by applying first shading regarding the identified first image content. The modified 3D information can be provided to a second 3D system. The scene can contain an object in the images, and generating the modified 3D information can include determining a surface normal for second image content of the object, and applying second shading regarding the second image content based on the determined surface normal. A portion of the object can have a greater depth value than another portion, and second shading can be applied regarding a portion of the images where the second portion is located.

Abstract: Various implementations disclosed herein include devices, systems, and methods that use event camera data to track deformable objects such as faces, hands, and other body parts. One exemplary implementation involves receiving a stream of pixel events output by an event camera. The device tracks the deformable object using this data. Various implementations do so by generating a dynamic representation of the object and modifying the dynamic representation of the object in response to obtaining additional pixel events output by the event camera. In some implementations, generating the dynamic representation of the object involves identifying features disposed on the deformable surface of the object using the stream of pixel events. The features are determined by identifying patterns of pixel events. As new event stream data is received, the patterns of pixel events are recognized in the new data and used to modify the dynamic representation of the object.

Abstract: Methods and systems include neural network-based image processing and blending circuitry to blend an output of the neural network to compensate for potential artifacts from the neural network-based image processing. The neural network(s) apply image processing to image data using one or more neural networks as processed data. Enhance circuitry enhances the image data in a scaling circuitry to generate enhanced data. Blending circuitry receives the processed image data and the enhanced data along with an image plane of the processed data. The blending circuitry also determines whether the image processing using the one or more neural networks has applied a change to the image data greater than a threshold amount. The blending circuitry then, based at least in part in response to the change being greater than the threshold amount and/or edge information of the image data, blends the processed data with the enhanced data.

Abstract: In one embodiment, a method includes receiving a first frame associated with a first time and one or more second frames of a video having a resolution lower than a target resolution, wherein each second frame is associated with a second time prior to the first time, generating a first feature map for the first frame and one or more second feature maps for the one or more second frames, up-sampling the first feature map and the one or more second feature maps to the target resolution, warping each of the up-sampled second feature maps according to a motion estimation between the associated second time and the first time, and generating a reconstructed frame having the target resolution corresponding to the first frame by using a machine-learning model to process the up-sampled first feature map and the one or more up-sampled and warped second feature maps.

Abstract: The present disclosure provides an apparatus and method for evaluate the quality of a three dimensional (3D) point cloud. The apparatus comprises an image segmenter to generate a segmented two-dimensional (2D) image for each of the plurality of images; a 2D mask generator to generate a 2D mask for each of the plurality of images from the 3D point cloud; a comparator to compare the segmented 2D image with the 2D mask to obtain a comparison result for each image; and an evaluator to evaluate the quality of the 3D point cloud based on aggregated comparison results for the plurality of images.