Abstract: Correcting motion-based inaccuracy in point cloud data generated by one or more sensors carried by a scanning platform, and associated systems and methods are disclosed herein. A representative method includes associating a motion model with a target object of the point cloud, estimating adjusting factors based on the motion model, and adjusting scanning points in the point cloud using the adjusting factors.

Abstract: A computer-implemented method (100) for generating a 3-dimensional wireframe model (13) of an object (2) comprising a plurality of parts, comprising: Scanning (10) the object (2), preferably with a computer tomography method (10a), so as to generate a voxel map; —Computing (12) a 3 dimensional wireframe model (13) of the object; —Segmenting (14) into a plurality of 3 dimensional wireframe part models (15), each 3 dimensional part model corresponding to one part of the object; Using a self-learning machine for classifying (16) said parts into part classes (17); —Adapting (18) the quality of the 3 dimensional wireframe part models depending on at least one quality parameter selected by the user (70) independently for at least one/one or more class (c1, c2, . . . cn). A texture may be applied to the model by photogrammetry and/or PBR.

Abstract: A track-based scanning system uses an ultrasound probe that is mechanically guided through incremental scans over an area of interest. The scanning system can be configured for different patient applications using interchangeable track stands. Two-dimensional scan data and probe position information are fed back to a base unit for processing and assembly of a three-dimensional (3D) shape model. 3D abdominal aorta segmentation and other type of analysis may be performed based on a 3D vascular shape model and an intensity model.

Abstract: The invention relates to a method of creating a virtual game environment. The method comprises: selecting one or more physical objects according to pre-determined physical properties; providing a physical model of the game environment/scene using the selected physical objects; scanning the physical model to obtain a digital three dimensional representation of the physical model including information on the pre-determined physical properties; converting the digital three-dimensional representation of the physical model into a virtual toy construction model made up of virtual toy construction elements; and defining game-controlling elements in the virtual toy construction model using information on the pre-determined physical properties, thereby creating the virtual game environment/scene. The invention further relates to an interactive game system including an implementation of the method.

Abstract: In one embodiment of the disclosure, it is proposed a method for encoding a matrix of image views obtained from data acquired by a plenoptic camera. The method is remarkable in that it comprises: obtaining at least one reference image view from a combination of at least two image views from said matrix of image views; encoding image views that are different from said at least one reference image view, based on said at least one reference image view.

Abstract: Provided is a method that encompasses the determination of a virtual trajectory within a virtual representation of a patient's anatomy, wherein the trajectory is defined by a user's visual axis relative to the three-dimensional virtual representation of the patient's anatomy. The method includes acquiring image data which describes the three-dimensional virtual representation of the patient's body part, acquiring position data which describes a spatial position of a user's visual axis within a virtual-world co-ordinate system, determining visualization data based on the image data and the position data, and determining virtual-world trajectory data based on the position data.

Abstract: Provided herein are devices, systems, and methods for a three-dimensional registering, tracking, and/or guiding an object of interest, such a body part, a surgical tool, or an implant, during a surgical procedure. Such devices, systems, and methods may offer minimally invasive, high precision registering, tracking, and/or guiding of the object of interest using a patterned light beam and data processing using artificial intelligence. The methods, devices, and systems disclosed herein may be compatible with a simple marker or markers placed by a minimally invasive method on the object of interest.

Abstract: A system includes one or more processors coupled to a non-transitory memory, where the one or more processors are configured to generate, using a training set comprising one or more two-dimensional (2D) training images of a dental arch and a three-dimensional (3D) dental arch model, a machine-learning model configured to generate 3D models of dental arches from 2D images of the dental arches, receive one or more 2D images of a dental arch of a user obtained by a user device of the user, and execute the machine-learning model using the one or more 2D images as input to generate a 3D model of the dental arch of the user.

Abstract: A control apparatus includes a first information acquiring section that acquires three-dimensional information of a first region of surfaces of a plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a first location; a second information acquiring section that acquires three-dimensional information of a second region of surfaces of the plurality of objects, the information being obtained by imaging or scanning the plurality of objects from a second location; and a combining section that generates information indicating three-dimensional shapes of at least a portion of the surfaces of the plurality of objects, based on the three-dimensional information of the first region acquired by the first information acquiring section and the three-dimensional information of the second region acquired by the second information acquiring section.

Abstract: Space-based ground-imaging lidar has become increasingly feasible with recent advances in compact fiber lasers and single-photon-sensitive Geiger-mode detector arrays. A challenge with such a system is imperfect pointing knowledge caused by angular jitter, exacerbated by long distances between the satellite and the ground. Unless mitigated, angular jitter blurs the 3D lidar data. Unfortunately, using mechanical isolation, advanced IMUs, star trackers, or auxiliary passive optical sensors to reduce the error in pointing knowledge increases size, weight, power, and/or cost. Here, the 2-axis jitter time series is estimated from the lidar data. Simultaneously, a single-surface model of the ground is estimated as nuisance parameters. Expectation Maximization separates signal and background detections, while maximizing the joint posterior probability density of the jitter and surface states.

Abstract: Disclosed is a system for differentiating the selection of three-dimensional (“3D”) image data in a 3D space from other unselected 3D image data that may be positioned in front of the selected 3D image data, and for customizing editing operations that are presented in a user interface based on the object or material property represented in the selection. The system selects a set of 3D image data in response to a user input, and adjusts the transparency of unselected 3D image data that is positioned in front of the selected set of 3D image data. The system presents a differentiated visualization by rendering the selected set of 3D image data according to an original size, position, and visual characteristics defined for the selected set of 3D image, and by performing a partial or fully transparent rendering of the unselected 3D image as a result of the transparency adjustment.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for utilizing an encoder-decoder architecture to learn a volumetric 3D representation of an object using digital images of the object from multiple viewpoints to render novel views of the object. For instance, the disclosed systems can utilize patch-based image feature extraction to extract lifted feature representations from images corresponding to different viewpoints of an object. Furthermore, the disclosed systems can model view-dependent transformed feature representations using learned transformation kernels. In addition, the disclosed systems can recurrently and concurrently aggregate the transformed feature representations to generate a 3D voxel representation of the object. Furthermore, the disclosed systems can sample frustum features using the 3D voxel representation and transformation kernels.

Abstract: Techniques for determining a swept volume of an object moving along a trajectory in a 3D space are disclosed. In some examples, a computer graphics application accesses a representation of the object, such as the signed distance field (SDF), and the trajectory information describing the movement path in the 3D space over a time period. The 3D space is represented using a grid of voxels each having multiple vertices. The computer graphics application determines the swept volume of the object in the 3D space by evaluating a subset of the grid of voxels (e.g., the voxels surrounding the surface of the swept volume). The number of voxels in the subset of voxels is less than the number of voxels in the grid of voxels. The computer graphics application further generates a representation of the swept volume surface for output.

Abstract: A registration fixture for use with a surgical navigation system for registration of a medical image or images to a three-dimensional tracking space that includes a first ring having at least one tracking marker, a second ring coupled to the first ring having at least one tracking marker and wherein the first ring and the second ring are spaced apart from one another and further include optical markers.

Abstract: The present disclosure relates to an information processing apparatus and method that are capable of suppressing a decrease in encoding efficiency. When point cloud data is to be Octree-encoded, a point distribution of child nodes of a current node is updated in accordance with a peripheral point distribution around the current node, and then a sequence of signals is generated by performing Octree encoding with the updated point distribution. The present disclosure is applicable, for example, to information processing apparatuses, image processing apparatuses, electronic equipment, information processing methods, or programs.

Abstract: A hardware-based traversal coprocessor provides acceleration of tree traversal operations searching for intersections between primitives represented in a tree data structure and a ray. The primitives may include opaque and alpha triangles used in generating a virtual scene. The hardware-based traversal coprocessor is configured to determine primitives intersected by the ray, and return intersection information to a streaming multiprocessor for further processing. The hardware-based traversal coprocessor is configured to provide a deterministic result of intersected triangles regardless of the order that the memory subsystem returns triangle range blocks for processing, while opportunistically eliminating alpha intersections that lie further along the length of the ray than closer opaque intersections.

Abstract: Methods and systems for performing robot-assisted surgery, including methods for defining a boundary surface for a robotic surgery system, methods for operating a robotic arm in an image-guided surgery system, methods and systems for providing haptic feedback to a user during robot-assisted surgery, and a robotic arm for use in robot-assisted surgery.

Abstract: A system configured to improve the operations associated with generating virtual representations on limited resources of a mobile device. In some cases, the system may utilize viewpoint bundles that include collection of image data with an associated pose in relative physical proximity to each other to render a virtual scene. In other cases, the system may utilize 2.5D manifolds including 2D image data and a weighted depth value to render the 3D environment.

Abstract: A method and system for providing access to and control of parameters within a scenegraph includes redefining components or nodes' semantic within a scenegraph. The set of components or nodes (depending on the scenegraph structure) are required to enable access from the Application User Interface to selected scenegraph information. In one embodiment, a user interface is generated for controlling the scenegraph parameters. In addition, constraints can be implemented that allow or disallow access to certain scenegraph parameters and restrict their range of values.

Abstract: A computer-implemented simulation of a virtual hand in a virtual environment can include reading in a hand dataset that is representative of a shape of a real hand of a user, applying the hand dataset to a trained, artificial neural network, and determining an output dataset with the artificial neural network to which the dataset has been applied indicative of forces acting on a virtual object.

Abstract: A method for determining three-dimensional (3D) coordinates of an object surface with a 3D measuring device includes forming from the determined 3D coordinates a mesh having a first face, constructing a voxel array aligned to the first face, obtaining a plurality of images from a first camera having a corresponding plurality of poses, obtaining for each voxel in the voxel array a plurality of voxel values obtained from the corresponding plurality of images, determining for each voxel row a quality value determined based at least in part on an average value of a first quantity and a dispersion of the first quantity, the first quantity based at least in part on first voxel values determined as a function of pose, and determining a distance from a point on the first face to the object surface based at least in part on the determined quality values for the voxel rows.

Abstract: A programmatic arbitrary distribution of items in a modeling system may be provided. To perform the distribution, a surface may be received, and a point count of application points associated with locations on the surface may be determined. A density map may be applied over the surface to assign a density to portions of the surface for the point count. Application points are then assigned to locations on the surface according to the density map and a scattering function of the point count, where the scattering function is based on one or more repulsion forces between neighboring points. The one or more repulsion forces are treated as pushing each of the neighboring point apart. Thereafter, the surface may be provided having the application points scattered across the surface based on the one or more repulsion forces.

Abstract: The implementations of the subject matter described herein relate to an octree-based convolutional neural network. In some implementations, there is provided a computer-implemented method for processing a three-dimensional shape. The method comprises obtaining an octree for representing the three-dimensional shape. Nodes of the octree include empty nodes and non-empty nodes. The empty nodes exclude the three-dimensional shape and are leaf nodes of the octree, and the non-empty nodes include at least a part of the three-dimensional shape. The method further comprises for nodes in the octree with a depth associated with a convolutional layer of a convolutional neural network, performing a convolutional operation of the convolutional layer to obtain an output of the convolutional layer.

Abstract: Disclosed is an imaging system and associated methods for mapping in-focus image data from two-dimensional (“2D”) images of a scene to a three-dimensional (“3D”) model of the scene. The imaging system receives the 2D images and the 3D model, determines the depth of field (“DOF”) and the field of view (“FOV”) for each 2D image, and selects a subset of 3D model constructs that form the FOV and are within the DOF of a particular 2D image. The imaging system determines pixels of the particular 2D image that represent a same set of points in the scene as the subset of 3D model constructs, and maps the visual characteristics from those pixels to non-positional elements of the subset of 3D model constructs.

Abstract: A system and method are disclosed for representing and studying anatomy in the oral region such as parts of a subject's teeth and adjoining tissues. Types of inputs are used to form segmented outputs representing the teeth and can include segmented crown and root portions of the teeth. Machine learning methods are used for optimum and accurate results and to generate data objects corresponding to respective anatomical features of the subject.

Abstract: The present disclosure describes systems, non-transitory computer-readable media, and methods for accurately and efficiently removing objects from digital images taken from a camera viewfinder stream. For example, the disclosed systems access digital images from a camera viewfinder stream in connection with an undesired moving object depicted in the digital images. The disclosed systems generate a temporal window of the digital images concatenated with binary masks indicating the undesired moving object in each digital image. The disclosed systems further utilizes a 3D to 2D generator as part of a 3D to 2D generative adversarial neural network in connection with the temporal window to generate a target digital image with the region associated with the undesired moving object in-painted. In at least one embodiment, the disclosed systems provide the target digital image to a camera viewfinder display to show a user how a future digital photograph will look without the undesired moving object.

Abstract: A method and apparatus for performing preemptive rendering of a 3D dataset is disclosed. Some rendering engines are too slow to render a large dataset quickly. In this patent, a list of possible views is generated. For each possible view includes a viewing position and viewing angle. For each possible view, rendering of a 3D dataset is performed to generate a corresponding preemptively rendered image for said each possible view. Each possible view and corresponding preemptively rendered image are stored. A head display unit's position and orientation are tracked and at a time epoch when said head display unit's position and orientation correspond to a possible view in said list of possible views, the corresponding preemptively rendered image is displayed.

Abstract: A method and an intersection testing module in a ray tracing system for performing intersection testing for a ray with respect to a plurality of convex polygons, each of which is defined by an ordered set of vertices, wherein at least one of the vertices is a shared vertex which is used to define at least two of the convex polygons. The vertices of the convex polygons are projected onto a pair of axes orthogonal to the ray direction. A vertex ordering scheme defines an ordering of the projected vertices which is independent of the ordering of the vertices in the ordered sets.

Abstract: An apparatus for point cloud decoding includes processing circuitry. The processing circuitry receives, from a coded bitstream for a point cloud, encoded occupancy codes for nodes in an octree structure for the point cloud. The nodes in the octree structure correspond to three dimensional (3D) partitions of a space of the point cloud. Sizes of the nodes are associated with sizes of the corresponding 3D partitions. Further, the processing circuitry decodes, from the encoded occupancy codes, occupancy codes for the nodes. At least a first occupancy code for a child node of a first node is decoded without waiting for a decoding of a second occupancy code for a second node having a same node size as the first node. Then, the processing circuitry reconstructs the octree structure based on the decoded occupancy codes for the nodes, and reconstructs the point cloud based on the octree structure.

Abstract: A system and method for volume rendering a light field, wherein the light field data is subjected to a layering scheme introducing a partitioning of the hogels into subsets. Each subset corresponding to a sub-volume of the layer volume, corresponds to the sub-region of the layer. Novel partitioning of the data combined with an efficient local memory caching technique, plenoptic downsampling strategies to reduce memory bandwidth requirements and volume rendering algorithm to produce a rendered light field image. A reduction in the total number of samples required can be obtained while still maintaining the quality of the resulting image. A method is also provided to order memory accesses aligned with ray calculations in order to maximize access coherency. Real-time layered scene decomposition can be combined with surface rendering method to create a hybrid real-time rendering method that supports rendering of scenes containing superimposed volumes and surfaces.

Abstract: A method of multi-sensor data fusion includes determining a plurality of first data sets using a plurality of sensors, each of the first data sets being associated with a respective one of a plurality of sensor coordinate systems, and each of the sensor coordinate systems being defined in dependence of a respective one of a plurality of mounting positions for the sensors; transforming the first data sets into a plurality of second data sets using a transformation rule, each of the second data sets being associated with a unified coordinate system, the unified coordinate system being defined in dependence of at least one predetermined reference point; and determining at least one fused data set by fusing the second data sets.

Abstract: A system and method enabling location-based activation of applications. A server system stores and provides a persistent virtual world system comprising one or more applications virtually attached to virtual-world entities positioned according to real-world coordinates, the server system being configured to compute the state of the virtual-world entities and to process the applications. Upon detecting the user devices in a corresponding zone (e.g., by the wireless network detecting user devices connecting to the server), the server system sends a minimum amount of application graphical representation to the user devices. Upon user interaction with the application graphical representation, the user devices download a minimum amount of application data, prompting the server system to determine suitable application media streams.

Abstract: In an exemplary embodiment, on the basis of a first direction input performed on an operation device, a player object is caused to perform a posturing action of holding an item object in an orientation corresponding to an input direction according to the first direction input. Further, in accordance with cancelation of the first direction input, the player object is caused to perform a swinging action of swinging the item object.

Abstract: A method for operating a robotic system includes determining a discretized object model representative of a target object; determining a discretized platform model representative of a task location; determining height measures based on real-time sensor data representative of the task location; and dynamically deriving a placement location based on (1) overlapping the discretized object model and the discretized platform model for stacking objects at the task location and (2) calculating a placement score associated with the overlapping based on the height measures.

Abstract: A method may include providing geometry data and attribute information of the model in one or more data structures on a data storage device and performing a rendering process for providing a visualization of the model on the display for each of the multitude of viewpoints. The rendering process may include rendering a multitude of like 3D arrays in a single draw call, each array comprising at least eight voxels, removing voxels that are not to be visualized by folding vertices to a centre of the array, computing a distortion on a mapping from voxel space to attribute volume space in dependence on the removed voxels, assigning attributes to the voxels of the multitude of arrays using distorted values for retrieving attribute values from attribute volume space, the attributes being assigned to the remaining voxels according to the computed distortion, and visualizing the voxels of the multitude of arrays.

Abstract: Embodiments of the present invention are directed to gaming devices that provide audio-visual animated characters in response to game play. The character has a personality that may be encouraging, taunting or another quality. A plurality of expressions of the personality is presented, between one extreme and another, dependant upon the history of game outcomes.

Abstract: An automated truck unloader for unloading/unpacking product, such as boxes or cases, from trailers and containers is disclosed. In one embodiment, a mobile base structure provides a support framework for a drive subassembly, conveyance subassembly, an industrial robot, a distance measurement subassembly, and a control subassembly. Under the operation of the control subassembly, an industrial robot having a suction cup-based gripper arm selectively removes boxes from the trailer and places the boxes on a powered transportation path. The control subassembly coordinates the selective articulated movement of the industrial robot and the activation of the drive subassembly based upon a perception-based robotic manipulation system.

Abstract: A user, a manipulator such as a hand, and at least one entity such as a virtual or augmented reality object are in an interface such as a 3D environmental interface. The manipulation distance is the distance between a reference feature of the user and a manipulation feature of the manipulator. The entity distance is the distance between the reference feature and an entity feature of the entity. When the manipulation distance becomes greater than the entity distance, the entity is caused to fade, disappear, move out of the way, shrink, etc. so as to be less of an obstruction to the user's field of view, for example to avoid obstructing more distant entities. Other factors than the manipulation distance and entity distance may be considered in determining whether to reduce the obstructivity of the entity, and exceptions to the obstruction relation may be considered.

Abstract: 3D reconstruction method based on deep learning includes the following steps: (1) The potential vector constrained in the input image is used to reconstruct the complete 3D shape of the target, and the mapping between the part and the complete 3D shape is learned, then the 3D reconstruction of a single depth image is realized. (2) Learn the intermediate feature representation between the 3D real object and the reconstructed object to obtain the target potential variables in step (1). (3) The voxel floating value predicted in step (1) is transformed into binary value by using the limit learning machine to complete high-precision reconstruction.

Abstract: Techniques for improving remote rendering and reprojection are disclosed herein. A color image is generated, where this color image includes overlapping content regions. A depth buffer is generated for the color image and includes depth values for the pixels in the color image. The depth buffer includes both essential and non-essential depth discontinuities. While preserving the essential depth discontinuities, the non-essential depth discontinuities are eliminated from the depth buffer. New non-essential discontinuities are prevented from being included in the final version of the depth buffer. The color image is encoded into a color image video stream, and the modified depth buffer is encoded into a depth buffer stream. The color image video stream and the depth buffer stream are then transmitted to a remotely located HMD. The HMD then reprojects the color image based on the depth values in the depth buffer.

Abstract: A method of transmitting three-dimensional (3D) video content, according to an embodiment of the present disclosure, includes: defining a 3D space; obtaining placement information about at least one 3D object to be placed in the 3D space; dividing the at least one 3D object; compressing 3D video content including the divided at least one 3D object into a two-dimensional (2D) video frame; obtaining, from the 2D video frame, location information about compressed data corresponding to the divided at least one 3D object; encoding the 2D video frame; generating a media file including the encoded 2D video frame and the location information; and transmitting the media file.

Abstract: A system and method are provided for determining vascular velocity using non-invasively acquired medical images. The method includes reconstructing CT angiography (CTA) data into a plurality of images of the subject by producing a composite image using the CTA data corresponding to a set of the plurality of view angles, backprojecting each view angle in the CTA data and weighting a value backprojected into at image pixel by an attenuation value of a corresponding pixel in the composite image, and summing backprojected values for each image pixel to produce a CT image of the subject. The method also includes determining a flow direction or a velocity of flow within a vessel, calculating, using the flow direction or velocity, a pressure in the vessel, and generating a quantitative map of the subject indicating the flow direction, velocity, or pressure in the vessel against an image of the subject including the vessel.

Abstract: A surround view system for a vehicle includes a detection unit and an evaluation unit. The detection unit is designed to detect data relating to the surroundings. The evaluation unit is designed to identify an object in the detected surroundings data and to determine the 3D shape of this object. The evaluation unit is additionally designed to add the determined 3D shape to a projection surface of the surround view system to produce a modified projection surface. The evaluation unit is designed to project the surroundings data onto the modified projection surface.

Abstract: Systems and methods for generating a three-dimensional (3D) model of a user's dental arch based on two-dimensional (2D) images include a model training system that receives data packets of a training set. Each data packet may include data corresponding to training images of a respective dental arch and a 3D training model of the respective dental arch. The model training system identifies, for a data packet, correlation points between the one or more training images and the 3D training model of the respective dental arch. The model training system generates a machine learning model using the correlation points for the data packets of the training set. A model generation system receives one or more images of a dental arch. The model generation system generates a 3D model of the dental arch by applying the images of the dental arch to the machine learning model.

Abstract: A method for preemptively generating and rendering a view of a computerized image is provided. The method may include determining rendering parameters for the computerized image by predicting an action to be performed on the computerized image, wherein the action modifies a view of the computerized image, and wherein the determined rendering parameters are based on the modified view of the computerized image. The method may further include preemptively rendering the view of the computerized image that is based on the determined rendering parameters before the action is performed on the computerized image.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for utilizing an encoder-decoder architecture to learn a volumetric 3D representation of an object using digital images of the object from multiple viewpoints to render novel views of the object. For instance, the disclosed systems can utilize patch-based image feature extraction to extract lifted feature representations from images corresponding to different viewpoints of an object. Furthermore, the disclosed systems can model view-dependent transformed feature representations using learned transformation kernels. In addition, the disclosed systems can recurrently and concurrently aggregate the transformed feature representations to generate a 3D voxel representation of the object. Furthermore, the disclosed systems can sample frustum features using the 3D voxel representation and transformation kernels.

Abstract: In one example, a representation of a volume of a part to be printed by an additive manufacturing system is generated, where the representation depicts a plurality of voxels. A plurality of slices of the representation are generated. Each slice relates to a plurality of voxels within a first plane. A first process is performed in respect of a slice of the plurality of slices, wherein in the first process at least one voxel that has a predetermined colour and is located on a predetermined surface of the part is identified and a data file is updated to include data representative of the identified at least one voxel. The first process is repeated in respect of another slice. A three-dimensional preview of the part to be printed is generated based on the data file.

Abstract: The disclosure herein relates to systems, methods, and devices for medical image analysis, diagnosis, risk stratification, decision making and/or disease tracking. In some embodiments, the systems, devices, and methods described herein are configured to analyze non-invasive medical images of a subject to automatically and/or dynamically identify one or more features, such as plaque and vessels, and/or derive one or more quantified plaque parameters, such as radiodensity, radiodensity composition, volume, radiodensity heterogeneity, geometry, location, and/or the like. In some embodiments, the systems, devices, and methods described herein are further configured to generate one or more assessments of plaque-based diseases from raw medical images using one or more of the identified features and/or quantified parameters.

Abstract: There is provided a display control device including a display controller configured to place a virtual object within an augmented reality space corresponding to a real space in accordance with a recognition result of a real object shown in an image captured by an imaging part, and an operation acquisition part configured to acquire a user operation. When the user operation is a first operation, the display controller causes the virtual object to move within the augmented reality space.

Abstract: Disclosed herein is a point cloud data transmission method. The transmission method may include encoding point cloud data, and transmitting point cloud data. Disclosed herein is a point cloud data reception device. Disclosed herein is a point cloud data reception method. The reception method may include receiving point cloud data, decoding the point cloud data, and rendering the point cloud data.