Abstract: Methods and systems for comparing a 3D model of a target object to a shape-search database are provided. An example method includes using a mobile device to acquire a plurality of images of a target object, determining a 3D model based on the images, transmitting a search query that includes the 3D model, and receiving a search query result. In another example method, a server could receive a search query that includes a 3D model of an object, compare the 3D model to a shape-search database, generate a search query result based on the comparison, and transmit the search query result. The search query result could include one or more of: information regarding the target object, information regarding one or more objects similar to the target object, and a suggestion for acquiring additional images of the target object.

Abstract: Methods and systems for controlling light arrays to determine properties of an object are described. An example method includes causing illumination of a surface of an object from multiple illumination positions using a programmable array of lights, and receiving images from an image-capture device while the surface of the object is illuminated. For example, the programmable array of lights may be modulated to cause illumination of a portion of the surface of the object from first and second illumination positions, and a first and second image of the surface of the object captured during illumination from the first and second illumination positions respectively may be received. Subsequently, a processor may determine material information for the object based on an amount of specular reflectivity for the surface of the object and reference to a database of known amounts of specular reflectivity for a plurality of materials.

Abstract: Methods and systems for providing a three-dimensional (3D) image viewer in a webpage are provided. According to an example method, a webpage may be provided, and the webpage may include embedded language that identifies a 3D image viewer to be provided within the webpage. Based on the embedded language, a computer having a processor and a memory may request information associated with rendering a 3D object data model in the 3D image viewer. The method may also include providing the 3D image viewer within the webpage, and receiving information associated with rendering the 3D object data model. Additionally, the 3D object data model may be rendered in the 3D image viewer based on the received information. Additional example systems and methods are described herein.

Abstract: Examples disclose a method and system for merging textures. The method may be executable to receive one or more images of an object, identify a texture value for a point in a first image of the one or more images, and determine a metric indicative of a relation between a view reference point vector and a normal vector of a position of a point on the object relative to the image capturing device. Based on the metrics, the method may be executable to determine a weighted average texture value to apply to a corresponding point of a three-dimensional mesh of the object.

Abstract: System and methods for rendering three-dimensional (3D) object data models based on a comparison of images. A 3D object data model of an object can be characterized by parameters defining rendering features of the 3D object data model. A comparison can be made of a first rendering of the 3D object data model to one or more reference images related to the object and, based on the comparison, the parameters of the 3D object data model can be modified. Following the modification, the 3D object data model can be rendered to generate a second rendering. Based on the second rendered 3D object data model, statistical information can be obtained and based on the statistical information, the parameters of the 3D object data model can be modified again to further adjust the appearance of the second rendering of the 3D object data model.

Abstract: Methods and systems are provided for determining and transmitting applicable lighting information, applicable viewing perspective, and a 3D model for an object in response to a search query. An example method includes receiving, at a server, a search query regarding an object. A 3D model for the object is determined. The 3D model includes three-dimensional shape information about the object. The method also includes determining, based on a plurality of stored images of the object, at least one applicable light field and at least one applicable viewing perspective. A search query result is transmitted from the server. The search query result may include the 3D model, the applicable light field(s), and the applicable viewing perspective(s). A server and a non-transitory computer readable medium are also disclosed that could perform a similar method.

Abstract: A cloud computing system is configured to (i) receive image and environmental data from a computing device, (ii) apply a plurality of image processing algorithms to the received image a plurality of times to generate a corresponding plurality of image processing results, where each application of an image processing algorithm to the received image is executed with a different corresponding parameter set, and (iii) based on the image processing results, select an image processing algorithm and corresponding parameter set for the computing device to use for image processing operations. The cloud computing device may also correlate the results of its analysis with the environmental data received from the computing device, and store the correlation in a machine vision knowledge base for future reference. In some embodiments, the computing device is a component of a robot.

Abstract: Examples disclose a method and system to transform a colored point cloud to a 3D textured mesh. The method may be executable to identify a location on a 2D image of an object, identify a location on a 3D image of the object, and determine a color associated with the location on the 2D image. Determining a color may include receiving data associated with a simulation of a plurality of rays cast on the 3D image, identifying a color of the location on the 3D image associated with the received data, identifying a confidence level associated with the identified color of the location on the 3D image, and associating the identified color of the location on the 3D image with the location on the 2D image.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for encoding and compressing 3D object data models are provided. An example method may involve receiving 3D mesh data for an object that includes geometry coordinates for a surface of the object. Additionally, material properties may be associated with the geometry coordinates. The method may also include identifying multiple portions of the mesh data based on the material properties associated with the geometry coordinates. For example, a given group of adjacent geometry coordinates having common material properties may be identified as a given portion. For at least some of the identified portions of the mesh data, the method may further include encoding information related to an identified portion of the mesh data and compressing the encoded information into a file of compressed geometric data.

Abstract: Methods and systems for multi-modal three-dimensional (3D) scanning of objects are described. An example method may include receiving, from a scanning system, 3D information associated with an object having a first resolution. A region of interest of the object may be determined by a processor based on the 3D information associated with the object. Additionally, instructions for operating the scanning system to determine additional information associated with the region of interest of the object may be determined. The additional information associated with the region of interest may have a second resolution that is higher than the first resolution. The instructions may be provided to the scanning system to determine the additional information associated with the region of interest of the object.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for encoding and compressing 3D object data models are provided. An example method may involve receiving 3D mesh data for an object that includes geometry coordinates for a surface of the object. Additionally, material properties may be associated with the geometry coordinates. The method may also include identifying multiple portions of the mesh data based on the material properties associated with the geometry coordinates. For example, a given group of adjacent geometry coordinates having common material properties may be identified as a given portion. For at least some of the identified portions of the mesh data, the method may further include encoding information related to an identified portion of the mesh data and compressing the encoded information into a file of compressed geometric data.

Abstract: Methods and systems for writing, interpreting, and translating three-dimensional (3D) scenes are provided. An example method may involve accessing data associated with a three-dimensional (3D) scene that includes one or more objects of the 3D scene and one or more rendering effects for the one or more objects. Requests for assets and instructions associated with rendering the one or more objects based on the data associated with the 3D scene may be determined and sent to a server. Additionally, the method may include receiving from the server assets and instructions that facilitate rendering the one or more objects based on the one or more rendering effects. According to the method, the one or more objects of the 3D scene may be rendered based on the received instructions and the received assets.

Abstract: Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Abstract: Methods and systems for determining three-dimensional (3D) object data models of an object based on movement of the object. A processor may receive first scan data and second scan data of an object within in a space. The first scan data may comprise a first position of the object in the space and a first mesh image defining a first view of a shape of the object at the first position. The second scan data may comprise movement information defining a movement of the object from the first position to a second position within the space and a second mesh image defining a second view of the shape of the object at the second position. The first scan data and the second scan data may be aligned to generate a 3D object data model based on the first mesh image, the movement information, and the second mesh image.

Abstract: The present application discloses shared robot knowledge bases for use with cloud computing systems. In one embodiment, the cloud computing system collects data from a robot about an object the robot has encountered in its environment, and stores the received data in the shared robot knowledge base. In another embodiment, the cloud computing system sends instructions for interacting with an object to a robot, receives feedback from the robot based on its interaction with the object, and updates data in the shared robot knowledge base based on the feedback. In yet another embodiment, the cloud computing system sends instructions to a robot for executing an application based on information stored in the shared robot knowledge base. In the disclosed embodiments, information in the shared robot knowledge bases is updated based on robot experiences so that any particular robot may benefit from prior experiences of other robots.

Abstract: Methods and systems for determining three-dimensional (3D) object data models of an object based on movement of the object. A processor may receive first scan data and second scan data of an object within in a space. The first scan data may comprise a first position of the object in the space and a first mesh image defining a first view of a shape of the object at the first position. The second scan data may comprise movement information defining a movement of the object from the first position to a second position within the space and a second mesh image defining a second view of the shape of the object at the second position. The first scan data and the second scan data may be aligned to generate a 3D object data model based on the first mesh image, the movement information, and the second mesh image.

Abstract: Methods and systems are provided for determining and transmitting applicable lighting information, applicable viewing perspective, and a 3D model for an object in response to a search query. An example method includes receiving, at a server, a search query regarding an object. A 3D model for the object is determined. The 3D model includes three-dimensional shape information about the object. The method also includes determining, based on a plurality of stored images of the object, at least one applicable light field and at least one applicable viewing perspective. A search query result is transmitted from the server. The search query result may include the 3D model, the applicable light field(s), and the applicable viewing perspective(s). A server and a non-transitory computer readable medium are also disclosed that could perform a similar method.

Abstract: Methods and systems for material refinement for portions of a three-dimensional (3D) object data model are provided. An example method may include rendering a portion of a 3D object data model, and determining a first appearance metric between an appearance of the portion in the rendered view and a two-dimensional (2D) image. For one or more iterations, a modification to material properties associated with the portion may be determined based on the first appearance metric, and another view of the portion of the 3D object data model may be rendered. Also for the one or more iterations, another appearance metric between and an appearance of the portion in the rendered another view and the 2D image may be determined. Additionally, modified material properties for the portion that are associated with a minimum appearance metric of the one or more iterations may be stored for the 3D object data model.