Patents by Inventor Zhe Fan
Zhe Fan has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20250029274Abstract: The present disclosure provides methods and systems of sampling-based object pose determination. An example method includes obtaining, for a time frame, sensor data of the object acquired by a plurality of sensors; generating a two-dimensional bounding box of the object in a projection plane based on the sensor data of the time frame; generating a three-dimensional pose model of the object based on the sensor data of the time frame and a model reconstruction algorithm; generating, based on the sensor data, the pose model, and multiple sampling techniques, a plurality of pose hypotheses of the object corresponding to the time frame, generating a hypothesis projection of the object for each of the pose hypotheses by projecting the pose hypothesis onto the projection plane; determining evaluation results by comparing the hypothesis projections with the bounding box; and determining, based on the evaluation results, an object pose for the time frame.Type: ApplicationFiled: October 17, 2023Publication date: January 23, 2025Inventors: Yizhe ZHAO, Zhe CHEN, Ye FAN, Lingting GE, Zhe HUANG, Panqu WANG, Xue MEI
-
Patent number: 10061715Abstract: The present invention relates to solving the issues related to subgraph query services with tunable preservation of privacy of structural information. More particularly, it relates to a novel cyclic group based encryption (CGBE) method for private matrix operations.Type: GrantFiled: August 26, 2015Date of Patent: August 28, 2018Assignee: Hong Kong Baptist UniversityInventors: Koon Kau Choi, Zhe Fan
-
Patent number: 9626082Abstract: Embodiments relate to selecting textures for a user-supplied photographic image in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method includes a method for inputting a user-supplied photographic image that uses a photogrammetry algorithm to adjust a plurality of camera parameters for the user-supplied photographic image. In the method, a user-supplied photographic image inputted by a user is received. A set of geographic characteristics inputted by the user that correspond to a geographic location of a camera that took the user-supplied photographic image is received. A plurality of camera parameters that correspond to the geographic location of the camera that took the user-supplied photographic image is determined. The user-supplied photographic image to be texture mapped to the three-dimensional model is enabled.Type: GrantFiled: October 16, 2014Date of Patent: April 18, 2017Assignee: Google Inc.Inventors: Zhe Fan, Scott Shattuck, Mark A. Limber, Matthew W. Lowrie, Brian Gammon Brown, Matthew Robert Simpson, Tilman Reinhardt
-
Publication number: 20160357799Abstract: The present invention relates to solving the issues related to subgraph query services with tunable preservation of privacy of structural information. More particularly, it relates to a novel cyclic group based encryption (CGBE) method for private matrix operations.Type: ApplicationFiled: August 26, 2015Publication date: December 8, 2016Inventors: Koon Kau CHOI, Zhe FAN
-
Patent number: 9269196Abstract: A system that runs in web browsers of mobile devices that allows mobile users to take photos of building exteriors and interiors or other real world objects, upload photos, share photos with others, and use the photo images to model the 3D models with the system's image-based modeling interface.Type: GrantFiled: October 21, 2014Date of Patent: February 23, 2016Assignee: Google Inc.Inventors: Zhe Fan, Mark Limber, Brian Gammon Brown, Tilman Reinhardt, Matthew Simpson
-
Publication number: 20150213590Abstract: Embodiments relate to determining pose data for a user-provided image. A user may model a building in a web browser plug in by mapping positions on two-dimensional images to a three-dimensional model of a building shown in the image. A geometry of the model of the building may be determined. The user may then provide an image that includes the building. One or more features of the selected building in the user-provided image may be detected using computer vision techniques. Detected features are correlated with features of the geometry of the three-dimensional model. Based on the correlation, pose data may be associated with the user-provided image.Type: ApplicationFiled: July 29, 2011Publication date: July 30, 2015Applicant: Google Inc.Inventors: Brian Gammon BROWN, Zhe FAN, Scott SHATTUCK, Matt LOWRIE
-
Publication number: 20150172628Abstract: Embodiments enable alteration of automatically-generated three-dimensional models using photogrammetry. In an embodiment, a method creates a three-dimensional model using a two-dimensional photographic image. An automatically generated three-dimensional model geocoded within a field of view of a camera that took the two-dimensional photographic image is received. A perspective of the camera that took the photographic image is represented by a set of camera parameters for the first two-dimensional photographic image. A user input constraint indicating that a feature of the automatically generated three-dimensional model corresponds to a position on two-dimensional photographic image is also received. In response to the user input constraint, the three-dimensional model is altered, using photogrammetry, according to the user input constraint and the set of camera parameters.Type: ApplicationFiled: June 30, 2011Publication date: June 18, 2015Applicant: Google Inc.Inventors: Brian Gammon BROWN, Tilman Reinhardt, Zhe Fan, Scott Shattuck
-
Patent number: 8902288Abstract: A system that runs in web browsers of mobile devices that allows mobile users to take photos of building exteriors and interiors or other real world objects, upload photos, share photos with others, and use the photo images to model the 3D models with the system's image-based modeling interface.Type: GrantFiled: June 16, 2011Date of Patent: December 2, 2014Assignee: Google Inc.Inventors: Zhe Fan, Mark Limber, Brian Gammon Brown, Tilman Reinhardt, Matthew Robert Simpson
-
Patent number: 8884950Abstract: Embodiments improve pose data for images via user interaction. A user may model a building in a web browser plug in by mapping positions on two-dimensional images to a three-dimensional model of a building shown in the image. Each of the images may have associated original pose data, which includes information related to a position and orientation of a camera which took the image. Upon modeling the building, pose data may be adjusted to more accurately depict the building as a three-dimensional model. If the adjusted pose data is superior to the original pose data, the adjusted pose data may be associated with the image.Type: GrantFiled: July 29, 2011Date of Patent: November 11, 2014Assignee: Google Inc.Inventors: Brian Gammon Brown, Zhe Fan, Tilman Reinhardt
-
Patent number: 8866850Abstract: Embodiments relate to selecting textures for a user-supplied photographic image in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method includes a method for inputting a user-supplied photographic image that uses a photogrammetry algorithm to adjust a plurality of camera parameters for the user-supplied photographic image. In the method, a user-supplied photographic image inputted by a user is received. A set of geographic characteristics inputted by the user that correspond to a geographic location of a camera that took the user-supplied photographic image is received. A plurality of camera parameters that correspond to the geographic location of the camera that took the user-supplied photographic image is determined. The user-supplied photographic image to be texture mapped to the three-dimensional model is enabled.Type: GrantFiled: March 13, 2012Date of Patent: October 21, 2014Assignee: Google Inc.Inventors: Zhe Fan, Scott Shattuck, Mark Limber, Matt Lowrie, Brian Gammon Brown, Matthew Robert Simpson, Tilman Reinhardt
-
Patent number: 8818768Abstract: Embodiments relate to modeling three-dimensional interiors from photographic images. In a first embodiment, a computer-implemented method creates a three-dimensional model of an interior of a building from a photographic image. Input data is received from a user specifying a floor plan of the interior of the building. A first constraint input by a user indicating that a position on the floor plan corresponds to a position on the three-dimensional model is received. A second constraint input by a user indicating that a position on a photographic image of the interior corresponds to a position on the three-dimensional model is received. Finally, the three-dimensional model and camera parameters representing a camera that took the photographic image of the interior are altered based on the first and second constraints.Type: GrantFiled: October 12, 2010Date of Patent: August 26, 2014Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Tilman Reinhardt, Matthew Robert Simpson
-
Patent number: 8817018Abstract: Embodiments relate to constructing a three-dimensional model using a plurality of two-dimensional photographic images. In a first embodiment, a first user input specifying a first position on a first photographic image may be received. A second user input specifying a second position on a second photographic image may be received, the second photographic image having a second perspective, specified by a second set of camera parameters, being different from a first perspective of the first image specified by a first set of camera parameters. The user inputs may indicate corresponding features in the images. Using photogrammetry, a point in a three-dimensional space may be determined according to the first position, the second position, the first set of camera parameters, and the second set of camera parameters. A surface of the three-dimensional model may be determined, whereby the surface may be curved to the point in three-dimensional space determined earlier.Type: GrantFiled: June 13, 2011Date of Patent: August 26, 2014Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Tilman Reinhardt, Matthew Robert Simpson
-
Patent number: 8817067Abstract: Embodiments relate to selecting textures for a panoramic photographic image in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method includes a method for inputting a panoramic photographic image that uses a photogrammetry algorithm to adjust a plurality of camera parameters for the panoramic photographic image. In the method, a street-level panoramic photographic image is selected by a user from a street-level panoramic image display site. An area of the street-level panoramic image is also identified by the user. A photographic image at the area of the street-level panoramic image is captured. Camera parameters representing a camera that took the photographic image is determined. A position on the photographic image is mapped to a position on a three-dimensional model. The camera parameters are adjusted so that the three-dimensional model is rendered onto the photographic image from a perspective specified by the adjusted camera parameters.Type: GrantFiled: July 29, 2011Date of Patent: August 26, 2014Assignee: Google Inc.Inventors: Zhe Fan, Scott Shattuck, Mark Limber, Matt Lowrie, Brian Gammon Brown, Matthew Robert Simpson, Tilman Reinhardt
-
Patent number: 8687877Abstract: Embodiments relate to locking geometric and camera parameters in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method modifies a three-dimensional model using a set of two-dimensional photographic images. In the method, at least one constraint, input by a user, for a two-dimensional photographic image from the set of two-dimensional photographic images are received. Each constraint indicates that a position on the two-dimensional photographic image corresponds to a position on the three-dimensional model. When the number of constraints received for the two-dimensional photographic image is less than or equal to a first threshold value, camera parameters representing a camera that took the constrained photographic image. Finally, a photogrammetry algorithm is applied to at least the set of parameters to determine at least one unlocked parameter in the set of parameters and to modify the three-dimensional model based on the constraints.Type: GrantFiled: January 13, 2012Date of Patent: April 1, 2014Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Tilman Reinhardt
-
Selecting and verifying textures in image-based three-dimensional modeling, and applications thereof
Patent number: 8669976Abstract: Embodiments relate to selecting and verifying textures in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method for texturing a three-dimensional model. In the method, a polygon orientation of a polygon defining a face of a three-dimensional model is determined. A first camera orientation corresponding to a camera that took a first photographic image in a set of photographic images is also determined. The first photographic image is unconstrained to the three-dimensional model. The polygon orientation is compared with the first camera orientation to determine whether the first photographic image is directed toward the face of the three-dimensional model. If the first photographic image is directed toward the face of the three-dimensional model, a notification is provided for display to a user indicating that the first photographic image could be texture mapped to the three-dimensional model if the first photographic image is constrained to the three-dimensional model.Type: GrantFiled: October 12, 2010Date of Patent: March 11, 2014Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Matthew Robert Simpson, Tilman Reinhardt -
Selecting and verifying textures in image-based three-dimensional modeling, and applications thereof
Patent number: 8279218Abstract: Embodiments relate to selecting and verifying textures in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method for texturing a three-dimensional model. In the method, a polygon orientation of a polygon defining a face of a three-dimensional model is determined. A first camera orientation corresponding to a camera that took a first photographic image in a set of photographic images is also determined. The first photographic image is unconstrained to the three-dimensional model. The polygon orientation is compared with the first camera orientation to determine whether the first photographic image is directed toward the face of the three-dimensional model. If the first photographic image is directed toward the face of the three-dimensional model, a notification is provided for display to a user indicating that the first photographic image could be texture mapped to the three-dimensional model if the first photographic image is constrained to the three-dimensional model.Type: GrantFiled: September 26, 2011Date of Patent: October 2, 2012Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Matthew Robert Simpson, Tilman Reinhardt -
Patent number: 8125522Abstract: A filter for filtering out spurious motion from a sequence of video images, for use in video image signal processing to identify objects in motion in the sequence of video images. Spurious motion is chaotic, repetitive, jittering portions of an image that constitute noise and interfere with motion detection in video signals. The filter keeps track of the location and the strengths of spurious motion, applies appropriate low pass filtering strengths according to the spurious motion strengths in real-time. Regular pixels without spurious motion will pass through the filter unaltered, while pixels with spurious motion will be “smoothed” to avoid being detected as noise.Type: GrantFiled: March 21, 2007Date of Patent: February 28, 2012Assignee: Siemens Industry, Inc.Inventor: Zhe Fan
-
Patent number: 8115762Abstract: Embodiments relate to locking geometric and camera parameters in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method modifies a three-dimensional model using a set of two-dimensional photographic images. In the method, at least one constraint, input by a user, for a two-dimensional photographic image from the set of two-dimensional photographic images are received. Each constraint indicates that a position on the two-dimensional photographic image corresponds to a position on the three-dimensional model. When the number of constraints received for the two-dimensional photographic image is less than or equal to a first threshold value, camera parameters representing a camera that took the constrained photographic image. Finally, a photogrammetry algorithm is applied to at least the set of parameters to determine at least one unlocked parameter in the set of parameters and to modify the three-dimensional model based on the constraints.Type: GrantFiled: September 26, 2011Date of Patent: February 14, 2012Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Tilman Reinhardt
-
Patent number: 8115761Abstract: Embodiments relate to locking geometric and camera parameters in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method modifies a three-dimensional model using a set of two-dimensional photographic images. In the method, at least one constraint, input by a user, for a two-dimensional photographic image from the set of two-dimensional photographic images are received. Each constraint indicates that a position on the two-dimensional photographic image corresponds to a position on the three-dimensional model. When the number of constraints received for the two-dimensional photographic image is less than or equal to a first threshold value, camera parameters representing a camera that took the constrained photographic image. Finally, a photogrammetry algorithm is applied to at least the set of parameters to determine at least one unlocked parameter in the set of parameters and to modify the three-dimensional model based on the constraints.Type: GrantFiled: October 12, 2010Date of Patent: February 14, 2012Assignee: Google Inc.Inventors: Zhe Fan, Brian Gammon Brown, Tilman Reinhardt
-
Publication number: 20070223596Abstract: A filter for filtering out spurious motion from a sequence of video images, for use in video image signal processing to identify objects in motion in the sequence of video images. Spurious motion is chaotic, repetitive, jittering portions of an image that constitute noise and interfere with motion detection in video signals. The filter keeps track of the location and the strengths of spurious motion, applies appropriate low pass filtering strengths according to the spurious motion strengths in real-time. Regular pixels without spurious motion will pass through the filter unaltered, while pixels with spurious motion will be “smoothed” to avoid being detected as noise.Type: ApplicationFiled: March 21, 2007Publication date: September 27, 2007Inventor: Zhe Fan