Patents by Inventor Stephen Charles Hsu

Stephen Charles Hsu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190234749
    Abstract: Described is a computer-implemented method which comprises receiving a plurality of images captured by at least one user device (120), wherein each image is associated with one of a corresponding plurality of geographic locations (A-F, L-R); determining a path between the plurality of geographic locations (201, 202); determining a confidence indicator (1050, 1150) representative of whether the determined path corresponds to a demarked path, wherein determining the confidence indicator comprises determining a time of capture of each of the plurality of images (601, 602); identifying the path as corresponding to a demarked route (1231), based on the confidence indicator; and marking the plurality of images for display as a demarked route.
    Type: Application
    Filed: October 7, 2016
    Publication date: August 1, 2019
    Inventor: Stephen Charles Hsu
  • Patent number: 10326933
    Abstract: This technology relates to optimizing location and orientation information of an image using known locations of places captured within the image. For example, an image and associated pose data including the image's orientation and location may be received. One or more places captured within the image may be determined, with each place having a respective known location. The image may be annotated with the one or more places. A difference between each annotation and its respective known location to obtain updated pose data of the image may be minimized and the associated pose data may be updated to the updated pose data.
    Type: Grant
    Filed: June 14, 2017
    Date of Patent: June 18, 2019
    Assignee: Google LLC
    Inventors: Tianqiang Liu, Meng Yi, Xin Mao, Jacqueline Anne Lai, Daniel Joseph Filip, Stephen Charles Hsu
  • Publication number: 20180367730
    Abstract: This technology relates to optimizing location and orientation information of an image using known locations of places captured within the image. For example, an image and associated pose data including the image's orientation and location may be received. One or more places captured within the image may be determined, with each place having a respective known location. The image may be annotated with the one or more places. A difference between each annotation and its respective known location to obtain updated pose data of the image may be minimized and the associated pose data may be updated to the updated pose data.
    Type: Application
    Filed: June 14, 2017
    Publication date: December 20, 2018
    Applicant: Google Inc.
    Inventors: Tianqiang Liu, Meng Yi, Xin Mao, Jacqueline Anne Lai, Daniel Joseph Filip, Stephen Charles Hsu
  • Publication number: 20170278293
    Abstract: Systems and methods for processing textures to be applied to surface of a three-dimensional model, such as a three-dimensional model of a geographic area, are provided. According to aspects of the present disclosure, a two-dimensional image processing operation can be performed in the two-dimensional texture atlas space defined by a texture atlas using manifold neighborhoods defined for pixels in the texture atlas. The manifold neighborhood for a pixel can be the set of texture atlas pixels whose corresponding position on the surface of the three-dimensional model lies within a threshold distance of the surface position corresponding to the pixel in the three-dimensional model. The two-dimensional image processing operations can be performed using the set of texture atlas pixels in the manifold neighborhood.
    Type: Application
    Filed: July 18, 2013
    Publication date: September 28, 2017
    Inventor: Stephen Charles Hsu
  • Patent number: 9454803
    Abstract: Systems and methods for generating a composite image from a plurality of source images using a scene dependent multi-band blending operation are provided. The multi-band blending operation implements a filtering operation to reduce blending between objects or surfaces that have natural color and/or brightness differences. More particularly, the typical space invariant upsampling that occurs during multi-band blending can be replaced by a scene dependent filtering operation during upsampling that omits or reduces contributions from pixels associated with different objects in a scene during the multi-band blending process. The scene dependent filtering can be based on scene dependent data, such as height data or slope data, which can be used to identify different objects in a scene.
    Type: Grant
    Filed: July 15, 2014
    Date of Patent: September 27, 2016
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu
  • Patent number: 9240055
    Abstract: The aspects described herein relate to replacing pixels in images in order to remove obstructions from images. In one example, an image of a scene having symmetrical features and image information identifying at least one pixel of the image corresponding to a hole to be filled may be received. This hole may correspond to an obstruction in the image. A set of symmetry axes may be identified based on the symmetrical features. A symmetry map identifying correspondences between different pixels in the image based on the set of symmetry axes may be generated. A correspondence between the at least one pixel corresponding to a hole to be filled and a second pixel of the image is identified based at least in part on the symmetry map and the image information. The at least one pixel may be altered based on the identified correspondence in order to remove the obstruction.
    Type: Grant
    Filed: September 16, 2014
    Date of Patent: January 19, 2016
    Assignee: Google Inc.
    Inventors: Yanxi Liu, Arthur Robert Pope, Vivek Verma, Stephen Charles Hsu
  • Patent number: 9224233
    Abstract: An example method and system for blending textures of a composite image formed by a plurality of source images mapped onto a three dimensional model are presented. The composite image is projected to obtain an unblended projected image having textures. The textures are blended to obtain a blended projected image. Both the unblended and the blended projected images are backprojected onto the three dimensional model. A difference is determined between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image. The determined difference is then applied to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.
    Type: Grant
    Filed: May 24, 2012
    Date of Patent: December 29, 2015
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu
  • Publication number: 20150170398
    Abstract: An exemplary method for simplifying a texture of a three-dimensional model includes simplifying a first three-dimensional model to determine a second three-dimensional model. The first three-dimensional model has a higher resolution than the second three-dimensional model. The method also includes allocating a texture atlas for the second three-dimensional model. The method further includes filling in the texture atlas for the second three-dimensional model.
    Type: Application
    Filed: May 8, 2012
    Publication date: June 18, 2015
    Applicant: Google Inc.
    Inventor: Stephen Charles HSU
  • Publication number: 20150154805
    Abstract: An example method and system for blending textures of a composite image formed by a plurality of source images mapped onto a three dimensional model are presented. The composite image is projected to obtain an unblended projected image having textures. The textures are blended to obtain a blended projected image. Both the unblended and the blended projected images are backprojected onto the three dimensional model. A difference is determined between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image. The determined difference is then applied to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.
    Type: Application
    Filed: May 24, 2012
    Publication date: June 4, 2015
    Applicant: Google Inc.
    Inventor: Stephen Charles HSU
  • Patent number: 9041711
    Abstract: An exemplary method for simplifying a texture of a three-dimensional model includes simplifying a first three-dimensional model to determine a second three-dimensional model. The first three-dimensional model has a higher resolution than the second three-dimensional model. The method also includes allocating a texture atlas for the second three-dimensional model. The method further includes filling in the texture atlas for the second three-dimensional model.
    Type: Grant
    Filed: May 8, 2012
    Date of Patent: May 26, 2015
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu
  • Patent number: 8884955
    Abstract: An exemplary method for simplifying the geometry of a three-dimensional mesh includes dividing a first three-dimensional mesh into a first plurality of tiles along initial boundary lines. The method also includes for at least one tile of the first plurality of tiles, reducing the vertices toward a center of the tile while maintaining vertices along a perimeter of the tile, and assembling the first plurality of tiles to determine a second three-dimensional mesh. The method also includes dividing the second three-dimensional mesh into a second plurality of tiles along boundary lines offset from the initial boundary lines. The method further includes for at least one tile of the second plurality of tiles, reducing the vertices toward a center of the tile while maintaining vertices along a perimeter of the tile, and assembling the second plurality of tiles to determine a simplified three-dimensional mesh.
    Type: Grant
    Filed: May 4, 2012
    Date of Patent: November 11, 2014
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu
  • Patent number: 8811764
    Abstract: Systems and methods for generating a composite image from a plurality of source images using a scene dependent multi-band blending operation are provided. The multi-band blending operation implements a filtering operation to reduce blending between objects or surfaces that have natural color and/or brightness differences. More particularly, the typical space invariant upsampling that occurs during multi-band blending can be replaced by a scene dependent filtering operation during upsampling that omits or reduces contributions from pixels associated with different objects in a scene during the multi-band blending process. The scene dependent filtering can be based on scene dependent data, such as height data or slope data, which can be used to identify different objects in a scene.
    Type: Grant
    Filed: October 25, 2012
    Date of Patent: August 19, 2014
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu
  • Publication number: 20140125667
    Abstract: System, methods and computer program product embodiments for roof generation of 3D models from ground acquired data. An embodiment includes grouping a plurality of 3D model facades and aligning each group of facades with aerial imagery. The embodiment further comprises identifying if a space between the facades is located within or outside a 3D model, generating a roof that connects plurality of facades and texturing the generated roof with the aerial imagery. In this way, when 3D data is limited to 3D model facades and lacks roof (and vertical face) information, embodiments of the invention create complete 3D models by generating a roof that connects plurality of facades and texturing the generated roof with the aerial imagery. This greatly improves user experience as 3D models closely approximate their respective real world counterparts.
    Type: Application
    Filed: November 11, 2010
    Publication date: May 8, 2014
    Applicant: Google Inc.
    Inventors: Emil PRAUN, Stephen Charles HSU, Xiaqing WU
  • Patent number: 7741961
    Abstract: An obstacle detection and tracking system identifies objects in the path of a vehicle equipped with the system and issues a visual, audible, and/or control system warning. The system includes a depth imaging system that acquires depth data from objects in the field of view of a detection zone encompassing at least a portion of the road to be driven upon. It is assumed most of the acquired data represents road plane information. Statistical analysis of the depth image data identifies in (X,Y,Z) space at least one plane of the road being driven, after which identification threshold normal heights above and below the road plane are defined. Imaged objects within the detection zone that are higher or lower than a threshold normal are deemed of potential concern and will generate a warning to the vehicle operator or vehicle.
    Type: Grant
    Filed: September 28, 2007
    Date of Patent: June 22, 2010
    Assignee: Canesta, Inc.
    Inventors: Abbas Rafii, Colin Tracey, Stephen Charles Hsu
  • Patent number: 7728833
    Abstract: A method and apparatus for automatically generating a three-dimensional computer model from a “point cloud” of a scene produced by a laser radar (LIDAR) system. Given a point cloud of an indoor or outdoor scene, the method extracts certain structures from the imaged scene, i.e., ceiling, floor, furniture, rooftops, ground, and the like, and models these structures with planes and/or prismatic structures to achieve a three-dimensional computer model of the scene. The method may then add photographic and/or synthetic texturing to the model to achieve a realistic model.
    Type: Grant
    Filed: August 18, 2005
    Date of Patent: June 1, 2010
    Assignee: Sarnoff Corporation
    Inventors: Vivek Verma, Rakesh Kumar, Stephen Charles Hsu, Harpreet Sawhney
  • Patent number: 7509241
    Abstract: A method and apparatus for automatically combining aerial images and oblique images to form a three-dimensional (3D) site model. The apparatus or method is supplied with aerial and oblique imagery. The imagery is processed to identify building boundaries and outlines as well as to produce a depth map. The building boundaries and the depth map may be combined to form a 3D plan view model or used separately as a 2D plan view model. The imagery and plan view model is further processed to determine roof models for the buildings in the scene. The result is a 3D site model having buildings represented rectangular boxes with accurately defined roof shapes.
    Type: Grant
    Filed: July 8, 2002
    Date of Patent: March 24, 2009
    Assignee: Sarnoff Corporation
    Inventors: Yanlin Guo, Rakesh Kumar, Stephen Charles Hsu, Harpreet Singh Sawhney
  • Publication number: 20080181487
    Abstract: A method and apparatus for high-resolution 3D imaging ladar system which can penetrate foliage and camouflage to sample fragments of concealed surfaces of interest is disclosed. Samples collected while the ladar moves can be integrated into a coherent object shape. In one embodiment, a system and method for automatic data-driven registration of ladar frames, comprises a coarse search stage, a pairwise fine registration stage using an iterated closest points algorithm, and a multi-view registration strategy. After alignment and aggregation, it is often difficult for human observers to find, assess and recognize objects from a point cloud display. Basic display manipulations, surface fitting techniques, and clutter suppression to enhance visual exploitation of 3D imaging ladar data may be utilized.
    Type: Application
    Filed: June 25, 2007
    Publication date: July 31, 2008
    Inventors: Stephen Charles Hsu, Supun Samarasekera, Rakesh Kumar, Wen-Yi Zhao, Keith J. Hanna
  • Patent number: 7242460
    Abstract: A method and apparatus for high-resolution 3D imaging ladar system which can penetrate foliage and camouflage to sample fragments of concealed surfaces of interest is disclosed. Samples collected while the ladar moves can be integrated into a coherent object shape. In one embodiment, a system and method for automatic data-driven registration of ladar frames, comprises a coarse search stage, a pairwise fine registration stage using an iterated closest points algorithm, and a multi-view registration strategy. After alignment and aggregation, it is often difficult for human observers to find, assess and recognize objects from a point cloud display. Basic display manipulations, surface fitting techniques, and clutter suppression to enhance visual exploitation of 3D imaging ladar data may be utilized.
    Type: Grant
    Filed: April 16, 2004
    Date of Patent: July 10, 2007
    Assignee: Sarnoff Corporation
    Inventors: Stephen Charles Hsu, Supun Samarasekera, Rakesh Kumar, Wen-Yi Zhao, Keith J. Hanna
  • Patent number: 7187809
    Abstract: A method and apparatus for performing two-dimensional video alignment onto three-dimensional point clouds. The system recovers camera pose from camera video, determines a depth map, converts the depth map to a Euclidean video point cloud, and registers two-dimensional video to the three-dimensional point clouds.
    Type: Grant
    Filed: June 10, 2005
    Date of Patent: March 6, 2007
    Assignee: Sarnoff Corporation
    Inventors: Wenyi Zhao, David Nister, Stephen Charles Hsu
  • Patent number: D777734
    Type: Grant
    Filed: April 10, 2015
    Date of Patent: January 31, 2017
    Assignee: Google Inc.
    Inventor: Stephen Charles Hsu