Patents by Inventor Zehang Sun

Zehang Sun has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20220353432
    Abstract: Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or “AR selfies.” In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
    Type: Application
    Filed: July 8, 2022
    Publication date: November 3, 2022
    Inventors: Xiaohuan Corina Wang, Zehang Sun, Joe Weil, Omid Khalili, Stuart Mark Pomerantz, Marc Robins, Toshihiro Horie, Eric Beale, Nathalie Castel, Jean-Michel Berthoud, Brian Walsh, Kevin O'Neil, Andy Harding, Greg Dudey
  • Patent number: 11481097
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Grant
    Filed: January 28, 2021
    Date of Patent: October 25, 2022
    Assignee: Apple Inc.
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Patent number: 11394898
    Abstract: Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or “AR selfies.” In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
    Type: Grant
    Filed: September 6, 2018
    Date of Patent: July 19, 2022
    Assignee: Apple Inc.
    Inventors: Xiaohuan Corina Wang, Zehang Sun, Joe Weil, Omid Khalili, Stuart Mark Pomerantz, Marc Robins, Toshihiro Horie, Eric Beale, Nathalie Castel, Jean-Michel Berthoud, Brian Walsh, Kevin O'Neil, Andy Harding, Greg Dudey
  • Publication number: 20210149549
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Application
    Filed: January 28, 2021
    Publication date: May 20, 2021
    Applicant: Apple Inc.
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Patent number: 10942634
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Grant
    Filed: December 24, 2019
    Date of Patent: March 9, 2021
    Assignee: Apple Inc.
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Patent number: 10839577
    Abstract: Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or “AR selfies.” In an embodiment, a method comprises: capturing, by a first camera of a mobile device, image data, the image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
    Type: Grant
    Filed: October 31, 2018
    Date of Patent: November 17, 2020
    Assignee: Apple Inc.
    Inventors: Toshihiro Horie, Kevin O'Neil, Zehang Sun, Xiaohuan Corina Wang, Joe Weil, Omid Khalili, Stuart Mark Pomerantz, Marc Robins, Eric Beale, Nathalie Castel, Jean-Michel Berthoud, Brian Walsh, Andy Harding, Greg Dudey
  • Publication number: 20200133469
    Abstract: Sonic embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Application
    Filed: December 24, 2019
    Publication date: April 30, 2020
    Applicant: Apple Inc.
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Patent number: 10552016
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Grant
    Filed: December 23, 2016
    Date of Patent: February 4, 2020
    Assignee: Apple Inc.
    Inventors: Timothy D. Cherna, Randy Ubillos, Zehang Sun
  • Publication number: 20190080498
    Abstract: Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or “AR selfies.” In an embodiment, a method comprises: capturing, by a first camera of a mobile device, image data, the image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
    Type: Application
    Filed: October 31, 2018
    Publication date: March 14, 2019
    Applicant: Apple Inc.
    Inventors: Toshihiro Horie, Kevin O'Neil, Zehang Sun, Xiaohuan Corina Wang, Joe Weil, Omid Khalili, Stuart Mark Pomerantz, Marc Robins, Eric Beale, Nathalie Castel, Jean-Michel Berthoud, Brian Walsh, Andy Harding, Greg Dudey
  • Publication number: 20190082118
    Abstract: Systems, methods, apparatuses and non-transitory, computer-readable storage mediums are disclosed for generating AR self-portraits or “AR selfies.” In an embodiment, a method comprises: capturing, by a first camera of a mobile device, live image data, the live image data including an image of a subject in a physical, real-world environment; receiving, by a depth sensor of the mobile device, depth data indicating a distance of the subject from the camera in the physical, real-world environment; receiving, by one or more motion sensors of the mobile device, motion data indicating at least an orientation of the first camera in the physical, real-world environment; generating a virtual camera transform based on the motion data, the camera transform for determining an orientation of a virtual camera in a virtual environment; and generating a composite image data, using the image data, a matte and virtual background content selected based on the virtual camera orientation.
    Type: Application
    Filed: September 6, 2018
    Publication date: March 14, 2019
    Applicant: Apple Inc.
    Inventors: Xiaohuan Corina Wang, Zehang Sun, Joe Weil, Omid Khalili, Stuart Mark Pomerantz, Marc Robins, Toshihiro Horie, Eric Beale, Nathalie Castel, Jean-Michel Berthoud, Brian Walsh, Kevin O'Neil, Andy Harding, Greg Dudey
  • Patent number: 9734587
    Abstract: In some implementations, a computing device can track an object from a first image frame to a second image frame using a self-correcting tracking method. The computing device can select points of interest in the first image frame. The computing device can track the selected points of interest from the first image frame to the second image frame using optical flow object tracking. The computing device can prune the matching pairs of points and generate a transform based on the remaining matching pairs to detect the selected object in the second image frame. The computing device can generate a tracking confidence metric based on a projection error for each point of interest tracked from the first frame to the second frame. The computing device can correct tracking errors by reacquiring the object when the tracking confidence metric is below a threshold value.
    Type: Grant
    Filed: September 30, 2015
    Date of Patent: August 15, 2017
    Assignee: APPLE INC.
    Inventors: Zehang Sun, Toshihiro Horie, Xin Tong, Peter Chou
  • Publication number: 20170109023
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Application
    Filed: December 23, 2016
    Publication date: April 20, 2017
    Inventors: Timothy D. Cherna, Randy Ubillos, Zehang Sun
  • Publication number: 20170091952
    Abstract: In some implementations, a computing device can track an object from a first image frame to a second image frame using a self-correcting tracking method. The computing device can select points of interest in the first image frame. The computing device can track the selected points of interest from the first image frame to the second image frame using optical flow object tracking. The computing device can prune the matching pairs of points and generate a transform based on the remaining matching pairs to detect the selected object in the second image frame. The computing device can generate a tracking confidence metric based on a projection error for each point of interest tracked from the first frame to the second frame. The computing device can correct tracking errors by reacquiring the object when the tracking confidence metric is below a threshold value.
    Type: Application
    Filed: September 30, 2015
    Publication date: March 30, 2017
    Inventors: Zehang Sun, Toshihiro Horie, Xin Tong, Peter Chou
  • Patent number: 9569078
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Grant
    Filed: September 27, 2012
    Date of Patent: February 14, 2017
    Assignee: APPLE INC.
    Inventors: Timothy D. Cherna, Randy Ubillos, Zehang Sun
  • Publication number: 20160173726
    Abstract: Some embodiments provide an image-viewing application. The image-viewing application displays a thumbnail display area that includes several thumbnails that each correspond to an image. With a set of at least one thumbnail selected, the application receives selection input over a particular unselected thumbnail in the thumbnail display area. When the selection input lasts less than a threshold duration, application selects only the particular thumbnail and displays the image corresponding to the particular thumbnail in an image display area. When the selection input lasts at least the threshold duration, the application adds the particular thumbnail to the set of selected thumbnails and displays the images corresponding to each of the selected thumbnails in the image display area.
    Type: Application
    Filed: June 24, 2015
    Publication date: June 16, 2016
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Patent number: 9245194
    Abstract: An efficient line detection technique pre-processes an image to remove edge pixels that are not on straight lines, before performing line detection. By removing edge pixels not on straight lines, the complexity of the task of line detection can be significantly reduced, while increasing the accuracy. Various embodiments preprocess the edge maps of an image by anisotropic line filtering, kernel density estimation-based edge pruning, and connected component analysis. The resulting pruned edge map may then be processed using a Hough transform to detect straight lines in the image, having removed much of the noise in the edge map.
    Type: Grant
    Filed: February 6, 2012
    Date of Patent: January 26, 2016
    Assignee: Apple Inc.
    Inventor: Zehang Sun
  • Publication number: 20130239063
    Abstract: Some embodiments provide an image-viewing application. The image-viewing application displays a thumbnail display area that includes several thumbnails that each correspond to an image. With a set of at least one thumbnail selected, the application receives selection input over a particular unselected thumbnail in the thumbnail display area. When the selection input lasts less than a threshold duration, application selects only the particular thumbnail and displays the image corresponding to the particular thumbnail in an image display area. When the selection input lasts at least the threshold duration, the application adds the particular thumbnail to the set of selected thumbnails and displays the images corresponding to each of the selected thumbnails in the image display area.
    Type: Application
    Filed: September 27, 2012
    Publication date: September 12, 2013
    Applicant: APPLE INC.
    Inventors: Randy Ubillos, Timothy D. Cherna, Zehang Sun
  • Publication number: 20130235076
    Abstract: Some embodiments provide an image editing application that edits an image. The image editing application displays an image in a display area. The image editing application displays a rotatable user interface item. In response to receiving a user input, the image editing application rotates the rotatable user interface item. The image editing application rotates the image displayed in the display area based on the rotation of the rotatable user interface item.
    Type: Application
    Filed: September 27, 2012
    Publication date: September 12, 2013
    Applicant: APPLE INC.
    Inventors: Timothy D. Cherna, Randy Ubillos, Zehang Sun
  • Publication number: 20130201358
    Abstract: An efficient line detection technique pre-processes an image to remove edge pixels that are not on straight lines, before performing line detection. By removing edge pixels not on straight lines, the complexity of the task of line detection can be significantly reduced, while increasing the accuracy. Various embodiments preprocess the edge maps of an image by anisotropic line filtering, kernel density estimation-based edge pruning, and connected component analysis. The resulting pruned edge map may then be processed using a Hough transform to detect straight lines in the image, having removed much of the noise in the edge map.
    Type: Application
    Filed: February 6, 2012
    Publication date: August 8, 2013
    Applicant: APPLE INC.
    Inventor: Zehang Sun
  • Patent number: 7636105
    Abstract: The method and apparatus for providing motion control signals between a fixed lens static camera and a PTZ camera that includes an adjustable-in-use focal length adapted to provide improved surveillance. In one aspect of the present invention, the improved method and apparatus includes an improved PTZ process in which fuzzy logic based information is utilized to achieve a set of more reliable P/T/Z parameters. In another aspect of the invention, the fixed lens static camera and the PTZ camera are mounted vertically on top of each other. In another aspect of the present invention, there is provided an automatic self-calibration between the fixed lens static camera and the PTZ camera. In another aspect of the present invention, there is provided a cost-effective standalone single board DSP solution.
    Type: Grant
    Filed: April 5, 2006
    Date of Patent: December 22, 2009
    Assignee: eTreppid Technologies LLC
    Inventors: Zehang Sun, Yongmian Zhang