Patents by Inventor Chih-Yao Hsieh
Chih-Yao Hsieh has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230132180Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that upsample and refine segmentation masks. Indeed, in one or more implementations, a segmentation mask refinement and upsampling system upsamples a preliminary segmentation mask utilizing a patch-based refinement process to generate a patch-based refined segmentation mask. The segmentation mask refinement and upsampling system then fuses the patch-based refined segmentation mask with an upsampled version of the preliminary segmentation mask. By fusing the patch-based refined segmentation mask with the upsampled preliminary segmentation mask, the segmentation mask refinement and upsampling system maintains a global perspective and helps avoid artifacts due to the local patch-based refinement process.Type: ApplicationFiled: January 26, 2022Publication date: April 27, 2023Inventors: Chih-Yao Hsieh, Yilin Wang
-
Publication number: 20230129341Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate preliminary object masks for objects in an image, surface the preliminary object masks as object mask previews, and on-demand converts preliminary object masks into refined object masks. Indeed, in one or more implementations, an object mask preview and on-demand generation system automatically detects objects in an image. For the detected objects, the object mask preview and on-demand generation system generates preliminary object masks for the detected objects of a first lower resolution. The object mask preview and on-demand generation system surfaces a given preliminary object mask in response to detecting a first input. The object mask preview and on-demand generation system also generates a refined object mask of a second higher resolution in response to detecting a second input.Type: ApplicationFiled: January 25, 2022Publication date: April 27, 2023Inventors: Betty Leong, Hyunghwan Byun, Alan L Erickson, Chih-Yao Hsieh, Sarah Kong, Seyed Morteza Safdarnejad, Salil Tambe, Yilin Wang, Zijun Wei, Zhengyun Zhang
-
Patent number: 11551338Abstract: The present disclosure is directed toward intelligently mixing and matching faces and/or people to generate an enhanced image that reduces or minimize artifacts and other defects. For example, the disclosed systems can selectively apply different alignment models to determine a relative alignment between a references image and a target image having an improved instance of the person. Upon aligning the digital images, the disclosed systems can intelligently identify a replacement region based on a boundary that includes the target instance and the reference instance of the person without intersecting other objects or people in the image. Using the size and shape of the replacement region around the target instance and the reference instance, the systems replace the instance of the person in the reference image with the target instance. The alignment of the images and the intelligent selection of the replacement region minimizes inconsistencies and/or artifacts in the final image.Type: GrantFiled: November 23, 2020Date of Patent: January 10, 2023Assignee: Adobe Inc.Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Patent number: 11544831Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short-exposure images without additional information.Type: GrantFiled: August 4, 2020Date of Patent: January 3, 2023Assignee: Adobe Inc.Inventors: Yilin Wang, Zhe Lin, Zhaowen Wang, Xin Lu, Xiaohui Shen, Chih-Yao Hsieh
-
Patent number: 11537518Abstract: Constraining memory use for overlapping virtual memory operations is described. The memory use is constrained to prevent memory from exceeding an operational threshold, e.g., in relation to operations for modifying content. These operations are implemented according to algorithms having a plurality of instructions. Before the instructions are performed in relation to the content, virtual memory is allocated to the content data, which is then loaded into the virtual memory and is also partitioned into data portions. In the context of the described techniques, at least one of the instructions affects multiple portions of the content data loaded in virtual memory. When this occurs, the instruction is carried out, in part, by transferring the multiple portions of content data between the virtual memory and a memory such that a number of portions of the content data in the memory is constrained to the memory that is reserved for the operation.Type: GrantFiled: September 26, 2017Date of Patent: December 27, 2022Assignee: Adobe Inc.Inventors: Chih-Yao Hsieh, Zhaowen Wang
-
Patent number: 11257491Abstract: This application relates generally to modifying visual data based on audio commands and more specifically, to performing complex operations that modify visual data based on one or more audio commands. In some embodiments, a computer system may receive an audio input and identify an audio command based on the audio input. The audio command may be mapped to one or more operations capable of being performed by a multimedia editing application. The computer system may perform the one or more operations to edit to received multimedia data.Type: GrantFiled: November 29, 2018Date of Patent: February 22, 2022Assignee: ADOBE INC.Inventors: Sarah Kong, Yinglan Ma, Hyunghwan Byun, Chih-Yao Hsieh
-
Patent number: 11216961Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that analyze feature points of digital images to selectively apply a pixel-adjusted-gyroscope-alignment model and a feature-based-alignment model to align the digital images. For instance, the disclosed systems can select an appropriate alignment model based on feature-point-deficiency metrics specific to an input image and reference-input image. Moreover, in certain implementations, the pixel-adjusted-gyroscope-alignment model utilizes parameters from pixel-based alignment and gyroscope-based alignment to align such digital images. Together with the feature-based-alignment model, the disclosed methods, non-transitory computer readable media, and systems can use a selective image-alignment algorithm that improves computational efficiency, accuracy, and flexibility in generating enhanced digital images from a set of input images.Type: GrantFiled: August 27, 2020Date of Patent: January 4, 2022Assignee: ADOBE INC.Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Patent number: 11196939Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that generate a virtual long exposure image from a sequence of short exposure images portraying a moving object. In various embodiments, the image transformation system aligns two digital images in the sequence of short exposure images. The image transformation system can determine a motion vector path for the moving object between the first digital image and the second digital image. The image transformation system can also blend pixels along the motion vector path to generate a blended image representative of the motion of the moving object between the first digital image and the second digital image. The image transformation system can generate additional blended images based on consecutive pairs of images in the sequence of digital images and generates a virtual long exposure image by combining the first blended image with the additional blended images.Type: GrantFiled: May 18, 2020Date of Patent: December 7, 2021Assignee: ADOBE INC.Inventors: Chih-Yao Hsieh, Sylvain Paris, Seyed Morteza Safdarnejad, Gregg Wilensky
-
Publication number: 20210073961Abstract: The present disclosure is directed toward intelligently mixing and matching faces and/or people to generate an enhanced image that reduces or minimize artifacts and other defects. For example, the disclosed systems can selectively apply different alignment models to determine a relative alignment between a references image and a target image having an improved instance of the person. Upon aligning the digital images, the disclosed systems can intelligently identify a replacement region based on a boundary that includes the target instance and the reference instance of the person without intersecting other objects or people in the image. Using the size and shape of the replacement region around the target instance and the reference instance, the systems replace the instance of the person in the reference image with the target instance. The alignment of the images and the intelligent selection of the replacement region minimizes inconsistencies and/or artifacts in the final image.Type: ApplicationFiled: November 23, 2020Publication date: March 11, 2021Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Patent number: 10896493Abstract: The present disclosure is directed toward intelligently mixing and matching faces and/or people to generate an enhanced image that reduces or minimize artifacts and other defects. For example, the disclosed systems can selectively apply different alignment models to determine a relative alignment between a references image and a target image having an improved instance of the person. Upon aligning the digital images, the disclosed systems can intelligently identify a replacement region based on a boundary that includes the target instance and the reference instance of the person without intersecting other objects or people in the image. Using the size and shape of the replacement region around the target instance and the reference instance, the systems replace the instance of the person in the reference image with the target instance. The alignment of the images and the intelligent selection of the replacement region minimizes inconsistencies and/or artifacts in the final image.Type: GrantFiled: November 13, 2018Date of Patent: January 19, 2021Assignee: ADOBE INC.Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Publication number: 20200394808Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that analyze feature points of digital images to selectively apply a pixel-adjusted-gyroscope-alignment model and a feature-based-alignment model to align the digital images. For instance, the disclosed systems can select an appropriate alignment model based on feature-point-deficiency metrics specific to an input image and reference-input image. Moreover, in certain implementations, the pixel-adjusted-gyroscope-alignment model utilizes parameters from pixel-based alignment and gyroscope-based alignment to align such digital images. Together with the feature-based-alignment model, the disclosed methods, non-transitory computer readable media, and systems can use a selective image-alignment algorithm that improves computational efficiency, accuracy, and flexibility in generating enhanced digital images from a set of input images.Type: ApplicationFiled: August 27, 2020Publication date: December 17, 2020Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Publication number: 20200372622Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short- exposure images without additional information.Type: ApplicationFiled: August 4, 2020Publication date: November 26, 2020Inventors: Yilin Wang, Zhe Lin, Zhaowen Wang, Xin Lu, Xiaohui Shen, Chih-Yao Hsieh
-
Patent number: 10796421Abstract: Embodiments of the present invention are directed to facilitating images with selective application of the long-exposure effect. In accordance with some embodiments of the present invention, virtual long-exposure image comprising a plurality of aligned frames is provided and a selection of a region of pixels in the virtual long-exposure image is received. The virtual long-exposure image is combined with one of the frames forming the virtual long-exposure image to create a selective virtual long-exposure image. The selective virtual long-exposure image comprises a visible portion of the original virtual long-exposure image and a visible portion of the individual frame that corresponds to the selected region of pixels. Additional frames may be combined with the virtual long-exposure image to create a plurality of selective virtual long-exposure image options, and the user may select one for continued use or for saving.Type: GrantFiled: February 13, 2018Date of Patent: October 6, 2020Inventors: Seyed Morteza Safdarnejad, Sarah Aye Kong, Gregg Darryl Wilensky, Chih-Yao Hsieh
-
Patent number: 10783649Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that analyze feature points of digital images to selectively apply a pixel-adjusted-gyroscope-alignment model and a feature-based-alignment model to align the digital images. For instance, the disclosed systems can select an appropriate alignment model based on feature-point-deficiency metrics specific to an input image and reference-input image. Moreover, in certain implementations, the pixel-adjusted-gyroscope-alignment model utilizes parameters from pixel-based alignment and gyroscope-based alignment to align such digital images. Together with the feature-based-alignment model, the disclosed methods, non-transitory computer readable media, and systems can use a selective image-alignment algorithm that improves computational efficiency, accuracy, and flexibility in generating enhanced digital images from a set of input images.Type: GrantFiled: September 17, 2018Date of Patent: September 22, 2020Assignee: ADOBE INC.Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh
-
Patent number: 10783622Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short-exposure images without additional information.Type: GrantFiled: April 25, 2018Date of Patent: September 22, 2020Assignee: ADOBE INC.Inventors: Yilin Wang, Zhe Lin, Zhaowen Wang, Xin Lu, Xiaohui Shen, Chih-Yao Hsieh
-
Publication number: 20200280670Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that generate a virtual long exposure image from a sequence of short exposure images portraying a moving object. In various embodiments, the image transformation system aligns two digital images in the sequence of short exposure images. The image transformation system can determine a motion vector path for the moving object between the first digital image and the second digital image. The image transformation system can also blend pixels along the motion vector path to generate a blended image representative of the motion of the moving object between the first digital image and the second digital image. The image transformation system can generate additional blended images based on consecutive pairs of images in the sequence of digital images and generates a virtual long exposure image by combining the first blended image with the additional blended images.Type: ApplicationFiled: May 18, 2020Publication date: September 3, 2020Inventors: Chih-Yao Hsieh, Sylvain Paris, Seyed Morteza Safdarnejad, Gregg Wilensky
-
Patent number: 10706512Abstract: Methods and systems are provided for adjusting the brightness of images. In some implementations, an exposure bracketed set of input images produced by a camera is received. A brightness adjustment is determined for at least one input image from the set of input images. The determined brightness adjustment is applied to the input image. An output image is produced by exposure fusion from the set of input images, using the input image having the determined brightness adjustment. The output image is transmitted where, the transmitting causes display of the output image on a user device.Type: GrantFiled: March 7, 2017Date of Patent: July 7, 2020Assignee: ADOBE INC.Inventors: Yinglan Ma, Sylvain Philippe Paris, Chih-Yao Hsieh
-
Patent number: 10701279Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that generate a virtual long exposure image from a sequence of short exposure images portraying a moving object. In various embodiments, the image transformation system aligns two digital images in the sequence of short exposure images. The image transformation system can determine a motion vector path for the moving object between the first digital image and the second digital image. The image transformation system can also blend pixels along the motion vector path to generate a blended image representative of the motion of the moving object between the first digital image and the second digital image. The image transformation system can generate additional blended images based on consecutive pairs of images in the sequence of digital images and generates a virtual long exposure image by combining the first blended image with the additional blended images.Type: GrantFiled: October 2, 2018Date of Patent: June 30, 2020Assignee: ADOBE INC.Inventors: Chih-Yao Hsieh, Sylvain Paris, Seyed Morteza Safdarnejad, Gregg Wilensky
-
Publication number: 20200175975Abstract: This application relates generally to modifying visual data based on audio commands and more specifically, to performing complex operations that modify visual data based on one or more audio commands. In some embodiments, a computer system may receive an audio input and identify an audio command based on the audio input. The audio command may be mapped to one or more operations capable of being performed by a multimedia editing application. The computer system may perform the one or more operations to edit to received multimedia data.Type: ApplicationFiled: November 29, 2018Publication date: June 4, 2020Inventors: Sarah Kong, Yinglan Ma, Hyunghwan Byun, Chih-Yao Hsieh
-
Publication number: 20200151860Abstract: The present disclosure is directed toward intelligently mixing and matching faces and/or people to generate an enhanced image that reduces or minimize artifacts and other defects. For example, the disclosed systems can selectively apply different alignment models to determine a relative alignment between a references image and a target image having an improved instance of the person. Upon aligning the digital images, the disclosed systems can intelligently identify a replacement region based on a boundary that includes the target instance and the reference instance of the person without intersecting other objects or people in the image. Using the size and shape of the replacement region around the target instance and the reference instance, the systems replace the instance of the person in the reference image with the target instance. The alignment of the images and the intelligent selection of the replacement region minimizes inconsistencies and/or artifacts in the final image.Type: ApplicationFiled: November 13, 2018Publication date: May 14, 2020Inventors: Seyed Morteza Safdarnejad, Chih-Yao Hsieh