Patents by Inventor Legong Sun

Legong Sun has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240129642
    Abstract: The achromatic region extraction unit calculates achromatic determination information for each color component of the color polarized image using the polarization information acquired from the color polarized image, and extracts a region satisfying a condition that the achromatic determination information is achromatic as an achromatic region. The gain setting unit sets the white balance gain used in the white balance adjustment of the color polarized image on the basis of the white balance gain with which the achromatic region extracted by the achromatic region extraction unit becomes achromatic. For example, the gain setting unit sets the white balance gain for the entire color polarized image on the basis of the white balance gain of the achromatic region, or sets the white balance gain of a region different from the achromatic region on the basis of the white balance gains of a plurality of achromatic regions.
    Type: Application
    Filed: January 4, 2022
    Publication date: April 18, 2024
    Applicant: Sony Group Corporation
    Inventors: Taishi ONO, Yuhi KONDO, Legong SUN
  • Patent number: 11930160
    Abstract: A polarization imaging unit 20 acquires a polarized image including polarization pixels with a plurality of polarization directions. An information compression unit 40 sets reference image information based on polarized image information of reference polarization pixels with at least a plurality of polarization directions in the polarized image and generates difference information between the polarized image information of each of polarization pixels different from the reference polarization pixels in the polarized image and the reference image information. In addition, the information compression unit 40 reduces the amount of information of the difference information generated for each of the polarization pixels with the plurality of polarization directions to generate compressed image information including the reference image information and the difference information with the reduced amount of information.
    Type: Grant
    Filed: June 6, 2019
    Date of Patent: March 12, 2024
    Assignee: SONY CORPORATION
    Inventors: Yuhi Kondo, Yasutaka Hirasawa, Takefumi Nagumo, Toshinori Ihara, Teppei Kurita, Legong Sun
  • Publication number: 20230410457
    Abstract: A polarized imaging unit 20 captures time-series polarized images of an object. A polarization information calculating unit 30 calculates polarization information for each polarized image from the time-series polarized images captured by the polarized imaging unit 20. An object specifying unit 50 specifies the image region of the object on the basis of, for example, a non-polarized image generated by a non-polarized image generating unit 40 on the basis of the polarized image. A state-change detecting unit 60 detects a state change of the object on the basis of a time series change of the polarization information in the image region of the object specified by the object specifying unit 50. An output unit 70 outputs the detection result of a state change such that the detection result is associated with the non-polarized image generated by the non-polarized image generating unit 40, the detection result being obtained by the state-change detecting unit 60. A state change of the object can be easily detected.
    Type: Application
    Filed: September 16, 2021
    Publication date: December 21, 2023
    Inventors: YUHI KONDO, TAISHI ONO, LEGONG SUN
  • Publication number: 20230410464
    Abstract: A black-and-white polarized image acquirer 21 of an image acquirer 20 acquires a black-and-white polarized image, and a color image acquirer 22 acquires a color image. A reflection-removed image generator 40 calculates characteristic information related to black-and-white reflection removal results generated based on a black-and-white polarized image. For example, the reflection-removed image generator calculates information indicating a relationship between the black-and-white polarized image and an average luminance calculated from an image for each polarization direction in the black-and-white polarized image, or information used for obtaining the black-and-white reflection removal results from the black-and-white polarized image. The reflection-removed image generator 40 generates a reflection-removed color image in which specular reflection is removed from the color image using the calculated characteristic information or the optimized characteristic information.
    Type: Application
    Filed: October 19, 2021
    Publication date: December 21, 2023
    Inventors: TAISHI ONO, YUHI KONDO, LEGONG SUN
  • Publication number: 20230316708
    Abstract: A birefringence imaging unit 20 has a birefringent material and a polarized imaging unit, and the polarized imaging unit generates polarized images based on subject light incident through a birefringent material. The parallax image generator 30 separates images with different polarization angles using the polarized images generated by the polarized imaging unit of the birefringence imaging unit 20, and generates an ordinary ray image and an extraordinary ray image as parallax images. The distance measuring unit 40 calculates a distance to a distance measurement position based on a parallax of the distance measurement position of the subject in the ordinary ray image and the extraordinary ray image generated by the parallax image generator 30. By using polarized images, it is possible to measure distances in portions other than edges, and distance information with high resolution can be obtained more easily than when measuring distances by matching corresponding points in edge images.
    Type: Application
    Filed: October 19, 2021
    Publication date: October 5, 2023
    Inventors: LEGONG SUN, YUHI KONDO, TAISHI ONO
  • Patent number: 11758290
    Abstract: The invalid pixel detection unit 32 detects an invalid pixel from a non-polarized image and a plurality of polarized images for different polarization directions obtained by performing image pickup using the polarization image pickup unit 20. For example, the invalid pixel detection unit 32 detects a saturated pixel having a pixel value larger than a preset saturation detection threshold and a black-crushed pixel having a pixel value smaller than a preset black crushing detection threshold as invalid pixels from a non-polarized image. The polarization information generation unit 33 performs processing of generating polarization information on the basis of the non-polarized image and polarized images and switches the processing of generating the polarization information depending on the detection result of the invalid pixels in the invalid pixel detection unit 32, so as to generate the polarization information without using the invalid pixels. Correct polarization information can be acquired.
    Type: Grant
    Filed: December 18, 2020
    Date of Patent: September 12, 2023
    Assignee: SONY GROUP CORPORATION
    Inventors: Yasutaka Hirasawa, Yuhi Kondo, Legong Sun, Taishi Ono
  • Publication number: 20230244929
    Abstract: The present technology relates to a learning device, a generation method, an inference device, an inference method, and a program that enable learning data suitable for learning to be selected without a manual operation and enable learning of an inference model to be efficiently performed by using the selected learning data. In a learning device according to one aspect of the present technology, on the basis of a learning data group including learning data having a correct answer and a processing target data group including processing target data for learning, the processing target data for learning having no correct answer and corresponding to data to be processed at the time of inference, the learning device selects, from the learning data group, the learning data suitable for learning of an inference model used at the time of inference, and outputs the selected learning data together with the inference model obtained by performing learning using the selected learning data.
    Type: Application
    Filed: May 7, 2021
    Publication date: August 3, 2023
    Inventors: YUHI KONDO, LEGONG SUN, TAISHI ONO, YASUTAKA HIRASAWA
  • Publication number: 20230239580
    Abstract: A same region detector detects a same region of a processing target for each of a plurality of different viewpoints from polarization images in a plurality of polarization directions acquired for each of the viewpoints. The polarization images in the plurality of polarization directions acquired for each of the plurality of different viewpoints are, for example, polarization images acquired by imaging over a period of a plurality of frames in which a positional relationship between the processing target and a polarization image acquisition unit that acquires the polarization images changes. A polarization degree calculation unit calculates a polarization degree of the same region for each of the viewpoints on the basis of the polarization images in the plurality of polarization directions.
    Type: Application
    Filed: May 13, 2021
    Publication date: July 27, 2023
    Inventors: YASUTAKA HIRASAWA, YUHI KONDO, LEGONG SUN, TAISHI ONO
  • Publication number: 20230196534
    Abstract: A polarization rendering setting unit 30 sets a plurality of parameters to be used for generating a polarized rendered image of an abnormality detection target. A polarized rendered image generation unit 40 generates the polarized rendered image of the abnormality detection target on the basis of the parameters set by the polarization rendering setting unit 30. An abnormality detection unit 50 detects an abnormal region of the abnormality detection target on the basis of a difference between a polarized captured image acquired by imaging the abnormality detection target and the polarized rendered image generated by the polarized rendered image generation unit 40. Abnormalities that are difficult to detect on the basis of luminance information and color information becomes able to be detected on the basis of polarized images.
    Type: Application
    Filed: March 30, 2021
    Publication date: June 22, 2023
    Inventors: LEGONG SUN, YASUTAKA HIRASAWA, YUHI KONDO, TAISHI ONO
  • Patent number: 11676245
    Abstract: A normal candidate information generation unit of an information processor generates normal candidate information for each pixel indicating, for example, a zenith angle, or an azimuth angle, or a zenith angle and an azimuth angle, on the basis of a polarization image in a plurality of polarization directions obtained by a polarization imaging unit. The in-plane pixel selection unit selects a plurality of pixels indicating the plane to be observed in the polarization image. A normal calculation unit calculates a normal of the plane to be observed on the basis of the normal candidate information of the pixels selected by the in-plane pixel selection unit.
    Type: Grant
    Filed: February 18, 2019
    Date of Patent: June 13, 2023
    Assignee: SONY CORPORATION
    Inventors: Legong Sun, Yasutaka Hirasawa, Yuhi Kondo, Ying Lu, Tuo Zhuang
  • Patent number: 11637967
    Abstract: Provided is a polarization imaging unit that acquires a polarization image using a lens. In a parameter storage unit of an information processing unit, parameters related to a change in polarization state due to the lens which are estimated in advance from polarization state information acquired by imaging a light source in a predetermined polarization state using the lens and polarization state information indicating the polarization state of the light source, for example, change parameters indicating the change in polarization state due to the lens or correction parameters for correcting the change in polarization state due to the lens are stored. A polarization state correction unit of the information processing unit corrects the change in polarization state due to the lens, using the parameters stored in advance in the parameter storage unit.
    Type: Grant
    Filed: January 15, 2019
    Date of Patent: April 25, 2023
    Assignee: SONY CORPORATION
    Inventors: Ying Lu, Yasutaka Hirasawa, Yuhi Kondo, Tuo Zhuang, Legong Sun
  • Publication number: 20230053038
    Abstract: The invalid pixel detection unit 32 detects an invalid pixel from a non-polarized image and a plurality of polarized images for different polarization directions obtained by performing image pickup using the polarization image pickup unit 20. For example, the invalid pixel detection unit 32 detects a saturated pixel having a pixel value larger than a preset saturation detection threshold and a black-crushed pixel having a pixel value smaller than a preset black crushing detection threshold as invalid pixels from a non-polarized image. The polarization information generation unit 33 performs processing of generating polarization information on the basis of the non-polarized image and polarized images and switches the processing of generating the polarization information depending on the detection result of the invalid pixels in the invalid pixel detection unit 32, so as to generate the polarization information without using the invalid pixels. Correct polarization information can be acquired.
    Type: Application
    Filed: December 18, 2020
    Publication date: February 16, 2023
    Applicant: SONY GROUP CORPORATION
    Inventors: Yasutaka HIRASAWA, Yuhi KONDO, Legong SUN, Taishi ONO
  • Publication number: 20230003583
    Abstract: A quantization range setting unit (31) of an encoder (30) sets a quantization range for each Stokes parameter of a Stokes vector acquired from a Stokes vector calculation unit (20). The quantization range is set for the Stokes parameter indicating intensity, and is then set for the other Stokes parameters. A quantization unit (32) calculates the Stokes parameter indicating intensity as a predetermined quantization bit number, and calculates the quantization bit numbers of the other Stokes parameters on the basis of the predetermined quantization bit number and the quantization ranges set for the respective Stokes parameters. The quantization unit (32) performs a quantization process on the Stokes parameters on the basis of the quantization ranges and the quantization bit numbers, to generate quantized polarization information. A decoder (40) performs inverse quantization compatible with the encoder 30 on the quantized polarization information, and generates the Stokes vectors before quantization.
    Type: Application
    Filed: November 30, 2020
    Publication date: January 5, 2023
    Inventors: LEGONG SUN, YASUTAKA HIRASAWA, YUHI KONDO, TAISHI ONO
  • Publication number: 20220210322
    Abstract: An imaging apparatus (10) includes an imaging unit (11) that has a first pixel block and a second pixel block provided in a pixel region, the first pixel block including polarization pixels with two different polarization directions and a non-polarization pixel, the second pixel block including polarization pixels with a plurality of polarization directions including a different polarization direction from the first pixel block. An image processing apparatus (20) acquires a captured image generated by the imaging apparatus (10), and a correction coefficient calculation unit (40) calculates a correction coefficient that absorbs a sensitivity difference between polarization pixels and non-polarization pixels, on the basis of an image of the second pixel block.
    Type: Application
    Filed: February 7, 2020
    Publication date: June 30, 2022
    Inventors: YASUTAKA HIRASAWA, YUHI KONDO, LEGONG SUN, TAISHI ONO
  • Publication number: 20220146434
    Abstract: A determination environment information acquisition unit 31 acquires incident polarized light information of a light source in a material determination environment. A determination target information acquisition unit 32 acquires emitted polarized light information from a polarized image obtained by capturing an image of a material determination target in the material determination environment.
    Type: Application
    Filed: January 15, 2020
    Publication date: May 12, 2022
    Inventors: LEGONG SUN, YASUTAKA HIRASAWA, YUHI KONDO, TAISHI ONO
  • Patent number: 11275230
    Abstract: A detection apparatus has a detector for detecting, on the basis of an object area that is an area corresponding to an object in an image, an effective focus area from two or more focus areas set to the image.
    Type: Grant
    Filed: June 27, 2016
    Date of Patent: March 15, 2022
    Assignee: Sony Group Corporation
    Inventors: Legong Sun, Seijiro Inaba
  • Publication number: 20210374906
    Abstract: A normal candidate information generation unit 31 of an information processor 30 generates normal candidate information for each pixel indicating, for example, a zenith angle, or an azimuth angle, or a zenith angle and an azimuth angle, on the basis of a polarization image in a plurality of polarization directions obtained by a polarization imaging unit 20. The in-plane pixel selection unit 32 selects a plurality of pixels indicating the plane to be observed in the polarization image. A normal calculation unit 33 calculates a normal of the plane to be observed on the basis of the normal candidate information of the pixels selected by the in-plane pixel selection unit 32.
    Type: Application
    Filed: February 18, 2019
    Publication date: December 2, 2021
    Inventors: LEGONG SUN, YASUTAKA HIRASAWA, YUHI KONDO, YING LU, TUO ZHUANG
  • Publication number: 20210243437
    Abstract: A polarization imaging unit 20 acquires a polarized image including polarization pixels with a plurality of polarization directions. An information compression unit 40 sets reference image information based on polarized image information of reference polarization pixels with at least a plurality of polarization directions in the polarized image and generates difference information between the polarized image information of each of polarization pixels different from the reference polarization pixels in the polarized image and the reference image information. In addition, the information compression unit 40 reduces the amount of information of the difference information generated for each of the polarization pixels with the plurality of polarization directions to generate compressed image information including the reference image information and the difference information with the reduced amount of information.
    Type: Application
    Filed: June 6, 2019
    Publication date: August 5, 2021
    Applicant: SONY CORPORATION
    Inventors: Yuhi KONDO, Yasutaka HIRASAWA, Takefumi NAGUMO, Toshinori IHARA, Teppei KURITA, Legong SUN
  • Patent number: 11044419
    Abstract: [Object] To reproduce an image to which an effect desired by a user is added. [Solution] There is provided an image processing device including: an image reverse stabilization processing unit configured to add an effect of expressing shaking to an image on the basis of shaking information on shaking of the image. In addition, there is provided an image processing device including: an image reverse stabilization processing unit configured to decide a degree of an effect of expressing shaking to be added to an image, on the basis of an expectation value of an immersed feeling of an observer with respect to the image.
    Type: Grant
    Filed: November 7, 2018
    Date of Patent: June 22, 2021
    Assignee: Sony Corporation
    Inventors: Legong Sun, Kouji Yamamoto, Masafumi Wakazono, Yoichiro Sato, Kazuhiro Shimauchi
  • Publication number: 20210152748
    Abstract: A polarization imaging unit 20 acquires a polarization image using a lens. In a parameter storage unit 32 of an information processing unit 30, parameters related to a change in polarization state due to the lens which are estimated in advance from polarization state information acquired by imaging a light source in a predetermined polarization state using the lens and polarization state information indicating the polarization state of the light source, for example, change parameters indicating the change in polarization state due to the lens or correction parameters for correcting the change in polarization state due to the lens are stored. A polarization state correction unit 33 of the information processing unit 30 corrects the change in polarization state due to the lens, using the parameters stored in advance in the parameter storage unit 32.
    Type: Application
    Filed: January 15, 2019
    Publication date: May 20, 2021
    Inventors: YING LU, YASUTAKA HIRASAWA, YUHI KONDO, TUO ZHUANG, LEGONG SUN