Patents by Inventor Ghanshyam Rathi

Ghanshyam Rathi has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10780827
    Abstract: A method for stitching images captured by multiple vehicular cameras includes disposing a plurality of cameras at the vehicle so as to have respective fields of view exterior the vehicle. Image data captured by first and second cameras of the plurality of cameras is processed to detect an object present in an overlapping portion of the fields of view of the first and second cameras. Image data captured by the first and second cameras is stitched, via processing provided captured image data, to form stitched images. Stitching of captured image data is adjusted responsive to determination of a difference between a characteristic of a feature of a detected object as captured by the first camera and the characteristic of the feature of the detected object as captured by the second camera in order to mitigate misalignment of stitched images.
    Type: Grant
    Filed: November 25, 2019
    Date of Patent: September 22, 2020
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Publication number: 20200276939
    Abstract: A method for calibrating a vehicular camera includes providing at least a front or rear camera and a side camera with overlapping fields of view, and calibrating the front or rear camera, capturing a calibrated frame of image data with the front or rear camera, and capturing a sideward frame of image data with a side camera. At least one feature is determined present in the overlapping region of the calibrated frame, and pixel positions of the determined feature are predicted for the side camera. Misalignment of the side camera is determined based on a comparison of the predicted pixel positions of the determined feature to the pixel positions of the determined feature in the sideward frame of image data captured by the side camera. Processing of image data captured by the side camera is adjusted to accommodate the determined misalignment.
    Type: Application
    Filed: May 18, 2020
    Publication date: September 3, 2020
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20200262347
    Abstract: A method for dynamically calibrating a vehicular camera includes disposing a camera at a vehicle and operating the camera to capture multiple frames of image data while the vehicle is in motion and is steered within at least two ranges of steering angles. Feature points are determined in an image frame when the vehicle is steered within a respective range of steering angles, and motion trajectories of those feature points are tracked in subsequent image frames for the respective range of steering angles. A horizon line is determined based on the tracked feature points. Responsive to determination that the determined horizon line is non-parallel to the horizontal axis of the image plane, at least one of pitch, roll or yaw of the camera is adjusted. Image data captured by the camera is processed at the control for object detection.
    Type: Application
    Filed: May 4, 2020
    Publication date: August 20, 2020
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Patent number: 10654423
    Abstract: A method for dynamically ascertaining alignment of a vehicular camera relative to a vehicle to which the camera is attached includes determining a plurality of steering angle ranges for the vehicle, each of which is a range of steering angles that approximates straight vehicle motion over less than two seconds of travel time of the vehicle while the vehicle is in motion and turning. Image data captured by a camera of the vehicle is processed to determine a central vanishing point when the vehicle is in motion and moving straight. A plurality of feature points are selected in the image frames for each steering angle range, and a vanishing point for a plurality of tracked motion trajectories for each steering angle range is determined. An alignment of the camera is determined based at least in part on the determined central vanishing point and a determined vanishing line.
    Type: Grant
    Filed: December 4, 2017
    Date of Patent: May 19, 2020
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Patent number: 10640041
    Abstract: A method for dynamically calibrating without manual intervention a forward viewing vehicular camera with respect to its three rotational degrees of freedom includes disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data as the vehicle is moving and steered through a plurality of steering angles. Feature points are determined in an image frame when the vehicle is steered within a respective range of steering angles and motion trajectories of those feature points are tracked in subsequent image frames for the respective range of steering angles. A vanishing point is established in the image plane for the feature points for the respective range of steering angles. Based on established vanishing points, a vanishing line is determined. When the vanishing line is determined to be non-horizontal, at least one of pitch, roll or yaw of the camera is adjusted.
    Type: Grant
    Filed: February 4, 2019
    Date of Patent: May 5, 2020
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20200101899
    Abstract: A method for stitching images captured by multiple vehicular cameras includes disposing a plurality of cameras at the vehicle so as to have respective fields of view exterior the vehicle. Image data captured by first and second cameras of the plurality of cameras is processed to detect an object present in an overlapping portion of the fields of view of the first and second cameras. Image data captured by the first and second cameras is stitched, via processing provided captured image data, to form stitched images. Stitching of captured image data is adjusted responsive to determination of a difference between a characteristic of a feature of a detected object as captured by the first camera and the characteristic of the feature of the detected object as captured by the second camera in order to mitigate misalignment of stitched images.
    Type: Application
    Filed: November 25, 2019
    Publication date: April 2, 2020
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Patent number: 10486596
    Abstract: A dynamic image stitching system for stitching images captured by cameras of a vision system for a vehicle includes a plurality of cameras disposed at the vehicle and having respective fields of view exterior the vehicle. The field of view of each side camera partially overlaps the field of view of a front camera and partially overlaps the field of view of a rear camera. Image data captured by the plurality of cameras is processed at a processor to detect an object present in an overlapping portion of the fields of view of two cameras. The processor stitches, via a stitching algorithm, image data captured by the cameras. The processor adjusts the stitching algorithm responsive to a determination of a difference between a characteristic of the feature as captured by one of the two cameras and the characteristic of the feature as captured by another of the two cameras.
    Type: Grant
    Filed: January 14, 2019
    Date of Patent: November 26, 2019
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Publication number: 20190168670
    Abstract: A method for dynamically calibrating without manual intervention a forward viewing vehicular camera with respect to its three rotational degrees of freedom includes disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data as the vehicle is moving and steered through a plurality of steering angles. Feature points are determined in an image frame when the vehicle is steered within a respective range of steering angles and motion trajectories of those feature points are tracked in subsequent image frames for the respective range of steering angles. A vanishing point is established in the image plane for the feature points for the respective range of steering angles. Based on established vanishing points, a vanishing line is determined. When the vanishing line is determined to be non-horizontal, at least one of pitch, roll or yaw of the camera is adjusted.
    Type: Application
    Filed: February 4, 2019
    Publication date: June 6, 2019
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20190143896
    Abstract: A dynamic image stitching system for stitching images captured by cameras of a vision system for a vehicle includes a plurality of cameras disposed at the vehicle and having respective fields of view exterior the vehicle. The field of view of each side camera partially overlaps the field of view of a front camera and partially overlaps the field of view of a rear camera. Image data captured by the plurality of cameras is processed at a processor to detect an object present in an overlapping portion of the fields of view of two cameras. The processor stitches, via a stitching algorithm, image data captured by the cameras. The processor adjusts the stitching algorithm responsive to a determination of a difference between a characteristic of the feature as captured by one of the two cameras and the characteristic of the feature as captured by another of the two cameras.
    Type: Application
    Filed: January 14, 2019
    Publication date: May 16, 2019
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Patent number: 10202077
    Abstract: A method for dynamically correcting misalignment of a vehicular camera includes fixedly disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data while the vehicle is moving generally in a straight line. A plurality of sets of feature points are selected in an image frame, with each set including a first feature point and a second feature point. For each set of feature points, a motion trajectory of that set's feature points is tracked in subsequent image frames. For each tracked first and second feature points, a vanishing point is established in the image plane. Based on the established vanishing point, a vanishing line is determined in the image plane. When the vanishing line is determined to be non-horizontal in the image plane, at least one of pitch, roll or yaw of the camera is adjusted to correct rotational misalignment of the camera.
    Type: Grant
    Filed: May 23, 2016
    Date of Patent: February 12, 2019
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Patent number: 10179543
    Abstract: A dynamic image stitching system for stitching images captured by multiple cameras of a vision system of a vehicle includes a first camera disposed at a vehicle and having a first field of view exterior the vehicle and a second camera disposed at the vehicle and having a second field of view exterior the vehicle. The first and second fields of view at least partially overlap. A processor is operable to process image data captured by the first and second cameras. The processor processes captured image data to determine characteristics of features or objects present in the overlapping region of the first and second fields of view. The processor is operable to adjust a stitching algorithm responsive to a determination of a difference between a characteristic of a feature as captured by the first camera and the characteristic of the feature as captured by the second camera.
    Type: Grant
    Filed: February 27, 2014
    Date of Patent: January 15, 2019
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Publication number: 20180086284
    Abstract: A method for dynamically ascertaining alignment of a vehicular camera relative to a vehicle to which the camera is attached includes determining a plurality of steering angle ranges for the vehicle, each of which is a range of steering angles that approximates straight vehicle motion over less than two seconds of travel time of the vehicle while the vehicle is in motion and turning. Image data captured by a camera of the vehicle is processed to determine a central vanishing point when the vehicle is in motion and moving straight. A plurality of feature points are selected in the image frames for each steering angle range, and a vanishing point for a plurality of tracked motion trajectories for each steering angle range is determined. An alignment of the camera is determined based at least in part on the determined central vanishing point and a determined vanishing line.
    Type: Application
    Filed: December 4, 2017
    Publication date: March 29, 2018
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Patent number: 9834153
    Abstract: A method of dynamically calibrating a given camera relative to a reference camera of a vehicle includes identifying an overlapping region in an image frame provided by the given camera and an image frame provided by the reference camera and selecting at least a portion of an object in the overlapped region of the reference image frame. Expected pixel positions of the selected object portion in the given image frame is determined based on the location of the selected object portion in the reference image frame, and pixel positions of the selected object portion are located as detected in the given image frame. An alignment of the given camera is determined based on a comparison of the pixel positions of the selected object portion in the given image frame to the expected pixel positions of the selected object portion in the given image frame.
    Type: Grant
    Filed: April 25, 2012
    Date of Patent: December 5, 2017
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20160267657
    Abstract: A method for dynamically correcting misalignment of a vehicular camera includes fixedly disposing a camera at a vehicle and operating the camera to acquire multiple frames of image data whilst the vehicle is moving generally in a straight line. A plurality of sets of feature points are selected in an image frame, with each set including a first feature point and a second feature point. For each set of feature points, a motion trajectory of that set's feature points is tracked in subsequent image frames. For each tracked first and second feature points, a vanishing point is established in the image plane. Based on the established vanishing point, a vanishing line is determined in the image plane. When the vanishing line is determined to be non-horizontal in the image plane, at least one of pitch, roll or yaw of the camera is adjusted to correct rotational misalignment of the camera.
    Type: Application
    Filed: May 23, 2016
    Publication date: September 15, 2016
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Patent number: 9357208
    Abstract: A method of dynamically ascertaining the alignment of a vehicular camera. The method involves acquiring a sequence of images provided by the camera whilst the vehicle is in motion. For each range of steering angles, the method (i) selects a plurality of feature points in the images, (ii) tracks a motion trajectory for each selected feature point, and (iii) determines a vanishing point in the image plane based on the tracked motion trajectories. The method determines a vanishing line in the image plane based on a locus of these vanishing points and determines the alignment of the camera based on the position of a central vanishing point (corresponding to the zero degree angle) and the vanishing line.
    Type: Grant
    Filed: January 20, 2012
    Date of Patent: May 31, 2016
    Assignee: MAGNA ELECTRONICS INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20150312565
    Abstract: A method for calibrating a vehicular vision system includes providing a camera at a vehicle, with the camera having a field of view. Images are captured with the camera and a set of resultant images are acquired for a classification. Information is extracted related to image features in the set of resultant images, and an appropriate subset of coefficients is determined. For each classification, a classification vector of at least one appropriate weight is stored that corresponds to the determined subset of coefficients. The determined subset of coefficients is determined by processing sets of coefficients produced from a selection of calibration images and determining a subset of coefficients which acceptably discriminate between defined classifications. A set of resultant images is acquired by limiting the dynamic range of acquired images to obtain resultant images that include at least one region of interest.
    Type: Application
    Filed: July 6, 2015
    Publication date: October 29, 2015
    Inventors: Bin Shi, Ghanshyam Rathi, Sharon Zibman, Tom Perovic, Nikhil Gupta, Hilda Faraji, Yong Zhou
  • Patent number: 9077962
    Abstract: A method for calibrating a vehicular vision system includes providing a camera at a vehicle, with the camera having a field of view. Images are captured with the camera and a set of resultant images are acquired for a classification. Information is extracted related to image features in the set of resultant images, and an appropriate subset of coefficients is determined. For each classification, a classification vector of at least one appropriate weight is stored that corresponds to the determined subset of coefficients.
    Type: Grant
    Filed: November 11, 2013
    Date of Patent: July 7, 2015
    Assignee: MAGNA INTERNATIONAL, INC.
    Inventors: Bin Shi, Ghanshyam Rathi, Sharon Zibman, Tom Perovic, Nikhil Gupta, Hilda Faraji, Yong Zhou
  • Publication number: 20150049193
    Abstract: A method of dynamically calibrating a given camera relative to a reference camera of a vehicle includes identifying an overlapping region in an image frame provided by the given camera and an image frame provided by the reference camera and selecting at least a portion of an object in the overlapped region of the reference image frame. Expected pixel positions of the selected object portion in the given image frame is determined based on the location of the selected object portion in the reference image frame, and pixel positions of the selected object portion are located as detected in the given image frame. An alignment of the given camera is determined based on a comparison of the pixel positions of the selected object portion in the given image frame to the expected pixel positions of the selected object portion in the given image frame.
    Type: Application
    Filed: April 25, 2012
    Publication date: February 19, 2015
    Applicant: MAGNA INTERNATIONAL INC.
    Inventors: Nikhil Gupta, Hilda Faraji, Daan He, Ghanshyam Rathi
  • Publication number: 20140247352
    Abstract: A dynamic image stitching system for stitching images captured by multiple cameras of a vision system of a vehicle includes a first camera disposed at a vehicle and having a first field of view exterior the vehicle and a second camera disposed at the vehicle and having a second field of view exterior the vehicle. The first and second fields of view at least partially overlap. A processor is operable to process image data captured by the first and second cameras. The processor processes captured image data to determine characteristics of features or objects present in the overlapping region of the first and second fields of view. The processor is operable to adjust a stitching algorithm responsive to a determination of a difference between a characteristic of a feature as captured by the first camera and the characteristic of the feature as captured by the second camera.
    Type: Application
    Filed: February 27, 2014
    Publication date: September 4, 2014
    Applicant: Magna Electronics Inc.
    Inventors: Ghanshyam Rathi, Hilda Faraji, Nikhil Gupta, Christian Traub, Michael Schaffner, Goerg Pflug
  • Publication number: 20140063254
    Abstract: A method for calibrating a vehicular vision system includes providing a camera at a vehicle, with the camera having a field of view. Images are captured with the camera and a set of resultant images are acquired for a classification. Information is extracted related to image features in the set of resultant images, and an appropriate subset of coefficients is determined. For each classification, a classification vector of at least one appropriate weight is stored that corresponds to the determined subset of coefficients.
    Type: Application
    Filed: November 11, 2013
    Publication date: March 6, 2014
    Applicant: MAGNA INTERNATIONAL INC.
    Inventors: Bin Shi, Ghanshyam Rathi, Sharon Zibman, Tom Perovic, Nikhil Gupta, Hilda Faraji, Yong Zhou