Patents by Inventor Srikumar Ramalingam

Srikumar Ramalingam has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10416681
    Abstract: Systems and methods for an imaging system having a memory with a historical localization dictionary database having geo-located driving image sequences, such that each reference image is applied to a threshold to produce a binary representation. A sensor to acquire a sequence of input images of a dynamic scene. An encoder to determine, for each input image in the sequence, a histogram of each input image indicating a number of vertical edges at each bin of the input image and to threshold the histogram to produce a binary representation of the input image. A visual odometer to compare the binary representations of each input image and each reference image, by matching an input image against a reference image. Wherein the visual odometer determines a location of the input image based on a match between the input image and the reference image.
    Type: Grant
    Filed: July 12, 2017
    Date of Patent: September 17, 2019
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Teng-Yok Lee, Srikumar Ramalingam, Yuichi Taguchi, Sonali Patil
  • Patent number: 10410353
    Abstract: A image processing system for multi-label semantic edge detection in an image includes an image interface to receive an image of a scene including at least one object, a memory to store a neural network trained for performing a multi-label edge classification of input images assigning each pixel of edges of objects in the input images into one or multiple semantic classes, a processor to transform the image into a multi-label edge-map using the neural network detecting an edge of the object in the image and assigning multiple semantic labels to at least some pixels forming the edge, and an output interface to render the multi-label edge-map.
    Type: Grant
    Filed: September 28, 2017
    Date of Patent: September 10, 2019
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Chen Feng, Zhiding Yu, Srikumar Ramalingam
  • Publication number: 20190213790
    Abstract: A system for assigning a semantic label to a three-dimensional (3D) point of a point cloud includes an interface to transmit and receive data via a network, a processor connected to the interface, a memory storing program modules including a communicator and a labeler executable by the processor. In this case, the communicator causes the processor to perform operations including establishing, using the interface, a communication with an external map server storing a map defined in a map coordinate system, determining a location on the map corresponding to the 3D point, and querying the map using the location to obtain an information relevant to the semantic label at the location in the map, wherein the labeler causes the processor to determine the semantic label of the 3D point using the information.
    Type: Application
    Filed: January 11, 2018
    Publication date: July 11, 2019
    Inventors: Yuichi Taguchi, Teng-Yok Lee, Srikumar Ramalingam, Chen Feng
  • Patent number: 10260862
    Abstract: A method and system estimates a three-dimensional (3D) pose of a sensor by first acquiring scene data of a 3D scene by the sensor. Two-dimensional (2D) lines are detected in the scene data and the 2D lines are matched to 3D lines of a 3D model of the scene to produce matching lines. Then, the 3D pose of the sensor is estimated using the matching lines.
    Type: Grant
    Filed: November 2, 2015
    Date of Patent: April 16, 2019
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Srikumar Ramalingam, Jay Thornton, Wenzhen Yuan
  • Publication number: 20190018423
    Abstract: Systems and methods for an imaging system having a memory with a historical localization dictionary database having geo-located driving image sequences, such that each reference image is applied to a threshold to produce a binary representation. A sensor to acquire a sequence of input images of a dynamic scene. An encoder to determine, for each input image in the sequence, a histogram of each input image indicating a number of vertical edges at each bin of the input image and to threshold the histogram to produce a binary representation of the input image. A visual odometer to compare the binary representations of each input image and each reference image, by matching an input image against a reference image. Wherein the visual odometer determines a location of the input image based on a match between the input image and the reference image.
    Type: Application
    Filed: July 12, 2017
    Publication date: January 17, 2019
    Inventors: Teng-Yok Lee, Srikumar Ramalingam, Yuichi Taguchi, Sonali Patil
  • Publication number: 20180336683
    Abstract: A image processing system for multi-label semantic edge detection in an image includes an image interface to receive an image of a scene including at least one object, a memory to store a neural network trained for performing a multi-label edge classification of input images assigning each pixel of edges of objects in the input images into one or multiple semantic classes, a processor to transform the image into a multi-label edge-map using the neural network detecting an edge of the object in the image and assigning multiple semantic labels to at least some pixels forming the edge, and an output interface to render the multi-label edge-map.
    Type: Application
    Filed: September 28, 2017
    Publication date: November 22, 2018
    Inventors: Chen Feng, Zhiding Yu, Srikumar Ramalingam
  • Patent number: 10118796
    Abstract: Systems and Methods for controlling a movement of cars of an elevator system. A processor determines for each car an individual waiting time of each hall call. Determines for each pair of hall calls assigned for each car, a pairwise delay over the individual waiting time of each hall call in the pair caused by a joint assignment of the car to the pair of the hall calls. Approximate a cumulative waiting time of an assignment of the cars to the hall calls as a sum of individual waiting times for each hall call with the assigned car and a sum of all pairwise delays determined between all pairs of hall calls assigned to the same car. Determine the assignment of the cars using a greedy optimization algorithm that greedily assigns hall calls to the cars to minimize the approximated cumulative waiting time, and control the movement of the cars.
    Type: Grant
    Filed: March 3, 2017
    Date of Patent: November 6, 2018
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Daniel Nikolaev Nikovski, Arvind U Raghunathan, Srikumar Ramalingam
  • Publication number: 20180251335
    Abstract: Systems and Methods for controlling a movement of cars of an elevator system. A processor determines for each car an individual waiting time of each hall call. Determines for each pair of hall calls assigned for each car, a pairwise delay over the individual waiting time of each hall call in the pair caused by a joint assignment of the car to the pair of the hall calls. Approximate a cumulative waiting time of an assignment of the cars to the hall calls as a sum of individual waiting times for each hall call with the assigned car and a sum of all pairwise delays determined between all pairs of hall calls assigned to the same car. Determine the assignment of the cars using a greedy optimization algorithm that greedily assigns hall calls to the cars to minimize the approximated cumulative waiting time, and control the movement of the cars.
    Type: Application
    Filed: March 3, 2017
    Publication date: September 6, 2018
    Inventors: Daniel Nikolaev Nikovski, Arvind U. Raghunathan, Srikumar Ramalingam
  • Publication number: 20180249144
    Abstract: A system for reconstructing a three-dimensional (3D) model of a scene including a point cloud having points identified by 3D coordinates includes at least one sensor to acquire a set of images of the scene from different poses defining viewpoints of the images and a memory to store the set of images and the 3D model of the scene. The system also includes a processor operatively connected to the memory and coupled with stored instructions to transform the images from the set of images to produce a set of virtual images of the scene viewed from virtual viewpoints; compare at least some features from the images and the virtual images to determine the viewpoint of each image in the set of images; and update 3D coordinates of at least one point in the model of the scene to match coordinates of intersections of ray back-projections from pixels of at least two images corresponding to the point according to the viewpoints of the two images.
    Type: Application
    Filed: February 28, 2017
    Publication date: August 30, 2018
    Inventors: Chen Feng, Yuichi Taguchi, Esra Cansizoglu, Srikumar Ramalingam, Khalid Yousif, Haruyuki Iwama
  • Patent number: 10013765
    Abstract: An image registrations includes determining a first binary descriptor of a first key point in a first image, determining a second binary descriptor of a second key point in a second image, determining a weighted Hamming distance between the first binary descriptor and the second binary descriptor, and registering the first key point with the second key point when the weighted Hamming distance is below a noise threshold. At least one element in the first or the second binary descriptor is a result of a comparison of a difference between intensities of at least two pixels of the first or the second image with a threshold. At least two weights of the weighted Hamming distance for comparing at least two elements of the first or the second binary descriptors are different.
    Type: Grant
    Filed: August 19, 2016
    Date of Patent: July 3, 2018
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Srikumar Ramalingam, Yuichi Taguchi, Bharath Sankaran
  • Publication number: 20180056998
    Abstract: A method generates a time-series signal indicative of a variation of the environment in vicinity of the vehicle with respect to a motion of the vehicle and determines, using the time-series signal, a trajectory of the vehicle to a location different from a current location of the vehicle, the trajectory of the vehicle is a function of time. The method transmits the trajectory of the vehicle to a remote vehicle and controls a motion of the vehicle according to the trajectory of the vehicle.
    Type: Application
    Filed: August 29, 2016
    Publication date: March 1, 2018
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Mouhacine Benosman, Srikumar Ramalingam, Ulugbek Kamilov
  • Publication number: 20180053293
    Abstract: An image registrations includes determining a first binary descriptor of a first key point in a first image, determining a second binary descriptor of a second key point in a second image, determining a weighted Hamming distance between the first binary descriptor and the second binary descriptor, and registering the first key point with the second key point when the weighted Hamming distance is below a noise threshold. At least one element in the first or the second binary descriptor is a result of a comparison of a difference between intensities of at least two pixels of the first or the second image with a threshold. At least two weights of the weighted Hamming distance for comparing at least two elements of the first or the second binary descriptors are different.
    Type: Application
    Filed: August 19, 2016
    Publication date: February 22, 2018
    Inventors: Srikumar Ramalingam, Yuichi Taguchi, Bharath Sankaran
  • Patent number: 9898822
    Abstract: A method reconstructs a scene as a three-dimensional (3D) model by first acquiring a sequence of images of the scene with a camera. Then, feature point matches or line matches are extracted from the sequence of images, variables for camera optical centers and 3D points are initialized using random values, and n copies of the variables are made depending on an availability of n constraints. The n copies are projected to satisfy each of the n constraints. Then, the n copies are replaced with averages of the copies and the projecting and the replacing are repeated until convergence to provide the 3D model.
    Type: Grant
    Filed: February 3, 2016
    Date of Patent: February 20, 2018
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventor: Srikumar Ramalingam
  • Patent number: 9811756
    Abstract: A method labels an image of a street view by first extracting, for each pixel, an appearance feature for inferring a semantic label, a depth feature for inferring a depth label. Then, a column-wise labeling procedure is applied to the features to jointly determine the semantic label and the depth label for each pixel using the appearance feature and the depth feature, wherein the column-wise labeling procedure is according to a model of the street view, and wherein each column of pixels in the images includes at most four layers.
    Type: Grant
    Filed: February 23, 2015
    Date of Patent: November 7, 2017
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Ming-Yu Liu, Srikumar Ramalingam, Oncel Tuzel
  • Patent number: 9807365
    Abstract: A method and system for registering data by first acquiring the data from a scene by a sensor at different viewpoints, and extracting, from the data, three-dimensional (3D) points and 3D lines, and descriptors associated with the 3D points and the 3D lines. A first set of primitives represented in a first coordinate system of the sensor is selected, wherein the first set of primitives includes at least three 3D points. A second set of primitives represented in a second coordinate system is selected, wherein the second set of primitives includes any combination of 3D points and 3D lines to obtain at least three primitives. Then, using the first set of primitives and the second set of primitives, the 3D points are registered with each other, and the 3D points with the 3D lines to obtain registered primitives, wherein the registered primitives are used in a simultaneous localization and mapping (SLAM) system.
    Type: Grant
    Filed: December 8, 2015
    Date of Patent: October 31, 2017
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Esra Cansizoglu, Yuichi Taguchi, Srikumar Ramalingam
  • Publication number: 20170221211
    Abstract: A method reconstructs a scene as a three-dimensional (3D) model by first acquiring a sequence of images of the scene with a camera. Then, feature point matches or line matches are extracted from the sequence of images, variables for camera optical centers and 3D points are initialized using random values, and n copies of the variables are made depending on an availability of n constraints. The n copies are projected to satisfy each of the n constraints. Then, the n copies are replaced with averages of the copies and the projecting and the replacing are repeated until convergence to provide the 3D model.
    Type: Application
    Filed: February 3, 2016
    Publication date: August 3, 2017
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventor: Srikumar Ramalingam
  • Publication number: 20170161901
    Abstract: A method and system for registering data by first acquiring the data from a scene by a sensor at different viewpoints, and extracting, from the data, three-dimensional (3D) points and 3D lines, and descriptors associated with the 3D points and the 3D lines. A first set of primitives represented in a first coordinate system of the sensor is selected, wherein the first set of primitives includes at least three 3D points. A second set of primitives represented in a second coordinate system is selected, wherein the second set of primitives includes any combination of 3D points and 3D lines to obtain at least three primitives. Then, using the first set of primitives and the second set of primitives, the 3D points are registered with each other, and the 3D points with the 3D lines to obtain registered primitives, wherein the registered primitives are used in a simultaneous localization and mapping (SLAM) system.
    Type: Application
    Filed: December 8, 2015
    Publication date: June 8, 2017
    Inventors: Esra Cansizoglu, Yuichi Taguchi, Srikumar Ramalingam
  • Publication number: 20170124693
    Abstract: A method and system estimates a three-dimensional (3D) pose of a sensor by first acquiring scene data of a 3D scene by the sensor. Two-dimensional (2D) lines are detected in the scene data and the 2D lines are matched to 3D lines of a 3D model of the scene to produce matching lines. Then, the 3D pose of the sensor is estimated using the matching lines.
    Type: Application
    Filed: November 2, 2015
    Publication date: May 4, 2017
    Inventors: Srikumar Ramalingam, Jay Thornton, Wenzhen Yuan
  • Patent number: 9595134
    Abstract: A method reconstructs at three-dimensional (3D) real-world scene from a single two-dimensional (2D) image by identifying junctions satisfying geometric constraint of the scene based on intersecting lines, vanishing points, and vanishing lines that are orthogonal to each other. Possible layouts of the scene are generated by sampling the 2D image according to the junctions. Then, an energy function is maximized to select an optimal layout from the possible layouts. The energy function use's a conditional random field (CRF) model to evaluate the possible layouts.
    Type: Grant
    Filed: May 11, 2013
    Date of Patent: March 14, 2017
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Srikumar Ramalingam, Yuichi Taguchi, Jaishanker K Pillai, Dan J Burns, Christopher Reed Laughman
  • Patent number: 9558424
    Abstract: A method determines motion between first and second coordinate systems by first extracting first and second sets of keypoints from first and second images acquired of a scene by a camera arranged on a moving object. First and second poses are determined from the first and second sets of keypoints. A score for each possible motion between the first and the second poses is determined using a scoring function and a pose-transition graph constructed from training data where each node in the post-transition graph represents a relative pose and each edge represents a motion between two consecutive relative poses. Then, based on the score, a best motion is selected as the motion between the first and second coordinate systems.
    Type: Grant
    Filed: June 30, 2015
    Date of Patent: January 31, 2017
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Srikumar Ramalingam, Gim Hee Lee