Patents by Inventor Stergios I. Roumeliotis

Stergios I. Roumeliotis has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240230335
    Abstract: A vision-aided inertial navigation system (VINS) comprises an image source for producing image data along a trajectory. The VINS further comprises an inertial measurement unit (IMU) configured to produce IMU data indicative of motion of the VINS and an odometry unit configured to produce odometry data. The VINS further comprises a processor configured to compute, based on the image data, the IMU data, and the odometry data, state estimates for a position and orientation of the VINS for poses of the VINS along the trajectory. The processor maintains a state vector having states for a position and orientation of the VINS and positions within the environment for observed features for a sliding window of poses. The processor applies a sliding window filter to compute, based on the odometry data, constraints between the poses within the sliding window and compute, based on the constraints, the state estimates.
    Type: Application
    Filed: February 15, 2024
    Publication date: July 11, 2024
    Applicant: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu, Chao Guo
  • Patent number: 11994392
    Abstract: A vision-aided inertial navigation system (VINS) implements a square-root multi-state constraint Kalman filter (SR-MSCKF) for navigation. In one example, a processor of a VINS receives image data and motion data for a plurality of poses of a frame of reference along a trajectory. The processor executes an Extended Kalman Filter (EKF)-based estimator to compute estimates for a position and orientation for each of the plurality of poses of the frame of reference along the trajectory. For features observed from multiple poses along the trajectory, the estimator computes constraints that geometrically relate the multiple poses of the respective feature. Using the motion data and the computed constraints, the estimator computes state estimates for the position and orientation of the frame of reference. Further, the estimator determines uncertainty data for the state estimates and maintains the uncertainty data as a square root factor of a covariance matrix.
    Type: Grant
    Filed: October 10, 2022
    Date of Patent: May 28, 2024
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu
  • Patent number: 11940277
    Abstract: A vision-aided inertial navigation system (VINS) comprises an image source for producing image data along a trajectory. The VINS further comprises an inertial measurement unit (IMU) configured to produce IMU data indicative of motion of the VINS and an odometry unit configured to produce odometry data. The VINS further comprises a processor configured to compute, based on the image data, the IMU data, and the odometry data, state estimates for a position and orientation of the VINS for poses of the VINS along the trajectory. The processor maintains a state vector having states for a position and orientation of the VINS and positions within the environment for observed features for a sliding window of poses. The processor applies a sliding window filter to compute, based on the odometry data, constraints between the poses within the sliding window and compute, based on the constraints, the state estimates.
    Type: Grant
    Filed: May 29, 2019
    Date of Patent: March 26, 2024
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu, Chao Guo, Georgios Georgiou
  • Publication number: 20240011776
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Application
    Filed: October 31, 2022
    Publication date: January 11, 2024
    Applicant: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Publication number: 20230408262
    Abstract: Vision-aided inertial navigation techniques are described. In one example, a vision-aided inertial navigation system (VINS) comprises an image source to produce image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein the image data captures features within the 3D environment at each of the first time instances. An inertial measurement unit (IMU) to produce IMU data for the VINS along the trajectory at a second set of time instances that is misaligned with the first set of time instances, wherein the IMU data indicates a motion of the VINS along the trajectory. A processing unit comprising an estimator that processes the IMU data and the image data to compute state estimates for 3D poses of the IMU at each of the first set of time instances and 3D poses of the image source at each of the second set of time instances along the trajectory.
    Type: Application
    Filed: August 1, 2023
    Publication date: December 21, 2023
    Applicant: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Chao Guo
  • Patent number: 11719542
    Abstract: Vision-aided inertial navigation techniques are described. In one example, a vision-aided inertial navigation system (VINS) comprises an image source to produce image data at a first set of time instances along a trajectory within a three-dimensional (3D) environment, wherein the image data captures features within the 3D environment at each of the first time instances. An inertial measurement unit (IMU) to produce IMU data for the VINS along the trajectory at a second set of time instances that is misaligned with the first set of time instances, wherein the IMU data indicates a motion of the VINS along the trajectory. A processing unit comprising an estimator that processes the IMU data and the image data to compute state estimates for 3D poses of the IMU at each of the first set of time instances and 3D poses of the image source at each of the second set of time instances along the trajectory.
    Type: Grant
    Filed: July 2, 2018
    Date of Patent: August 8, 2023
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Chao Guo
  • Publication number: 20230194266
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Application
    Filed: October 31, 2022
    Publication date: June 22, 2023
    Applicant: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Patent number: 11519729
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Grant
    Filed: May 11, 2020
    Date of Patent: December 6, 2022
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Patent number: 11486707
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Grant
    Filed: November 30, 2021
    Date of Patent: November 1, 2022
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Patent number: 11466990
    Abstract: A vision-aided inertial navigation system (VINS) implements a square-root multi-state constraint Kalman filter (SR-MSCKF) for navigation. In one example, a processor of a VINS receives image data and motion data for a plurality of poses of a frame of reference along a trajectory. The processor executes an Extended Kalman Filter (EKF)-based estimator to compute estimates for a position and orientation for each of the plurality of poses of the frame of reference along the trajectory. For features observed from multiple poses along the trajectory, the estimator computes constraints that geometrically relate the multiple poses of the respective feature. Using the motion data and the computed constraints, the estimator computes state estimates for the position and orientation of the frame of reference. Further, the estimator determines uncertainty data for the state estimates and maintains the uncertainty data as a square root factor of a covariance matrix.
    Type: Grant
    Filed: July 21, 2017
    Date of Patent: October 11, 2022
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu
  • Publication number: 20220082386
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Application
    Filed: November 30, 2021
    Publication date: March 17, 2022
    Applicant: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Patent number: 10907971
    Abstract: A vision-aided inertial navigation system comprises an image source to produce image data for poses of reference frames along a trajectory, a motion sensor configured to provide motion data of the reference frames, and a hardware-based processor configured to compute estimates for a position and orientation of the reference frames for the poses. The processor executes a square-root inverse Schmidt-Kalman Filter (SR-ISF)-based estimator to compute, for features observed from poses along the trajectory, constraints that geometrically relate the poses from which the respective feature was observed. The estimator determines, in accordance with the motion data and the computed constraints, state estimates for position and orientation of reference frames for poses along the trajectory and computes positions of the features that were each observed within the environment.
    Type: Grant
    Filed: December 7, 2018
    Date of Patent: February 2, 2021
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu, Tong Ke
  • Publication number: 20200300633
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The processor computes, from the image data, one or more constraints based on features observed from multiple poses of the electronic device along the trajectory, and computes updated state estimates for the position of the electronic device in accordance with the motion data and the one or more computed constraints without computing updated state estimates for the features for which the one or more constraints were computed.
    Type: Application
    Filed: May 11, 2020
    Publication date: September 24, 2020
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Patent number: 10670404
    Abstract: Localization and navigation systems and techniques are described. An electronic device comprises a processor configured to apply an extended Kalman filter (EKF) as the electronic device traverses the trajectory. The extended Kalman filter is configured to maintain a state vector storing estimates for a position of the electronic device at poses along a trajectory within an environment along with estimates for positions for one or more features within the environment. The EKF computes constraints based on features observed from multiple poses along the trajectory, and updates, in accordance with motion data and the one or more computed constraints, the estimates within the state vector of the extended Kalman filter while excluding, from the state vector, state estimates for positions within the environment for the features that were observed from the multiple poses and for which the constraints were computed.
    Type: Grant
    Filed: September 15, 2017
    Date of Patent: June 2, 2020
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Anastasios I. Mourikis
  • Publication number: 20190368879
    Abstract: A vision-aided inertial navigation system (VINS) comprises an image source for producing image data along a trajectory. The VINS further comprises an inertial measurement unit (IMU) configured to produce IMU data indicative of motion of the VINS and an odometry unit configured to produce odometry data. The VINS further comprises a processor configured to compute, based on the image data, the IMU data, and the odometry data, state estimates for a position and orientation of the VINS for poses of the VINS along the trajectory. The processor maintains a state vector having states for a position and orientation of the VINS and positions within the environment for observed features for a sliding window of poses. The processor applies a sliding window filter to compute, based on the odometry data, constraints between the poses within the sliding window and compute, based on the constraints, the state estimates.
    Type: Application
    Filed: May 29, 2019
    Publication date: December 5, 2019
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu, Chao Guo, Georgios Georgiou
  • Patent number: 10371529
    Abstract: This disclosure describes inverse filtering and square root inverse filtering techniques for optimizing the performance of a vision-aided inertial navigation system (VINS). In one example, instead of keeping all features in the system's state vector as SLAM features, which can be inefficient when the number of features per frame is large or their track length is short, an estimator of the VINS may classify the features into either SLAM or MSCKF features. The SLAM features are used for SLAM-based state estimation, while the MSCKF features are used to further constrain the poses in the sliding window. In one example, a square root inverse sliding window filter (SQRT-ISWF) is used for state estimation.
    Type: Grant
    Filed: May 22, 2017
    Date of Patent: August 6, 2019
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu
  • Publication number: 20190178646
    Abstract: A vision-aided inertial navigation system comprises an image source to produce image data for poses of reference frames along a trajectory, a motion sensor configured to provide motion data of the reference frames, and a hardware-based processor configured to compute estimates for a position and orientation of the reference frames for the poses. The processor executes a square-root inverse Schmidt-Kalman Filter (SR-ISF)-based estimator to compute, for features observed from poses along the trajectory, constraints that geometrically relate the poses from which the respective feature was observed. The estimator determines, in accordance with the motion data and the computed constraints, state estimates for position and orientation of reference frames for poses along the trajectory and computes positions of the features that were each observed within the environment.
    Type: Application
    Filed: December 7, 2018
    Publication date: June 13, 2019
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu, Tong Ke
  • Publication number: 20190154449
    Abstract: A vision-aided inertial navigation system (VINS) implements a square-root multi- state constraint Kalman filter (SR-MSCKF) for navigation. In one example, a processor of a VINS receives image data and motion data for a plurality of poses of a frame of reference along a trajectory. The processor executes an Extended Kalman Filter (EKF)- based estimator to compute estimates for a position and orientation for each of the plurality of poses of the frame of reference along the trajectory. For features observed from multiple poses along the trajectory, the estimator computes constraints that geometrically relate the multiple poses of the respective feature. Using the motion data and the computed constraints, the estimator computes state estimates for the position and orientation of the frame of reference. Further, the estimator determines uncertainty data for the state estimates and maintains the uncertainty data as a square root factor of a covariance matrix.
    Type: Application
    Filed: July 21, 2017
    Publication date: May 23, 2019
    Inventors: Stergios I. Roumeliotis, Kejian J. Wu
  • Patent number: 10254118
    Abstract: This disclosure describes various techniques for use within a vision-aided inertial navigation system (VINS). A VINS comprises an image source to produce image data comprising a plurality of images, and an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the vision-aided inertial navigation system while producing the image data, wherein the image data captures features of an external calibration target that is not aligned with gravity. The VINS further includes a processing unit comprising an estimator that processes the IMU data and the image data to compute calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target, wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the vision-aided inertial navigation system.
    Type: Grant
    Filed: February 21, 2014
    Date of Patent: April 9, 2019
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Dimitrios G. Kottas, Kejian J. Wu
  • Patent number: 10203209
    Abstract: A method includes: receiving, with a computing platform, respective trajectory data and map data independently generated by each of a plurality of vision-aided inertial navigation devices (VINS devices) traversing an environment, wherein the trajectory data specifies poses along a path through the environment for the respective VINS device and the map data specifies positions of observed features within the environment as determined by an estimator executed by the respective VINS device; determining, with the computing platform and based on the respective trajectory data and map data from each of the VINS devices, estimates for relative poses within the environment by determining transformations that geometrically relate the trajectory data and the map data between one or more pairs of the VINS devices; and generating, with the computing platform and based on the transformations, a composite map specifying positions within the environment for the features observed by the VINS devices.
    Type: Grant
    Filed: May 25, 2017
    Date of Patent: February 12, 2019
    Assignee: Regents of the University of Minnesota
    Inventors: Stergios I. Roumeliotis, Esha D. Nerurkar, Joel Hesch, Chao Guo, Ryan C. DuToit, Kourosh Sartipi, Georgios Georgiou