Abstract: In various examples, sensor calibration for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that calibrate image sensors, such as cameras, using images captured by the image sensors at different time instances. For instance, a first image sensor may generate first image data representing at least two images and a second image sensor may generate second image data representing at least one image. One or more feature points may then be tracked between the images represented by the first image data and the image represented by the second image data. Additionally, the feature point(s), timestamps associated with the images, poses associated with image sensors (e.g., poses of a vehicle), and/or other information may be used to determine one or more values of one or more parameters that calibrate the first image sensor with the second image sensor.

Abstract: In various examples, a trailer angle may be estimated using one or more machine learning models to predict one or more keypoints on the center axis of the trailer drawbar (e.g., a keypoint representing the drawbar junction around which the drawbar pivots, one or more other keypoints along the center axis), back-projecting the predicted keypoint(s) onto a three-dimensional (3D) representation of the ground, and calculating the angle between the longitudinal axis of the towing vehicle and a line or ray formed by or fitted to the projected keypoints. The trailer angle may be estimated at any frame rate. For each frame, keypoints may be predicted from that frame and/or optical flow or some other type of feature tracking may be used to propagate predicted keypoint(s) from a preceding frame in lieu of predicting keypoint(s), and the resulting keypoint(s) may be used to estimate the trailer angle for that frame.

Abstract: In various examples, parking area detection for autonomous and/or semi-autonomous systems and applications is described herein. Systems and methods described herein may use a hybrid method to more accurately determine geometries of the parking areas within environments. For instance, sensor data (e.g., image data, etc.) may be processed using one or more edge detection techniques to determine edge features (e.g., two-dimensional pixels, three-dimensional points, etc.) associated with an environment that includes a parking area. A predicted geometry associated with the parking area, as determined using one or more machine learning models, may then be used to filter the edge features in order to identify a portion of the edge features that represent the parking area. One or more optimization techniques may then be used to determine the final geometry associated with the parking area based at least on the filtered edge features.

Abstract: In various examples, epipolar constraint-based cross-camera calibration validation is disclosed. For a pair of cameras that have partially overlapping fields of view, a shared region of their overlapping fields of view may be extracted and used as the basis to perform an epipolar constraint-guided feature descriptor matching process. A camera calibration metric may be computed based on the degree to which a feature descriptor appearing at a pixel of the first image aligns as expected in the second image with an epipolar line associated with the pixel of the first image, where the epipolar line is computed using extrinsic camera calibration parameters associated with the pair of cameras. Epipolar matching may be performed for a plurality of feature points and an aggregate validation score computed based on measuring the computed deviations for each feature. A sensitivity analysis may be applied to better assess the usefulness of the validation score.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: In various examples, a trailer angle may be estimated using one or more machine learning models to predict one or more keypoints on the center axis of the trailer drawbar (e.g., a keypoint representing the drawbar junction around which the drawbar pivots, one or more other keypoints along the center axis), back-projecting the predicted keypoint(s) onto a three-dimensional (3D) representation of the ground, and calculating the angle between the longitudinal axis of the towing vehicle and a line or ray formed by or fitted to the projected keypoints. The trailer angle may be estimated at any frame rate. For each frame, keypoints may be predicted from that frame and/or optical flow or some other type of feature tracking may be used to propagate predicted keypoint(s) from a preceding frame in lieu of predicting keypoint(s), and the resulting keypoint(s) may be used to estimate the trailer angle for that frame.

Abstract: In various examples, sensor calibration for autonomous or semi-autonomous systems and applications is described herein. Systems and methods are disclosed that calibrate image sensors, such as cameras, using images captured by the image sensors at different time instances. For instance, a first image sensor may generate first image data representing at least two images and a second image sensor may generate second image data representing at least one image. One or more feature points may then be tracked between the images represented by the first image data and the image represented by the second image data. Additionally, the feature point(s), timestamps associated with the images, poses associated with image sensors (e.g., poses of a vehicle), and/or other information may be used to determine one or more values of one or more parameters that calibrate the first image sensor with the second image sensor.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: In various examples, sensor configuration for autonomous or semi-autonomous systems and applications is described. Systems and methods are disclosed that may use image feature correspondences between camera images along with an assumption that image features are locally planar to determine parameters for calibrating an image sensor with a LiDAR sensor and/or another image sensor. In some examples, an optimization problem is constructed that attempts to minimize a geometric loss function, where the geometric loss function encodes the notion that corresponding image features are views of a same point on a locally planar surface (e.g., a surfel or mesh) that is constructed from LiDAR data generated using a LiDAR sensor. In some examples, performing such processes to determine the calibration parameters may remove structure estimation from the optimization problem.

Abstract: A multi-view (MV) network bridge device includes an upstream interface, multiple downstream interfaces, and a controller. The controller receives, from the upstream interface, a specification of one or more viewing zones and a specification of one or more content streams. Also, the controller sends, on at least one of the downstream interfaces, at least a subset of each of the specifications received from the upstream interface. The upstream interface may be coupled to a computer that provides the specifications. Each of the downstream interfaces may be coupled to a different MV display panel. One of the downstream interfaces may be coupled to an MV display panel that is coupled to another MV display panel. One of the downstream interfaces may be coupled to an upstream interface of another MV network bridge device having a downstream interface coupled to an MV display panel.

Abstract: A multi-view (MV) transportation hub information system is provided, which includes: a MV display including one or more multi-view (MV) pixels, wherein each MV pixel is configured to emit beamlets in different directions; a sensing system configured to detect a first location of a first blob and a second location of a second blob; an input node configured to receive a first attribute of a first viewer and a second attribute of a second viewer; and a system controller configured to perform user tagging to tag the first blob with the first attribute and to tag the second blob with the second attribute. The system controller controls the MV pixels to project a first image based on the first attribute to the first viewer tagged with the first blob, and to project a second image based on the second attribute to the second viewer tagged with the second blob.

Abstract: A multi-view (MV) network bridge device includes an upstream interface, multiple downstream interfaces, and a controller. The controller receives, from the upstream interface, a specification of one or more viewing zones and a specification of one or more content streams. Also, the controller sends, on at least one of the downstream interfaces, at least a subset of each of the specifications received from the upstream interface. The upstream interface may be coupled to a computer that provides the specifications. Each of the downstream interfaces may be coupled to a different MV display panel. One of the downstream interfaces may be coupled to an MV display panel that is coupled to another MV display panel. One of the downstream interfaces may be coupled to an upstream interface of another MV network bridge device having a downstream interface coupled to an MV display panel.

Abstract: A multi-view (MV) transportation hub information system is provided, which includes: a MV display including one or more multi-view (MV) pixels, wherein each MV pixel is configured to emit beamlets in different directions; a sensing system configured to detect a first location of a first blob and a second location of a second blob; an input node configured to receive a first attribute of a first viewer and a second attribute of a second viewer; and a system controller configured to perform user tagging to tag the first blob with the first attribute and to tag the second blob with the second attribute. The system controller controls the MV pixels to project a first image based on the first attribute to the first viewer tagged with the first blob, and to project a second image based on the second attribute to the second viewer tagged with the second blob.

Abstract: Systems and methods for determining position and orientation of an object using captured images.

Abstract: Systems and methods for tracking using a tracking camera. For each frame of image data generated by the tracking camera, each blob of the frame is determined. For each determined blob, a 2D image coordinate of a centroid of the blob is determined in a coordinate space of the frame. A tracking system processor generates a first tag identifier from the determined 2D image coordinates. The tracking system processor uses the first tag identifier to access stored first tag information that is stored in association with the first tag identifier. The tracking system processor determines an absolute 3-space position and orientation of the tracking camera by performing a motion tracking process using the determined 2D image coordinates and the accessed first tag information.

Abstract: Systems and methods for tracking using a tracking camera. For each frame of image data generated by the tracking camera, each blob of the frame is determined. For each determined blob, a 2D image coordinate of a centroid of the blob is determined in a coordinate space of the frame. A tracking system processor generates a first tag identifier from the determined 2D image coordinates. The tracking system processor uses the first tag identifier to access stored first tag information that is stored in association with the first tag identifier. The tracking system processor determines an absolute 3-space position and orientation of the tracking camera by performing a motion tracking process using the determined 2D image coordinates and the accessed first tag information.