Abstract: A method for calibrating sensors devices mounted on a machine is disclosed. The method includes transforming a first 3D point cloud to generate a transformed first 3D point cloud based on an alignment of the first 3D point cloud with a 3D model of the machine. The first 3D point cloud includes 3D point cloud of one or more features of a terrain around the machine, and the machine. The method further includes aligning a second 3D point cloud with the transformed first 3D point cloud based on the one or more features of the terrain, to determine one or more transformation parameters. Thereafter, one or more calibration parameters for the second sensor device is extracted from the one or more transformation parameters. The second sensor device is calibrated based on the one or more calibration parameters.

Abstract: A system for monitoring an implement of a work machine is provided. The system may include one or more image sensors mounted on the work machine configured to capture one or more images of a field of view associated with the implement, and an implement controller in electrical communication with the image sensors. The implement controller may be configured to receive the images from the image sensors, identify one or more interactive targets within the images, select one of the interactive targets based on proximity, and align the implement to the selected interactive target.

Abstract: A system for monitoring an implement of a work machine is provided. The system may include one or more image sensors mounted on the work machine configured to capture one or more images of a field of view associated with the implement, and an implement controller in electrical communication with the image sensors. The implement controller may be configured to receive the images from the image sensors, identify one or more interactive targets within the images, select one of the interactive targets based on proximity, and align the implement to the selected interactive target.

Abstract: A work tool recognition system for a work tool coupled to a machine is provided. The system includes an image capturing assembly configured to capture an image feed of the work tool. A controller is communicably coupled to the image capturing assembly. The controller receives the image feed of the work tool from the image capturing assembly. The controller extracts a plurality of features of the work tool from the image feed. The controller classifies the extracted image and determine a confidence value. The controller estimates a size of the work tool. The controller estimates a position and an orientation of the work tool. The controller determines verification features of the work tool and compares the image feed of the work tool with a predetermined dataset based on the classification. The controller identifies a type of the work tool based on the comparison.

Abstract: A work tool recognition system for a work tool coupled to a machine is provided. The system includes an image capturing assembly configured to capture an image feed of the work tool. A controller is communicably coupled to the image capturing assembly. The controller receives the image feed of the work tool from the image capturing assembly. The controller extracts a plurality of features of the work tool from the image feed. The controller classifies the extracted image and determine a confidence value. The controller estimates a size of the work tool. The controller estimates a position and an orientation of the work tool. The controller determines verification features of the work tool and compares the image feed of the work tool with a predetermined dataset based on the classification. The controller identifies a type of the work tool based on the comparison.

Abstract: A method for calibrating sensors devices mounted on a machine is disclosed. The method includes transforming a first 3D point cloud to generate a transformed first 3D point cloud based on an alignment of the first 3D point cloud with a 3D model of the machine. The first 3D point cloud includes 3D point cloud of one or more features of a terrain around the machine, and the machine. The method further includes aligning a second 3D point cloud with the transformed first 3D point cloud based on the one or more features of the terrain, to determine one or more transformation parameters. Thereafter, one or more calibration parameters for the second sensor device is extracted from the one or more transformation parameters. The second sensor device is calibrated based on the one or more calibration parameters.

Abstract: A terrain mapping system includes an aerial system including at least one camera configured to capture a plurality of aerial images of an area and transmit the plurality of aerial images to an image database, a plurality of machines each having a ground control point (GCP) disposed thereon, and each being configured to periodically record a global location of the GCP in a location history database, and a mapping unit configured to construct a preliminary three-dimensional (3D) terrain map based on the plurality of aerial images, detect at least one GCP within at least one aerial image, determine an estimated global location and an accurate global location of the at least one GCP, and calibrate the preliminary 3D terrain map based on the estimated global location and the accurate global location of the at least one GCP.

Abstract: A system for augmenting wireless communication and satellite positioning for machines at a worksite includes one or more unmanned aerial vehicles (UAV) configured to be remotely operated above an area encompassing the worksite. Each of the UAV includes a real time kinematic (RTK) global positioning system (GPS) onboard the UAV for determining the position of the UAV relative to a base station located at a known location, and a machine vision module for detecting an object on the ground at the worksite. The RTK GPS onboard each UAV determines the global coordinates of the detected object in 3D space using the position of the UAV.

Abstract: A system related to synthetic colorization of real-time immersive environments is disclosed. The system discloses converting real-time camera data on-board a machine into signatures which describe the real-time camera data. The signatures are based on image properties, such as color, intensity, and illumination. The signatures, which require much less bandwidth than the real-time camera data to transmit, are transmitted to a remote operator station. The remote operator station utilizes the signatures to synthetically color and texture the environment surrounding the machine to provide an immersive environment to a user at the remote operator station.

Abstract: A terrain mapping system includes an aerial system including at least one camera configured to capture a plurality of aerial images of an area and transmit the plurality of aerial images to an image database, a plurality of machines each having a ground control point (GCP) disposed thereon, and each being configured to periodically record a global location of the GCP in a location history database, and a mapping unit configured to construct a preliminary three-dimensional (3D) terrain map based on the plurality of aerial images, detect at least one GCP within at least one aerial image, determine an estimated global location and an accurate global location of the at least one GCP, and calibrate the preliminary 3D terrain map based on the estimated global location and the accurate global location of the at least one GCP.

Abstract: A calibration system for a machine is provided. The calibration system includes an unmanned aerial vehicle provided in association with a perception sensor. The unmanned aerial vehicle includes a target attached thereto. The unmanned aerial vehicle is configured to move along a predetermined path sweeping across a field of view of the perception sensor. The unmanned aerial vehicle is configured to present the target to the perception sensor, such that the target covers the field of view of the perception sensor based on the movement of the unmanned aerial vehicle along the predetermined path. The unmanned aerial vehicle is configured to determine and communicate an orientation and a position of the target to the calibration system.

Abstract: A system related to synthetic colorization of real-time immersive environments is disclosed. The system discloses converting real-time camera data on-board a machine into signatures which describe the real-time camera data. The signatures are based on image properties, such as color, intensity, and illumination. The signatures, which require much less bandwidth than the real-time camera data to transmit, are transmitted to a remote operator station. The remote operator station utilizes the signatures to synthetically color and texture the environment surrounding the machine to provide an immersive environment to a user at the remote operator station.

Abstract: A calibration system for a machine is provided. The calibration system includes an unmanned aerial vehicle provided in association with a perception sensor. The unmanned aerial vehicle includes a target attached thereto. The unmanned aerial vehicle is configured to move along a predetermined path sweeping across a field of view of the perception sensor. The unmanned aerial vehicle is configured to present the target to the perception sensor, such that the target covers the field of view of the perception sensor based on the movement of the unmanned aerial vehicle along the predetermined path. The unmanned aerial vehicle is configured to determine and communicate an orientation and a position of the target to the calibration system.

Abstract: An image processing system is disclosed for an articulated machine. The system includes a plurality of cameras mounted on the machine for capturing images, and a machine state sensor for obtaining machine state data of machine. The system also includes a processor connected to the plurality of cameras and the machine state sensor, the processor being configured to establish camera extrinsic models for the plurality of cameras based on positions and orientations of the cameras, obtain the machine state data from the machine state sensor, establish a machine geometric model based on the machine state data, and establish image mapping rules for the plurality of cameras based on the machine geometric model and the camera extrinsic models. The processor is further configured to generate a unified image from the images captured by the plurality of cameras based on the image mapping rules, and render the unified image on a display.

Abstract: An imaging system is disclosed for use with a mobile machine. The imaging system may have at least one onboard camera configured to generate image data for an actual environment of the mobile machine, and an onboard sensor configured to generate object data regarding detection and ranging of an object in the actual environment. The imaging system may also have a display mounted on the machine, and a processor in communication with the at least one camera, the sensor, and the display. The processor may be configured to generate a virtual geometry, and generate a virtual object within the virtual geometry based on the object data. The processor may further be configured to generate a unified image of the actual environment based on the image data, to map a projection of the unified image onto the virtual geometry and the virtual object, and to render a selected portion of the projection on the display.