Abstract: Presented herein are embodiments of a two-stage methodology that integrates data-driven imitation learning and model-based trajectory optimization to generate optimal trajectories for autonomous excavators. In one or more embodiments, a deep neural network using demonstration data to mimic the operation patterns of human experts under various terrain states, including their geometry shape and material type. A stochastic trajectory optimization methodology is used to improve the trajectory generated by the neural network to ensure kinematics feasibility, improve smoothness, satisfy hard constraints, and achieve desired excavation volumes. Embodiments were tested on a Franka robot arm equipped with a bucket end-effector. Embodiments were also evaluated on different material types, such as sand and rigid blocks. Experimental results showed that embodiments of the two-stage methodology that comprises combining expert knowledge and model optimization increased the excavation weights by up to 24.

Abstract: The present application provides a model training method and apparatus, and a prediction method and apparatus, and it relates to fields of artificial intelligence, deep learning, image processing, and autonomous driving. The model training method includes: inputting a first sample image of sample images into a depth information prediction model, and acquiring depth information of the first sample image; acquiring inter-image posture information based on a second sample image of the sample images and the first sample image; acquiring a projection image corresponding to the first sample image, at least according to the inter-image posture information and the depth information; and acquiring a loss function by determining a function for calculating a similarity between the second sample image and the projection image, and training the depth information prediction model using the loss function.

Abstract: A radar point cloud data processing method and device, an apparatus, and storage medium are provided, which are related to technical fields of radar point cloud, automatic driving, and deep learning. An implementation includes: determining a target location area where a target object is located by utilizing a target detection box in the radar point cloud data; removing each point of the target object in the target location area from the radar point cloud data; and adding an object model to the target location area. By applying embodiments of the present disclosure, richer radar point cloud data may be obtained by removing the target object from the radar point cloud data and adding the needed three-dimensional model to the target location area in the radar point cloud data.

Abstract: The present disclosure discloses an engineering machinery equipment, and a method, system, and storage medium for safety control thereof, and relates to the field of artificial intelligence, automatic control, and engineering machinery technologies. A method can include: acquiring spatial sensing data of a work area; performing obstacle detection based on the spatial sensing data to determine a position of an obstacle within the work area; determining a safe working range of a mechanical structural component of the engineering machinery equipment based on the position of the obstacle within the work area; and controlling a working range of the mechanical structural component of the engineering machinery equipment based on the safe working range.

Abstract: Provided are a vehicle attitude estimation method, an electronic device and a storage medium, relates to a technical field of data processing, and in particular to fields of automatic driving, intelligent transportation, Internet of Things, big data and the like. A specific implementation solution includes: obtaining first target data, based on point cloud data of a vehicle, the first target data being capable of constituting a target surface of the vehicle; performing attitude estimation on a target body for surrounding the vehicle, based on the first target data, to obtain an estimation result; and estimating an attitude of the vehicle, based on the estimation result. According to the implementation solution, precise or accurate estimation of the attitude of the vehicle may be achieved.

Abstract: A scalable solution to robot behavioral navigation following natural language instructions is presented. An example of the solution includes: receiving, by a pre-trained sequential prediction model, a navigation graph of the task environment, instructions in natural language and an initial location of the robot in the navigation graph, wherein the navigation graph comprises nodes indicating locations in the task environment, coordinates of the nodes, and edges indicating connectivity between the locations; and predicting sequentially, by the pre-trained sequential prediction model, a sequence of single-step behaviors executable by the robot to navigate the robot from the initial location to a destination.

Abstract: A speed determination method, an electronic device and a computer storage medium are provided, relates to the field of computer technology, and may be applied to the field of artificial intelligence, especially the field of automated driving. The method includes: determining an expected speed direction of a controlled point of a first controlled target according to an actual location of the controlled point of the first controlled target and a preset trajectory of the controlled point of the first controlled target, wherein the first controlled target is one of a plurality of controlled targets having a kinematic relationship; and determining a target speed of at least one controlled target of the plurality of controlled targets according to the expected speed direction of the controlled point of the first controlled target and the kinematic relationship.

Abstract: The present disclosure discloses an engineering machinery equipment, and a method, system, and storage medium for operation trajectory planning thereof, and relates to the field of artificial intelligence, automatic control, and engineering machinery technologies.

Abstract: Presented herein are embodiments of a vision-based object perception system for activity analysis, safety monitoring, or both. Embodiments of the perception subsystem detect multi-class objects (e.g., construction machines and humans) in real-time while estimating the poses and actions of the detected objects. Safety monitoring embodiments and object activity analysis embodiments may be based on the perception result. To evaluate the performance of embodiments, a dataset was collected including multi-class of objects in different lighting conditions with human annotations. Experimental results show that the proposed action recognition approach outperforms the state-of-the-art approaches on top-1 accuracy by about 5.18%.

Abstract: Autonomous excavator has developed rapidly in recent years because of a shortage of labor and hazardous working environments for operating excavators. Presented herein are embodiments of a novel hierarchical planning system for autonomous machines, such as excavators. In one or more embodiments, the overall planning system comprises a high-level task planner for task division and base movement planning, and general sub-task planners with motion primitives, which include both arm and base movement in the case of an excavator. Using embodiments of the system architecture, experiments were performed for the trench and pile removal tasks in the real world and for the large-scale material loading tasks in a simulation environment. The results show that the system architecture embodiments and planner method embodiments generate effective task and motion plans that perform well in autonomous excavation.

Abstract: Presented herein are novel approaches to synthesize video of the speech from text. In a training phase, embodiments build a phoneme-pose dictionary and train a generative neural network model using a generative adversarial network (GAN) to generate video from interpolated phoneme poses. In deployment, the trained generative neural network in conjunction with the phoneme-pose dictionary convert an input text into a video of a person speaking the words of the input text. Compared to audio-driven video generation approaches, the embodiments herein have a number of advantages: 1) they only need a fraction of the training data used by an audio-driven approach; 2) they are more flexible and not subject to vulnerability due to speaker variation; and 3) they significantly reduce the preprocessing, training, and inference times.

Abstract: Described herein are systems and methods for inverse reinforcement learning to leverage the benefits of model-based optimization method and model-free learning method. Embodiments of a framework combining human behavior model with model predictive control are presented. The framework takes advantage of feature identification capability of a neural network to determine the reward function of model predictive control. Furthermore, embodiments of the present approach are implemented to solve the practical autonomous driving longitudinal control problem with simultaneous preference on safe execution and passenger comfort.

Abstract: Embodiments of a learning-based excavation planning method are disclosed for excavating rigid objects in clutter, which is challenging due to high variance of geometric and physical properties of objects, and large resistive force during the excavation. A convolutional neural network is utilized to predict a probability of excavation success. Embodiments of a sampling-based optimization method are disclosed for planning high-quality excavation trajectories by leveraging the learned prediction model. To reduce simulation-to-real gap for excavation learning, voxel-based representations of an excavation scene are used. Excavation experiments were performed in both simulation and real world to evaluate the learning-based excavation planners. Experimental results show that embodiments of the disclosed method may plan high-quality excavations for rigid objects in clutter and outperform baseline methods by large margins.

Abstract: A method for detecting an operating terrain is provided. The method includes obtaining point cloud data of an operating region that are collected by a laser radar at a current time, including three-dimensional coordinates of a plurality of sampling points. The operating region is divided into a plurality of grids, each having a corresponding height value. The method includes for any grid determining an input point of the grid from the plurality of sampling points, based on the three-dimensional coordinates. The method includes determining a type of the input point, based on a height coordinate of the input point and the height value of the grid. The type includes a noise point and a ground point. The method includes in response to determining that the input point is the ground point, updating the height value of the grid based on the height coordinate of the input point.

Abstract: Embodiments of the present disclosure disclose a method, apparatus, device, and storage medium for controlling a guide robot, and relate to the field of artificial intelligence, robots, and multi-sensor fusion technologies. A specific embodiment of the method includes: acquiring a state of the guide robot, a state of a user, and a position of an obstacle; generating a state update equation for a combined system of the guide robot and the user based on the state of the guide robot and the state of the user; generating a collision-free global path based on the position of the obstacle; generating a control command based on the state update equation for the combined system and the collision-free global path; and driving the guide robot to move based on the control command.

Abstract: Described herein are embodiments of a neural network-based task planner (TaskNet) for autonomous vehicle. Given a high-level task, the TaskNet planner decomposes it into a sequence of sub-tasks, each of which is further decomposed into task primitives with specifications. TaskNet comprises a first model for predicating the global sequence of working area to cover large terrain, and a second model for determining local operation order and specifications for each operation. The neural models may include convolutional layers for extracting features from grid map-based environment representation, and fully connected layers to combine extracted features with past sequences and predict the next sub-task or task primitive. Embodiments of the TaskNet are trained using an excavation trace generator and evaluate its performance using a 3D physically-based terrain and excavator simulator.

Abstract: A method of determining a vehicle travel trajectory, an electronic device, a storage medium and a vehicle, which relate to a field of an artificial intelligence technology, in particular to a field of autonomous driving and intelligent transportation. A specific implementation solution includes: determining an initial path information for a vehicle; optimizing the initial path information to generate a target optimized path information; determining an optimized mapping relationship for velocity according to the target optimized path information and a first energy consumption constraint parameter; and determining an optimized trajectory for the vehicle according to the target optimized path information and the optimized mapping relationship for velocity.

Abstract: A method, an apparatus, and a storage medium for determining a destination on a map are provided. In the method, N segments of a text are acquired; a recursive order of the N segments is determined based on a grammatical relationship between the N segments of the text; a matching model is selected for each of the N segments from multiple models based on a meaning of the segment; each of the segments is input in the recursive order with an initial region of the map or an update region output by the matching model for the last segment prior to the segment in the recursive order, into the matching model of the segment; and the update region output by the matching model for the last segment in the recursive order is used as the destination in the map.

Abstract: A scalable solution to robot behavioral navigation following natural language instructions is presented. An example of the solution includes: receiving, by a pre-trained sequential prediction model, a navigation graph of the task environment, instructions in natural language and an initial location of the robot in the navigation graph, wherein the navigation graph comprises nodes indicating locations in the task environment, coordinates of the nodes, and edges indicating connectivity between the locations; and predicting sequentially, by the pre-trained sequential prediction model, a sequence of single-step behaviors executable by the robot to navigate the robot from the initial location to a destination.

Abstract: Provided are a depth information processing method, an apparatus, and a storage medium, which relate to the field of image processing and, in particular, to computer vision, deep learning and autonomous driving. A specific implementation includes: determining intermediate depth information of a target scene according to sparse depth information of the target scene by a sub-model unit in a depth information supplementing model; and using intermediate depth information determined by a tail sub-model unit in the depth information supplementing model as dense depth information of the target scene.