Abstract: ETA (estimated time of arrival) calculation for a mobile machine is disclosed. The ETA of the mobile machine to a destination is calculated by obtaining a current pose of the mobile machine, obtaining a global path from the current pose of the mobile machine to the destination, obtaining a local path, calculating a dynamic ETA for each pair of the consecutive poses in the local path and summing the calculated dynamic ETAs, calculating a baseline ETA for each pair of consecutive poses from a pose in the global path that is closest to the last pose in the local path to the last pose in the global path and summing the calculated baseline ETAs, and obtaining a total ETA to the destination based on the dynamic ETA and the baseline ETA.

Abstract: Image-based trajectory planning for a mobile machine having a camera is disclosed. A trajectory for the mobile machine to move to a referenced target is planned by obtaining, a desired image of the referenced target captured at a desired pose of the trajectory and an initial image of the referenced target captured at an initial pose of the trajectory through the camera of the mobile machine, calculating a homography of the desired image and the initial image; decomposing the homography into an initial translation component and an initial rotation matrix, obtaining optimized translation component(s) corresponding to constraint(s) for the mobile machine based on the initial translation component, obtaining optimized homography(s) based on the optimized translation component(s) and the initial rotation matrix, and obtaining objective image features corresponding to the trajectory based on the optimized homography(s).

Abstract: A human-object scene recognition method includes: acquiring an input RGB image and a depth image corresponding to the RGB image; detecting objects and humans in the RGB image using a segmentation classification algorithm based on a sample database; in response to detection of objects and/or humans, performing a segment detection to each of the detected objects and/or humans based on the ROB image and the depth image, and acquiring a result of the segment detection; calculating 3D hounding boxes for each of the detected objects and/or humans according to the result of the segment detection, and determining a position of each of the detected objects and/or humans according to the 3D bounding boxes.

Abstract: Human posture determination is disclosed. Human posture is determined by obtaining range image(s) through a range camera, detecting key points of an estimated skeleton of a human in color data of the range image(s) and calculating positions of the detected key points based on depth data of the range image(s), choosing a feature map from a set of predefined feature maps based on the detected key points among a set of predefined key points, obtaining two features of a body of the human corresponding to the chosen feature map based on the positions of the detected key points, and determining a posture of the human according to the two features in the chosen feature map.

Abstract: Collision avoidance for a mobile machine having a plurality of sensors is disclosed. The mobile machine is avoided from colliding with a collision object by fusing sensor data received from the plurality of sensors to obtain a plurality of data points corresponding to the collision object, calculating a closed-form solution of a distance between the mobile machine and each of the plurality of data points, calculating a maximum allowed velocity of the mobile machine based on the shortest distance between the mobile machine and the plurality of data points and a current velocity of the mobile machine, and controlling the mobile machine to move according to the maximum allowed velocity.

Abstract: A design method of serial manipulator that comprises an end effector, a number of random-access links, and a number of motors, includes: obtaining a desired motion profile of the end effector; discretizing the desired motion profile into a plurality of points, wherein each of the points carries information of speed, acceleration, and force/torque of the end effector at the point; determining the number of degrees of freedom of the serial manipulator, and initializing the length of each of the links and the motor type of each of the motors; and at each of the points, optimizing the initialized lengths of the links and the motor types of the motors by calculating a dynamic manipulability ellipsoid at the end effector, to obtain desired lengths of the links and desired motor types of the motors, which allows the end effector to execute the desired motion profile under predetermined constraints.

Abstract: On-floor obstacle detection using an RGB-D camera is disclosed. An obstacle on a floor is detected by receiving an image including depth channel data and RGB channel data through the RGB-D camera, estimating a ground plane corresponding to the floor based on the depth channel data, obtaining a foreground of the image corresponding to the ground plane based on the depth channel data, performing a distribution modeling on the foreground of the image based on the RGB channel data to obtain a 2D location of the obstacle, and transforming the 2D location of the obstacle into a 3D location of the obstacle based on the depth channel data.

Abstract: Embodiments of the disclosure provide methods and systems for implementing a function for an application using a middleware on a computer. An exemplary method may include initializing an application node in the middleware corresponding to the application, the initializing comprising binding the function to the application node and associating the application node with a cryptor, connecting the middleware to a remote device through a transport layer using the cryptor, publishing a message to invoke the function from the application node, through the middleware, to the cryptor, and communicating with the remote device to invoke the function, using the cryptor, based on the message.

Abstract: A human-object scene recognition method includes: acquiring an input RGB image and a depth image corresponding to the RGB image; detecting objects and humans in the RGB image using a segmentation classification algorithm based on a sample database; in response to detection of objects and/or humans, performing a segment detection to each of the detected objects and/or humans based on the ROB image and the depth image, and acquiring a result of the segment detection; calculating 3D hounding boxes for each of the detected objects and/or humans according to the result of the segment detection, and determining a position of each of the detected objects and/or humans according to the 3D bounding boxes.

Abstract: Human lying posture detections are disclosed. A human lying on a bed is detected by obtaining an image through a depth camera, detecting objects in the image and marking the objects in the image using 2D bounding boxes by deep learning, determining the human being in a lying posture in response to a width and a height of the 2D bounding box of the human meeting a predetermined condition, detecting one or more skin areas in the image and generating skin area 2D bounding boxes to mark each of the one or more skin areas using a skin detection algorithm, and determining the human being in the lying posture in response to the skin area 2D bounding boxes and the 2D bounding box of the bed meeting a predetermined positional relationship.

Abstract: Human posture determination is disclosed. Human posture is determined by obtaining range image(s) through a range camera, detecting key points of an estimated skeleton of a human in color data of the range image(s) and calculating positions of the detected key points based on depth data of the range image(s), choosing a feature map from a set of predefined feature maps based on the detected key points among a set of predefined key points, obtaining two features of a body of the human corresponding to the chosen feature map based on the positions of the detected key points, and determining a posture of the human according to the two features in the chosen feature map.

Abstract: Embodiments of the disclosure provide methods and systems for continuous regulation of a nonholonomic mobile robot. An exemplary method may include identifying a current pose of the nonholonomic mobile robot in a world frame, where the current pose is represented by a first set of values defining a first set of states of the nonholonomic mobile robot in the world frame; receiving a final goal pose of the nonholonomic mobile robot, where the final goal pose is represented by a second set of values defining a second set of states of nonholonomic mobile robot in the world frame; determining a moving path for moving the nonholonomic mobile robot from the current pose to the final goal pose; and controlling the nonholonomic mobile robot to move from the current pose to the final goal pose according to the moving path, where the nonholonomic mobile robot moves to the final goal pose by converging the nonholonomic mobile robot from the first set of states to the second set of states simultaneously.

Abstract: Carpet detection using an RGB-D camera and mobile machine movement control based thereon are disclosed. Carpets and carpet curls are detected by obtaining a RGB-D image pair including an RGB image and a depth image through an RGB-D camera, detecting carpet and carpet-curl areas in the RGB image and generating a 2D bounding box to mark each area using a deep learning model, and generating groups of carpet and carpet-curt points corresponding to each of the carpet and carpet-curl areas by matching each pixel of the RGB image within each 2D bounding box corresponding to the carpet and carpet curl areas to each pixel in the depth image.

Abstract: Navigation of a mobility aid robot having a camera and gripping part(s) disposed toward different directions is disclosed. The mobility aid robot is navigated to approach a user by identifying a posture of the user through the camera, determining a mode of the robot according to a type of the specified task to be performed on the user and the identified posture of the user, controlling the robot to move according to a planned trajectory corresponding to the determined mode of the robot, and turning the robot upon reaching the desired pose such that the gripping part faces the user, in response to the determined mode of the robot corresponding to the specified task of an assisting type and the user at one of a standing posture and a sitting posture.

Abstract: A design method of serial manipulator that comprises an end effector, a number of random-access links, and a number of motors, includes: obtaining a desired motion profile of the end effector; discretizing the desired motion profile into a plurality of points, wherein each of the points carries information of speed, acceleration, and force/torque of the end effector at the point; determining the number of degrees of freedom of the serial manipulator, and initializing the length of each of the links and the motor type of each of the motors; and at each of the points, optimizing the initialized lengths of the links and the motor types of the motors by calculating a dynamic manipulability ellipsoid at the end effector, to obtain desired lengths of the links and desired motor types of the motors, which allows the end effector to execute the desired motion profile under predetermined constraints.

Abstract: A moving target following method, which is executed by one or more processors of a robot that includes a camera and a sensor electrically coupled to the one or more processors, includes: performing a body detection to a body of a target based on images acquired by the camera to obtain a body detection result; performing a leg detection to legs of the target based on data acquired by die sensor to obtain a leg detection result; and fusing the body detection result and the leg detection result to obtain a fusion result, and controlling the robot to follow the target based on the fusion result.

Abstract: On-floor obstacle detection using an RGB-D camera is disclosed. An obstacle on a floor is detected by receiving an image including depth channel data and RGB channel data through the RGB-D camera, estimating a ground plane corresponding to the floor based on the depth channel data, obtaining a foreground of the image corresponding to the ground plane based on the depth channel data, performing a distribution modeling on the foreground of the image based on the RGB channel data to obtain a 2D location of the obstacle, and transforming the 2D location of the obstacle into a 3D location of the obstacle based on the depth channel data.

Abstract: Image-based trajectory planning for a mobile machine having a camera is disclosed. A trajectory for the mobile machine to move to a referenced target is planned by obtaining, a desired image of the referenced target captured at a desired pose of the trajectory and an initial image of the referenced target captured at an initial pose of the trajectory through the camera of the mobile machine, calculating a homography of the desired image and the initial image; decomposing the homography into an initial translation component and an initial rotation matrix, obtaining optimized translation component(s) corresponding to constraint(s) for the mobile machine based on the initial translation component, obtaining optimized homography(s) based on the optimized translation component(s) and the initial rotation matrix, and obtaining objective image features corresponding to the trajectory based on the optimized homography(s).

Abstract: Embodiments of the disclosure provide methods and systems for implementing a function for an application using a middleware on a computer. An exemplary method may include initializing an application node in the middleware corresponding to the application, the initializing comprising binding the function to the application node and associating the application node with a cryptor, connecting the middleware to a remote device through a transport layer using the cryptor, publishing a message to invoke the function from the application node, through the middleware, to the cryptor, and communicating with the remote device to invoke the function, using the cryptor, based on the message.

Abstract: A mobile robot control method includes: acquiring a first image that is captured by a camera on a robot when the robot is in a desired pose; acquiring a second image that is captured by the camera on the robot when the robot is in a current pose; extracting multiple pairs of matching feature points from the first image and the second image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points, wherein a center of the virtual unitary sphere is coincident with an optical center of coordinates of the camera; acquiring an invariant image feature and a rotation vector feature based on the multiple projection feature points, and controlling the robot to move until the robot is in the desired pose according to the invariant image feature and the rotation vector feature.