Abstract: A method for facilitating a multiparty conversation is disclosed. An electronic device using the method may facilitate a multiparty conversation by identifying participants of a conversation, localizing relative positions of the participants, detecting speeches of the conversation, matching one of the participants to each of the detected speeches according to the relative positions of the participants, counting participations of the matched participant in the conversation, identifying a passive subject from all the participants according to the participations of all the participants in the conversation, finding a topic of the conversation between the participants, and engaging the passive subject by addressing the passive subject and speaking a sentence related to the topic.

Abstract: A human-object scene recognition method includes: acquiring an input RGB image and a depth image corresponding to the RGB image; detecting objects and humans in the RGB image using a segmentation classification algorithm based on a sample database; in response to detection of objects and/or humans, performing a segment detection to each of the detected objects and/or humans based on the ROB image and the depth image, and acquiring a result of the segment detection; calculating 3D hounding boxes for each of the detected objects and/or humans according to the result of the segment detection, and determining a position of each of the detected objects and/or humans according to the 3D bounding boxes.

Abstract: Response methods to human abnormal behaviors for a mobility aid robot having a user-facing camera are disclosed. The mobility aid robot responds to human abnormal behaviors by detecting a face of a human during the robot aiding the human to move through the camera, comparing an initial size of the face and an immediate size of the face in response to the face of the human having detected during the robot aiding the human to move, determining the human as in abnormal behavior(s) in response to the immediate size of the face being smaller than the initial size of the face, and performing response(s) corresponding to the abnormal behavior(s) in response to the human being in the abnormal behavior(s), where the response(s) include slowing down the robot.

Abstract: A robotic assistant includes a base; an elevation mechanism positioned on the base; a display rotatably mounted on the elevation mechanism; a camera positioned on the display; and a control system that receives command instructions. In response to the command instructions, the control system is to detect movement of a face of the user in a vertical direction based on the images captured by the camera. In response to detection of the movement of the face of the user in the vertical direction, the control system is to rotate the display and actuate the elevation mechanism to the move the display up and down to allow the camera to face the face of the user during the movement of the face of the user in the vertical direction.

Abstract: Human posture determination is disclosed. Human posture is determined by obtaining range image(s) through a range camera, detecting key points of an estimated skeleton of a human in color data of the range image(s) and calculating positions of the detected key points based on depth data of the range image(s), choosing a feature map from a set of predefined feature maps based on the detected key points among a set of predefined key points, obtaining two features of a body of the human corresponding to the chosen feature map based on the positions of the detected key points, and determining a posture of the human according to the two features in the chosen feature map.

Abstract: Human lying posture detections are disclosed. A human lying on a bed is detected by obtaining an image through a depth camera, detecting objects in the image and marking the objects in the image using 2D bounding boxes by deep learning, determining the human being in a lying posture in response to a width and a height of the 2D bounding box of the human meeting a predetermined condition, detecting one or more skin areas in the image and generating skin area 2D bounding boxes to mark each of the one or more skin areas using a skin detection algorithm, and determining the human being in the lying posture in response to the skin area 2D bounding boxes and the 2D bounding box of the bed meeting a predetermined positional relationship.

Abstract: A method for controlling a robotic walking assistant that includes a wheeled base having one or more wheels, two handles and a foldable seat that are coupled to the wheeled base, includes: detecting whether two hands of a user have held the two handles of the robotic walking assistant; receiving a command from the user to select an operation mode in response to detection of the two hands holding the two handles; controlling the wheeled base to move in response to a walking assistive mode being selected; providing resistance to at least one of the one or more wheels according to selection of the user, in response to a walking training mode being selected; and locking the one or more wheels in response to a static training mode being selected.

Abstract: A robotic walking assistant includes a wheeled base having a base and one or more position adjustable wheels connected to the base, a body disposed in a vertical direction, positioned on the wheeled base and having a handle, and a control system that receives command instructions. Each of the one or more wheels is slidable with respect to the base between a retracted position and an extended position in a direction that is substantially parallel to a surface where the wheeled base moves. In response to the command instructions, the control system moves the one or more wheels between the retracted positions and the extended positions.

Abstract: Embodiments of the disclosure provide methods and systems for continuous regulation of a nonholonomic mobile robot. An exemplary method may include identifying a current pose of the nonholonomic mobile robot in a world frame, where the current pose is represented by a first set of values defining a first set of states of the nonholonomic mobile robot in the world frame; receiving a final goal pose of the nonholonomic mobile robot, where the final goal pose is represented by a second set of values defining a second set of states of nonholonomic mobile robot in the world frame; determining a moving path for moving the nonholonomic mobile robot from the current pose to the final goal pose; and controlling the nonholonomic mobile robot to move from the current pose to the final goal pose according to the moving path, where the nonholonomic mobile robot moves to the final goal pose by converging the nonholonomic mobile robot from the first set of states to the second set of states simultaneously.

Abstract: A robotic assistant includes a wheeled base, a storage unit including drawers, a foldable arm connected to a top of the storage unit and including an end of arm tooling (EOAT) connected to a distal end of the foldable arm, an elevation mechanism positioned on the wheeled base and used to move the storage unit up and down, and a control system that receives command instructions. In response to the command instructions, the control system is configured to move the wheeled base, open or close the one or more drawers, actuate movement of the foldable arm and the EOAT to pick up and place external objects from/to a determined location, and control the storage unit to move up/down.

Abstract: Navigation of a mobility aid robot having a camera and gripping part(s) disposed toward different directions is disclosed. The mobility aid robot is navigated to approach a user by identifying a posture of the user through the camera, determining a mode of the robot according to a type of the specified task to be performed on the user and the identified posture of the user, controlling the robot to move according to a planned trajectory corresponding to the determined mode of the robot, and turning the robot upon reaching the desired pose such that the gripping part faces the user, in response to the determined mode of the robot corresponding to the specified task of an assisting type and the user at one of a standing posture and a sitting posture.

Abstract: Carpet detection using an RGB-D camera and mobile machine movement control based thereon are disclosed. Carpets and carpet curls are detected by obtaining a RGB-D image pair including an RGB image and a depth image through an RGB-D camera, detecting carpet and carpet-curl areas in the RGB image and generating a 2D bounding box to mark each area using a deep learning model, and generating groups of carpet and carpet-curt points corresponding to each of the carpet and carpet-curl areas by matching each pixel of the RGB image within each 2D bounding box corresponding to the carpet and carpet curl areas to each pixel in the depth image.

Abstract: A wheeled base includes a housing, two driven wheeled mechanisms positioned on a bottom of the housing and on opposite sides of the housing, at least one passive wheel positioned on the bottom of the housing, actuated feet positioned on the bottom of the housing and configured to move up and down, sensors, and a battery pack arranged within the housing. The two driven wheeled mechanisms each includes a damping mechanism, and each damping mechanism includes at least two dampers configured to absorb impact caused by an upward movement of the housing, and absorb impact caused by a downward movement of the housing.

Abstract: Embodiments of the disclosure provide systems and methods for speech detection. The method may include receiving a multichannel audio input that includes a set of audio signals from a set of audio channels in an audio detection array. The method may further include processing the multichannel audio input using a neural network classifier to generate a series of classification results in a series of time windows for the multichannel audio input. The neural network classifier includes a causal temporal convolutional network (TCN) configured to determine a classification result for each time window based on portions of the multichannel audio input in the corresponding time window and one or more time windows before the corresponding time window. The method may additionally include determining whether the multichannel audio input includes one or more speech segments in the series of time windows based on the series of classification results.

Abstract: A design method of serial manipulator that comprises an end effector, a number of random-access links, and a number of motors, includes: obtaining a desired motion profile of the end effector; discretizing the desired motion profile into a plurality of points, wherein each of the points carries information of speed, acceleration, and force/torque of the end effector at the point; determining the number of degrees of freedom of the serial manipulator, and initializing the length of each of the links and the motor type of each of the motors; and at each of the points, optimizing the initialized lengths of the links and the motor types of the motors by calculating a dynamic manipulability ellipsoid at the end effector, to obtain desired lengths of the links and desired motor types of the motors, which allows the end effector to execute the desired motion profile under predetermined constraints.

Abstract: A moving target following method, which is executed by one or more processors of a robot that includes a camera and a sensor electrically coupled to the one or more processors, includes: performing a body detection to a body of a target based on images acquired by the camera to obtain a body detection result; performing a leg detection to legs of the target based on data acquired by die sensor to obtain a leg detection result; and fusing the body detection result and the leg detection result to obtain a fusion result, and controlling the robot to follow the target based on the fusion result.

Abstract: In accordance with certain implementations of the present approach, a reduced, element-by-element, data set is transmitted between a robot having a sensor suite and a control system remote from the robot that is configured to display a representation of the environment local to the robot. Such a scheme may be useful in allowing a human operator remote from the robot to perform an inspection using the robot while the robot is on-site with an asset and the operator is off-site. In accordance with the present approach, an accurate representation of the environment in which the robot is situated is provided for the operator to interact with.

Abstract: Image-based trajectory planning for a mobile machine having a camera is disclosed. A trajectory for the mobile machine to move to a referenced target is planned by obtaining, a desired image of the referenced target captured at a desired pose of the trajectory and an initial image of the referenced target captured at an initial pose of the trajectory through the camera of the mobile machine, calculating a homography of the desired image and the initial image; decomposing the homography into an initial translation component and an initial rotation matrix, obtaining optimized translation component(s) corresponding to constraint(s) for the mobile machine based on the initial translation component, obtaining optimized homography(s) based on the optimized translation component(s) and the initial rotation matrix, and obtaining objective image features corresponding to the trajectory based on the optimized homography(s).

Abstract: On-floor obstacle detection using an RGB-D camera is disclosed. An obstacle on a floor is detected by receiving an image including depth channel data and RGB channel data through the RGB-D camera, estimating a ground plane corresponding to the floor based on the depth channel data, obtaining a foreground of the image corresponding to the ground plane based on the depth channel data, performing a distribution modeling on the foreground of the image based on the RGB channel data to obtain a 2D location of the obstacle, and transforming the 2D location of the obstacle into a 3D location of the obstacle based on the depth channel data.

Abstract: Embodiments of the disclosure provide systems and methods for speech detection. The method may include receiving a multichannel audio input that includes a set of audio signals from a set of audio channels in an audio detection array. The method may further include processing the multichannel audio input using a neural network classifier to generate a series of classification results in a series of time windows for the multichannel audio input. The neural network classifier includes a causal temporal convolutional network (TCN) configured to determine a classification result for each time window based on portions of the multichannel audio input n the corresponding time window and one or more time windows before the corresponding time window. The method may additionally include determining whether the multichannel audio input includes one or more speech segments in the series of time windows based on the series of classification results.