Abstract: Conventional robotic systems often fail to provide effective escorting services as they lack awareness of human motion dynamics. Present disclosure provides method and system for providing robot-based escorting service. The system tracks a user utilizing the robot-based escort service using a human re-identification technique and a human movement tracking technique. The human re-identification technique ensures that the same user is identified every time in crowded spaces and human movement tracking technique predicts a user state at intervals indicating whether user is following, lagging, or stopping based on re-identification performed by the human re-identification technique. Thereafter, the system adjust a speed of the robot in case it is determined that the user is either lagging or stopping, thereby enabling the robot to adapt its speed according to user's movements which further helps in providing seamless experience to user.

Abstract: State of art techniques relate to telepresence robots expressing internal states using emotional expressions and corresponding text messages are based on the perceived input from the co-located user or the participant, and not related to the teleoperator. Embodiments of the present disclosure provide a method and system for expressing telepresence robot internal states using combination of multiple modalities. The telepresence robot expresses its own internal states in the form of a plurality of emotional expressions and corresponding text messages to the teleoperator, using a robot internal state predictor model. Unlike the state of art techniques which express an emotion as its internal state, the disclosed method expresses the internal state as the emotional expression with respect to a task processing of the telepresence robot.

Abstract: This disclosure relates generally to system and methods for dynamic scheduling and rescheduling using heterogeneous multi-agent fleet. The embodiments of present disclosure herein address unresolved problem of task allocation using a single solution which is not sufficient for handling various scenarios of multi-agent task allocation problems. For instance, the task allocation becomes more challenging in a scenario where each task has a deadline associated with it and execution time of the tasks cannot be pre-computed as they are dependent on previous tasks. The method of present disclosure provides a scalable solution for dynamic scheduling and rescheduling that handles tasks with multiple pickup and drop locations and dynamic execution time using the agents with heterogeneous speed in a more efficient manner, reducing indirect operating costs and increasing revenue potential while minimizing additional penalty due to run time delays that an agent may encounter.

Abstract: The disclosure generally relates to methods and systems for complex natural language task understanding for embodied robots or agents. Conventional works on relation extraction generally find relevant triplets in a natural language phrase, but neither ground the task nor ground the arguments. The present disclosure implements a Grounded Argument and Task Extraction (GATE) technique that extracts a set of tasks and relevant arguments from the complex natural language instruction. The GATE uses an encoder-decoder neural network with nested decoding technique. The extracted tasks are mapped (grounded) to the known skill set of the robot and arguments are mapped (grounded) to objects within the environment, classifies the tokens as many times as possible which existing sequence labeling cannot do.

Abstract: Accuracy of transcript is of foremost importance in Automatic Speech Recognition (ASR). State of the art system mostly rely on spelling correction based contextual improvement in ASR, which is generally a static vocabulary based biasing approach. Embodiments of the present disclosure provide a method and system for visual context aware ASR. The method provides biasing using shallow fusion biasing approach with a modified beam search decoding technique, which introduces a non-greedy pruning strategy to allow biasing at the sub-word level. The biasing algorithm brings in the visual context of the robot to the speech recognizer based on a dynamic biasing vocabulary, improving the transcription accuracy. The dynamic biasing vocabulary, comprising objects in a current environment accompanied by their self and relational attributes, is generated using a bias prediction network that explicitly adds label to objects, which are detected and captioned via a state of the art dense image captioning network.

Abstract: This disclosure addresses the unresolved problems of tackling object disambiguation task for an embodied agent. The embodiments of present disclosure provide a method and system for disambiguation of referred objects for embodied agents. With a phrase-to-graph network disclosed in the system of the present disclosure, any natural language object description indicating the object disambiguation task can be converted into a semantic graph representation. This not only provides a formal representation of the referred object and object instances but also helps to find an ambiguity in disambiguating the referred object using a real-time multi-view aggregation algorithm. The real-time multi-view aggregation algorithm processes multiple observations from an environment and finds the unique instances of the referred object.

Abstract: The embodiments of present disclosure herein address unresolved problem of cognitive navigation strategies for a telepresence robotic system. This includes giving instruction remotely over network to go to a point in an indoor space, to go an area, to go to an object. Also, human robot interaction to give and understand interaction is not integrated in a common telepresence framework. The embodiments herein provide a telepresence robotic system empowered with a smart navigation which is based on in situ intelligent visual semantic mapping of the live scene captured by a robot. It further presents an edge-centric software architecture of a teledrive comprising a speech recognition based HRI, a navigation module and a real-time WebRTC based communication framework that holds the entire telepresence robotic system together. Additionally, the disclosure provides a robot independent API calls via device driver ROS, making the offering hardware independent and capable of running in any robot.

Abstract: Existing cognitive robotic applications follow a practice of building specific applications for specific use cases. However, the knowledge of the world and the semantics are common for a robot for multiple tasks. In this disclosure, to enable usage of knowledge across multiple scenarios, a method and system for ontology guided indoor scene understanding for cognitive robotic tasks is described where in scenes are processed based on techniques filtered based on querying ontology with relevant objects in perceived scene to generate a semantically rich scene graph. Herein, an initially manually created ontology is updated and refined in online fashion using external knowledge-base, human robot interaction and perceived information. This knowledge helps in semantic navigation, aids in speech, and text based human robot interactions.

Abstract: The disclosure generally relates to methods and systems for complex natural language task understanding for embodied robots or agents. Conventional works on relation extraction generally find relevant triplets in a natural language phrase, but neither ground the task nor ground the arguments. The present disclosure implements a Grounded Argument and Task Extraction (GATE) technique that extracts a set of tasks and relevant arguments from the complex natural language instruction. The GATE uses an encoder-decoder neural network with nested decoding technique. The extracted tasks are mapped (grounded) to the known skill set of the robot and arguments are mapped (grounded) to objects within the environment, classifies the tokens as many times as possible which existing sequence labeling cannot do.

Abstract: Any technical error with robotic arms that are used to automatically perform object packing can affect quality and efficiency with which the packing is being carried out, and this in turn affects space utilization when a large quantity of objects are to be accommodated in tight packing spaces. This disclosure relates generally to automated object packing and more specifically to an object packing mechanism in which corrections are made when placement of object is identified as violating one or more regulations. The system packs objects by calculating ICP-BCP pairs for each empty space in a packing space. After packing each object, the system checks whether placement of the object violates one or more regulations, and if any violation is found, then the system determines and executes one or more corrective action to correct placement of the object that violates the regulation.

Abstract: Accuracy of transcript is of foremost importance in Automatic Speech Recognition (ASR). State of the art system mostly rely on spelling correction based contextual improvement in ASR, which is generally a static vocabulary based biasing approach. Embodiments of the present disclosure provide a method and system for visual context aware ASR. The method provides biasing using shallow fusion biasing approach with a modified beam search decoding technique, which introduces a non-greedy pruning strategy to allow biasing at the sub-word level. The biasing algorithm brings in the visual context of the robot to the speech recognizer based on a dynamic biasing vocabulary, improving the transcription accuracy. The dynamic biasing vocabulary, comprising objects in a current environment accompanied by their self and relational attributes, is generated using a bias prediction network that explicitly adds label to objects, which are detected and captioned via a state of the art dense image captioning network.

Abstract: This disclosure addresses the unresolved problems of tackling object disambiguation task for an embodied agent. The embodiments of present disclosure provide a method and system for disambiguation of referred objects for embodied agents. With a phrase-to-graph network disclosed in the system of the present disclosure, any natural language object description indicating the object disambiguation task can be converted into a semantic graph representation. This not only provides a formal representation of the referred object and object instances but also helps to find an ambiguity in disambiguating the referred object using a real-time multi-view aggregation algorithm. The real-time multi-view aggregation algorithm processes multiple observations from an environment and finds the unique instances of the referred object.

Abstract: The embodiments of present disclosure herein address unresolved problem of cognitive navigation strategies for a telepresence robotic system. This includes giving instruction remotely over network to go to a point in an indoor space, to go an area, to go to an object. Also, human robot interaction to give and understand interaction is not integrated in a common telepresence framework. The embodiments herein provide a telepresence robotic system empowered with a smart navigation which is based on in situ intelligent visual semantic mapping of the live scene captured by a robot. It further presents an edge-centric software architecture of a teledrive comprising a speech recognition based HRI, a navigation module and a real-time WebRTC based communication framework that holds the entire telepresence robotic system together. Additionally, the disclosure provides a robot independent API calls via device driver ROS, making the offering hardware independent and capable of running in any robot.

Abstract: Online 3-dimensional bin packing problem (O3D-BPP) is getting renewed prominence due to the industrial automation brought by Industry 4.0. However, due to limited attention in the past and its challenging nature, a good approximate technique is in scarcity as compared to 1D or 2D problems. Present disclosure provides system and method that considers real-time O3D-BPP of cuboidal boxes with partial information (look-ahead) in an automated robotic sorting center. System presents two rolling-horizon mixed-integer linear programming (MILP) cum-heuristic based algorithms wherein a framework is provided that adapts and improves performance of BP heuristics by utilizing information in an online setting with look-ahead.

Abstract: This disclosure relates generally to system and methods for dynamic scheduling and rescheduling using heterogeneous multi-agent fleet. The embodiments of present disclosure herein address unresolved problem of task allocation using a single solution which is not sufficient for handling various scenarios of multi-agent task allocation problems. For instance, the task allocation becomes more challenging in a scenario where each task has a deadline associated with it and execution time of the tasks cannot be pre-computed as they are dependent on previous tasks. The method of present disclosure provides a scalable solution for dynamic scheduling and rescheduling that handles tasks with multiple pickup and drop locations and dynamic execution time using the agents with heterogeneous speed in a more efficient manner, reducing indirect operating costs and increasing revenue potential while minimizing additional penalty due to run time delays that an agent may encounter.

Abstract: Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.

Abstract: Robotic platform for tele-presence applications has gained paramount importance, such as for remote meetings, group discussions, and the like and has sought much attention. There exist some robotic platforms for such tele-presence applications, these lack efficacy in communication and interaction between remote person and avatar robot deployed in another geographic location thus adding network overhead. Embodiments of the present disclosure for edge centric communication protocol for remotely maneuvering tele-presence robot in geographically distributed environment.

Abstract: The disclosure herein relates to methods and systems for enabling human-robot interaction (HRI) to resolve task ambiguity. Conventional techniques that initiates continuous dialogue with the human to ask a suitable question based on the observed scene until resolving the ambiguity are limited. The present disclosure use the concept of Talk-to-Resolve (TTR) which initiates a continuous dialogue with the user based on visual uncertainty analysis and by asking a suitable question that convey the veracity of the problem to the user and seek guidance until all the ambiguities are resolved. The suitable question is formulated based on the scene understanding and the argument spans present in the natural language instruction. The present disclosure asks questions in a natural way that not only ensures that the user can understand the type of confusion, the robot is facing; but also ensures minimal and relevant questioning to resolve the ambiguities.

Abstract: This disclosure provides systems and methods for robotic task planning when a complex task instruction is provided in natural language. Conventionally robotic task planning relies on a single task or multiple independent or serialized tasks in the task instruction. Alternatively, constraints on space of linguistic variations, ambiguity and complexity of the language may be imposed. In the present disclosure, firstly dependencies between multiple tasks are identified. The tasks are then ordered such that a dependent task is always scheduled for planning after a task it is dependent upon. Moreover, repeated tasks are masked. Thus, resolving task dependencies and ordering dependencies, a complex instruction with multiple interdependent tasks in natural language facilitates generation of a viable task execution plan. Systems and methods of the present disclosure finds application in human-robot interactions.

Abstract: The systems and methods provide for optimizing the scheduling of non-preemptive tasks in the multi-robot environment. The embodiment provides for scheduling, by implementing an Online Minimum Performance Loss Scheduling (OMPLS) technique, initially, tasks with a higher performance loss value and then secondly, tasks that can be scheduled within their deadline and having a low performance loss value amongst the merged tasks; and finally minimizing, a performance loss value of a remaining subset of tasks that cannot be scheduled within a pre-defined deadline. Moreover, the embodiments herein further provides executing the priority update on each of the remaining subset of tasks for optimizing the scheduling of the non-pre-emptive tasks.