Abstract: Accuracy of transcript is of foremost importance in Automatic Speech Recognition (ASR). State of the art system mostly rely on spelling correction based contextual improvement in ASR, which is generally a static vocabulary based biasing approach. Embodiments of the present disclosure provide a method and system for visual context aware ASR. The method provides biasing using shallow fusion biasing approach with a modified beam search decoding technique, which introduces a non-greedy pruning strategy to allow biasing at the sub-word level. The biasing algorithm brings in the visual context of the robot to the speech recognizer based on a dynamic biasing vocabulary, improving the transcription accuracy. The dynamic biasing vocabulary, comprising objects in a current environment accompanied by their self and relational attributes, is generated using a bias prediction network that explicitly adds label to objects, which are detected and captioned via a state of the art dense image captioning network.

Abstract: This disclosure addresses the unresolved problems of tackling object disambiguation task for an embodied agent. The embodiments of present disclosure provide a method and system for disambiguation of referred objects for embodied agents. With a phrase-to-graph network disclosed in the system of the present disclosure, any natural language object description indicating the object disambiguation task can be converted into a semantic graph representation. This not only provides a formal representation of the referred object and object instances but also helps to find an ambiguity in disambiguating the referred object using a real-time multi-view aggregation algorithm. The real-time multi-view aggregation algorithm processes multiple observations from an environment and finds the unique instances of the referred object.

Abstract: The embodiments of present disclosure herein address unresolved problem of cognitive navigation strategies for a telepresence robotic system. This includes giving instruction remotely over network to go to a point in an indoor space, to go an area, to go to an object. Also, human robot interaction to give and understand interaction is not integrated in a common telepresence framework. The embodiments herein provide a telepresence robotic system empowered with a smart navigation which is based on in situ intelligent visual semantic mapping of the live scene captured by a robot. It further presents an edge-centric software architecture of a teledrive comprising a speech recognition based HRI, a navigation module and a real-time WebRTC based communication framework that holds the entire telepresence robotic system together. Additionally, the disclosure provides a robot independent API calls via device driver ROS, making the offering hardware independent and capable of running in any robot.

Abstract: Online 3-dimensional bin packing problem (O3D-BPP) is getting renewed prominence due to the industrial automation brought by Industry 4.0. However, due to limited attention in the past and its challenging nature, a good approximate technique is in scarcity as compared to 1D or 2D problems. Present disclosure provides system and method that considers real-time O3D-BPP of cuboidal boxes with partial information (look-ahead) in an automated robotic sorting center. System presents two rolling-horizon mixed-integer linear programming (MILP) cum-heuristic based algorithms wherein a framework is provided that adapts and improves performance of BP heuristics by utilizing information in an online setting with look-ahead.

Abstract: This disclosure relates generally to system and methods for dynamic scheduling and rescheduling using heterogeneous multi-agent fleet. The embodiments of present disclosure herein address unresolved problem of task allocation using a single solution which is not sufficient for handling various scenarios of multi-agent task allocation problems. For instance, the task allocation becomes more challenging in a scenario where each task has a deadline associated with it and execution time of the tasks cannot be pre-computed as they are dependent on previous tasks. The method of present disclosure provides a scalable solution for dynamic scheduling and rescheduling that handles tasks with multiple pickup and drop locations and dynamic execution time using the agents with heterogeneous speed in a more efficient manner, reducing indirect operating costs and increasing revenue potential while minimizing additional penalty due to run time delays that an agent may encounter.

Abstract: Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.

Abstract: Robotic platform for tele-presence applications has gained paramount importance, such as for remote meetings, group discussions, and the like and has sought much attention. There exist some robotic platforms for such tele-presence applications, these lack efficacy in communication and interaction between remote person and avatar robot deployed in another geographic location thus adding network overhead. Embodiments of the present disclosure for edge centric communication protocol for remotely maneuvering tele-presence robot in geographically distributed environment.

Abstract: The disclosure herein relates to methods and systems for enabling human-robot interaction (HRI) to resolve task ambiguity. Conventional techniques that initiates continuous dialogue with the human to ask a suitable question based on the observed scene until resolving the ambiguity are limited. The present disclosure use the concept of Talk-to-Resolve (TTR) which initiates a continuous dialogue with the user based on visual uncertainty analysis and by asking a suitable question that convey the veracity of the problem to the user and seek guidance until all the ambiguities are resolved. The suitable question is formulated based on the scene understanding and the argument spans present in the natural language instruction. The present disclosure asks questions in a natural way that not only ensures that the user can understand the type of confusion, the robot is facing; but also ensures minimal and relevant questioning to resolve the ambiguities.

Abstract: This disclosure provides systems and methods for robotic task planning when a complex task instruction is provided in natural language. Conventionally robotic task planning relies on a single task or multiple independent or serialized tasks in the task instruction. Alternatively, constraints on space of linguistic variations, ambiguity and complexity of the language may be imposed. In the present disclosure, firstly dependencies between multiple tasks are identified. The tasks are then ordered such that a dependent task is always scheduled for planning after a task it is dependent upon. Moreover, repeated tasks are masked. Thus, resolving task dependencies and ordering dependencies, a complex instruction with multiple interdependent tasks in natural language facilitates generation of a viable task execution plan. Systems and methods of the present disclosure finds application in human-robot interactions.

Abstract: The systems and methods provide for optimizing the scheduling of non-preemptive tasks in the multi-robot environment. The embodiment provides for scheduling, by implementing an Online Minimum Performance Loss Scheduling (OMPLS) technique, initially, tasks with a higher performance loss value and then secondly, tasks that can be scheduled within their deadline and having a low performance loss value amongst the merged tasks; and finally minimizing, a performance loss value of a remaining subset of tasks that cannot be scheduled within a pre-defined deadline. Moreover, the embodiments herein further provides executing the priority update on each of the remaining subset of tasks for optimizing the scheduling of the non-pre-emptive tasks.

Abstract: This disclosure relates to system and method for enabling a robot to perceive and detect socially interacting groups. Various known systems have limited accuracy due to prevalent rule-driven methods. In case of few data-driven learning methods, they lack datasets with varied conditions of light, occlusion, and backgrounds. The disclosed method and system detect the formation of a social group of people, or, f-formation in real-time in a given scene. The system also detects outliers in the process, i.e., people who are visible but not part of the interacting group. This plays a key role in correct f-formation detection in a real-life crowded environment. Additionally, when a collocated robot plans to join the group it has to detect a pose for itself along with detecting the formation. Thus, the system provides the approach angle for the robot, which can help it to determine the final pose in a socially acceptable manner.

Abstract: The disclosure herein relates to methods and systems for enabling human-robot interaction (HRI) to resolve task ambiguity. Conventional techniques that initiates continuous dialogue with the human to ask a suitable question based on the observed scene until resolving the ambiguity are limited. The present disclosure use the concept of Talk-to-Resolve (TTR) which initiates a continuous dialogue with the user based on visual uncertainty analysis and by asking a suitable question that convey the veracity of the problem to the user and seek guidance until all the ambiguities are resolved. The suitable question is formulated based on the scene understanding and the argument spans present in the natural language instruction. The present disclosure asks questions in a natural way that not only ensures that the user can understand the type of confusion, the robot is facing; but also ensures minimal and relevant questioning to resolve the ambiguities.

Abstract: This disclosure relates generally to human-robot interaction (HRI) to enable a robot to execute tasks that are conveyed in a natural language. The state-of-the-art is unable to capture human intent, implicit assumptions and ambiguities present in the natural language to enable effective robotic task identification. The present disclosure provides accurate task identification using classifiers trained to understand linguistic and semantic variations. A mixed-initiative dialogue is employed to resolve ambiguities and address the dynamic nature of a typical conversation. In accordance with the present disclosure, the dialogues are minimal and directed to the goal to ensure human experience is not degraded. The method of the present disclosure is also implemented in a context sensitive manner to make the task identification effective.

Abstract: This disclosure relates to system and method for enabling a robot to perceive and detect socially interacting groups. Various known systems have limited accuracy due to prevalent rule-driven methods. In case of few data-driven learning methods, they lack datasets with varied conditions of light, occlusion, and backgrounds. The disclosed method and system detect the formation of a social group of people, or, f-formation in real-time in a given scene. The system also detects outliers in the process, i.e., people who are visible but not part of the interacting group. This plays a key role in correct f-formation detection in a real-life crowded environment. Additionally, when a collocated robot plans to join the group it has to detect a pose for itself along with detecting the formation. Thus, the system provides the approach angle for the robot, which can help it to determine the final pose in a socially acceptable manner.

Abstract: Systems and methods for generating control system solutions for robotics environments is provided. The traditional systems and methods provide robotics solutions but specialized to only a particular robotic application, domain, and selected structure.

Abstract: Conventional tele-presence robots have their own limitations with respect to task execution, information processing and management. Embodiments of the present disclosure provide a tele-presence robot (TPR) that communicates with a master device associated with a user via an edge device for task execution wherein control command from the master device is parsed for determining instructions set and task type for execution. Based on this determination, the TPR queries for information across storage devices until a response is obtained enough to execute task. The task upon execution is validated with the master device and user. Knowledge acquired, during querying, task execution and validation of the executed task, is dynamically partitioned by the TPR across storage devices namely, on-board memory of the tele-present robot, an edge device, a cloud and a web interface respectively depending upon the task type, operating environment of the tele-presence robot, and other performance affecting parameters.

Abstract: This disclosure relates to attention shifting of a robot in a group conversation with two or more attendees, wherein at least one of them is a speaker. State of the art has dealt with several aspects of Human-Robot Interaction (HRI) including responding to a source of sound at a time, addressing a fixed viewing area or determining who is the speaker based on eye gaze direction. However, attention shifting to make the conversation human-like is a challenge. The present disclosure uses audio-visual perception for speaker localization. Only qualified direction of arrivals (DOAs) are used for the audio perception. Further the audio perception is complimented by visual perception employing real time face detection and lip movement detection. Use of HRI rules, clustering of the DOAs, dynamic adjustment of rotation of the robot and a dynamically updated knowledge repository enriches the robot with intelligence to shift attention with minimum human intervention.

Abstract: Any technical error with robotic arms that are used to automatically perform object packing can affect quality and efficiency with which the packing is being carried out, and this in turn affects space utilization when a large quantity of objects are to be accommodated in tight packing spaces. This disclosure relates generally to automated object packing and more specifically to an object packing mechanism in which corrections are made when placement of object is identified as violating one or more regulations. The system packs objects by calculating ICP-BCP pairs for each empty space in a packing space. After packing each object, the system checks whether placement of the object violates one or more regulations, and if any violation is found, then the system determines and executes one or more corrective action to correct placement of the object that violates the regulation.

Abstract: This disclosure provides systems and methods for robotic task planning when a complex task instruction is provided in natural language. Conventionally robotic task planning relies on a single task or multiple independent or serialized tasks in the task instruction. Alternatively, constraints on space of linguistic variations, ambiguity and complexity of the language may be imposed. In the present disclosure, firstly dependencies between multiple tasks are identified. The tasks are then ordered such that a dependent task is always scheduled for planning after a task it is dependent upon. Moreover, repeated tasks are masked. Thus, resolving task dependencies and ordering dependencies, a complex instruction with multiple interdependent tasks in natural language facilitates generation of a viable task execution plan. Systems and methods of the present disclosure finds application in human-robot interactions.

Abstract: Robotic platform for tele-presence applications has gained paramount importance, such as for remote meetings, group discussions, and the like and has sought much attention. There exist some robotic platforms for such tele-presence applications, these lack efficacy in communication and interaction between remote person and avatar robot deployed in another geographic location thus adding network overhead. Embodiments of the present disclosure for edge centric communication protocol for remotely maneuvering tele-presence robot in geographically distributed environment.