Abstract: Methods and systems of building a knowledge graph based on event-based ontology of a scene and vehicle trajectory in the scene. Image data corresponding to a plurality of scenes captured by one or more cameras is received. Event-based ontology data corresponding to events occurring in the plurality of scenes is received. Via an object-tracking machine-learning model, the system determines (i) a presence of a plurality of vehicles in the image data, and (ii) a plurality of vehicle trajectories, each vehicle trajectory associated with a respective one of the vehicles. Using a clustering model, the vehicle trajectories are clustered. A knowledge graph is augmented based on the clustered vehicle trajectories and the event-based ontology.

Abstract: Methods and system of building and augmenting a knowledge graph regarding ontology of events occurring in images. Image data corresponding to a plurality of scenes captured by one or more cameras is received. A knowledge graph is built with event-based ontology data corresponding to events occurring in the plurality of scenes. One or more of the scenes is displayed to a plurality of crowdsourcing workers which provide natural-language input including event-based semantic annotations corresponding to the scene. Using natural language processing on the input, triples are generated. The knowledge graph is augmented with the generated triples to yield an augmented knowledge graph for use in determining event-based ontology associated with the plurality of scenes.

Abstract: A method includes receiving input dialog including a text string corresponding to at least one question and extracting at least one keyword from the text string. The method also includes generating at least one action prediction and providing one or more sub-questions associated with the at least one question. The method also includes receiving one or more answers to the one or more sub-questions, generating at least one sub-goal based on the one or more answers, and traversing an environment based on the at least one sub-goal. The method also includes receiving one or more images associated with the environment, predicting, using the one or more images, an answer to the at least one question, and providing, at an output mechanism, the answer to the at least one question.

Abstract: A method includes receiving a first set of characteristics associated with an engine emissions calibration project and identifying, in a knowledge graph corresponding to engine emissions calibration, a second set of characteristics that corresponds to the first set of characteristics.

Abstract: A human-assisted neuro-symbolic system for outputting fine-grained classifications and corresponding images or video of a desired object or scene. The system includes one or more cameras configured to generate a video feed of a scene. One or more processors are programmed to generate video analytics data from the video feed, including coarse-grained classification data regarding one or more objects in the scene. A knowledge graph is built with instantiated (e.g., time-based) domain ontology of the one or more objects in the scene. The domain ontology can be augmented via human-in-the-loop. Once augmented, the knowledge graph can be infused into a deep learning model, such as a natural language model. An input (e.g., in natural language) can seek fine-grained input characteristics, and the deep learning model infused with the knowledge graph retrieves a corresponding portion of the video feed with the fine-grained input characteristics.

Abstract: A system for image processing includes a first sensor configured to capture at least one or more images, a second sensor configured to capture sound information, a processor in communication with the first sensor and second sensor, wherein the processor is programmed to receive the one or more images and sound information, extract one or more data features associated with the images and sound information utilizing an encoder, output metadata via a decoder to a spatiotemporal reasoning engine, wherein the metadata is derived utilizing the decoder and the one or more data features, determine one or more scenes utilizing the spatiotemporal reasoning engine and the metadata, and output a control command in response to the one or more scenes.

Abstract: A computer-implemented system and method relates to natural language processing. The computer-implemented system and method are configured to obtain a current data structure from a global knowledge graph, which comprises various knowledge graphs. The current data structure includes a current head element, a current relationship element, and a current tail element. A sentence is obtained based on the current data structure. A question is generated by removing the current tail element from the sentence. A correct answer is generated for the question. The correct answer includes the current tail element. A pool of data structures is extracted from the global knowledge graph based on a set of distractor criteria. The set of distractor criteria ensures that each extracted data structure includes the current relationship element. Tail elements from the pool of data structures are extracted to create a pool of distractor candidates. A set of distractors are selected from the pool of distractor candidates.

Abstract: A dialogue computer and method of using the dialogue computer is disclosed. The method may comprise: receiving a query from a user; providing the query to an input layer of a neural network; injecting one or more triples of a knowledge graph into a plurality of nodes of an output layer of the neural network; and determining an answer to the query based on the output layer.

Abstract: A decision-support system for a public transportation system includes a computing system programmed to generate public transit route options for the commuter based on at least one decision factor that estimates an expected impact on other commuters resulting from the commuter choosing each of the public transit route options, and output, for display, values associated with the at least one decision factor to influence the commuter in making a route selection.

Abstract: A crowdsourced dialogue system includes a first and second computer system, a training system, and an automated response system. The first computer system includes a caller interface operated by a non-expert user and to generate caller dialogue data based on inputs to the first computer system from the non-expert user. The second computer system includes an expert interface operated by an expert user and to generate expert dialogue data based on inputs to the second computer system from the expert user. The training system includes a general domain dialogue database. The training system is configured to generate a domain-specific dialogue database based on the caller dialogue data and the expert dialogue data. The automated response system includes a chatbot that is trained with the general domain dialogue database and the domain-specific dialogue database to generate natural language dialogue data at least in a domain of the domain-specific dialogue database.

Abstract: A system and method is disclosed for adjusting control of an autonomous vehicle based on crowd-source data. The autonomous vehicle may be designed to receive crowd-source data relating to a driving condition located along a travel route the autonomous vehicle is travelling. The control of the autonomous vehicle may then be adjusted in response to the crowd-source data provided. The autonomous vehicle may also request crowd-source data related to how the autonomous vehicle should proceed along a travel route. Based on the request, the autonomous vehicle may receive crowd-source data instructing the autonomous vehicle how to proceed along the travel route. The autonomous vehicle may also adjust how the autonomous vehicle proceeds along the travel route in response to the crowd-source data.

Abstract: A decision-support system for a public transportation system includes a computing system programmed to generate public transit route options for the commuter based on at least one decision factor that estimates an expected impact on other commuters resulting from the commuter choosing each of the public transit route options, and output, for display, values associated with the at least one decision factor to influence the commuter in making a route selection.

Abstract: A crowdsourced dialogue system includes a first and second computer system, a training system, and an automated response system. The first computer system includes a caller interface operated by a non-expert user and to generate caller dialogue data based on inputs to the first computer system from the non-expert user. The second computer system includes an expert interface operated by an expert user and to generate expert dialogue data based on inputs to the second computer system from the expert user. The training system includes a general domain dialogue database. The training system is configured to generate a domain-specific dialogue database based on the caller dialogue data and the expert dialogue data. The automated response system includes a chatbot that is trained with the general domain dialogue database and the domain-specific dialogue database to generate natural language dialogue data at least in a domain of the domain-specific dialogue database.