Abstract: A robot arm control device is proposed. The robot arm control device may include a memory. The device may also include an acquisition unit acquiring an arbitrary target image, a virtual canvas image in which a virtual drawing operation of a robot arm for the target image is reflected, and a virtual nib image of the robot arm whose position is changed as the virtual drawing operation is reflected in the virtual canvas image. The device may further include a processor configured to input the target image to the pre-trained learning model, determine a position change amount of the virtual nib image in the virtual canvas image using the pre-trained learning model, and output a joint angle change amount for driving the robot arm on the basis of the position change amount.

Abstract: Disclosed herein are a reasoning method based on a structural attention mechanism for knowledge-based question answering and a computing apparatus for performing the reasoning method. The reasoning method includes: recognizing one or more entities in a query including content and a question, and linking the recognized entities to a knowledge base; constructing a question hypergraph and a query-aware knowledge hypergraph by performing a multi-hop graph walk on a question graph and the knowledge base; and inferring a correct answer to the question by applying as attention mechanism to a query hyperedge and a knowledge hyperedge included in the question hypergraph and the query-aware knowledge hypergraph, respectively.

Abstract: A method of constructing a transformer model for answering questions about a video story according to an embodiment includes: extracting feature vectors related to each character of a video from video data including vision data and subtitle data and question data for video questions and answers, and generating an input embedding using the feature vectors related to the character; and training a transformer model using the input embedding.

Abstract: Disclosed herein are a computing apparatus and method for performing reinforcement learning using a multimodal artificial intelligence agent. The method for performing reinforcement learning using a multimodal artificial intelligence agent includes: dividing frames, included in images acquired by capturing a virtual environment, into a plurality of sections; and performing reinforcement learning by applying any one of a plurality of guidance types to each of the plurality of sections and then allowing a multimodal artificial intelligence agent to interact with the virtual environment through the images. The plurality of guidance types is classified into three or more types according to their guidance level. Performing the reinforcement learning is performing reinforcement learning by applying a moderate-level guidance type to the sections of predetermined critical periods and also applying any one of the plurality of guidance types to the other sections.

Abstract: A map merging method for an electronic apparatus which includes: obtaining information about a first local map of a first apparatus, a pose of the first apparatus in the first local map, a second local map of a second apparatus, a pose of the second apparatus in the second local map, and an image of the second apparatus obtained by the first apparatus; identifying a relative pose of the second apparatus relative to the first apparatus from the image using a first trained artificial neural network; transforming the second local map to correspond to the first local map based on the relative pose, the pose of the first apparatus, and the pose of the second apparatus; and merging the first local map and a transformed second local map transformed in the transforming the second local map to output a merged map is provided.

Abstract: Disclosed herein are a visual story generation method and apparatus for generating a story from a plurality of images by using a deep learning network. The visual story generation method includes: extracting features from a plurality of respective images by using the first extraction unit of a deep learning network; generating the structure of a story based on the overall feature of the plurality of images by using the second extraction unit of the deep learning network; and generating the story by using outputs of the first and second extraction units.

Abstract: A target-oriented reinforcement learning method according to an embodiment includes: collecting data related to the target of reinforcement learning as target data in the process of performing the reinforcement learning; learning the collected target data as auxiliary learning for the reinforcement learning; and incorporating the results of the learning of the target data into the performance of the reinforcement learning.

Abstract: A question answering method that is performed by a question answering apparatus includes: receiving a data set including video content and question-answer pairs; generating input time-series sequences from the video content of the input data set and also generating a question-answer time-series sequence from the question-answer pair of the input data set; calculating weights by associating the input time-series sequence with the question-answer time-series sequence and also calculating first result values by performing operations on the calculated weights and the input time-series sequences; calculating second result values by paying attention to portions of the input time-series sequences that are directly related to characters appearing in questions and answers; and calculating third result values by concatenating the time-series sequences, the first result values, the second result values, and Boolean flags and selecting a final answer based on the third result values.

Abstract: A query response method includes: dividing image frames, audio data and caption data included in video data of a data set on a per-shot basis based on the same single caption; extracting a shot feature vector by calculating the feature vectors of image frames, audio data and caption data included in each shot; extracting feature vectors of query data and a plurality of pieces of option data corresponding to the query data from each query-response pair included in the data set; calculating a video feature vector by inputting the shot feature vectors into a multilayer neural network, assigning an attention weight, calculated based on the feature vector of the query data, to output vectors of respective layers, and then summing the weighted output vectors; and selecting a final response from among the plurality of pieces of option data based on similarities between the video feature vector and option feature vectors.

Abstract: A toddler-inspired Bayesian learning method according to an embodiment includes: collecting information related to at least one task while an agent is performing exploration; and performing Bayesian inference regarding the at least one task by using the collected information as Bayesian informative priors.

Abstract: An equipment diagnosis system and method, which use abnormal data and normal data of equipment and accurately and effectively perform diagnosis on the equipment, are provided. The equipment diagnosis system includes a data acquisition unit to acquire time series data of equipment, a preprocessing unit convert the time series data into frequency data including a temporal component through a Fourier transform, a deep learning unit to perform deep learning through a convolution neural network (CNN) by using the frequency data, and a diagnosis unit to determine a state of the equipment to be a normal state or a breakdown state based on the deep learning.

Abstract: There are proposed an attention memory method and system for locating an object through visual dialogue. The attention memory system for identifying an object on an image includes: a control unit which generates a question for identifying a preset object on the image, derives an answer to the generated question, and identifies the preset object on the image based on the question and the answer; and memory which stores the image. The control unit generates the question by incorporating information about objects included in the image into the question, and updates the information about the objects based on the answer.

Abstract: There are proposed a method and system for information theory-based questioning for a goal-oriented dialog system. An information theory-based question provision system includes an approximate answer generation unit configured to generate an approximate answer by approximating the generation of an answer to a question by an answer module constituting a part of the goal-oriented dialog system; an intention inference unit configured to infer an intention value, which is the degree of proximity of the question to a goal, based on the answer of the answer module, and a question provision unit configured to provide a question to the answer module based on the intention value.

Abstract: A query response method includes: dividing image frames, audio data and caption data included in video data of a data set on a per-shot basis based on the same single caption; extracting a shot feature vector by calculating the feature vectors of image frames, audio data and caption data included in each shot; extracting feature vectors of query data and a plurality of pieces of option data corresponding to the query data from each query-response pair included in the data set; calculating a video feature vector by inputting the shot feature vectors into a multilayer neural network, assigning an attention weight, calculated based on the feature vector of the query data, to output vectors of respective layers, and then summing the weighted output vectors; and selecting a final response from among the plurality of pieces of option data based on similarities between the video feature vector and option feature vectors.

Abstract: The present invention relates to a wearable sensor-based automatic scheduling device and method. The wearable sensor-based automatic scheduling device includes: a typical schedule setting part that sets a typical schedule received from the user and specifying a specific event; an atypical schedule inferring part that infers an atypical schedule by analyzing the user's current location and the user's behavioral patterns occurring over a specific period of time in the past; and a schedule determining part that determines the schedule from this point on based on the typical schedule and the atypical schedule. Accordingly, the present invention may provide a technology that can predict the user's expected schedule in real time by taking into account both a typical schedule recorded in a scheduler and an atypical schedule mapped out based on daily behavioral patterns.

Abstract: A home robot device includes a memory, a movement module, and a processor. The processor is configured to execute a motion based on specified motion execution information stored in the memory, obtain feedback information of a user, generate modified motion execution information by modifying at least a portion of the specified motion execution information based on the feedback information of the user, where the modified motion execution information includes a movement value of at least one joint unit of the home robot device or at least one support linked to the at least one joint unit selected from a probability model of the specified motion execution information, and execute a motion of the home robot device based on the modified motion execution information.

Abstract: Disclosed herein are a visual story generation method and apparatus for generating a story from a plurality of images by using a deep learning network. The visual story generation method includes: extracting features from a plurality of respective images by using the first extraction unit of a deep learning network; generating the structure of a story based on the overall feature of the plurality of images by using the second extraction unit of the deep learning network; and generating the story by using outputs of the first and second extraction units.

Abstract: There are proposed an attention memory method and system for locating an object through visual dialogue. The attention memory system for identifying an object on an image includes: a control unit which generates a question for identifying a preset object on the image, derives an answer to the generated question, and identifies the preset object on the image based on the question and the answer; and memory which stores the image. The control unit generates the question by incorporating information about objects included in the image into the question, and updates the information about the objects based on the answer.

Abstract: An equipment diagnosis system and method, which use abnormal data and normal data of equipment and accurately and effectively perform diagnosis on the equipment, are provided. The equipment diagnosis system includes a data acquisition unit to acquire time series data of equipment, a preprocessing unit convert the time series data into frequency data including a temporal component through a Fourier transform, a deep learning unit to perform deep learning through a convolution neural network (CNN) by using the frequency data, and a diagnosis unit to determine a state of the equipment to be a normal state or a breakdown state based on the deep learning.

Abstract: The present invention relates to a wearable sensor-based automatic scheduling device and method. The wearable sensor-based automatic scheduling device includes: a typical schedule setting part that sets a typical schedule received from the user and specifying a specific event; an atypical schedule inferring part that infers an atypical schedule by analyzing the user's current location and the user's behavioral patterns occurring over a specific period of time in the past; and a schedule determining part that determines the schedule from this point on based on the typical schedule and the atypical schedule. Accordingly, the present invention may provide a technology that can predict the user's expected schedule in real time by taking into account both a typical schedule recorded in a scheduler and an atypical schedule mapped out based on daily behavioral patterns.