Abstract: In one aspect, a device includes at least one processor, a display accessible to the at least one processor, and storage accessible to the at least one processor. The storage includes instructions executable by the at least one processor to identify a context associated with a user, identify a location of the user, and adjust the orientation of the display to direct the display toward the location of the user based on the context associated with the user and the location of the user.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for using shortcut command phrases to operate an automated assistant. A user of the automated assistant can request that a shortcut command phrase be established for causing the automated assistant to perform a variety of different actions. In this way, the user does not necessarily have to provide an individual command for each action to be performed but, rather, can use a shortcut command phrase to cause the automated assistant to perform the actions. The shortcut command phrases can be used to control peripheral devices, IoT devices, applications, websites, and/or any other apparatuses or processes capable of being controlled through an automated assistant.

Abstract: An electronic apparatus including a processor configured to receive a first audio signal, obtain a second audio signal by relating noise to the received first audio signal, identify whether the second audio signal matches a second command obtained by relating the noise to a first command of first reference data, based on second reference data, obtained by relating the noise to the first reference data, and perform an operation based on identification in response to the identifying that the second audio signal matches the second command.

Abstract: Disclosed are systems and methods for voice-based control of augmented reality (AR) objects on a wearable device. The systems and methods perform operations comprising: instructing a display element of the AR wearable device to present a visual indicator representing a cursor; receiving voice input representing a first virtual object; determining a real-world position within a real-world environment being viewed through the AR wearable device based on a current position of the visual indicator; and instructing the display element of the AR wearable device to present the first virtual object within the real-world environment at the real-world position.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to both perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. For example, some embodiments can automatically and intelligently cause a switch to a page the user needs and automatically and intelligently cause a population of particular fields of the page the user needs based on a user view context and the voice utterance.

Abstract: A system and method for authenticating sound verbalized or otherwise generated by a live source within a monitored setting for voice-controlled or sound-controlled automation of a responsive process. One or more classifiers each generate a decision value according to values of predetermined signal features extracted from a received digital stream, and a sound type classification is computed according to an aggregate score of a predetermined number of decision values. The actuation of the responsive process is authenticated when the system discriminately indicates the captured sound signals to be verbalized or generated by a live source. The responsive process is thereby suppressed when the sound is instead determined to be reproduced or otherwise previously transduced, for example by a transmission or recording.

Abstract: Various techniques are disclosed, including receiving at a multiplatform management system a communication from a computing device via a groupware platform, the multiplatform management system interfacing with multiple disparate platforms including the groupware platform and an image processing platform, determining an event type based on the communication from the computing device to identify a cloud platform to be selected from among the plurality of disparate platforms based on a detection of one of the image or the text in the communication from the groupware platform; and identifying an action to be performed by the selected cloud platform based on the determined event type.

Abstract: A head-mounted device may use head pose changes for user input. In particular, a display in the head-mounted device may display a slider with an indicator. The slider may be a visual representation of a scalar quantity of a device setting such as volume or brightness. Based on head pose changes, the scalar quantity of the device setting and the position of the indicator on the slider may be updated. The direction of a head movement may correspond to the direction of movement of the indicator in the slider. The scalar quantity of a device setting may only be updated when gaze input from a user targets the slider. The slider may be displayed in response to gaze input targeting an icon associated with the slider.

Abstract: A method and apparatus for controlling a social robot includes operating an electronic output device based on social interactions between the social robot and a user. The social robot utilizes an algorithm or other logical solution process to infer a user mental state, for example a mood or desire, based on observation of the social interaction. Based on the inferred mental state, the social robot causes an action of the electronic output device to be selected. Actions may include, for example, playing a selected video clip, brewing a cup of coffee, or adjusting window blinds.

Abstract: Embodiment of this application provide a method and apparatus for displaying an unread message, an electronic device, and a storage medium. The method includes: displaying a program corner mark in a case that an unread message is received through a target application; receiving a first input performed by a user on the program corner mark; displaying an information window in response to the first input, where the information window includes an identifier of a target contact, the target contact is a contact meeting a preset condition in contacts corresponding to the unread message, and receiving a second input performed by the user on the identifier of the target contact; and starting the target application in response to the second input.

Abstract: In general, the subject matter described in this specification can be embodied in methods, systems, and program products for receiving a voice query at a mobile computing device and generating data that represents content of the voice query. The data is provided to a server system. A textual query that has been determined by a speech recognizer at the server system to be a textual form of at least part of the data is received at the mobile computing device. The textual query is determined to include a carrier phrase of one or more words that is reserved by a first third-party application program installed on the computing device. The first third-party application is selected, from a group of one or more third-party applications, to receive all or a part of the textual query. All or a part of the textual query is provided to the selected first application program.

Abstract: A display apparatus, including a microphone configured to receive a voice signal and a controller configured to generate a resulting screen corresponding to the voice signal different from a first displayed screen and according to a status of the content of the first displayed screen; to display the resulting screen on the display. Accordingly, the display apparatus provides a different resulting screen according to a status of the display apparatus.

Abstract: Disclosed is a display device capable of classifying clickable content in a web application screen, and the display device can perform primary classification on all nodes existing in a document object model (DOM) and perform secondary classification for determining whether a node exists in the screen.

Abstract: A machine implemented method and system, including: receiving at a near real-time processor module, one or more tenant-specific business objects from a message handler module; receiving at the near real-time processor module, contextual data related to the received one or more tenant-specific business objects from a platform analytics module; forming at the near real-time processor module, one or more events by applying one or more pre-defined analytic models to the received contextual data and the received one or more tenant-specific business objects; receiving at a message publisher module, one or more events from the near real-time processor module; and transmitting the received one or more events to one or more subscribers for the one or more events.

Abstract: A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform receiving a voice command from a user; transforming the voice command; transforming the voice command can include using a natural language understanding and rules execution engine into (a) an intent of the user to add recipe ingredients to a cart and (b) a recipe descriptor; determining a matching recipe from a set of ingested recipes based on the recipe descriptor; determining items and quantities associated with the items that correspond to a set of ingredients included in the matching recipe using a quantity inference algorithm; and automatically adding all of the items and the quantities associated with the items to the cart. Other embodiments are disclosed.

Abstract: An information processing apparatus includes circuitry that displays, on a display, form management information including form information indicating content of a form and management information for managing the form information, the form information having been acquired based on an embedment code included in form image data of the form, and in response to an instruction to output the form management information, outputs the form management information to an extraneous source.

Abstract: A method for computer control of online social interactions based on conversation processing. A portion of a conversation with a given user is recorded, stored in memory, and processed so as to influence subsequent interactions. These may include audio or other contextualized annunciations of subsequent telephone calls. Other modes addressing conversational processing for social purposes can result in interwoven conversation guided among threads of interest, advertising and incented participation of conversational content and placement. The invention is capable of implementation in telecommunications systems such as cellular, local exchange, and VOIP, and in combination with other forms of internet-based telecommunication, including smart phones and adaptive forums chat rooms.

Abstract: The present invention may be embodied in a natural language processing system that conducts user interaction to identify and refine requests for data analyses, and automatically conducts data mining and prepares data visualizations in response to natural language queries. Similarly natural language processing system can be utilized for updating business system data in response to natural language requests. The system greatly improves the ease of use, intuitiveness, variety, and responsiveness of the data analytics system by converting natural language requests into requests for data analyses. This allows a much wider range of users to conduct commercially relevant data analytics without relying on specialists in the field data analytics specialists and at much lower cost than the conventional approach.

Abstract: A method for decoding a 360-degree image includes: receiving a bitstream obtained by encoding a 360-degree image; generating a prediction image by making reference to syntax information obtained from the received bitstream; combining the generated prediction image with a residual image obtained by dequantizing and inverse-transforming the bitstream, so as to obtain a decoded image; and reconstructing the decoded image into a 360-degree image according to a projection format. Here, generating the prediction image includes: checking, from the syntax information, prediction mode accuracy for a current block to be decoded; determining whether the checked prediction mode accuracy corresponds to most probable mode (MPM) information obtained from the syntax information; and when the checked prediction mode accuracy does not correspond to the MPM information, reconfiguring the MPM information according to the prediction mode accuracy for the current block.

Abstract: A wearable electronic device is provided. The electronic device includes a camera, a communication circuit, a display including a transparent lens and displaying content through the lens, and a processor, wherein the processor may be configured to obtain first information about a first device indicated by a first object through the communication circuit when the first object is selected from among a plurality of objects for controlling devices displayed on an external electronic device, identify a first position for displaying the first information corresponding to the first object as an augmented reality image, based on a gaze direction of a user identified through the camera, determine a property of the augmented reality image, based on a user input, and control the display to display the augmented reality image having the property at the first position.

Abstract: Provided is an information processing apparatus including an output control unit that controls an output based on a result of recognition processing for a user input. Further, in a situation where a plurality of users is present, the output control unit dynamically controls, for each of the users, an output of an executability state of the recognition processing for the user input. Furthermore, provided is an information processing method including, by a processor, controlling an output based on a result of recognition processing for a user input. The controlling further including, in a situation where a plurality of users is present, dynamically controlling, for each of the users, an output of an executability state of the recognition processing for the user input.

Abstract: A user interacts with a Voice User Interface (VUI) software application such as a digital assistant on a first user device such as a smart phone to access information about different products by communicating with a VUI server over a first communications channel. The VUI server relays indications of the products to an assistance server, for example a server in a wireless telecommunications network of a telecommunications service provider for the user device, and the assistance server provides additional information regarding the different products including images to the first user device and/or one or more additional user devices such as a smart television connected to the wireless telecommunications network user device via at least a second communications channel that is distinct from the first communications channel.

Abstract: Systems, methods, and computer readable media for voice-controlled user interfaces (UIs) for augmented reality (AR) wearable devices are disclosed. Embodiments are disclosed that enable a user to interact with the AR wearable device without using physical user interface devices. An application has a non-voice-controlled UI mode and a voice-controlled UI mode. The user selects the mode of the UI. The application running on the AR wearable device displays UI elements on a display of the AR wearable device. The UI elements have types. Predetermined actions are associated with each of the UI element types. The predetermined actions are displayed with other information and used by the user to invoke the corresponding UI element.

Abstract: To provide an information processing apparatus (100) including: a detection unit (110) that detects an object selected by a user with a finger-pointing gesture as a selected object, in which the detection unit detects the selected object on the basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side.

Abstract: Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.

Abstract: Provided are a display method and apparatus, a smart wearable device, and a computer-readable storage medium, the display method being used on the smart wearable device, and the smart wearable device including at least two screen display regions and at least two corresponding audio collection units; when a user is wearing the smart wearable device, the screen display regions are not simultaneously in the same plane. The display method includes according to voice signals collected by all of the audio collection units, determining an audio collection unit nearest a sound source; turning on a screen display region corresponding to the audio collection unit nearest the sound source to display current content.

Abstract: Systems and methods enable a computing system to recognize a sequence of repeated actions and offer to automatically repeat any such recognized actions. An example method includes determining a current sequence of user actions is similar to previous sequence of user actions, determining whether the previous sequence is reproducible and, when reproducible, initiating display of a prompt that requests approval for completing the current sequence based on the previous sequence and, responsive to receiving an indication of approval, completing the previous sequence. Another example method included determining that a first current sequence of user interactions is complete and is not similar to any saved sequence of user interactions, saving the first current sequence as a previous sequence, identifying a second sequence as satisfying a similarity threshold with the previous sequence, and initiating display of a prompt that requests approval for saving the previous sequence as a shortcut.

Abstract: This specification is generally directed to techniques for robust natural language (NL) based control of computer applications. In many implementations, the NL control is at least selectively interactive in that the user feedback input is solicited, and received, in resolving action(s), resolving action set(s), generating domain specific knowledge, and/or in providing feedback on implemented action set(s). The user feedback input can be utilized in further training of machine learning model(s) utilized in the NL based control of the computer applications.

Abstract: Systems and methods to implement commands based on selection sequences to a user interface are disclosed. Exemplary implementations may: store, electronic storage, a library of terms utterable by users that facilitate implementation of intended results; obtain audio information representing sounds captured by a client computing platform; detect the spoken terms uttered by the user present within the audio information; determine whether the spoken terms detected are included in the library of terms; responsive to determination that the spoken terms are not included in the library of terms, effectuate presentation of an error message via the user interface; record a selection sequence that the user performs subsequent to the presentation of the error message that causes a result; correlate the selection sequence with the spoken terms based on the selection sequence recorded subsequent to error message to generate correlation; and store the correlation to the electronic storage.

Abstract: An assistant executing at, at least one processor, is described that determines content for a conversation with a user of a computing device and selects, based on the content and information associated with the user, a modality to signal initiating the conversation with the user. The assistant is further described that causes, in the modality, a signaling of the conversation with the user.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content in association with travel. The program and method provide for receiving, by a messaging application, a request to perform a scan operation in association with an image captured by a device camera; determining a travel parameter associated with the request and an attribute of an object depicted in the image; selecting an augmented reality content item based on at least one of the travel parameter or the attribute, the augmented reality content item being configured to present augmented reality content based on speech input; receiving the speech input; obtaining at least one of a transcription or translation of the speech input; and presenting the augmented reality content item, including the transcription or translation, in association with the image.

Abstract: An agent control device that acquires a request signal requesting establishment of communication between a mobile terminal and an onboard device; responds to the request signal so as to present, on a display device, a display screen showing procedure information used to set up and establish communication between the mobile terminal and the onboard device; and performs control to continue operation of an agent that is operational inside a vehicle while the display screen is being presented.

Abstract: An apparatus, method, and computer program for identifying a user of a display unit. The apparatus comprises a processor configured to receive a physiological measurement for a first user; compare the received physiological measurement with a plurality of time series of physiological measurements, each time series corresponding to one of a plurality of users; and determine, based on the comparison, an identity of the first user.

Abstract: An apparatus includes: a display; and a controller that: causes the display to display a plurality of operation items that can be instructed by a user, receives an instruction by a voice operation with respect to at least one of the operation items, and receives an instruction by a manual operation with respect to at least one of the operation items. The display displays a first operation item identifiable from the remaining operation items, and the first operation item can be instructed by a user voice.

Abstract: A method and system are disclosed for outputting augmented reality information to a first user. In an embodiment, the method includes acquiring first information, including image information, depth information, coordinate information and combinations thereof, the first information relating to at least one of a medical device and a medical examination of a patient; creating the augmented reality information, relating to the medical device and/or the medical examination of the patient, based on the first information; and outputting the augmented reality information such that the augmented reality information is perceivable in a field of view of the first user.

Abstract: A method including embedding, by a trained issue MLM (machine learning model), a new natural language issue statement into an issue vector. An inner product of the issue vector with an actions matrix is calculated. The actions matrix includes centroid-vectors calculated using a clustering method from a second output of a trained action MLM which embedded prior actions expressed in natural language action statements taken as a result of prior natural issue statements. Calculating the inner product results in probabilities associated with the prior actions. Each of the probabilities represents a corresponding estimate that a corresponding prior action is relevant to the issue vector. A list of proposed actions relevant to the issue vector is generated by comparing the probabilities to a threshold value and selecting a subset of the prior actions with corresponding probabilities above the threshold. The list of proposed actions is transmitted to a user device.

Abstract: A highlight moment within a video, music to accompany a looping presentation of the video, and a looping effect for the video may be determined. A segment of the video to be used for the looping presentation of the video may be selected based on highlight moment, the music, and the looping effect. The looping presentation of the video may be generated to have the segment edited based on a style of the looping effect and to include accompaniment of the music.

Abstract: Techniques include a method of providing a user interface on a device. The user interface has at least a first display portion, a second display portion and a third display portion, the second display portion including a link to the third display portion that, when activated by a user, cause the device to present the third display portion of the user interface, the first display portion not including the link to the third display portion. The method includes causing the first display portion to be displayed. The method further includes receiving audible input while the first display portion is being displayed. The method further includes determining that the third display portion corresponds to an utterance in the audible input based at least in part on labels determined to match the utterance. The method further includes causing the third display portion to be displayed.

Abstract: A reception apparatus, including processing circuitry that is configured to receive a voice command related to content from a user during presentation of the content to the user. The processing circuitry is configured to transmit the voice command to a server system for processing. The processing circuitry is configured to receive a response to the voice command from the server system. The response to the voice command is generated based on the voice command and content information for identifying the content related to the voice command.

Abstract: At least some embodiments are directed to a system to compute uniform structured summarization of customer chats. In at least some embodiments, the system may operate a processor and receive a corpus of chats between customers and customer service representatives of an enterprise. Grouping the corpus of chats into subgroup task types and then extracting chat keywords and chat related words for each subgroup task type. Generating an expandable template data structures for each subgroup task type. Processing at least one chat to extract chat utterances and chat snippets ranking the chat utterances and chat snippets. Populating the expandable template data structure based on rankings to generate a chat summary data structure.

Abstract: Systems and methods to implement commands based on selection sequences to a user interface are disclosed. Exemplary implementations may: store, electronic storage, a library of terms utterable by users that facilitate implementation of intended results; obtain audio information representing sounds captured by a client computing platform; detect the spoken terms uttered by the user present within the audio information; determine whether the spoken terms detected are included in the library of terms; responsive to determination that the spoken terms are not included in the library of terms, effectuate presentation of an error message via the user interface; record a selection sequence that the user performs subsequent to the presentation of the error message that causes a result; correlate the selection sequence with the spoken terms based on the selection sequence recorded subsequent to error message to generate correlation; and store the correlation to the electronic storage.

Abstract: The present disclosure discloses an image processing device including: a receiving module configured to receive a voice signal and an image to be processed; a conversion module configured to convert the voice signal into an image processing instruction and determine a target area according to a target voice instruction conversion model, in which the target area is a processing area of the image to be processed; and a processing module configured to process the target area according to the image processing instruction and a target image processing model. The examples may realize the functionality of using voice commands to control image processing, which may save users' time spent in learning image processing software prior to image processing, and improve user experience.

Abstract: An endpoint system including one or more computing devices presents an object in a virtual environment (e.g., a shared virtual environment); receives gaze input corresponding to a gaze of a user of the endpoint system; calculates a gaze vector based on the gaze input; receives motion input corresponding to an action of the user; determines a path adjustment (e.g., by changing motion parameters such as trajectory and velocity) for the object based at least in part on the gaze vector and the motion input; and simulates motion of the object within the virtual environment based at least in part on the path adjustment. The object may be presented as being thrown by an avatar, with a flight path based on the path adjustment. The gaze vector may be based on head orientation information, eye tracking information, or some combination of these or other gaze information.

Abstract: The present disclosure relates to resolving natural language ambiguities with respect to a simulated reality setting. In an exemplary embodiment, a simulated reality setting having one or more virtual objects is displayed. A stream of gaze events is generated from the simulated reality setting and a stream of gaze data. A speech input is received within a time period and a domain is determined based on a text representation of the speech input. Based on the time period and a plurality of event times for the stream of gaze events, one or more gaze events are identified from the stream of gaze events. The identified one or more gaze events is used to determine a parameter value for an unresolved parameter of the domain. A set of tasks representing a user intent for the speech input is determined based on the parameter value and the set of tasks is performed.

Abstract: Provided are a device and a method for providing a response message to a voice input of a user. The method, performed by a device, of providing a response message to a voice input of a user includes: receiving the voice input of the user; determining a destination of the user and an intention of the user, by analyzing the received voice input; obtaining association information related to the destination; generating the response message that recommends a substitute destination related to the intention of the user, based on the obtained association information; and displaying the generated response message.

Abstract: A sensor transformation attention network (STAN) model including sensors, attention modules, a merge module and a task-specific module is provided. The attention modules calculate attention scores of feature vectors corresponding to the input signals collected by the sensors. The merge module calculates attention values of the attention scores, and generates a merged transformation vector based on the attention values and the feature vectors. The task-specific module classifies the merged transformation vector.

Abstract: An electronic device includes circuitry, firmware, and software that determines identification information associated with a first performer-of-interest at a live event and retrieves a first set of audio tracks from a plurality of audio tracks based on the determined identification information. The circuitry receives a first audio segment associated with the first performer-of-interest from an audio capturing device. The circuitry compares a first audio characteristic of the first audio segment with a second audio characteristic of a first audio portion of each of the first set of audio tracks. The circuitry determines a first audio track based on the comparison between the first audio characteristic and the second audio characteristic. The circuitry identifies a start position of the first audio track based on the first audio segment associated with the first audio track. The circuitry controls a display of the first lyrics information of the first audio track.

Abstract: A method for performing multi-touch (MT) data fusion is disclosed in which multiple touch inputs occurring at about the same time are received to generating first touch data. Secondary sense data can then be combined with the first touch data to perform operations on an electronic device. The first touch data and the secondary sense data can be time-aligned and interpreted in a time-coherent manner. The first touch data can be refined in accordance with the secondary sense data, or alternatively, the secondary sense data can be interpreted in accordance with the first touch data. Additionally, the first touch data and the secondary sense data can be combined to create a new command.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for using shortcut command phrases to operate an automated assistant. A user of the automated assistant can request that a shortcut command phrase be established for causing the automated assistant to perform a variety of different actions. In this way, the user does not necessarily have to provide an individual command for each action to be performed but, rather, can use a shortcut command phrase to cause the automated assistant to perform the actions. The shortcut command phrases can be used to control peripheral devices, IoT devices, applications, websites, and/or any other apparatuses or processes capable of being controlled through an automated assistant.