Abstract: Systems and methods for automatic generation of free-form conversational interfaces are disclosed. In one embodiment, a system receives an input from a user device through a conversational graphical user interface (GUI). An intent of the user may be determined based on the received input. Based on the intent of the user, the system may identify, from a plurality of objects available to the system, one or more objects. Each of the plurality of objects has annotations corresponding to one or more elements of the object and one or more functions of the object. The one or more functions corresponding to the one or more elements are executable to perform an action upon corresponding elements. Based on the identified one or more objects and the annotations of the identified one or more objects, the system may generate a dynamic dialogue flow for the conversational GUI, where the dynamic dialogue flow is generated in real-time during a conversational GUI session.

Abstract: Methods for rapid micro-payment transactions with deferred payments employ a network-connected AR device (“HMD”) having sensors for detecting a user's gaze/physical commands.

Abstract: Techniques for coordinating output of inferred content using various components and systems are described. A supplemental content system and a notification system may each receive inferred content to be output. When the supplemental content system or the notification system outputs the inferred content, the respective system stores a record of the output of the content in a historical output storage. Thereafter, when the other system is ready to output the inferred content, the other system may prevent the inferred content from being output based on the inferred content having already been output, as represented in the historical output storage.

Abstract: This application provides an application starting method, an electronic device, and a readable storage medium, and relates to the field of terminal application technologies. The method includes: when an application is switched from a foreground to a background for running, saving a final interface (namely, a first interface) displayed before the application is switched to the background for running as a snapshot; when a user clicks an icon of the application on a system desktop to start the application again, in a case that a process of the application does not end, a first activity corresponding to the first interface is not destroyed, and anew activity needs to be created when the application is started. Therefore, from the user's point of view, the start response speed of application software is improved.

Abstract: Techniques for configuring device-specific skills as top-level skills are described. When a system receives a user input the system performs NLU processing to determine an intent of the user input. In some instances, the system may identify a device-specific skill associated with the device interacted with by the user. At least partially in parallel to performing NLU processing to determine the intent of the user input, the system may also perform NLU processing to determine a likelihood that the user input corresponds to an intent actionable by the device-specific skill. Once the system has finished NLU processing, the system may implement one or more prioritization rules to determine whether the user input should be sent to the device-specific skill or another skill of the system.

Abstract: Embodiments provide a voice interaction method and an apparatus, and relate to the field of terminal technologies. Common voice skill commands in a first application scenario may be determined based on the first application scenario and a historical voice skill usage record, and displayed in a display interface. This can implement scenario-based recommendation of voice skill commands, to cover as many application scenarios as possible. In this application, after being woken up, the voice assistant determines the first application scenario based on one or more information items. The voice assistant determines the common voice skill commands in the first application scenario based on the first application scenario and the historical voice skill usage record. The voice assistant displays the common voice skill commands in the first application scenario in the display interface.

Abstract: An icon display method includes: receiving a first input performed by a user on a target icon, where the target icon is an icon on at least one first interface; and in response to the first input, displaying the target icon on a second interface in a case that a target quantity reaches a target value. The target quantity is a quantity of icons displayed in fixed icon placeholders on the second interface, and the target value is a quantity of fixed icon placeholders on the second interface. After the target icon is displayed on the second interface, P icons on the second interface are located in non-fixed icon placeholders, P is a quantity of target icons, and P is a positive integer.

Abstract: A vehicle lamp system and method with adjustable lights that are configurable and configured to display a variety of colors and blinking patterns. Moreover, the lamp comprises a mobile user interface that commands the lights wirelessly, receives light status from the physical lights, and displays and animates the lights in real time.

Abstract: An electronic device, method, and computer program product support automatic sharing of a cross-view by a network device hosting a communication session for first and second communication devices. The first communication device presents first visual content that is selected by a first participant. The second communication device presents second visual content that is selected by a second participant. First semantic inputs by the first participant are communicated to the second communication device for presenting to the second participant. Second semantic inputs by the second participant are communicated to the first communication device for presenting to the first participant. In response to determining that the semantic input(s) are associated with the visual content, the network device communicates a duplicate copy of the first visual content to the second communication device and a duplicate copy of the second visual content to the first communication device, as a cross view to enhance discussion.

Abstract: Systems and methods to implement commands based on selection sequences to a user interface are disclosed. Exemplary implementations may: store, electronic storage, a library of terms utterable by users that facilitate implementation of intended results; obtain audio information representing sounds captured by a client computing platform; detect the spoken terms uttered by the user present within the audio information; determine whether the spoken terms detected are included in the library of terms; responsive to determination that the spoken terms are not included in the library of terms, effectuate presentation of an error message via the user interface; record a selection sequence that the user performs subsequent to the presentation of the error message that causes a result; correlate the selection sequence with the spoken terms based on the selection sequence recorded subsequent to error message to generate correlation; and store the correlation to the electronic storage.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content in association with travel. The program and method provide for receiving, by a messaging application, a request to perform a scan operation in association with an image captured by a device camera; determining a travel parameter associated with the request and an attribute of an object depicted in the image; selecting an augmented reality content item based on at least one of the travel parameter or the attribute, the augmented reality content item being configured to present augmented reality content based on speech input; receiving the speech input; obtaining at least one of a transcription or translation of the speech input; and presenting the augmented reality content item, including the transcription or translation, in association with the image.

Abstract: A display apparatus includes presence detection circuitry for detecting an individual in proximity to the display apparatus; a display for displaying video content and a user interface; a processor in communication with the user input circuitry, the display, and the search history database; and non-transitory computer readable media in communication with the processor that stores instruction code. The instruction code is executed by the processor and causes the processor to: a) determine, from the presence detection circuitry, a user in proximity of the display apparatus; b) determine one or more program types associated with the user; c) determine available programs that match the predicted one or more programs; and d) update the user interface to depict a listing of one or more of the available programs that match the predicted one or more programs.

Abstract: Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.

Abstract: Provided is an input device capable of facilitating an operation for inputting, a command such as characters. The input device includes an operation unit of which an operation amount is variable and a control unit. The control unit selects one command group among a plurality of command groups and decides on a command on the basis of the command group selected and the operation amount of the operation unit. The control unit switches the command group selected when the operation amount of the operation unit is in a prescribed range.

Abstract: An electronic device according to various embodiments includes a display, a camera module, and at least one processor, wherein the at least one processor may be set to: display one or more real objects and one or more virtual objects through the display; receive a first input for selecting any one object from among the objects, and then receive a second input related to the picked object; analyze information about grouped objects when a third input is received for grouping the objects selected by repeating the first input and the second input; and provide a function extracted on the basis of analyzing the information about the grouped objects, when a fourth input related to the grouped objects is received.

Abstract: In one aspect, a device includes at least one processor, a display accessible to the at least one processor, and storage accessible to the at least one processor. The storage includes instructions executable by the at least one processor to identify a context associated with a user, identify a location of the user, and adjust the orientation of the display to direct the display toward the location of the user based on the context associated with the user and the location of the user.

Abstract: An electronic apparatus including a processor configured to receive a first audio signal, obtain a second audio signal by relating noise to the received first audio signal, identify whether the second audio signal matches a second command obtained by relating the noise to a first command of first reference data, based on second reference data, obtained by relating the noise to the first reference data, and perform an operation based on identification in response to the identifying that the second audio signal matches the second command.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for using shortcut command phrases to operate an automated assistant. A user of the automated assistant can request that a shortcut command phrase be established for causing the automated assistant to perform a variety of different actions. In this way, the user does not necessarily have to provide an individual command for each action to be performed but, rather, can use a shortcut command phrase to cause the automated assistant to perform the actions. The shortcut command phrases can be used to control peripheral devices, IoT devices, applications, websites, and/or any other apparatuses or processes capable of being controlled through an automated assistant.

Abstract: Disclosed are systems and methods for voice-based control of augmented reality (AR) objects on a wearable device. The systems and methods perform operations comprising: instructing a display element of the AR wearable device to present a visual indicator representing a cursor; receiving voice input representing a first virtual object; determining a real-world position within a real-world environment being viewed through the AR wearable device based on a current position of the visual indicator; and instructing the display element of the AR wearable device to present the first virtual object within the real-world environment at the real-world position.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to both perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. For example, some embodiments can automatically and intelligently cause a switch to a page the user needs and automatically and intelligently cause a population of particular fields of the page the user needs based on a user view context and the voice utterance.

Abstract: A system and method for authenticating sound verbalized or otherwise generated by a live source within a monitored setting for voice-controlled or sound-controlled automation of a responsive process. One or more classifiers each generate a decision value according to values of predetermined signal features extracted from a received digital stream, and a sound type classification is computed according to an aggregate score of a predetermined number of decision values. The actuation of the responsive process is authenticated when the system discriminately indicates the captured sound signals to be verbalized or generated by a live source. The responsive process is thereby suppressed when the sound is instead determined to be reproduced or otherwise previously transduced, for example by a transmission or recording.

Abstract: Various techniques are disclosed, including receiving at a multiplatform management system a communication from a computing device via a groupware platform, the multiplatform management system interfacing with multiple disparate platforms including the groupware platform and an image processing platform, determining an event type based on the communication from the computing device to identify a cloud platform to be selected from among the plurality of disparate platforms based on a detection of one of the image or the text in the communication from the groupware platform; and identifying an action to be performed by the selected cloud platform based on the determined event type.

Abstract: A head-mounted device may use head pose changes for user input. In particular, a display in the head-mounted device may display a slider with an indicator. The slider may be a visual representation of a scalar quantity of a device setting such as volume or brightness. Based on head pose changes, the scalar quantity of the device setting and the position of the indicator on the slider may be updated. The direction of a head movement may correspond to the direction of movement of the indicator in the slider. The scalar quantity of a device setting may only be updated when gaze input from a user targets the slider. The slider may be displayed in response to gaze input targeting an icon associated with the slider.

Abstract: Embodiment of this application provide a method and apparatus for displaying an unread message, an electronic device, and a storage medium. The method includes: displaying a program corner mark in a case that an unread message is received through a target application; receiving a first input performed by a user on the program corner mark; displaying an information window in response to the first input, where the information window includes an identifier of a target contact, the target contact is a contact meeting a preset condition in contacts corresponding to the unread message, and receiving a second input performed by the user on the identifier of the target contact; and starting the target application in response to the second input.

Abstract: A method and apparatus for controlling a social robot includes operating an electronic output device based on social interactions between the social robot and a user. The social robot utilizes an algorithm or other logical solution process to infer a user mental state, for example a mood or desire, based on observation of the social interaction. Based on the inferred mental state, the social robot causes an action of the electronic output device to be selected. Actions may include, for example, playing a selected video clip, brewing a cup of coffee, or adjusting window blinds.

Abstract: A display apparatus, including a microphone configured to receive a voice signal and a controller configured to generate a resulting screen corresponding to the voice signal different from a first displayed screen and according to a status of the content of the first displayed screen; to display the resulting screen on the display. Accordingly, the display apparatus provides a different resulting screen according to a status of the display apparatus.

Abstract: In general, the subject matter described in this specification can be embodied in methods, systems, and program products for receiving a voice query at a mobile computing device and generating data that represents content of the voice query. The data is provided to a server system. A textual query that has been determined by a speech recognizer at the server system to be a textual form of at least part of the data is received at the mobile computing device. The textual query is determined to include a carrier phrase of one or more words that is reserved by a first third-party application program installed on the computing device. The first third-party application is selected, from a group of one or more third-party applications, to receive all or a part of the textual query. All or a part of the textual query is provided to the selected first application program.

Abstract: Disclosed is a display device capable of classifying clickable content in a web application screen, and the display device can perform primary classification on all nodes existing in a document object model (DOM) and perform secondary classification for determining whether a node exists in the screen.

Abstract: A machine implemented method and system, including: receiving at a near real-time processor module, one or more tenant-specific business objects from a message handler module; receiving at the near real-time processor module, contextual data related to the received one or more tenant-specific business objects from a platform analytics module; forming at the near real-time processor module, one or more events by applying one or more pre-defined analytic models to the received contextual data and the received one or more tenant-specific business objects; receiving at a message publisher module, one or more events from the near real-time processor module; and transmitting the received one or more events to one or more subscribers for the one or more events.

Abstract: A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform receiving a voice command from a user; transforming the voice command; transforming the voice command can include using a natural language understanding and rules execution engine into (a) an intent of the user to add recipe ingredients to a cart and (b) a recipe descriptor; determining a matching recipe from a set of ingested recipes based on the recipe descriptor; determining items and quantities associated with the items that correspond to a set of ingredients included in the matching recipe using a quantity inference algorithm; and automatically adding all of the items and the quantities associated with the items to the cart. Other embodiments are disclosed.

Abstract: A method for computer control of online social interactions based on conversation processing. A portion of a conversation with a given user is recorded, stored in memory, and processed so as to influence subsequent interactions. These may include audio or other contextualized annunciations of subsequent telephone calls. Other modes addressing conversational processing for social purposes can result in interwoven conversation guided among threads of interest, advertising and incented participation of conversational content and placement. The invention is capable of implementation in telecommunications systems such as cellular, local exchange, and VOIP, and in combination with other forms of internet-based telecommunication, including smart phones and adaptive forums chat rooms.

Abstract: An information processing apparatus includes circuitry that displays, on a display, form management information including form information indicating content of a form and management information for managing the form information, the form information having been acquired based on an embedment code included in form image data of the form, and in response to an instruction to output the form management information, outputs the form management information to an extraneous source.

Abstract: The present invention may be embodied in a natural language processing system that conducts user interaction to identify and refine requests for data analyses, and automatically conducts data mining and prepares data visualizations in response to natural language queries. Similarly natural language processing system can be utilized for updating business system data in response to natural language requests. The system greatly improves the ease of use, intuitiveness, variety, and responsiveness of the data analytics system by converting natural language requests into requests for data analyses. This allows a much wider range of users to conduct commercially relevant data analytics without relying on specialists in the field data analytics specialists and at much lower cost than the conventional approach.

Abstract: A method for decoding a 360-degree image includes: receiving a bitstream obtained by encoding a 360-degree image; generating a prediction image by making reference to syntax information obtained from the received bitstream; combining the generated prediction image with a residual image obtained by dequantizing and inverse-transforming the bitstream, so as to obtain a decoded image; and reconstructing the decoded image into a 360-degree image according to a projection format. Here, generating the prediction image includes: checking, from the syntax information, prediction mode accuracy for a current block to be decoded; determining whether the checked prediction mode accuracy corresponds to most probable mode (MPM) information obtained from the syntax information; and when the checked prediction mode accuracy does not correspond to the MPM information, reconfiguring the MPM information according to the prediction mode accuracy for the current block.

Abstract: A wearable electronic device is provided. The electronic device includes a camera, a communication circuit, a display including a transparent lens and displaying content through the lens, and a processor, wherein the processor may be configured to obtain first information about a first device indicated by a first object through the communication circuit when the first object is selected from among a plurality of objects for controlling devices displayed on an external electronic device, identify a first position for displaying the first information corresponding to the first object as an augmented reality image, based on a gaze direction of a user identified through the camera, determine a property of the augmented reality image, based on a user input, and control the display to display the augmented reality image having the property at the first position.

Abstract: Provided is an information processing apparatus including an output control unit that controls an output based on a result of recognition processing for a user input. Further, in a situation where a plurality of users is present, the output control unit dynamically controls, for each of the users, an output of an executability state of the recognition processing for the user input. Furthermore, provided is an information processing method including, by a processor, controlling an output based on a result of recognition processing for a user input. The controlling further including, in a situation where a plurality of users is present, dynamically controlling, for each of the users, an output of an executability state of the recognition processing for the user input.

Abstract: A user interacts with a Voice User Interface (VUI) software application such as a digital assistant on a first user device such as a smart phone to access information about different products by communicating with a VUI server over a first communications channel. The VUI server relays indications of the products to an assistance server, for example a server in a wireless telecommunications network of a telecommunications service provider for the user device, and the assistance server provides additional information regarding the different products including images to the first user device and/or one or more additional user devices such as a smart television connected to the wireless telecommunications network user device via at least a second communications channel that is distinct from the first communications channel.

Abstract: Systems, methods, and computer readable media for voice-controlled user interfaces (UIs) for augmented reality (AR) wearable devices are disclosed. Embodiments are disclosed that enable a user to interact with the AR wearable device without using physical user interface devices. An application has a non-voice-controlled UI mode and a voice-controlled UI mode. The user selects the mode of the UI. The application running on the AR wearable device displays UI elements on a display of the AR wearable device. The UI elements have types. Predetermined actions are associated with each of the UI element types. The predetermined actions are displayed with other information and used by the user to invoke the corresponding UI element.

Abstract: To provide an information processing apparatus (100) including: a detection unit (110) that detects an object selected by a user with a finger-pointing gesture as a selected object, in which the detection unit detects the selected object on the basis of a first extension line extending from a first line segment connecting a first reference point indicating a position of a wrist of a hand of the user corresponding to the finger-pointing gesture and a second reference point corresponding to a height of the wrist on a first perpendicular line perpendicularly extending from a base of an arm of the user corresponding to the finger-pointing gesture to a floor surface toward a wrist side.

Abstract: Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.

Abstract: Provided are a display method and apparatus, a smart wearable device, and a computer-readable storage medium, the display method being used on the smart wearable device, and the smart wearable device including at least two screen display regions and at least two corresponding audio collection units; when a user is wearing the smart wearable device, the screen display regions are not simultaneously in the same plane. The display method includes according to voice signals collected by all of the audio collection units, determining an audio collection unit nearest a sound source; turning on a screen display region corresponding to the audio collection unit nearest the sound source to display current content.

Abstract: This specification is generally directed to techniques for robust natural language (NL) based control of computer applications. In many implementations, the NL control is at least selectively interactive in that the user feedback input is solicited, and received, in resolving action(s), resolving action set(s), generating domain specific knowledge, and/or in providing feedback on implemented action set(s). The user feedback input can be utilized in further training of machine learning model(s) utilized in the NL based control of the computer applications.

Abstract: Systems and methods enable a computing system to recognize a sequence of repeated actions and offer to automatically repeat any such recognized actions. An example method includes determining a current sequence of user actions is similar to previous sequence of user actions, determining whether the previous sequence is reproducible and, when reproducible, initiating display of a prompt that requests approval for completing the current sequence based on the previous sequence and, responsive to receiving an indication of approval, completing the previous sequence. Another example method included determining that a first current sequence of user interactions is complete and is not similar to any saved sequence of user interactions, saving the first current sequence as a previous sequence, identifying a second sequence as satisfying a similarity threshold with the previous sequence, and initiating display of a prompt that requests approval for saving the previous sequence as a shortcut.

Abstract: Systems and methods to implement commands based on selection sequences to a user interface are disclosed. Exemplary implementations may: store, electronic storage, a library of terms utterable by users that facilitate implementation of intended results; obtain audio information representing sounds captured by a client computing platform; detect the spoken terms uttered by the user present within the audio information; determine whether the spoken terms detected are included in the library of terms; responsive to determination that the spoken terms are not included in the library of terms, effectuate presentation of an error message via the user interface; record a selection sequence that the user performs subsequent to the presentation of the error message that causes a result; correlate the selection sequence with the spoken terms based on the selection sequence recorded subsequent to error message to generate correlation; and store the correlation to the electronic storage.

Abstract: An assistant executing at, at least one processor, is described that determines content for a conversation with a user of a computing device and selects, based on the content and information associated with the user, a modality to signal initiating the conversation with the user. The assistant is further described that causes, in the modality, a signaling of the conversation with the user.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality content in association with travel. The program and method provide for receiving, by a messaging application, a request to perform a scan operation in association with an image captured by a device camera; determining a travel parameter associated with the request and an attribute of an object depicted in the image; selecting an augmented reality content item based on at least one of the travel parameter or the attribute, the augmented reality content item being configured to present augmented reality content based on speech input; receiving the speech input; obtaining at least one of a transcription or translation of the speech input; and presenting the augmented reality content item, including the transcription or translation, in association with the image.

Abstract: An agent control device that acquires a request signal requesting establishment of communication between a mobile terminal and an onboard device; responds to the request signal so as to present, on a display device, a display screen showing procedure information used to set up and establish communication between the mobile terminal and the onboard device; and performs control to continue operation of an agent that is operational inside a vehicle while the display screen is being presented.

Abstract: An apparatus, method, and computer program for identifying a user of a display unit. The apparatus comprises a processor configured to receive a physiological measurement for a first user; compare the received physiological measurement with a plurality of time series of physiological measurements, each time series corresponding to one of a plurality of users; and determine, based on the comparison, an identity of the first user.

Abstract: An apparatus includes: a display; and a controller that: causes the display to display a plurality of operation items that can be instructed by a user, receives an instruction by a voice operation with respect to at least one of the operation items, and receives an instruction by a manual operation with respect to at least one of the operation items. The display displays a first operation item identifiable from the remaining operation items, and the first operation item can be instructed by a user voice.