Abstract: Some speech processing systems may handle some commands on-device rather than sending the audio data to a second device or system for processing. The first device may have limited speech processing capabilities sufficient for handling common language and/or commands, while the second device (e.g., an edge device and/or a remote system) may call on additional language models, entity libraries, skill components, etc. to perform additional tasks. An intermediate data generator may facilitate dividing speech processing operations between devices by generating a stream of data that includes a first-pass ASR output (e.g., a word or sub-word lattice) and other characteristics of the audio data such as whisper detection, speaker identification, media signatures, etc. The second device can perform the additional processing using the data stream; e.g., without using the audio data. Thus, privacy may be enhanced by processing the audio data locally without sending it to other devices/systems.

Abstract: Some speech processing systems may handle some commands on-device rather than sending the audio data to a second device or system for processing. The first device may have limited speech processing capabilities sufficient for handling common language and/or commands, while the second device (e.g., an edge device and/or a remote system) may call on additional language models, entity libraries, skill components, etc. to perform additional tasks. An intermediate data generator may facilitate dividing speech processing operations between devices by generating a stream of data that includes a first-pass ASR output (e.g., a word or sub-word lattice) and other characteristics of the audio data such as whisper detection, speaker identification, media signatures, etc. The second device can perform the additional processing using the data stream; e.g., without using the audio data. Thus, privacy may be enhanced by processing the audio data locally without sending it to other devices/systems.

Abstract: A speech interface device is configured to detect an interrupt event and process a voice command without detecting a wakeword. The device includes on-device interrupt architecture configured to detect when device-directed speech is present and send audio data to a remote system for speech processing. This architecture includes an interrupt detector that detects an interrupt event (e.g., device-directed speech) with low latency, enabling the device to quickly lower a volume of output audio and/or perform other actions in response to a potential voice command. In addition, the architecture includes a device directed classifier that processes an entire utterance and corresponding semantic information and detects device-directed speech with high accuracy. Using the device directed classifier, the device may reject the interrupt event and increase a volume of the output audio or may accept the interrupt event, causing the output audio to end and performing speech processing on the audio data.

Abstract: A speech interface device is configured to detect an interrupt event and process a voice command without detecting a wakeword. The device includes on-device interrupt architecture configured to detect when device-directed speech is present and send audio data to a remote system for speech processing. This architecture includes an interrupt detector that detects an interrupt event (e.g., device-directed speech) with low latency, enabling the device to quickly lower a volume of output audio and/or perform other actions in response to a potential voice command. In addition, the architecture includes a device directed classifier that processes an entire utterance and corresponding semantic information and detects device-directed speech with high accuracy. Using the device directed classifier, the device may reject the interrupt event and increase a volume of the output audio or may accept the interrupt event, causing the output audio to end and performing speech processing on the audio data.

Abstract: This disclosure proposes systems and methods enabling on-device/hybrid processing of speech requests using a hub device. The hub device is capable of receiving audio data from surrounding devices and performing speech processing on the audio data to improve latency and/or provide functionality to other devices within a private network. The hub device may receive multiple requests corresponding to different utterances. If the hub device receives a second utterance while processing a first utterance, the hub device may send an error notification, process the first utterance and the second utterance sequentially, suspend processing of the first utterance to process the second utterance first, send the second utterance to another hub device or remote system, or suspend processing of the first utterance and send the first utterance to the remote system in order to process the second utterance.

Abstract: This disclosure proposes systems and methods enabling on-device/hybrid processing of speech requests using a hub device. The hub device is capable of receiving audio data from surrounding devices and performing speech processing on the audio data to improve latency and/or provide functionality to other devices within a private network. The hub device may receive multiple requests corresponding to different utterances. If the hub device receives a second utterance while processing a first utterance, the hub device may send an error notification, process the first utterance and the second utterance sequentially, suspend processing of the first utterance to process the second utterance first, send the second utterance to another hub device or remote system, or suspend processing of the first utterance and send the first utterance to the remote system in order to process the second utterance.

Abstract: A speech interface device is configured to detect an interrupt event and process a voice command without detecting a wakeword. The device includes on-device interrupt architecture configured to detect when device-directed speech is present and send audio data to a remote system for speech processing. This architecture includes an interrupt detector that detects an interrupt event (e.g., device-directed speech) with low latency, enabling the device to quickly lower a volume of output audio and/or perform other actions in response to a potential voice command. In addition, the architecture includes a device directed classifier that processes an entire utterance and corresponding semantic information and detects device-directed speech with high accuracy. Using the device directed classifier, the device may reject the interrupt event and increase a volume of the output audio or may accept the interrupt event, causing the output audio to end and performing speech processing on the audio data.

Abstract: Handling of touch events in a browser environment is disclosed. An example method includes, while a document is displayed on a touchscreen display of a device, detecting a touch event at the touchscreen display, and selectively processing the detected touch event using one of a default hander, a touch event handler, and a conversion to one or more mouse events, according to a touch event handling property defined for the document.

Abstract: A device, system and method are provided for processing structured documents, such as webpages, for display. Various elements within the structured document are parsed and rendered by an electronic device for outputting to an external or integrated display. In response to a detected scaling instruction, such as a zoom in instruction, a selected region of the displayed document indicated by the instruction is scaled to a first scaled size, including any text content therein. Any text content contained within the region may be reflowed according to the bounds of its containing element or a viewport. A dominant alignment is determined from the element or elements contained within the selected region, and the portion of the scaled region to be output to the display is determined based on the dominant alignment.

Abstract: A method and computing device are provided for processing user events received via a user interface, such as a touchscreen, in multiple threads. When a user event is received for a target element in a webpage, the user event is dispatched to both a main browser thread and a secondary thread. The secondary thread processes user events in accordance with established default actions defined within the browser, while the main thread processes the user events in accordance with any event handlers defined for that target element. The main thread processing may be delayed by other interleaved task, and the secondary thread may be given priority over the main thread. When the secondary thread completes processing, an updated webpage is displayed. When the main thread subsequently completes processing, its updated rendering of the webpage is displayed. The secondary thread thus provides an early user interface response to the user event.

Abstract: A method and computing device are provided for processing user events received via a user interface, such as a touchscreen, in multiple threads. When a user event is received for a target element in a webpage, the user event is dispatched to both a main browser thread and a secondary thread. The secondary thread processes user events in accordance with established default actions defined within the browser, while the main thread processes the user events in accordance with any event handlers defined for that target element. The main thread processing may be delayed by other interleaved task, and the secondary thread may be given priority over the main thread. When the secondary thread completes processing, an updated webpage is displayed. When the main thread subsequently completes processing, its updated rendering of the webpage is displayed. The secondary thread thus provides an early user interface response to the user event.

Abstract: A method and computing device are provided for processing user events received via a user interface, such as a touchscreen, in multiple threads. When a user event is received for a target element in a webpage, the user event is dispatched to both a main browser thread and a secondary thread. The secondary thread processes user events in accordance with established default actions defined within the browser, while the main thread processes the user events in accordance with any event handlers defined for that target element. The main thread processing may be delayed by other interleaved task, and the secondary thread may be given priority over the main thread. When the secondary thread completes processing, an updated webpage is displayed. When the main thread subsequently completes processing, its updated rendering of the webpage is displayed. The secondary thread thus provides an early user interface response to the user event.

Abstract: A device, system and method are provided for managing memory for rendering webpages and other structured documents that contain multiple regions. A backing store is created in memory for storing rendered document content. A main region of the structured document is rendered for display, divided into a set of tiles, and stored in the backing store. A subregion of the document is rendered and stored as tiles in the same backing store as well. At least a portion of the tiles for the main region and subregion intersecting with corresponding viewports are outputted to a display. When an active one of the viewports is changed and additional content of the document is to be rendered for display, tiles in the backing store used to store rendered but undisplayed data for the inactive viewport are released to store new rendered content for the active viewport.

Abstract: A method and computing device are provided for processing user events received via a user interface, such as a touchscreen, in multiple threads. When a user event is received for a target element in a webpage, the user event is dispatched to both a main browser thread and a secondary thread. The secondary thread processes user events in accordance with established default actions defined within the browser, while the main thread processes the user events in accordance with any event handlers defined for that target element. The main thread processing may be delayed by other interleaved task, and the secondary thread may be given priority over the main thread. When the secondary thread completes processing, an updated webpage is displayed. When the main thread subsequently completes processing, its updated rendering of the webpage is displayed. The secondary thread thus provides an early user interface response to the user event.

Abstract: A method and computing device are provided for processing user events received via a user interface, such as a touchscreen, in multiple threads. When a user event is received for a target element in a webpage, the user event is dispatched to both a main browser thread and a secondary thread. The secondary thread processes user events in accordance with established default actions defined within the browser, while the main thread processes the user events in accordance with any event handlers defined for that target element. The main thread processing may be delayed by other interleaved task, and the secondary thread may be given priority over the main thread. When the secondary thread completes processing, an updated webpage is displayed. When the main thread subsequently completes processing, its updated rendering of the webpage is displayed. The secondary thread thus provides an early user interface response to the user event.

Abstract: A device, system and method are provided for processing structured documents, such as webpages, for display. Various elements within the structured document are parsed and rendered by an electronic device for outputting to an external or integrated display. In response to a detected scaling instruction, such as a zoom in instruction, a selected region of the displayed document indicated by the instruction is scaled to a first scaled size, including any text content therein. Any text content contained within the region may be reflowed according to the bounds of its containing element or a viewport. A dominant alignment is determined from the element or elements contained within the selected region, and the portion of the scaled region to be output to the display is determined based on the dominant alignment.

Abstract: Properties defining modes for handling user events such as cursor events and touch events on a computing device such as a touchscreen device are provided. One property determines, for an entire webpage or for elements thereof, whether received cursor events should be processed in accordance with default handling routines configured in the browser environment, or passed for handling by an event handler defined for the element. Another property determines, for the entire webpage or elements thereof, whether received touch events should be processed in accordance with default handling routines, passed for handling by an event handler, or converted to cursor events or mouse events for handling by an event handler.

Abstract: A device, system and method are provided for processing structured documents for display. Content of a first viewable portion of the structured document having a fixed position in relation to a viewport is rendered as first rendered image data. Content of a second viewable portion that does not have a fixed position is rendered as second rendered image data. The first and second rendered image data are composited, and a resultant composited image is output for display. In response to a scroll or zoom command applied to the document, and in particular to the second viewable portion, the second rendered image data is updated and composited with the first rendered image data. Compositing can be carried out by a graphics processor separate from a main processor in the electronic device. When no fixed position elements are present in the structured document, the main processor renders the entire content without compositing.

Abstract: In a publish-subscribe system, a subscribing process may specify a condition relating to an object's attribute value. The condition may be retained in a data structure, and evaluated against a to be published event. If the condition is satisfied or occurred, the publish-subscribe system transmits a notification event to the subscribing process indicating the existence of the published event.

Abstract: An electronic device receives a request to zoom content displayed in a display screen. A width corresponding to the display screen is determined. The zoomed content is wrapped to cause the zoomed content to fit within the determined width.