Abstract: To generate substantially domain-invariant and speaker-discriminative features, embodiments are associated with a feature extractor to receive speech frames and extract features from the speech frames based on a first set of parameters of the feature extractor, a senone classifier to identify a senone based on the received features and on a second set of parameters of the senone classifier, an attention network capable of determining a relative importance of features extracted by the feature extractor to domain classification, based on a third set of parameters of the attention network, a domain classifier capable of classifying a domain based on the features and the relative importances, and on a fourth set of parameters of the domain classifier; and a training platform to train the first set of parameters of the feature extractor and the second set of parameters of the senone classifier to minimize the senone classification loss, train the first set of parameters of the feature extractor to maximize the dom

Abstract: The present disclosure proposes a federated speaker verification method based on differential privacy, including: 1. performing, by a server, UBM pre-training to obtain an initial UBM; 2. receiving, by the client, the pre-trained initial UBM, and performing initial UBM learning based on local private speech data; 3. performing, by the client, differential privacy protection based on learned statistics; 4. aggregating, by the server, statistics uploaded by multiple clients, and updating the initial UBM; and 5. receiving, by the client, the updated UBM, performing adjustment based on the local private speech data to obtain a GMM for a user of the client, and determining, based on the updated UBM and the GMM, whether a to-be-verified speech is generated by the user of the client.

Abstract: Disclosed are a display apparatus and a method for operating the display apparatus, the display apparatus being operated by executing an artificial intelligence (AI) algorithm and/or a machine learning algorithm in a 5G environment connected for Internet of Things. The method for operating the display apparatus includes the acts of receiving utterance information of a user who is watching the display apparatus, selecting an utterance intention corresponding to the user's utterance information according to a predefined rule, switching operation of the display apparatus on the basis of the selected utterance intention, collecting reaction information of the user corresponding to the switched operation of the display apparatus, and reconstructing the predefined rule by using the user's utterance information, the selected utterance intention, and the user's reaction information.

Abstract: Techniques for adjusting user experiences for participants of a multiuser session by deploying vocal-characteristic models to analyze audio streams received in association with the participants are disclosed herein. The vocal-characteristic models are used to identify emotional state indicators corresponding to certain vocal properties being exhibited by individual participants. Based on the identified emotional state indicators, probability scores are generated indicating a likelihood that individual participants are experiencing a predefined emotional state. For example, a specific participant's voice may be continuously received and analyzed using a vocal-characteristic model designed to detect whether vocal properties are consistent with a predefined emotional state. Probability scores may be generated based on how strongly the detected vocal properties correlate with the vocal-characteristic model.

Abstract: An information processing device including an emotion recognition unit that recognizes, on the basis of information concerning a user and information concerning another user having been sensed, an emotion of the other user, and a notification control unit that performs control such that the user is notified of information concerning the emotion of the other user having been recognized.

Abstract: Provided are a voice processing method and an apparatus, the method including: acquiring, during playback of a content of a first type, a first voice inputted by a user, where the first voice instructs a terminal to switch a played content to a content of a second type; and where the terminal plays a content of a predefined type before playing the content of the first type; playing a first reply voice according to the first voice, prompting the user to determine whether to continue to play the content of the second type after the content of the predefined type during a predefined period; and continuing to play a content of a target type after the content of the predefined type during the predefined period, where the target type is related to the user's feedback on the first reply voice, thus improving a reliability for the terminal.

Abstract: Disclosed is a method for separating a speech based on artificial intelligence in a vehicle. The method includes separating a speech signal using at least one sound beam-forming or source separation; and performing a speech recognition function by detecting a speaker (user) of the separated signal, and accordingly, other users in the vehicle in addition to a driver may use the speech recognition function. A device and method for separating speech based on artificial intelligence in vehicle of the present disclosure can be associated with artificial intelligence modules, drones (unmanned aerial vehicles (UAVs)), robots, augmented reality (AR) devices, virtual reality (VR) devices, devices related to 5G service, etc.

Abstract: A method of converting speech to text comprises receiving an audio recording from an input device comprising speech of a plurality of speakers. Extracting from the audio recording, a speaker audio recording comprising recorded audio of an individual speaker. Selecting, based on a characteristic of the speaker audio recording, a speech to text engine and a dictionary. Configuring the speech to text engine with the dictionary and executing a first conversion process to convert a first portion of the speaker audio recording to produce a first transcript. Evaluating a performance metric of the conversion process against a quality metric to reconfigure the speech to text engine and execute a second conversion process to convert a second portion of the speaker audio recording to produce a second transcript. Combining the first transcript and the second transcript to produce a transcript of the speaker audio recording.

Abstract: An attribute identification technology that can reject an attribute identification result if the reliability thereof is low is provided. An attribute identification device includes: a posteriori probability calculation unit 110 that calculates, from input speech, a posteriori probability sequence {q(c, i)} which is a sequence of the posteriori probabilities q(c, i) that a frame i of the input speech is a class c; a reliability calculation unit 120 that calculates, from the posteriori probability sequence {q(c, i)}, reliability r(c) indicating the extent to which the class c is a correct attribute identification result; and an attribute identification result generating unit 130 that generates an attribute identification result L of the input speech from the posteriori probability sequence {q(c, i)} and the reliability r(c).

Abstract: Systems and processes for selectively processing and responding to a spoken user input are provided. In one example, audio input containing a spoken user input can be received at a user device. The spoken user input can be identified from the audio input by identifying start and end-points of the spoken user input. It can be determined whether or not the spoken user input was intended for a virtual assistant based on contextual information. The determination can be made using a rule-based system or a probabilistic system. If it is determined that the spoken user input was intended for the virtual assistant, the spoken user input can be processed and an appropriate response can be generated. If it is instead determined that the spoken user input was not intended for the virtual assistant, the spoken user input can be ignored and/or no response can be generated.

Abstract: An electronic device can implement a zero-latency digital assistant by capturing audio input from a microphone and using a first processor to write audio data representing the captured audio input to a memory buffer. In response to detecting a user input while capturing the audio input, the device can determine whether the user input meets a predetermined criteria. If the user input meets the criteria, the device can use a second processor to identify and execute a task based on at least a portion of the contents of the memory buffer.

Abstract: Disclosed are various embodiments for segmenting and classifying video content using conversation. In one embodiment, a plurality of segments of a video content item are generated by analyzing audio accompanying the video content item. A subset of the plurality of segments that correspond to conversation segments are selected. Individual segments of the subset of the plurality of segments are processed to determine whether a classification applies to the individual segments. A list of segments of the video content item to which the classification applies is generated.

Abstract: Aspects of the present invention disclose a method for preventing adversarial audio attacks through detecting and isolating inconsistencies utilizing beamforming techniques and IoT devices. The method includes one or more processors identifying an audio command received by a listening device. The method further includes determining a source location of the audio command utilizing a sensor array of the listening device. The method further includes determining a location of a user in relation to the listening device based on data of an Internet of Things (IoT) device. The method further includes determining an inconsistency between the determines source location and the determined location of the user based at least in part on data of the sensor array and data of the IoT device.

Abstract: Described are techniques for linking generating a skill-stored user profile, and linking same with a natural language processing (NLP) system-stored user profile. In at least some examples, a user may provide a natural language input to a NLP system. The NLP system may determine a skill is to process to perform an action responsive to the natural language input. To perform the action, the skill may require the user have a user profile stored by the skill, but the user may not have such a user profile. However, the NLP system may store a user profile for the user. The NLP system may determine the user profile stored thereby and may send, with user permission, information in the user profile to the skill. The skill may use the received information to generate and store a user profile for the user. Thereafter, the skill may provide the NLP system with a user profile identifier that the skill may use to identify the user's profile stored thereby.

Abstract: The present disclosure relates to an information processing apparatus and an information processing method for enabling provision of a more convenient speech recognition service. The information processing apparatus includes a speech recognition unit that performs speech recognition for speech information based on an utterance of a user, and a correction portion estimation unit that collates content of a sentence obtained as a speech recognition result with collation information necessary for determining accuracy of the content to estimate, for the sentence, a correction portion that requires correction. The sentence obtained as a speech recognition result is displayed together with the correction portion estimated by the correction portion estimation unit and presented to the user.

Abstract: Systems and methods for training a voice morphing apparatus are described. The voice morphing apparatus is trained to morph input audio data to mask an identity of a speaker. Training is performed by evaluating an objective function that is a function of the input audio data and an output of the voice morphing apparatus. The objective function may have a first term that is based on speaker identification and a second term that is based on audio fidelity. By optimizing the objective function, parameters of the voice morphing apparatus may be adjusted so as to reduce a confidence of speaker identification and maintain an audio fidelity of the morphed audio data. The voice morphing apparatus, once trained, may be used as part of an automatic speech recognition system.

Abstract: A determination unit (7) determines whether or not a specific passenger in a car has spoken, on the basis of sound data collected in the car. A control unit (8) activates an in-car communication function, when it is determined by the determination unit (7) that the specific passenger has spoken.

Abstract: An access control apparatus includes a sensor and an authentication circuit coupled to the sensor. The sensor detects eye movement of a user. The authentication circuit stores predetermined access code data corresponding to the user. The authentication circuit compares the detected eye movement of the user to the predetermined access code data. Based on the comparison indicating that the detected eye movement matches the predetermined access code data, the authentication circuit permits access beyond an access control point.

Abstract: A speech interface device is configured to receive response data from a remote speech processing system for responding to user speech. This response data may be enhanced with information such as remote NLU data. The response data from the remote speech processing system may be compared to local NLU data to improve a speech processing model on the device. Thus, the device may perform supervised on-device learning based on the remote NLU data. The device may determine differences between the updated speech processing model and an original speech processing model received from the remote system and may send data indicating these differences to the remote system. The remote system may aggregate data received from a plurality of devices and may generate an improved speech processing model.

Abstract: A voice interaction device includes a processor configured to recognize an utterance content from data of voice of a speaker. The processor is configured to perform voice interaction with the speaker by repeating a process of generating data of an utterance sentence based on the utterance content of the speaker and outputting the data of an utterance sentence by voice. The processor is configured to generate the data of the utterance sentence for inquiring about conditions of the speaker and to output the data of the utterance sentence by voice during execution of the voice interaction with the speaker.

Abstract: Methods, devices, and systems for processing audio information are disclosed. An exemplary method includes receiving an audio stream. The audio stream may be monitored by a low power integrated circuit. The audio stream may be digitized by the low power integrated circuit. The digitized audio stream may be stored in a memory, wherein storing the digitized audio stream comprises replacing a prior digitized audio stream stored in the memory with the digitized audio stream. The low power integrated circuit may analyze the stored digitized audio stream for recognition of a keyword. The low power integrated circuit may induce a processor to enter an increased power usage state upon recognition of the keyword within the stored digitized audio stream. The stored digitized audio stream may be transmitted to a server for processing. A response received from the server based on the processed audio stream may be rendered.

Abstract: A computerize method for voice authentication of a customer in a self-service system is provided. A request for authentication of the customer is received and the customer is enrolled in the self-service system with a text-independent voice print. A passphrase from a plurality of passphrases to transmit to the customer is determined based on comparing each of the plurality of passphrases to a text-dependent or text-independent voice biometric model. The passphrase is transmitted to the customer, and when the customer responds, an audio stream of the passphrase is received. The customer is authenticated by comparing the audio stream of the passphrase against the text-independent voice print. If the customer is authenticated, then storing the audio stream of the passphrase and the topic of the passphrase.

Abstract: A voice-activated camera system for a computing device. The voice-activated camera system includes a processor, a camera module, a speech recognition module and a microphone for accepting user voice input. The voice-activated camera system includes authorized for only a specific user's voice, so that a camera function may be performed when the authorized user speaks the keyword, but the camera function is not performed when an unauthorized user speaks the keyword.

Abstract: Identification information is stored in association with a user. The identification information is received from an identification device, speech data is received from a speech input and output device, and speech identification information is recognized from the speech data. At this time, when the identification information and the speech identification information match each other, the user corresponding to the identification information is permitted to log in.

Abstract: A smart and scalable dementia assistant device is provided that converses with a patient in voices familiar with the patient. It utilizes content learned from the patient and content provided by family, friends, caregivers, and doctors, and autonomously adjusts conversations based on the changing state of the patient's dementia state. The device autonomously controls IoT devices (e.g. doors, elevators, tvs, medical dispensers) to help and assist the dementia patient using oral and IoT sensors.

Abstract: Detecting liveness of a speaker comprises: generating an ultrasound signal; receiving an audio signal comprising a reflection of the ultrasound signal; using the received audio signal comprising the reflection of the ultrasound signal to detect the liveness of a speaker; monitoring ambient ultrasound noise; and adjusting the operation of a system receiving the audio signal, based on a level of the reflected ultrasound and the monitored ambient ultrasound noise. The method can be used in a voice biometrics system, in which case detecting the liveness of a speaker comprises determining whether a received speech signal may be a product of a replay attack. The operation of the voice biometrics system may be adjusted based on a level of the reflected ultrasound and the monitored ambient ultrasound noise.

Abstract: A driving assistance device includes a gaze detection unit that detects a gaze of a driver of a vehicle, an imaging selection unit that selects a first imaging unit corresponding to a direction of the gaze from among a plurality of first imaging units that image the periphery of the vehicle, and a display control unit that processes first captured image data obtained by imaging using the selected first imaging unit on the basis of second captured image data obtained by imaging using a second imaging unit that images the front of eyes of the driver and displays an image based on the first captured image data after the processing on a transmission type display unit fixed at the front of the eyes of the driver of the vehicle.

Abstract: Systems and methods are described herein for controlling mobile devices with user equipment devices. A user equipment device may detect a mobile device is within a predetermined proximity to the user equipment device by establishing a bi-directional communication. The user equipment device may determine an input/output option of the mobile device. The input/output option may be a microphone, a cellular Internet connection, a camera, or any other sensor and/or actuator of the mobile device. The user equipment device may generate for display an icon corresponding to the input/output option of the mobile device.

Abstract: Detecting a replay attack on a voice biometrics system comprises: receiving a speech signal; generating an ultrasound signal; detecting a reflection of the generated ultrasound signal; detecting Doppler shifts in the reflection of the generated ultrasound signal; and identifying whether the received speech signal is indicative of the liveness of a speaker based on the detected Doppler shifts. Identifying whether the received speech signal is indicative of liveness based on the detected Doppler shifts comprises determining whether the detected Doppler shifts correspond to a speech articulation rate.

Abstract: Methods, systems, apparatus, including computer programs encoded on computer storage medium, to facilitate language independent-speaker verification. In one aspect, a method includes actions of receiving, by a user device, audio data representing an utterance of a user. Other actions may include providing, to a neural network stored on the user device, input data derived from the audio data and a language identifier. The neural network may be trained using speech data representing speech in different languages or dialects. The method may include additional actions of generating, based on output of the neural network, a speaker representation and determining, based on the speaker representation and a second representation, that the utterance is an utterance of the user. The method may provide the user with access to the user device based on determining that the utterance is an utterance of the user.

Abstract: Healthcare object management systems and methods are presented. Health Object Identifiers (HOI) representative of healthcare object (e.g., medical records, data, etc.) can be derived from biometric data associated with a patient. HOIs can resolve to a network location where the healthcare object can be accessed. Biometric data, or other types of object data, can be exchanged among computing devices and the healthcare object. On exemplary use of HOIs includes self assembling personal data records as sensor data is obtained.

Abstract: Examples of creating a device identifier that are based upon hardware components of a client device are discussed. An inaudible or high frequency reference audio sample is played. Audio capture is initiated using the microphone system. A sensor-based device identifier can be generated from the captured audio due the manufacturing variances in the hardware components used for the speaker and microphone systems.

Abstract: An authentication system prevents leakage of a key-reading speech during user authentication based on the key-reading speech of a user reading an authentication key. For each user ID, a storage stores a voiceprint of a user in association with a recorded sound including speech spoken previously by the user. A specifier specifies the user ID of a user attempting to receive authorization. An outputter outputs a masking sound that includes the recorded sound recorded in association with the specified user ID. An acquirer acquires a key-reading speech of the user reading the authentication key and the output masking sound. A remover acquires a second sound by removing the masking sound from the acquired first sound. A determiner determines whether the user has authority pertaining to the specified user ID based on the acquired second sound.

Abstract: An identity verification method performed at a terminal includes playing in an audio form action guide information including mouth shape guide information selected from a preset action guide information library at a speed corresponding to the action guide information, and collecting a corresponding set of action images within a preset time window; performing matching detection on the collected set of action images and the action guide information, to obtain a living body detection result indicating whether a living body exists in the collected set of action images; according to the living body detection result that indicates that a living body exists in the collected set of action images: collecting user identity information and performing verification according to the collected user identity information, to obtain a user identity information verification result; and determining the identity verification result according to the user identity information verification result.

Abstract: Detecting liveness of a speaker comprises: generating an ultrasound signal; receiving an audio signal comprising a reflection of the ultrasound signal; using the received audio signal comprising the reflection of the ultrasound signal to detect the liveness of a speaker; monitoring ambient ultrasound noise; and adjusting the operation of a system receiving the audio signal, based on a level of the reflected ultrasound and the monitored ambient ultrasound noise. The method can be used in a voice biometrics system, in which case detecting the liveness of a speaker comprises determining whether a received speech signal may be a product of a replay attack. The operation of the voice biometrics system may be adjusted based on a level of the reflected ultrasound and the monitored ambient ultrasound noise.

Abstract: The present invention relates to methods, apparatus and systems for authentication of a user based on ear biometric data, and voice biometric data or other authentication data. The ear biometric data may be combined with voice biometric data or with a security question and response.

Abstract: Detecting a replay attack on a voice biometrics system comprises: receiving a speech signal; generating an ultrasound signal; detecting a reflection of the generated ultrasound signal; detecting Doppler shifts in the reflection of the generated ultrasound signal; and identifying whether the received speech signal is indicative of the liveness of a speaker based on the detected Doppler shifts. Identifying whether the received speech signal is indicative of liveness based on the detected Doppler shifts comprises determining whether the detected Doppler shifts correspond to a speech articulation rate.

Abstract: A method, a computer system, and a computer program product for detecting voice commands. Audio is recorded by the computer system to form a recorded audio. The computer system then determines whether a voice command spoken by a first person is present in the recorded audio. If the voice command is present in the recorded audio, the computer system determines whether the voice command is directed to a second person by the first person. If the voice command is not being directed to the second person, the computer system processes the voice command, wherein processing of the voice command occurs without a wake word.

Abstract: A method for user authentication based on keystroke dynamics is provided. The user authentication method includes receiving a keystroke input implemented by a user; separating a sequence of pressed keys into a sequence of bigrams having bigram names simultaneously with the user typing free text; collecting a timing information for each bigram of the sequence of bigrams; extracting a feature vector for each bigram based on the timing information; separating feature vectors into subsets according to the bigram names; estimating a GMM user model using subsets of feature vectors for each bigram; providing real time user authentication using the estimated GMM user model for each bigram and bigram features from current real time user keystroke input. The corresponding system is also provided. The GMM based analysis of the keystroke data separated by bigrams provides strong authentication using free text input, while user additional actions (to be verified) are kept at a minimum.

Abstract: A process generates, at a computer-implemented service provider platform, a simulated user request for a service. Further, the process sends, from the computer-implemented service provider platform to a computing device associated with an agent, the simulated user request for a service. Additionally, the process performs, with a processor at the computer-implemented service provider platform, an assessment of agent responsiveness to the simulated user request for the service. Finally, the process automatically generates, with the processor at the computer-implemented service provider platform, one or more actions based on the assessment.

Abstract: Methods and systems are provided for generating automatic program recommendations based on user interactions. In some embodiments, control circuitry processes verbal data received during an interaction between a user of a user device and a person with whom the user is interacting. The control circuitry analyzes the verbal data to automatically identify a media asset referred to during the interaction by at least one of the user and the person with whom the user is interacting. The control circuitry adds the identified media asset to a list of media assets associated with the user of the user device. The list of media assets is transmitted to a second user device of the user.

Abstract: Implementations directed to providing a computer-implemented system for performing an action with a robot comprising receiving command information indicating a command related to performance of an action with a robot, identifying state information for a plurality of active routines that are actively running for the robot, the state information indicating a state for each of the active routines, determining contextual information for the command based on the accessed state information for the plurality of active routines, selecting one of the active routines as a handling routine to service the command based on the contextual information, determining an output module of the robot to perform the action based on the state of the handling routine and the contextual information, and executing one or more instructions to perform the action with the output module.

Abstract: A method, system and computer program product includes detecting a volume level for audio input of a first user in a multi-user conference call, and automatically adjusting a volume level for a second user receiving audio output of the first user based on at least one of preferences of the second user, historic data between the first and the second user, and geographic characteristics of the audio input of the first user.

Abstract: An electronic apparatus is provided. The electronic apparatus includes a memory, a microphone and a processor configured to compare a volume of a voice input through the microphone and a standard voice volume stored in the memory, corresponding to a space in which the electronic apparatus is located, and identify whether to perform a voice recognition on the voice based on the comparison.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for adjusting an eligibility value for transmitting a digital component. In one aspect, a computing system includes a server for identifying opportunities to transmit digital components to client devices. The server determines a first probability of a given outcome occurring following user interaction with the digital component when the digital component is transmitted to the client device. The server determines a second probability of the given outcome occurring if the digital component is not transmitted to the client device. The server generates an outcome incrementality factor for the digital component, including determining a ratio of the first probability relative to the second probability, and triggers adjustment of an eligibility value based on the outcome incrementality factor. The server then controls transmission of the digital component to the client device using the adjusted eligibility value.

Abstract: Techniques for performing command processing are described. A system receives, from a device, input data corresponding to a command. The input data may originate as audio data, as text data, or as other data. The system determines NLU processing results corresponding to the input data. The NLU processing results may be associated with multiple speechlets. The system also determines NLU confidences for the NLU processing results for each speechlet. The system sends NLU processing results and an indication to provide potential results to a portion of the multiple speechlets, and receives potential results from the portion of the speechlets. The system also receives indications whether the speechlets need to be re-called if the speechlets are selected to execute with respect to the command. The system ranks the portion of the speechlets based at least in part on the NLU processing results as well as the potential results provided by the portion of the speechlets.

Abstract: A system and method for enrolling a speaker in a speaker authentication and identification system (AIS), the method comprising: generating a user account, the user account comprising: a user identifier based on one or more metadata elements associated with an audio input received from an end device; generating a first i-vector from an audio frame of the audio input, a trained T-matrix, and a Universal Background Model (UBM), wherein the first i-vector generation comprises an optimized computation; and associating the user account with the first i-vector.

Abstract: A keyword confirmation method and apparatus are provided. A keyword confirmation method includes: obtaining first audio data, the first audio data being recognized as a keyword; obtaining a pronunciation similarity probability of a similar pronunciation unit corresponding to at least one fragment of the first audio data and second audio data; determining that multiple contiguous silence fragments exist in second audio data contiguous in time with the first audio data; utilizing the silence probability, as well as a pronunciation similarity probability corresponding to fragment(s) of the first audio data and/or a pronunciation similarity probability corresponding to fragment(s) of the second audio data, evaluating whether the second audio data is silence; and confirming the first audio data as an effective keyword.

Abstract: Systems and processes for proactive assistance based on dialog communication between devices are provided. In one example process, while voice communication between an electronic device and a second electronic device is established, a stream of audio data associated with the second electronic device can be received. In response to detecting a user input, a text representation of speech contained in a portion of the stream of audio data can be generated. The process can determine whether the text representation contains information corresponding to one of a plurality of types of information. In response to determining that the text representation contains information corresponding to one of a plurality of types of information, one or more tasks based on the information can be performed.

Abstract: A computer-implemented method for classifying an input data set within a data category using multiple data representation modes. The method includes identifying at least a first data representation source mode and a second data representation source mode; classifying the at least first data representation source mode via at least a first data recognition tool and the at least second data representation source mode via at least a second data recognition tool, the classifying including allocating a confidence factor for each data representation source mode in the data category; and combining outputs of the classifying into a single output confidence score by using a weighted fusion of the allocated confidence factors.