Apparatus and Methods to Update a Language Model in a Speech Recognition System
The technology of the present application provides a method and apparatus to allow for dynamically updating a language model across a large number of similarly situated users. The system identifies individual changes to user profiles and evaluates the change for a broader application, such as, a dialect correction for a speech recognition engine, as administrator for the system identifies similarly situated user profiles and downloads the profile change to effect a dynamic change to the language model of similarly situated users.
Latest NVOQ INCORPORATED Patents:
- Recognition of artifacts in computer displays
- Systems, methods, and apparatus for real-time dictation and transcription with multiple remote endpoints
- Systems and methods for a wireless microphone to access remotely hosted applications
- Apparatus and methods for dynamically changing a speech resource based on recognized text
- Systems and methods for a wireless microphone to access remotely hosted applications
None.
REFERENCE TO CO-PENDING APPLICATIONS FOR PATENTNone.
BACKGROUND1. Field
The technology of the present application relates generally to speech recognition systems, and more particularly, to apparatuses and methods to update a language model associated with speech recognition systems for a number of similarly situated users dynamically rather than statically.
2. Background
The primary means for communication between people is speech. Since the early 1980s, significant progress has been made to allow people to interface with machines using speech through interfaces such as speech to text engines and text to speech engines. The former converts speech to a machine (and user) readable format; the later converts machine readable code to audio signals for people to hear.
Early speech to text engines operated on a theory of pattern matching. Generally, these machines would record utterances spoken by a person, convert them into phoneme sequences and match these sequences to known words or phrases. For example, the audio of “cat” might produce the phoneme sequence “k ae t”, which matches the standard pronunciation of the word “cat”. Thus, the pattern matching speech recognition machine converts the audio to a machine readable version “cat.” Similarly, a text to speech engine would read the word “cat”, convert it into a sequence of phonemes, each of which have a known audio signal, and, when concatenated (and appropriately shaped) produce the sound of “cat” (phonetically: “k ae t”). Pattern matching machines, however, are not significantly robust. Generally, pattern matching machines either operate with a high number of recognizable utterances for a limited number of users or operate with a higher number of users but a more limited number of recognizable utterances.
More recently, speech recognition engines have moved to a continuous or natural language speech recognition. The focus of natural language systems is to match the utterance to a likely vocabulary and phraseology and determine how likely the sequence of language symbols would appear in speech. Determining the likelihood of a particular sequence of language symbols is generally called a language model. The language model provides a powerful statistical model to direct a word search based on predecessor words for a span of n words. Thus, the language model will use probability and statistically more likely words for similar utterances. For example, the words “see” and “sea” are pronounced substantially the same in the United States of America. Using a language model, the speech recognition engine would populate the phrase: “Ships sail on the sea” correctly because the probability indicates the word “sea” is more likely to follow the earlier words “ship” and “sail” in the sentence. The mathematics behind the natural language speech recognition system are conventionally known as a hidden Markov model. The hidden Markov model is a system that predicts the next state based on the previous states in the system and the limited number of choices available. The details of the hidden Markov model are reasonably well known in the industry of speech recognition and will not be further described herein.
Generally speaking, speech recognition engines using natural language have users register with an account. More often than not, the speech recognition downloads the recognition application and database to the local device making it a fat or thick client. In some instances, the user has a thin client where the audio is routed to a server that has the application and database that allows speech recognition to occur. The client account provides an audio profile and language model that is tuned to a particular user's voice and speech. The initial training of a natural language speech recognition engine generally uses a number of “known” words and phrases that the user dictates. The statistical algorithms which map audio signals to phonemes are modified to match the user's voice. Subsequent training of the speech recognition engine may be individualized by corrections entered by a user to transcripts when the transcribed speech is incorrect. While any individual user's speech recognition engine is effectively trained to the individual, the training of the language model is potentially inefficient in that common phrases and the like for similarly situated users must be input individually for each installed engine and/or each user. Moreover, changes in language modeling that a single user identifies that would be useful for multiple similarly situated users cannot be propagated through the speech recognition system without a new release of the application and database.
Thus, against this background, it is desirable to develop improved apparatuses and methods to update a language model in a speech recognition system.
SUMMARYTo attain the advantages and in accordance with the purpose of the technology of the present application, an apparatus and method is provided that receives or pulls changes to user profiles and evaluates the individual changes for whether the change would have application over a broader group of similarly situated users of the system. On identification of other similarly situated users, the system and method pushes the change to the language models to the destinations associated with the similarly situated users. By pulling the improvements to the language model for an individual and pushing the improvements across a large number of users, the user profiles for each of the similarly situated users are dynamically updated effectively dynamically changing the language model for the users.
In certain aspects, the profile change is identified by a query or poll from an administrator. In other aspects of the technology, as the user profile is changed, the change is transmitted to the administrator.
The technology in some cases provides an option for updating a user profile by providing a new word and pronunciation of the new word to change a user profile, or by providing a shortcut for a particular word or phrase, or a substitution which formats an existing speech recognition to a desired appearance. In other cases, the user profile may provide for macros or scripts to invoke or activate applications or system navigation based on certain words or commands.
In some embodiments, the technology determines similarly situated user profiles by identifying common attributes in user profiles. The common attributes may include, for example, tags to identify user profiles, common hierarchy in an organization, similar geographical locations, similar age groups, common language models and the like may all be attributes used to determine similarly situated users.
The foregoing and other features, utilities and advantages of the invention will be apparent from the following more particular description of a preferred embodiment of the invention as illustrated in the accompanying drawings.
Various examples of the technology of the present application will be discussed with reference to the appended drawings. These drawings depict only illustrative examples of the technology and are not to be considered limiting of its scope, which is defined by the claims.
The technology of the present application will now be explained with reference to the figures. While the technology of the present application is described with relation to a speech recognition system using natural language or continuous speech recognition, one of ordinary skill in the art will recognize on reading the disclosure that other configurations are possible. Moreover, the technology of the present application will be described with reference to particular discrete processors, modules, or parts, but one of ordinary skill in the art will recognize on reading the disclosure that processors may be integrated into a single processor or server or separated into multiple processors or servers. Moreover, the technology of the present application will be described generically and portions of the present application may be loaded onto a particular user's device (fat or thick client) or hosted by a server that is accessed by the device (thin client). Additionally, the technology of the present application is described with regard to certain exemplary embodiments. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments. All embodiments described herein should be considered exemplary unless otherwise stated.
Referring first to
Dictation manager 104 is connected to one or more dictation servers 110 by a second network connection 112. Second network connection 112 may be the same as first network connection 106, which may similarly be a cloud system. Dictation manager 104 and dictation server(s) 110 may be a single integrated unit connected by a bus, such as a PCI or PCI express protocol. Each dictation server 110 incorporates or accesses a natural language or continuous speech transcription engine as is generally understood in the art. In operation, the dictation manager 104 receives an audio file or stream for transcription from a client device 102. In this description, we use the term audio for both a discrete file as well as a stream because computer programming interfaces treat files and streams very similarly. In practice, an audio file may be transcribed in one step and the resulting text returned to the supplier of the audio, and a stream may be transcribed as audio arrives and the transcription returned incrementally to the caller as the audio is transcribed. Dictation manager 104 selects an appropriate dictation server 110, using conventional load balancing or the like, and transmits the audio to the dictation server 110. The dictation server 110 would have a processor that uses the appropriate algorithms to transcribe the speech using a natural language or continuous speech to text processor. In most instances, the dictation manager 104 uploads a user profile to the dictation server 110. The user profile, as explained above, modifies the language model of the speech to text processer for the user's particular dialect, speech patterns, or the like based on conventional training techniques as are generally known in the industry and not explained herein. The audio, once transcribed by the dictation server 110, is returned to the client device 102 as a transcription or data file. Alternatively, the transcription or data file may be saved for retrieval by the user at a convenient time and place.
Referring now to
Referring to
Dictation manager 104 would accept requests from client device 102 and direct the audio to dictation servers 110 in any of a number of conventional manners. One exemplary methodology is provided in U.S. Pat. No. 7,185,094, issued on Feb. 27, 2007, to Marquette et al., titled Media Session Framework Using a Control Module to Direct and Manage Application and Service Servers, the disclosure of which is incorporated herein by reference as if set out in full. Load balancing between the dictation servers 110 may be accomplished using conventional technology such as the technology associated with U.S. Pat. No. 8,105,689, issued Apr. 3, 2012, to Beach et al., titled Distributed Dictation/Transcription System, the disclosure of which is incorporated herein by reference as if set out in full. Using the methodology, dictation manager 104, for example, would receive a request for dictation services from client device 102. Dictation manager 104 would send a service request to the dictation servers 110 until one of the dictation servers 110 sends back an acceptance of the service request. Dictation server 110 has the user's profile uploaded. Audio would then be streamed from client device 102 to the dictation manager 104 which would in turn stream the audio to dictation server 110. Dictation server 110 would then transcribe the dictation by using the speech transcription engine's language model modified by the user's profile and associated with dictation server 110, and the transcription results would be sent back to client device 102 through the dictation manager 104 as textual data. Alternatively or in combination, dictation manager 104 may cause the textual data to be stored for later retrieval by the user. Also, once the handshake between client device 102 and dictation server 110 is established, dictation manager 104 may be removed from the communication link.
Referring now to
As can be appreciated by the above, while the above ability to fine tune a particular user's profile is functional, it is not efficient for propagating changes to similarly situated users. Using the above methodology, each individual user would need to recognize the utility of the potential new word, revised word, shortcut, or the like and make the same or a similar change to his profile. This may result in multiple variations on the same improvements, which could lead to problems, inconsistencies, and the like. Moreover, each new client would receive a language model that is the original language model without any of the potentially beneficial modifications. Thus, it is desirable for an administrator of the dictation system to monitor new words, training changes, shortcuts, or the like (generically referred to as “Profile Changes”) to a user profile (generically referred to as the originating user profile) to determine whether any originating user profiles have Profile Changes that would be useful to a group of similarly situated users.
With reference to
Referring now to
While the above
Another group includes a group of engineers under a project leader. As can be appreciated, any particular user can be in multiple groups. A direct report or tree structure as described above is a reasonably straightforward way to group users to identify similarly situated users. However, in today's cloud systems, it is sometimes difficult to identify direct hierarchies. Another way to group similarly situated users may include, for example, assigning a tag to the user. Thus, when pushing updates, the data administrator 502 would only cause the updates to be pushed to accounts having a particular tag. For example, if a user account includes a medical specialty language model, such as, for example, a neurosurgical language model, the account may be tagged. When any dynamic changes to the neurosurgical language model are authorized, the update is only pushed to the tagged users and not necessarily to all users. Again, any particular user may have several tags. In many instances, the groups can be established automatically by identifying features of the user. As shown above, the user account for the medical specialty may automatically tag a user. Other occupational groupings are possible as well. For example, iPhone application programmers may be grouped; Bloomingdale's buyers may be grouped, etc. Other groupings may include other indicia of the users in the group, such as regional (by a Postal Zip Code, for instance, in the United States) as users in the same region likely have similar language patterns, age, hobbies, activity groups, social media groups, etc. to name but a few indicia that may act to signal a potential grouping. The above are but some exemplary forms of attributes that may be used to identify similarly situated users, which are generically referred to as attributes.
The dynamic updates are described with reference to vocabulary, sentence structures, and shortcuts for a language model of a speech to text or text to speech engine. The rapid and dynamic propagation of updates does not need to be limited to language models. In certain aspects of the technology of the present application, a user in a group may develop a command, a key stroke application, a macro, or a script, etc. that facilitate common actions. Similar to the above, an administrator or manager associated with the group of users may be alerted to the macro or script and evaluate the efficiencies garnered. The administrator or manager may push the macro or script to similarly situated users that have similar needs and will experience similar efficiencies. In one example, users may frequent a plurality of news pages in a particular order for updated, online news information. Thus, a command, such as, for example, “News” may navigate the user to a particular Internet news page by opening the systems network browser, entering the URL, and displaying the information. In some aspects, the command “News” may take the user to multiple pages based on the URL after a certain delay at any particular URL or other unique address for the information. In other situations, the command may have a number, of similar commands, such as, for example, the command “News1” displays www.cnn.com, “News2” displays www.foxnews.com, “News3” displays www.msnbc.com, etc. Rather than an audio command as contemplated above, the same command could be associated with a particular key stroke, icon click, or the like. Each of these items may be pulled by the administrator or manager from one or more users in a group and subsequently pushed to all the similarly situated users.
While described above as the administrator pushing the dynamic update to the user, the administrator may instead publish the available updates pulled from the various users. The other similarly situated users may elect to subscribe or not to the update. Additionally, whether pushed by the administrator or accepted by the user, such a dynamic update may be provided with an explanation or other help regarding the dynamic update. Such help may be that XYZ tool is now available, a warning that the audio phrase “widget” will now launch a network browser and navigate to a particular URL, a description of a shortcut, or a vocabulary change.
Referring now to
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The above identified components and modules may be superseded by new technologies as advancements to computer technology continue.
The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method performed on at least one processor for dynamically updating a group of similarly situated user profiles, the method comprising the steps of:
- receiving over a communication link at least one profile change to an originating user profile entered by a user at a client workstation;
- determining whether the at least one profile change is applicable to at least one similarly situated user profile;
- transmitting over the communication link the at least one profile change to a destination user profile to automatically update the destination, wherein the user profile for the group of similarly situated user profiles is updated dynamically based on the at least one profile change.
2. The method of claim 1 wherein the step of receiving over a communication link at least one profile change comprises:
- polling a plurality of user profiles to determine whether the user profiles have received at least one user initiated profile change; and
- pulling the at least one user initiated profile change to be received.
3. The method of claim 2 wherein the step of polling comprises the step of querying for profile changes.
4. The method of claim 1 wherein the step of transmitting over the communication link the at least one profile change to the destination user profile includes the step of displaying a user profile update to alert the user of the at least one profile change.
5. The method of claim 1 wherein the destination profile comprises a plurality of user profiles similarly situated to the originating user profile.
6. The method of claim 5 further comprising the step of determining a first group of user profiles that are similarly situated to the originating user profile.
7. The method of claim 6 wherein the step of determining a first group of user profiles that are similarly situated to the originating user profile includes identifying common attributes.
8. The method of claim 7 wherein the common attributes are tags associated with the plurality of user profiles.
9. The method of claim 7 wherein the common attributes comprise a hierarchy of the plurality of user profiles in an organization.
10. The method of claim 7 wherein the common attribute of the plurality of user profiles comprises a geographical location.
11. The method of claim 7 wherein the common attribute of the plurality of user profiles comprises the language model.
12. The method of claim 1 wherein the at least one profile change is a change to a language model for a speech recognition system.
13. The method of claim 1 wherein the at least one profile change is selected from a group of profile changes consisting of: shortcuts, macros, hot key sequences upload, commands, navigation, or a combination thereof.
14. The method of claim 12 further comprising the steps of originating the profile change to the language model of a user profile, wherein the originating step comprising the steps of:
- inputting a new word to a language model;
- inputting a pronunciation of the new word to the language model; and
- training the originating user profile based on the new word and new pronunciation.
15. The method of claim 12 further comprising the steps of originating the profile change to the language model of a user profile, wherein the originating step comprising the steps of:
- inputting a shortcut to a language model; and
- inputting transcription text to be transcribed when the shortcut is spoken; and
- training the language model.
16. The method of claim 12 further comprising the steps of originating the profile change to the language model of a user profile, wherein the originating step comprising the steps of:
- correcting a transcription text with corrected text; and
- training the originating user profile based on the correction.
17. An apparatus for dynamically updating a plurality of user profiles for similarly situated users comprising:
- an administrator workstation coupled to a network and configured to receive at least one modification to at least a first user profile associated with a group of user profiles;
- the administrator workstation comprising a processor configured to identify from the group of user profiles at least one second user profile that has at least one common attribute as the first user profile; and
- the administrator workstation configured to modify the second user profile with the modification from the first user profile based on the common attribute, the language model of the second user profile is dynamically updated by the modification.
18. The apparatus according to claim 17 wherein the administrator workstation is configured to poll the group of user profiles for the at least one modification.
19. The apparatus according to claim 17 wherein the administrator workstation dynamically updates the second user profile without action by the user of the second user profile.
20. The apparatus according to claim 19 wherein the administrator workstation transmits an update summary to the workstation of the second user profile wherein the user at the workstation is alerted to the dynamic update.
21. The apparatus according to claim 17 wherein the administrator workstation transmits an update summary of the at least one modification to the workstation of the second user profile and dynamically updates the second user profile on acceptance by the second user.
22. The apparatus according to claim 17 wherein the administrator workstation determines the at least one common attribute by evaluating identification tags linked to each user profile of the group of user profiles.
Type: Application
Filed: Jun 11, 2012
Publication Date: Dec 12, 2013
Patent Grant number: 9489940
Applicant: NVOQ INCORPORATED (Boulder, CO)
Inventor: Charles Corfield (Boulder, CO)
Application Number: 13/493,761
International Classification: G10L 15/04 (20060101); G10L 19/00 (20060101);