SYSTEM AND METHOD FOR CONDUCTING A SEARCH USING A WIRELESS MOBILE DEVICE
A method and system are provided by which a wireless mobile device takes a vocally entered query and transmits it in a text message format over a wireless network to a search engine; receives search results based on the query from the search engine over the wireless network; and displays the search results.
Latest Nuance Communications, Inc. Patents:
- System and method for aligning ASR model weights with NLU concepts
- Automated clinical documentation system and method
- Using environment and user data to deliver advertisements targeted to user interests, e.g. based on a single command
- System and method for array data simulation and customized acoustic modeling for ambient ASR
- Automated clinical documentation system and method
The invention relates to a wireless mobile device based search system, and specifically to submitting queries on a wireless mobile device.BACKGROUND OF THE INVENTION
The need for faster and easier access to information has never been greater in this internet era. The popularization of wireless mobile devices has made it possible to obtain any information wherever and whenever. Currently, most of the mobile devices on the market have browser capability that provides internet access. But navigating through the various links and web pages to find the right information is not an easy task, especially on a small screen of a mobile device. In addition, only a relatively small percentage of the all websites on the internet offers a Wireless Application Protocol (WAP) version of themselves for display on wireless mobile devices.
Text messaging based information retrieval systems such as the Google™ Short Message Service eliminates the need for web pages and browsers on a mobile device by allowing users to send queries and receive answers as text messages. They provide a more dynamic and efficient way to get the desired information, but a major deficiency of these systems is that many of the mobile devices, such as a cellular phone, are tiny and have tiny numeric keypads and screens that make it very difficult to enter text for a Short Message Service (SMS) message. The only text input method available on most involves pressing numeric keys once, twice or three times to select one of the alphabetical characters associated with it. This makes for very slow input, and discourages users from utilizing features like the Google™ SMS.
The present invention addresses these problems.SUMMARY OF THE INVENTION
The invention relates to a system and method for obtaining and displaying information content from sources external on a wireless mobile device based on a spoken query.
In one aspect, a method of obtaining query results on a mobile device is provided. In one embodiment a user is presented with a list of available queries on a mobile device such as a cellular phone. The query list can be customized based on the current location of the user or in response to a prior search history stored on the mobile device. After the user selects a query verbally, the mobile device prompts the user for additional information required to complete the query. The user is able to provide the needed information verbally using speech recognition software. In one embodiment once the query is complete, it is automatically converted to a SMS format, addressed to a search engine, and transmitted over the mobile phone network to a search engine. Search results based on the query are sent back to the mobile device also in SMS format from the search engine. Once received, the search results are reformatted by the mobile device and displayed on the screen of the mobile device.
In another embodiment, the search results are sent to an intermediary server on the mobile phone network. In addition to formatting, in another embodiment the intermediary server also processes advertisements and appends the advertisements to the search results. In various embodiments, the added advertisements originate at the search engine; at an advertisement provider on the network; or directly from a particular vendor of goods and services. The formatted search results and advertisements are then forwarded to the mobile device for display. In yet another embodiment, upon receiving the search results, the mobile device uses text to speech synthesis to generate a voice speaking the received search results in addition to displaying them on the screen of the mobile device.
In another aspect, the invention relating to a system for obtaining query results on a mobile device is provided. In one embodiment the system includes a wireless mobile device, a wireless network, and a search engine. According to this embodiment, the wireless mobile device includes a voice input, a speech to text converter, a text to text protocol converter, a wireless transmitter, and a wireless receiver. The search engine includes a search engine receiver and a search engine transmitter. The voice input is converted to a message format and transmitted by the wireless transmitter through the wireless network to the search engine. The search engine then transmits search results to the wireless receiver for display.
In another aspect, the present invention relates to a method of monetizing a search performed by a wireless mobile device. The method includes the steps of speaking a query to the wireless mobile device; transmitting the query from the wireless mobile device over a wireless network to a search engine; receiving search results by the wireless mobile device from the search engine over the wireless network; receiving advertisements by the wireless mobile device over the wireless network; formatting the search results and the advertisements; displaying the search results and the advertisements on the wireless mobile device; and recovering a monetary remuneration from the originator of the advertisements in response to the query. In different embodiments, the originator may be the search engine, or an advertisement provider, or a particular vendor of goods and services.
These embodiments and other aspects of this invention will be readily apparent from the detailed description below and the appended drawings, which are meant to illustrate and not to limit the invention, and in which:
The present invention will be more completely understood through the following detailed description, which should be read in conjunction with the attached drawings. In this description, like numbers refer to similar elements within various embodiments of the present invention. Within this detailed description, the claimed invention will be explained with respect to preferred embodiments. However, the skilled artisan will readily appreciate that the methods and systems described herein are merely exemplary and that variations can be made without departing from the spirit and scope of the invention.
In various embodiments the wireless mobile device 100 is a cellular phone, a PDA, a BlackBerry®, or any other similar device equipped with voice recognition technology that allows a user to input information verbally. The wireless network 102 can be any existing telecommunication network to which the wireless mobile device 100 is connected. For example, if the wireless mobile device 100 is a cellular phone, the wireless network 102 is the mobile phone network providing service to that particular phone. While connected to the wireless network 102, the wireless mobile device 100 is able to send and receive information to other devices that have access to the same wireless network 102.
After receiving a query over the wireless network 102, the search engine 104 generates search results and sends the search results back over the wireless network 102 to the wireless mobile device 100. In addition to the results of the query, the search engine 104 is also able to send advertisements to the wireless mobile device 100. The search engine 104 may have the advertisements stored locally or may request the advertisements for a third party advertisement provider 108. Alternatively the search engine 104 may redirect the third party advertisement provider 108 to send the advertisements directly to the wireless mobile device 100.
Before the search results are displayed on the wireless mobile device 100, they are formatted to fit on the small screen of such device. Such formatting may take place on the wireless mobile device 100 itself, at the intermediary site 106, at the third party advertisement provider 108, or the search engine 104. In the process of formatting, additional information such as advertisements or other information can be also added to the search results. The additional information may or may not originate at the location where the formatting occurs. The additional information may be obtained concurrently with the search results if the information comes from the search engine 104, or may be already stored locally at the formatting site. The additional information is appended to the search results and transmitted over the wireless network 102 to the wireless mobile device 100. In one embodiment, the additional information also serves as the subject of future queries by embedding a link to another function. For example, the link could be a map query function that provides directions to a destination returned as a part of the search results. The intermediary site 106 and the third party advertisement provider 108 may communicate with each other and with the search engine 104 and the wireless mobile device 100 over the wireless network 102.
Formatting may also occur on the wireless mobile device 100 itself. For example, advertisements which are pre-stored on the wireless mobile device 100 may also be formatted along with the search results. In addition, pre-formatted advertisements provided by external entities may be sent to the wireless mobile device 100 periodically or in response to the occurrence of certain events. For example, the issuing of a query or the movement of the wireless mobile device 100 into a particular area may trigger a fresh download of advertisements.
After receiving the search results, either directly from the search engine 104 or from the intermediary site 106, the wireless mobile device 100 displays the search results on its screen. In one embodiment, the wireless mobile device uses a text to speech synthesis process to generate a voice speaking the search results.
The provider of the disclosed technology may collect a license fee for providing an advertisement platform from the various advertisement suppliers including the search engine 104, the intermediary site 106, and the third party advertisement provider 108. In addition, revenue generated by each of the suppliers from selling advertisements to its customers is shared with the provider of the disclosed technology in return for utilizing the technology to display advertisements with search results on the wireless mobile device 100.
The wireless mobile device also includes a wireless receiver 118 which receives the results of the query over the network 102 and which transmits the results to a formatter 120 for display on a display 122. In another embodiment the results are transmitted additionally or optionally to a text to speech synthesizer 124 for audible output 126.
The speech based input mechanism of the wireless mobile device 100 generally obviates the need for numerous key presses that would otherwise be necessary to address the SMS message, type in its contents in the proper query format expected by the search engine 104, and to organize the multiple SMS messages received from a single query. Since nothing is typed, the user is not aware SMS messaging is being used to issue queries and receive results. The voice recognition and speech synthesis technology provide a seamless way to input a query without the need to use the keypad or look at the display of the wireless mobile device 100.
In the case where the system does not recognize the response to a request for more information, the user can type in the requested information. In this embodiment, the formatting of the query utilizes both the input from the voice recognition portion of the system and the input from the keypad.
In operation, the wireless mobile device 100 receives via its voice input 110 a query spoken by a user. The spoken query is converted by the speech to text converter 112 to text. The text is then converted by the text to text protocol converter 114 to a text message. An existing protocol for transmitting such a text message is SMS, which transmits short text messages to and from a mobile device. An alternative protocol is Multimedia Message Service (MMS) which transmits graphics, sound files, and video clips in addition to plain text. The text message is transmitted by the wireless transmitter 116 to an external wireless network 102. When search results become available, they are received in a text message format from the wireless network 102 through the wireless receiver 118 of the wireless mobile device 100. The text message is sent to the formatter 120, which formats the search results for display on the display 122 of the mobile wireless device 100, and the display 122 displays them. Formatted information is ordered and presented in an easily digestible way. The search results in one embodiment are displayed in a window that occupies about two thirds of the top portion of the display 122. A scroll bar is shown on the right frame of the window if the search results do not all fit in the window. Advertisements and other information are displayed on the remaining portion of the display 122 below the search results.
After a selection is made vocally by the user, the wireless mobile device 100 continues a dialog with the user to obtain the requisite information to complete the query (Step 136). The wireless mobile device 100 may use speech synthesis, pre-recorded speech, and/or visual display to prompt the user for such information. For example, if the user selects the Restaurant category, the user will then be prompted to provide the type of restaurant 204 (
If all the required valid information is entered, the user instructs the device to send the query 208 (
In one embodiment, the wireless network 102 automatically detects the location of the wireless mobile device 100 and passes that information along with the text message to the search engine 104. The location may be used in selecting advertisements to be returned with the search results in the following steps. Alternatively mobile devices with Global Positioning System (GPS) capability can transfer its location directly to the search engine without the requirement of mobile network triangulation.
After the search engine 104 receives the text message specifying the query, it generates corresponding search results (Step 146), which are sent back to the wireless mobile device 100 (Step 148) over the wireless network 102 in text message format. The search results are then formatted (Step 150) and have advertisements added to them on the wireless mobile device 100 (Step 152). In another embodiment, the search results are sent to an intermediary site 106 (Step 160), where they are formatted (Step 150′) and appended with advertisements (Step 152′). The advertisements originate from either the search engine 104 or an advertisement provider 106. They are either transmitted via a SMS or MMS message over the wireless network 102 or stored in the form of bitmaps locally on the wireless mobile device 100 or the intermediary site 106 before being formatted. The selection of advertisements is based on factors including the particular search results returned and the location of the wireless mobile device 100. When a query generates multiple search results, each is appended with a different advertisement and sent via separate responsive SMS or MMS messages.
After receiving the search results, either directly from the search engine 104 (Step 148) or from the intermediary site 106 (Step 162), the wireless mobile device 100 displays the search results on its screen (Step 154). Alternatively, it uses a text to speech synthesis process (Step 156) to generate a voice speaking the search results (Step 158) in addition to the screen display (Step 154). When there are multiple search results received in separate SMS or MMS messages 172, 174 (
The invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
1. A method for obtaining query results on a wireless mobile device comprising the steps of:
- speaking a query to a wireless mobile device;
- converting said query into text;
- transmitting said text using a messaging protocol over a wireless network to a search engine;
- performing a search based on said text with said search engine;
- transmitting search results using said messaging protocol over said wireless network;
- formatting said search results; and
- displaying said search results on said wireless mobile device.