NAVIGATION AND ORIENTATION TOOLS FOR SPEECH SYNTHESIS
TTS is a well known technology for decades used for various applications from Artificial Call centers attendants to PC software that allows people with visual impairments or reading disabilities to listen to written works on a home computer. However to date TTS is not widely adopted for PC and Mobile users for daily reading tasks such as reading emails, reading pdf and word documents, reading through website content, and for reading books. The present invention offers new user experience for operating TTS for day to day usage. More specifically this invention describes a synchronization technique for following text being read by TTS engines and specific interfaces for touch pads, touch and multi touch screens. Nevertheless this invention also describes usage of other input methods such as touchpad, mouse, and keyboard.
Priority is claimed from U.S. provisional application No. 61/297,921 entitled “Navigation and orientation tools for speech synthesis” filed 25 Jan. 2010 and from U.S. Patent Application No. 61/347,575 entitled “Navigation and orientation tools for speech synthesis” filed 24 May, 2010.
FIELD OF THE INVENTIONThe present invention is in the field of navigation and orientation tools for speech synthesis.
BACKGROUND OF THE INVENTIONAccording to Wikipedia: Speech synthesis is the artificial production of human speech. A computer system used for this purpose is called a speech synthesizer, and can be implemented in software or hardware. A text-to-speech (TTS) system converts normal language text into speech; other systems render symbolic linguistic representations like phonetic transcriptions into speech.
Since invented, speech technology constantly improved it's abilities. Most efforts where around imitating a human voice and fluently reading while the user interface and text navigating abandoned. From the user point of view, it is still complicated to use since current common user interfaces are limited, for example:
The existing products/applications are far from being comfortable for the end users.
-
- a. In most cases, the user needs to select, by marking the text before listening to it.
- b. If the user stops in the middle of reading, playing text again will start from the beginning of the marked text.
- c. During reading there are no text pointers and the users lost their orientation very quickly.
- d. Not using device specific input methods and apparatuses such as touchpad's, touch and multitouch screens making navigation easier and more intuitive.
- e. reading large amounts of content are almost impossible.
- f. Current audio books navigation is cumbersome
There is a need in the art to provide new controls for text to speech navigation and reading orientation by adding new orientation abilities that will enable easy navigation through large documents, and will help readers to follow the text as it is being read by the TTS engine.
There is a need in the art to provide a solution that will work on any device Mac/PC, Mobile Smartphone or Tablets by touch, voice, mouse or keyboard.
According to Wikipedia: A text-to-speech (TTS) system (or “engine”) is composed of two parts: a front-end and a back-end. The front-end has two major tasks. First, it converts raw text containing symbols like numbers and abbreviations into the equivalent of written-out words. This process is often called text normalization, pre-processing, or tokenization. The front-end then assigns phonetic transcriptions to each word, and divides and marks the text into prosodic units, like phrases, clauses, and sentences. The process of assigning phonetic transcriptions to words is called text-to-phoneme or grapheme-to-phoneme conversion. Phonetic transcriptions and prosody information together make up the symbolic linguistic representation that is output by the front-end. The back-end—often referred to as the synthesizer—then converts the symbolic linguistic representation into sound.
In one embodiment of the present invention the engine will provide portrayed text indications every time a new sentence or a new word or a new character collectively referred hereunder as the “text” is being output by the back-end. Based on these indications the system will mark for example but not limited to portraying a magnifying glass over text being read, providing the user with orientation of the current text being read.
In a second embodiment of the present invention the engine will provide portrayed line indications every time the text being read, where that text is the next line or in the previous line relatively to the text that was read immediately before the current text. A line indication can be for example portraying a small needle at the beginning of the line that is currently being read.
In a third embodiment of the present invention the user may click, double click, drag, use a single touch or a multitouch gesture applied on over the portrayed text indicator in order to start or stop playback of the TTS engine.
In a fourth embodiment of the present invention the user may drag, use a single touch or a multitouch gesture applied on the portrayed text indicator in order to set a new reading point for playback of the TTS engine.
In a fifth embodiment of the present invention the user may drag, use a single touch or a multitouch gesture applied on the portrayed text indicator in order to set a new reading point for playback of the TTS engine. Where said reading point is not in the same page of the book.
SUMMARY OF THE INVENTIONIn accordance with an aspect of the invention, there is provided a method for outputting a text, comprising
-
- a. indicating read text on a screen by using a portrayed text indicator, for example portraying a magnifying glass on the read text;
- b. synchronizing the read text and audio playback of the indicated text.
- In accordance with an embodiment of the invention, there is provided a method, wherein the synchronization is at word boundary.
- In accordance with an embodiment of the invention, there is further provided a method, wherein the synchronization is at sentence boundary.
- In accordance with an embodiment of the invention, there is still further provided a method, further comprising providing a scroll indicator for scrolling the text by a user dragging the scroll indicator.
- In accordance with an embodiment of the invention, there is still further provided a method, further comprising providing a page flipping indicator for flipping page by means of a user swipe gesture on the flipping indicator.
- In accordance with an embodiment of the invention, there is still further provided a method, further comprising displaying the text in a screen layout that portrays a text book.
- In accordance with an embodiment of the invention, there is still further provided a method further comprising removing text controls when portraying flipping of a page in the text book.
- In accordance with an embodiment of the invention, there is still further provided a method configured to operate on the IPAD™, IPAD™, IPOD™, IPHONE™, Android™.
- In accordance with an aspect of the invention, there is provided a method for outputting a text, comprising:
- indicating read text on a touch screen by portraying a text indicator on the read text;
- applying a swipe gesture by a user touch on the text indicator to start or stop reading the text
- synchronizing the read text and audio playback of the indicated text.
- In accordance with an embodiment of the invention, there is provided a method, wherein the direction of the swipe gesture prescribes the start or stop playback, respectively.
- In accordance with an aspect of the invention, there is provided a method for outputting a text, comprising:
- indicating read text on a touch screen by portraying a text indicator on the read text;
- dragging the text indicator by a user touch to a different location in the text;
- synchronizing the read text starting from the new position and audio playback of the indicated text.
- In accordance with an embodiment of the invention, there is provided a method, further comprising changing the position of a time indicator to reflect the text that already been processed up to the new position, wherein the time indicator indicating on the proportion of the text that has already been processed compared to the entire text passage for reading.
- In accordance with an embodiment of the invention, there is provided a method comprising:
- indicating read text on a touch screen by portraying a text indicator on the read text;
- calculating the entire reading time of a text by being proportionally to multiplying the average time required to read a character by the total characters in the text;
- portraying a time indicator to reflect the text that already been processed up to the new position, wherein the time indicator indicating on the proportion of the text that has already been processed compared to said calculated entire reading time.
- In accordance with an embodiment of the invention, there is provided a method, wherein the average time required to read a character is configurable according to the a desired text playback rate.
In accordance with certain other aspects of the invention there are provided counterpart system configurations configured to perform the specified method steps.
In order to understand the invention and to see how it may be carried out in practice, embodiments will now be described, by way of non-limiting example only, with reference to the accompanying drawings, in which:
The subject matter of the present application can have features of different aspects described above, or their equivalents, in any combination thereof, which can also be combined with any feature/s of the subject matter described in the detailed description of embodiments presented below, or their equivalents.
DETAILED DESCRIPTION OF THE INVENTIONThe current invention describes a system and methods for users that need to read text from a display. This system is useful for mobile users that would like the computer to read the text for them since they are in constant movement, say walking, driving, on a train, were they often need to move their eyes from the display and therefore lose the last point they read in the text, making it impossible to have a continuous experience. Also when reading large text even when stationary there is a need for a pointer, instead of using the finger or the mouse, the system is reading the text for the user and the current word and line text is being highlighted keeping the listener with visual orientation to the text being read. It seems that readers that are reading through text that it not written in their mother tongue or readers that are not highly skilled in reading for example kids in kindergarten and kids during their first years as students find it hard to pronounce some of the words and generally reading slower than usual will find the TTS system with Navigation and Orientation tools detailed hereunder the user reads through the text faster, and easier while also expending his language skills.
According to
The Timeline 217 represents the duration it will take reading through the entire text, the knob 214 represent the current time of the text being read. For each word that is being read by the TTS engine the system is generating 1) A portray indicating the word being read 207, 2) portray indicating the line of the word being read 208 and, 3) Time indication by moving the knob 214 to the respective point in time of that word in the entire reading sequence. Also the time elapsed 220, time remaining 221, and page number 215 is updated accordingly.
The user can at any time see the current time in the reading 220 and the reading time 221 to complete the reading of the text. These indications are imperative for the user to understand where he is in the process of reading and how much time remained for the reading, and plan his time accordingly. Additionally the timeline has a knob 214 that represents the current time of the text being read, that knob 214 is moving along the timeline as the reading progresses, giving the user the feeling of continuity and that he is in progress while listening to the text, the progress bar gives the user further indication of the pace the text is being read and since progress is captured by the mind as something positive, it actually encourages the user to keep on listening to the text and complete the task of listening to the entire text. The knob 214 is also used for navigation by scrolling the knob 214 along the time line the user can access any point in the text. When scrolling the knob the text in the display area 206 is moved accordingly keeping sync with the knob 214.
The system may be operated in different ways, in some embodiments of the present invention the system maybe a generic capability of the device system (also known as a system service, drive, resident, etc.) where the user can use the system to read for example: his emails, word documents, pdf files and webpage's for the user and the system will read the text for the user including the highlight of the words, lines, timeline and control buttons within his application. In another embodiment of the present invention the system is implemented as an application where the user paste clips of at least text copied for example from his emails, word documents, pdf files and webpage's and then ask the application to read through the text using the highlight of the words, lines, timeline and control buttons.
Note that the description above was provided for understanding the need and typical use of the system of the invention. The invention is by no means bound by this exemplary description which is provided for illustrative purposes only.
Bearing this in mind, attention is drawn to
The present invention also has a Navigation and Orientation Unit 201 and Synchronization Unit 204. The Synchronization unit 204 is responsible for taking the output of the front-end 203 and feed it synchronically to the Wave Form Generation 205 and the Navigation and Orientation Unit 201. In one embodiment of the present invention the Synchronization unit 204 may Synchronize the Wave Form Generation 205 and the Navigation and Orientation Unit 201 for every new word. In another embodiment of the present invention the Synchronization Unit 204 may Synchronize the Wave Form Generation 205 and the Navigation and Orientation Unit 201 for every character, line, sentence, paragraph, bookmark, or page. In another embodiment of the present invention the Synchronization Unit 204 may Synchronize the Wave Form Generation 205 and the Navigation and Orientation Unit 201 for every segment of the text as defined by the application. Synchronization is achieved for example, in the case were the Synchronization Unit 204 synchronizes for every word, by having the Synchronization Unit 204 waiting for the word to be played by the Wave Form Generation 205, and only then continue to the next word if such word exists. By waiting for the word to be played the system achieve synchronization.
The Navigation and Orientation Unit 201 has Human Interface—UI, that enables the user to interact with the displayed text and other UI parts such as buttons. In one embodiment of the present invention the user may start a TTS session by ordering the Text Analysis Unit 202 to start reading text by selecting a “Play” button. In another embodiment of the present inventions the user interacts with the Navigation and Orientation Unit 201 UI, the Navigation and Orientation Unit 201 may change the text feed to the Text Analysis Unit 202 and start a new TTS session.
In one embodiment of the present invention the portrayed text indicator animation, in this case the portrayed magnifying glass over words 302 and 308, will start when the synchronization unit 204 will send “word is about to start” event to the Navigation and orientation tool unit 201. The “word is about to start” event is fired immediately after the waveform generator 205 completed synthesizing the current word 302. The time between the “word is about to start” event and the time the waveform generator will start synthesizing the next word 308 is the time duration for the magnifying glass to animate between the current word 302 to the next word 308. For example in case the time gap in between words is 200 ms, the time gap between the “word is about to start” event and actual speech synthesis of that word will be 200 ms, and that would also be the time for the magnifying glass to animate from word 302 to word 308. In another embodiment of the present invention during the time gap between the “word is about to start” event and actual speech synthesis of that word the word indicator and the line indicator will animate from their current position 313, 314 to their new position 318, 319.
In yet another preferred embodiment of the present invention a device with a touch, multi touch screen or similar apparatus the user shall use a drag by touching the magnifying glass 502 and move it to a new location 509 on the text in order to set the current position for text to be read by the TTS engine. In another embodiment of the present invention when the user drag the text indicator 502 using his finger 503 to a new location 509 the magnifying glass 502 will follow the finger 503 while it is being drag to the new location 509. For example in English the user will use his finger 503 touching the magnifying glass 502 and then the user will drag his finger 503 over the touch screen to a new location 509 over the text. When the user is removing his finger from the screen (i.e not touching it) that will mean that dragging is completed. The magnifying glass 502 will be placed on the text setting a new reading 509 point for playback of the TTS engine.
Drag gesture may be double tap followed by a drag motion, or a single tap and hold followed by a drag motion.
In another embodiment of the present invention when the user drag the text indicator 601 using his finger 608, the magnifying glass 601 will follow the finger 608 while it is being dragged. When the magnifying glass 601 is dragged from one position placed on one line to a new position 617 placed on a different line, the line indicator portray 602 will also change its position pointing to the new line 614. In another embodiment of the present invention when the magnifying glass 601 is dragged from one position to a new position 617, the current time indicator 606 the time elapsed 622 and time remaining 623 will also change their position and value respectively representing a new current time 612, and new time elapsed 625 and time remaining 625.
In another embodiment of the present invention when the user drag the magnifying glass 601 using his finger 608, the magnifying glass 617 will follow the finger 620 while it is being dragged. When the magnifying glass 601 is dragged from one position placed on one line to a new position 617 placed on a different line, the line indicator portray 602 will also change its position pointing to the new line 614. In another embodiment of the present invention when the magnifying glass 601 is dragged from one position to a new position 617, the current time indicator 606 and the time elapsed 622 and time remaining 623 will also change their position and value respectively representing a new current time 612, and new time elapsed 624 and time remaining 625.
Drag gesture may be double tap followed by a drag motion, or a single tap and hold followed by a drag motion.
In some embodiments of the present invention (not shown in the figures) in a book format, when the TTS engine reads the last word of a page the page will turn to a different page and the TTS engine will continue reading from the first word in the new page. In case the text layout is in scrolling format, when the TTS engine reach the last word in the visible screen area or When the TTS engine nearly reach the last word of the visible screen area the text on the screen will scroll revealing new text to be read by the TTS engine and enable the user continuous reading. In some embodiments of the present invention when the TTS engine on stop mode and the user is scrolling the text or flipping to a new page a new reading point will be set automatically in a visible place in the text viewing area. In a preferred embodiment of the present invention when the TTS engine on stop mode and the user is scrolling the text or flipping to a new page a new reading point will be set automatically at the first word of the text that is currently visible in the text viewing area.
In a preferred embodiment of the present invention Get next word 1002 will be triggered by a “word is about to start” event generated by the synchronization unit 204.
The present invention has been described with a certain degree of particularity, but those versed in the art will readily appreciate that various modifications and alterations may be carried out, without departing from the scope of the following Claims:
Claims
1. A method for outputting a text, comprising
- a. indicating read text on a touch screen by portraying a text indicator on the read text;
- b. Synchronizing the read text and audio playback of the indicated text.
2. The method according to claim 1, wherein the synchronization is at word boundary.
3. The method according to claim 1, wherein the synchronization is at sentence boundary.
4. The method according claim 1, further comprising providing a scroll indicator for scrolling the text by a user dragging the scroll indicator.
5. The method according to claim 1, further comprising providing a page flipping indicator for flipping page by means of a user swipe gesture on the flipping indicator.
6. The method according to claim 1, further comprising displaying the text in a screen layout that portrays a text book.
7. The method according to claim 6, further comprising removing text controls when portraying flipping of a page in the text book.
8. The method according to claim 1 configured to operate on any of the following devices: IPAD™ IPAD™ IPOD™ IPHONE™ Android™ Kindle™ Nook™
9. A method for outputting a text, comprising
- a. Indicating read text on a touch screen by portraying a text indicator on the read text;
- b. Applying a swipe gesture by a user touch on the text indicator to start or stop reading the text
- c. Synchronizing the read text and audio playback of the indicated text.
10. The method according to claim 9, wherein the direction of the swipe gesture prescribes the start or stop playback, respectively.
11. A method for outputting a text, comprising
- a. Indicating read text on a touch screen by portraying a text indicator on the read text;
- b. Dragging the text indicator by a user touch to a different position of the text;
- c. Synchronizing the read text starting from the new position and audio playback of the indicated text.
12. The method according to claim 11, further comprising changing the position of a time indicator to reflect the text that already been processed up to the new position, wherein the time indicator indicating on the proportion of the text that has already been processed compared to the entire text passage for reading.
13. A method according to claim 11, comprising:
- a. Indicating read text on a touch screen by portraying a text indicator on the read text;
- b. Calculating the entire reading time of a text by being proportionally to multiplying the average time required to read a character by the total characters in the text;
- c. Portraying a time indicator to reflect the text that already been processed up to the new position, wherein the time indicator indicating on the proportion of the text that has already been processed compared to said calculated entire reading time.
14. The method according to claim 13, wherein said average time required to read a character is configurable according to the desired text playback rate.
Type: Application
Filed: Jan 25, 2011
Publication Date: Jul 28, 2011
Inventors: Dror KALISKY (Raanana), Sharon CARMEL (Givatayim)
Application Number: 13/012,989
International Classification: G10L 13/00 (20060101);