Speech transmission method and apparatus and speech service system
A speech transmission method and apparatus include receiving at least one speech request packet sequentially sent by a terminal. Each speech request packet carries one speech data segment of the speech stream, an offset position, and a data length. The method includes generating a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries an expected offset position and an expected data length. The method includes feeding back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
Latest TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED Patents:
- Screen projection status determining method and apparatus
- Data transmission method, apparatus, and device, and computer storage medium
- Medical image classification method, model training method, computing device, and storage medium
- Medical image segmentation method, image segmentation method, and related apparatus and system
- Mini program production method and apparatus, terminal, and storage medium
This application is a continuation of International Application No. PCT/CN2015/072378, filed on Feb. 6, 2015. This application claims the benefit and priority of Chinese Application No. 201310661738.8, filed Dec. 9, 2013. The entire disclosures of each of the above applications are incorporated herein by reference.
FIELDThe present disclosure relates to communication technology field, and to speech transmission.
BACKGROUNDThis section provides background information related to the present disclosure which is not necessarily prior art.
As speech service technology matures and progresses, many applications provide the functionality of speech service. A speech service system typically includes a terminal and a server.
In speech recognition, for example, a terminal sends a server a speech request which carries speech data to be recognized. Correspondingly, after recognizing the speech data, the server feeds back a speech response carrying a recognition result to the terminal. In order to shorten the response time of the server for the speech request, streaming is a desirable mode for speech transmission. Through streaming, the transmission and recognition of a speech stream is not completed by one speech request, but by dividing the entire speech stream into a number of pieces of speech data segments according to certain rules, and when the user talks, the terminal, at the same time, begins to send the speech requests carrying the speech data segments one by one to the server for speech recognition. Thus, multiplexing is performed for talk time and time for transmitting the speech between the terminal and the server, i.e. when the user begins to talk, the server begins to perform speech recognition, thus the response time of the server for the speech requests is significantly shortened.
SUMMARYThis section provides a general summary of the disclosure, and is not a comprehensive disclosure of its full scope or all of its features.
The embodiments of the present disclosure provide a speech transmission method and apparatus and a speech service system, to improve the accuracy of processing the speech stream, or even where the speech stream cannot be processed during the speech transmission due to packet loss, duplication, and disorder. The technical approaches are as follow.
According to a first aspect, a speech transmission method is provided and the method includes:
receiving at least one speech request packet sequentially sent by a terminal, wherein each speech request packet carries one speech data segment of a speech stream, an offset position of the speech data segment in the speech stream, and a data length of the speech data segment;
for each speech request packet, generating a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries an expected offset position and an expected data length, wherein the expected offset position is the expected offset position in the speech stream of a speech data segment in a next speech request packet that is received by a server, and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server; and
feeding back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
According to a second aspect, a speech transmission method is provided, and the method includes:
sequentially sending a server at least one speech request packet, wherein each speech request packet carries one speech data segment of the speech stream, an expected offset position of the speech data segment in the speech stream, and a data length of the speech data segment;
receiving a speech response packet that is sent by the server and corresponds to the speech request packet, wherein the speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream; the speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position in the speech stream of a speech data segment in a next speech request packet that is received by the server, and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server; and
making a determination for sending the next speech request packet according to the speech response packet.
According to a third aspect, a speech transmission apparatus is provided, and the apparatus includes:
a request receiving module, adapted to receive at least one speech request packet sequentially sent by a terminal, wherein each speech request packet carries one speech data segment of the speech stream, an offset position of the speech data segment in the speech stream, and a data length of the speech data segment;
a response generating module, adapted to, for each speech request packet, generate a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of the speech data segment in a next speech request packet that is received by the server, and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server; and
a response feedback module, adapted to feed back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
According to a fourth aspect, a speech transmission apparatus is provided, and the apparatus includes:
a request sending module, adapted to sequentially send a server at least one speech request packet, wherein each speech request packet carries one speech data segment of the speech stream, an expected offset position of the speech data segment in the speech stream, and a data length of the speech data segment;
a response receiving module, adapted to receive the speech response packet that is sent by the server and corresponds to the speech request packet, wherein the speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream, the speech response packet carries the expected offset position, and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server; and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server; and
a request determination module, adapted to make a determination for sending the next speech request packet according to the speech response packet.
According to a fourth aspect, a speech service system is provided, which includes a server and a terminal. The server comprises the speech transmission apparatus according to the third aspect and the terminal comprises the speech transmission apparatus according to the fourth aspect.
Further areas of applicability will become apparent from the description provided herein. The description and examples in this summary are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.
The drawings described herein are for illustrative purposes only of selected embodiments and not all possible implementations, and are not intended to limit the scope of the present disclosure.
In order to more clearly illustrate the technical approaches of the present disclosure, hereinafter, the drawings used in the descriptions of the embodiments will be simply described. The following described drawings are only some embodiments of the present disclosure, and those with ordinary skill in the art can obtain other drawings based on these drawings without any creative effort.
Corresponding reference numerals indicate corresponding parts throughout the several views of the drawings.
DETAILED DESCRIPTIONExample embodiments will now be described more fully with reference to the accompanying drawings.
In the implementation of the present disclosure, the inventors have found the following issues in the above technique. With the popularity of mobile terminals, since the mobile terminals use a mobile network or a wireless network and the network environment of the mobile network or the wireless network can be relatively poor, packet loss, duplication and disorder occur when a speech transmission is performed between the mobile terminal and the server under the streaming transmission mode, which will result in t inaccurate processing results of the speech stream or the inability to process the speech stream.
In order to make the goal, the technical methods, and the benefits of the present disclosure more apparent, hereinafter, embodiments of the present disclosure will be described in further detail in conjunction with the drawings.
The speech server 240 may also include a speech access device (not shown in
Block 302: The terminal sequentially sends the server at least one speech request packet, each of which includes a piece of speech data segment of the speech stream, offset position of the speech data segment in the speech stream, and the data length of the speech data segment. Correspondingly, the server sequentially receives, from the terminal, the at least one speech request packet, wherein each speech request packet includes a piece of speech data segment of the speech stream, offset position of the speech data segment in the speech stream, and the data length of the speech data segment.
Block 304: For each speech request packet, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries an expected offset position and an expected data length. The expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
Block 306: The server feeds back to the terminal a speech response packet so that the terminal may make a determination for sending the next speech request packet according to this speech response packet. Correspondingly, the terminal receives the speech response packet that corresponds to the speech request packet and is fed back by the server, wherein the speech response packet is generated by the server based on the speech request packet and other received speech request packets corresponding to the speech stream, and the speech response packet carries the expected offset position and the expected data length. The expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
Block 308: The terminal makes a determination for sending a next speech request packet according to the speech response packet.
The above blocks 302 and 308 may be implemented separately as a speech transmission method on the terminal side, and the blocks 304 and 306 may be implemented separately as a speech transmission method on the server side
According to the speech transmission method according to various embodiments, for each speech request packet received from the terminal, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length and the server feeds back the terminal a speech response packet so that the terminal may make a determination about the next speech request packet to be sent according to this speech response packet. According to the above, the processing results of the speech stream may be inaccurate or the speech stream cannot be processed during the speech transmission due to packet loss, duplication, and disorder. These issues are addressed here, and can be solved, and the accuracy and process ability or the speech stream can be improved.
According to various embodiments, when a speech transmission is performed between a terminal and a server, the parameters involved in the speech request packet sent by the terminal and the parameters involved in the speech response packet fed back by the server are described below.
The parameters involved in the speech request packet are shown in the following Table 1:
Wherein, Speech stream data refers to speech data segment, Offset refers to the offset position to represent the offset position of the speech data segment in the entire speech stream, Len refers to data length to represent the length of the data of the speech data segment, and Is_end refers to the request end identifier to indicate whether the speech data segment carried in the current speech request packet is the last segment of the speech stream. For example, Is_end=1 represents True, i.e. the speech data segment carried in the current speech request packet is the last segment of the speech stream, while Is_end=0 represents False.
The parameters involved in the speech response packet are shown in the following Table 2:
Wherein, Ret refers to a processing result identifier, indicating whether the speech request packet processing is successful or not, for example, when the speech request packet processing is successful, 0 may be returned, otherwise an error code may be returned, Act_offset refers to an expected offset position to represent an expected offset position, in the entire speech stream, of a speech data segment of a next speech request packet that is received by the server, Ack_len refers to an expected length of the data to represent an expected length of the data of a speech data segment in the entire speech stream of a next speech request packet that is received by the server, and Recognition results refers to the processing results of the speech stream, for example, results=XXX.
Before the detailed introduction and description are given for the packet loss, duplication, and disorder which occur when a speech transmission is performed according to the speech transmission method provided in various embodiments of the present disclosure, this application first gives an introduction of the process flow of the speech transmission under normal conditions, i.e. under the conditions that the packet loss, duplication, and disorder do not occur.
The first speech request packet sent by the terminal in Step1 carries the first speech data fragment Speech stream data1 (not shown in
The second speech request packet sent by the terminal in Step3 carries a second speech data segment Speech stream data2 (not shown), an offset position Offset=100, data length Len=100, and the request end identifier Is_end=0. Since the speech data segment carried in the second speech request packet follows the speech data segment carried in the first speech request packet in the speech sequence, and the speech data segment carried in the first speech request packet has the offset position from 0 to 99, in the second speech request packet Offset=100; Len=100 denotes that the data length of the speech data segment carried in the second speech request packet is still 100; and Is_end=0 denotes that the speech data segment carried in the second speech request packet is not the last segment. As shown in
Similarly, in the third speech request packet sent by the terminal in Step5, Offset=200 and Len=100 denotes that the data length of the speech data segment carried in the third speech request packet is still 100. Different from the previous two speech request packets, the request end identifier Is_end=1, indicating that the speech data segment carried in the third speech request packet is the last segment. As shown in
The first speech response packet fed back by the server in Step2 is in response to the first speech request packet. The first speech response packet contains the processing result identifier Ret=0, the expected offset position Act_offset=100, and the expected data length Ack_len=0. Since the speech data segment of the first speech request packet is located at the beginning of the entire speech stream, and Offset=0, Len=100, that is, the first speech data segment includes the data arranged with the offset position from 0 to 99, the first speech response packet has an expected offset position Act_offset=100, which indicates that the server expects that offset position of the speech data segment in the next speech request packet in the speech stream is 100. Further, under normal conditions, the expected data length Ack_len is preset to be 0.
The second speech response packet fed back by the server in Step4 is in response to the second speech request packet. The second speech response packet contains the processing result identifier Ret=0, the expected offset position Act_offset=200, and the expected data length Ack_len=0. Since in the second speech request packet, Offset=100 and Len=100, namely, the second speech data segment includes the data arranged with the offset position from 100 to 199, the second speech response packet has an expected offset position Act_offset=200, which indicates that the server expects that offset position of the speech data segment in the next speech request packet in the speech stream is 200.
The third speech response packet fed back by the server in Step6 is in response to the third speech request packet. The third speech response packet contains the processing result identifier Ret=0, the expected offset position Act_offset=300, the expected data length Ack_len=0, and the processing result of the speech stream Results=XXX. Since in the third speech request packet, Offset=200 and Len=100, the third speech data segment includes the speech data arranged with the offset position from 200 to 299. So, the third speech response packet has an expected offset position Act_offset=300, which indicates that the server expects that offset position of the speech data segment in the next speech request packet in the speech stream is 300. Furthermore, since the third speech request packet carries the request end identifier Is_end=1, it indicates that the speech data segment carried in the third speech request packet is the last segment, and the third speech request packet needs to carry the processing result of the speech stream Results=XXX.
Hereinafter, from the server side and the terminal side, respectively, the speech transmission method provided in the various embodiments will be described in detail.
Block 501: Receive at least one speech request packet sequentially sent by a terminal. For each speech stream, the terminal will divide the speech stream into several speech data segments and sequentially carry these speech data segments in different speech request packets to be sent. Correspondingly, the server receives the at least one speech request packet sequentially sent by the terminal. As shown in
Further, each speech request packet can also carries the request end identifier. For example, request end identifier Is_end=1 represents that the speech data segment carried in the speech request packet is the last segment of the speech stream and Is_end=0 represents that the speech data segment carried in the speech request packet is not the last segment of the speech stream. The speech request packet may not carry the request end identifier, for example, only the speech request packet which carries the last speech data segment of the speech stream contains a predetermined identification for labeling.
Since the issue of packet disorder, loss, and duplication may occur during the transmission of each speech request packet due to the network conditions, after each speech request packet is received and processed, the server generates a corresponding speech response packet sent to the terminal as guidance for the sending of the next speech request packet. The details are described in the following Block 502 and its various sub-processes.
Block 502: For each speech request packet, generate a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. For each speech request packet, the server generates a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. As shown in
Further, each speech request packet may also carry a processing result identifier, which indicates whether the speech request packet is processed successfully. For example, if the processing result identifier Ret=0, it indicates that the speech request packet is processed successfully. If Ret shows error codes, it indicates that the processing of the speech request packet has failed due to discrepancies such as the internet failures or logic errors.
As shown in
Block 502a: Detect whether the speech data segment carried in the speech request packet is already received in other speech request packets. In order to handle duplicate speech request packets, when each speech request packet is received, the server detects whether the speech data segment carried in the speech request packet is already received in other speech request packets. The server may detect whether the speech data segment carried in the speech request packet is already received in other speech request packets according to the offset position and the data length of the speech data segment. For example, during the processing of a speech stream, when the offset positions and the data lengths carried in two received speech request packets are the same, it may determine that the speech data segments carried in the two speech request packets are the same.
Due to the different data length of different speech data segments, the speech data segments carried in two speech request packets may be partially overlapped, or the speech data segment of one speech request packet contains the speech data segment of the other one speech request packet. However, for any one of the above data segment duplication conditions, the server can find the duplication according to the offset position and the data length of the speech data segments.
Block 502b: When the detecting result shows that the speech data segment has been received in other speech request packets, de-duplication processing will be performed for the speech data segment carried in the speech request packet. When the detecting result shows that the speech data segment has been received in other speech request packet, de-duplication processing will be performed by the server for the speech data segment carried in the speech request packet. The server selects the duplicate speech data segments, and only one copy of the duplicate speech data segments is maintained. De-duplication processing is performed once, not repeatedly, for the speech data segments. In consideration of possible packet loss during the transmission of the speech response packet, the terminal will sends a speech request packet again when it did not receive a speech response packet corresponding to the speech request packet. Therefore, the server still sends a speech response packet corresponding to the speech request packet carrying a duplicate speech data segment.
In
Block 502c: For each speech request packet, sort the speech data segments carried in the speech request packet and other respective speech request packets according to the offset positions to obtain a speech sequence. The offset position carried in each speech request packet represents the offset position, in the entire speech stream, of the speech data segment in the speech request packet. The server sorts the speech data segments carried in the speech request packet and other respective speech request packets according to the offset positions to obtain a speech sequence.
Referring to
In
Block 502d: Determine whether the speech sequence lacks a speech data segment. When the speech sequence lacks a speech data segment, proceed to block 502e. When the speech sequence does not lack a speech data segment, proceed to block 502f.
The server determines whether the speech sequence lacks a speech data segment. The statement that the speech sequence lacks a speech data segment means that the speech sequence obtained by the server by the sorting action is not a complete sequence. For example, in the flow chart shown in
In another example, in
Block 502e: Generate an expected offset position according to the offset position, in the speech stream, of the missing speech data segment, and generate an expected data length according to the data length of the missing speech data segment.
When it is determined that the speech sequence lacks a speech data segment, the server generates an expected offset position according to the offset position, in the speech stream, of the missing speech data segment, and generates an expected data length according to the data length of the missing speech data segment.
In the first possible implementation, when the speech sequence lacks one speech data segment, namely, one speech data segment is missing, for example, in the speech sequence obtained by the server by sorting the speech data segment carried in the speech request packets received in Step4, the missing speech data segment is the speech data segment of which the offset position is from 100 to 199. The server may then configure the expected offset position as 100 in order to be the same as the beginning offset position of the missing speech data segment, and configure the expected data length as 100 in order to be the same as the data length of the missing speech data segment. Thereafter, as shown in Step5 of
In the second possible implementation, when the speech sequence lacks two or more speech data segments, namely, two or more speech data segments are missing, the server generates an expected offset position according to the offset position, in the speech stream, of the first missing speech data segment in the speech sequence, and generates an expected data length according to the data length of the first missing speech data segment in the speech sequence.
Referring to
Since during a speech transmission, especially when the speech transmission is between the mobile terminal or the like and a server, speech data contained in a speech stream is shorter. Thus, the number of speech data segments obtained by dividing the speech stream will not be large. Usually, it is sufficient to divide a speech stream into 10 speech request packets for transmission. Therefore, various embodiments use a relatively simple processing mechanism, that is, when two or more speech data segments are missing, the server computes only the offset position of the first missing speech data segment in the speech stream and its length data, and the following missing speech data segments are not in consideration. This can speed up the response speed of the server, as the server will also focus on making the calculation and processing of the speech data segments, thereby saving server overhead.
Block 502f: Generate an expected offset position according to the end position of the speech sequence and configure the expected data length as a preset value. When it is determined that the speech sequence does not lack a speech data segment, the server generates an expected offset position according to the end position of the speech sequence and configures the expected data length as a preset value.
Referring to
In the first possible implementation, when it is determined that the speech sequence does not lack a speech data segment, the server always adds 1 to the end position of the speech sequence to generate the expected offset position. The server adds 1 to the end position of the speech sequence to generate the expected offset position, that is, the server expect to receive the speech data segments in order. As shown in Step7 of
In the second possible implementation, when it is determined that the speech sequence does not lack a speech data segment and all the received request end identifiers are False, the server adds 1 to the end position of the speech sequence to generate the expected offset position.
As described in the above block 501, each speech request packet may carry the request end identifier, and the request end identifier denotes whether the speech data segment carried in the speech request packet is the last segment of the speech stream. When all the received request end identifiers are False, i.e. all Is_end=0, that means that the last speech data segment of the speech stream has not been received, and the server then adds 1 to the end position of the speech sequence to generate the expected offset position. When the received request end identifier is True, i.e. Is_end=1, the server may use a preset identifier instead of the expected offset position, or may not generate the expected offset position, because at this moment, the server has received the complete speech stream and just needs to deal with the speech stream and feed back the processing result to the terminal.
Block 502g: When the request end identifier carried in the speech request packet is true and all speech data segments of the speech stream have been received, add a processing result of the speech stream into the speech response packet. When the request end identifier carried in the speech request packet are true and all speech data segments of the speech stream have been received, it means that the server has received the complete speech stream. In such a case, the server adds a processing result of the speech stream into the speech response packet. As shown in Step7 of
Block 502h: Cache the processing result of the speech stream for a predetermined length of time. After generating the processing result of the speech stream, the server may cache the processing result of the speech stream for a predetermined length of time. The predetermined length of time can be pre-set according to actual requirements, such as 20 seconds. As such, when the speech response packet that is fed back from the server to the terminal and carries the processing result of the speech stream is lost, and the server has received a speech request packet that is resent by the terminal and carries the last segment, the server may retrieve the processing result of the speech stream directly from the cache and feed back the processing result to the terminal again. Otherwise, when the processing result of the speech stream is not cached, the server needs to retrieve the speech stream, and handle the speech stream again, which affects the efficiency of the server's response and increases the pressure on the server computing
In summary, in the case of abnormal network conditions, speech request packet loss, duplication, and disorder may occur. Speech response packets generated by the server may have three main functions: First, when the speech sequence lacks a speech data segment due to the packet loss and packet disorder; the speech response packet should be used to indicate the terminal to resend the missing speech data segment. Second, when no speech data segment is missing due to the packet loss or packet disorder, the speech response packet should be used to indicate the terminal to send the next speech data segment as normal. Third, when the speech request packet carries the last segment of the speech stream, feed back the processing result of the speech stream in the speech response packet.
Block 503: Feed back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet. The server feeds back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet. Regardless of whether it is a speech response packet generated for the duplicated speech request packet, a speech response packet generated when a speech request packet loss or disorder occurs, or a speech response packet generated when a speech request packet loss or disorder does not occur, the server always feeds back the speech response packet to the terminal.
After receiving the speech response packet sent by the server, the terminal makes a determination for sending the next speech request packet according to the speech response packet. The terminal may determine whether the next speech request packet should be sent, determine the speech data segment carried by the next speech request packet, determine the offset position and the data length, etc. Terminal side processing mechanism will be described and illustrated in detail according to various embodiments shown in
It should be noted that, in order to improve the concurrency of the server to make the number of the speech request packets processed by the server at the same time as large as possible, the server may not process the speech request packet immediately once it is received, but may cache the speech request packet for a while so that the response time is not affected, and may simultaneously process multiple speech request packets after a certain number of speech request packets have been received.
As mentioned above, according to the speech transmission method provided according to various embodiments, for each speech request packet received from the terminal, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length, and the server feeds back the terminal a speech response packet so that the terminal may make a determination about the next speech request packet to be sent according to this speech response packet. According to the above methods, the issue of inaccurate processing results of the speech stream or the inability to process the speech stream during the speech transmission due to packet loss, duplication and disorder can be solved and avoided in the future.
Further, in the case where the packet loss and disorder occur in a plurality of speech request packets, after obtaining the speech sequence, regardless of the number of the missing speech data segments of the speech sequence, the server generates only the expected offset position of the first missing speech data segment in the speech stream and its expected length data. This can speed up the response speed of the server and the server will also focus on making the calculation and processing of the speech data segments, thereby saving server overhead.
Block 601: Sequentially send at least one speech request packet to the server. For one speech stream, the terminal may divide the speech stream into several speech data segments, and sequentially carries these speech data segments in different speech request packets for transmission. The terminal sequentially send the server at least one speech request packet. As shown in
Further, each speech request packet may carry the request end identifier. For example, when Is_end=1, this indicates that the speech data segment carried in the current speech request packet is the last segment of the speech stream, and when Is_end=0, this indicates that the speech data segment carried in the current speech request packet is not the last segment of the speech stream. The speech request packet may not carry the request end identifier, for example, only the speech request packet which carries the last speech data segment of the speech stream contains a predetermined identification for labeling.
Block 602: Receive the speech response packet that is sent by the server and corresponds to the speech request packet. The terminal receives the speech response packet that is sent by the server and corresponds to the speech request packet. The speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream. As shown in
In addition, each speech request packet may also carry a process result identifier, which indicates whether the speech request packet is processed successfully. For example, when the process result identifier Ret=0, this indicates that the speech request packet is processed successfully. When Ret shows error codes, this indicates that the processing of the speech request packet has failed due to discrepancies such as the internet failures or logic errors.
The detailed contents of how the server generates the speech response packet according to the speech request packets have been described and illustrated in various embodiments shown in
Block 603: Make a determination for sending the next speech request packet according to the speech response packet. The process of making a determination for sending the next speech request packet may include the following. The terminal may determine whether the next speech request packet should be sent, determine the speech data segments carried by the next speech request packet, determine the expected offset position and the expected data length, etc.
Referring to
Block 603a: Detect whether the expected data length carried in the speech response packet is equal to a predetermined value. When the expected data length carried in the speech response packet is not equal to the predetermined value, proceed to Block 603b. When the expected data length carried in the speech response packet is equal to a predetermined value, proceed to Block 603c.
The terminal determines whether the expected data length carried in the speech response packet is equal to a predetermined value. By deciding whether the expected data length carried in the speech response packet is equal to a predetermined value, the terminal may determine whether a packet loss or disorder occurs in the speech request packet sent by itself during the transmission. The expected data length represents the expected length of the data of the speech data segment in the next speech request packet that is received by the server. When the expected data length is equal to a predetermined value, i.e. when Ack_len=0, this indicates that the speech sequence obtained by the server does not lack a speech data segment, that is, the server does not find a packet loss or packet disorder. When the expected data length is not equal to a predetermined value, i.e. when Ack_len≠0, this indicates that the speech sequence obtained by the server lacks a speech data segment, that is, a packet loss or packet disorder occurs.
Block 603b: Carry all or part of the speech data segment that has the expected offset position and expected data length in the next speech request packet for transmitting. When the expected data length does not equal a predetermined value, the server carries all or part of the speech data segment that has the expected offset position and expected data length in the next speech request packet for transmitting. The expected offset position and expected data length represent the missing speech data segment needed by the server, and the terminal may send the missing speech data segment by carrying it in one speech request packet or in multiple speech request packets.
Referring to
The terminal may also send the speech data segment in multiple speech request packets, such as one speech request packet carries one part of the speech data segment with Offset=100 and Len=50, and another speech request packet carries the other part of the speech data segment with Offset=150 and Len=50.
Block 603c: Determine whether the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent. When it determines that the expected data length is equal to the predetermined value, the terminal determines that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent. When it determines that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent, proceed to Block 603d. When it determines that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has not been sent, proceed to Block 603e.
Block 603d: When there is another speech data segment of the speech stream that has not been sent, carry the speech data segment in the next speech request packet for transmitting.
Referring to
After receiving the speech response packet that was fed back by the server in Step5 and corresponds to the speech request packet sent in Step 2, the terminal reads the expected offset position Ack_offset=200 in the speech response packet and finds that the speech request packet carries the speech data segment with offset position Ack_offset=200 has been sent, and therefore all of the speech stream has been sent. The terminal does not then send a speech request packet any more, and waits for a processing result of the speech stream to be sent by the server.
Block 603e: When a speech data segment exists with the expected offset position in the speech stream, carry the speech data segment with the expected offset position in the next speech request packet for transmitting.
Referring to
Further, after block 601, the following may be performed.
Block 604: For each speech request packet, within a predetermined time interval after the speech request packet is sent, detect whether a speech response packet corresponding to the speech request packet is received. In consideration of the possible condition that a packet loss and disorder may occur in the speech response packet fed back by the server during the transmission, within a predetermined time interval after the speech request packet is sent, the terminal detects whether a speech response packet corresponding to the received speech request packet is received.
In one possible implementation, the terminal detects whether a speech response packet corresponding to the received speech request packet is received according to the expected offset position carried in the speech response packet. In the condition that packet loss and disorder do not occur, for each speech request packet, the expected offset position carried in the speech response packet fed back by the server equals to the value obtained by adding 1 to the end position of the speech data segment carried in the speech request packet. Therefore, the terminal is able to detect whether a speech response packet corresponding to the received speech request packet is received according to the expected offset position carried in the received speech response packet.
In this implementation there will be a deviation in the case that speech request packet loss or disorder occurs, because when a packet loss or disorder occurs, the expected offset position sent back by the server will be equal to the offset position of the missing speech data segment, and at this point the expected offset position sent back by the server does not accurately reflect the corresponding relationship between the speech response packet and the speech request packet, and some speech request packets may be unnecessarily resent by the terminal. However, because the server side needs only to do de-duplication processing, it will not bring great computing pressure on the server. The present implementation may enable easier speech transmission.
In another possible implementation, when sending the speech request packets, the terminal can add different identifiers to different speech request packets, e.g. adding identifier A to the first speech request packet and adding identifier B to the second speech request packet. After the server processes the first speech request packet, the first speech response packet fed back by the server also carries the identifier A. After the server processes the second speech request packet, the second speech response packet fed back by the server also carries the identifier B. As such, the terminal can determine the corresponding relationship between the speech response packet and the speech request packet according to the identifier in the received speech response packet, in order to detect whether the speech response packet corresponding to the speech request packet has been received within a predetermined time interval. Although the present implementation can determine the corresponding relationship between the speech request packet and the speech response packet, but will increase the computing amount of the server.
In practice, any one of the above two implementations can be selected based on the computing capability of the server.
Block 605: When it is detected that the speech response packet corresponding to the speech request packet has not been received yet, resend the speech request packet. When the speech response packet corresponding to the speech request packet has not been received yet within a predetermined time interval, it means that the speech response packet may be lost during the transmission. The terminal then resends the speech request packet, notifies the server that the speech response packet is lost, and requests the server to reprocess the speech request packet.
Referring to
Further, the terminal may carry a request end identifier in each speech request packet, and the request end identifier indicates whether the speech data segment carried in the speech request packet is the last segment of the speech stream. For example, when Is_end=1, this indicates that the speech data segment carried in the current speech request packet is the last segment of the speech stream, and when Is_end=0, this indicates that the speech data segment carried in the current speech request packet is not the last segment of the speech stream. The speech request packet may not carry the request end identifier, for example, only the speech request packet which carries the last speech data segment of the speech stream contains a predetermined identification for labeling.
As mentioned above, according to the speech transmission method according to various embodiments, the terminal sends the server a speech request packet carrying the speech data segment, the offset position of the speech request packet in the speech stream, and the data length of the speech data segment, and after the terminal receives the speech response packet that is fed back by the server and carries the expected offset position and the expected data length, the terminal makes a determination about the next speech request packet to be sent according to this speech response packet. According to the above methods, the issue of inaccurate processing results of the speech stream or the inability to process the speech stream during the speech transmission due to packet loss, duplication and disorder can be solved and avoided in the future.
Further, various embodiments provide an improved speech transmission mechanism, wherein within a predetermined time interval after the terminal sends a speech request packet, the terminal detects whether the speech response packet corresponding to the speech request packet has been received, and resends the speech request packet when it detects that the speech response packet corresponding to the speech request packet has not been received, thereby avoiding the influence due to the occurrence of the speech response packet loss.
The following is the apparatus embodiment of the present disclosure, which can be used to perform the method embodiment of the present disclosure. For details that are not described in the apparatus embodiment of the present disclosure, please refer to the method embodiment of the present disclosure.
The request receiving module 710 receives at least one speech request packet sequentially sent by a terminal. Each speech request packet carries one speech data segment of the speech stream, the offset position of the speech data segment in the speech stream, and the data length of the speech data segment.
The response generating module 720, for each speech request packet, generates a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
The response feedback module 730 feeds back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
As mentioned above, according to the speech transmission apparatus according to various embodiments, for each speech request packet received from the terminal, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length. The server feeds back to the terminal a speech response packet so that the terminal may make a determination about the next speech request packet to be sent according to this speech response packet. According to the, the issue of inaccurate processing results of the speech stream or inability of the speech stream to be processed during the speech transmission due to packet loss, duplication and disorder can be solved and avoided in the future.
The request sending module 810 sequentially sends the server at least one speech request packet. Each speech request packet carries one speech data segment of the speech stream, the expected offset position of the speech data segment in the speech stream, and the data length of the speech data segment.
The response receiving module 820 receives the speech response packet that is sent by the server and corresponds to the speech request packet. The speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
The request determination module 830 makes a determination for sending the next speech request packet according to the speech response packet.
As mentioned above, according to the speech transmission apparatus provided according to various embodiments, the terminal sends the server a speech request packet carrying the speech data segment, the offset position of the speech request packet in the speech stream, and the data length of the speech data segment, and after the speech response packet that is fed back by the server receives and carries the expected offset position and the expected data length, the terminal makes a determination about the next speech request packet to be sent according to this speech response packet. According to the above methods, the issue of inaccurate processing results of the speech stream or the inability of the speech stream to be processed during the speech transmission due to packet loss, duplication, and disorder can be solved and avoided in the future.
The server 700 may include a speech transmission apparatus, which can be implemented by using software, hardware, or a combination thereof. The speech transmission apparatus may include a request receiving module 710, a response generating module 720, a result caching module 722, and a response feedback module 730.
The request receiving module 710 receives at least one speech request packet sequentially sent by a terminal. Each speech request packet carries one speech data segment of the speech stream, the offset position of the speech data segment in the speech stream, and the data length of the speech data segment.
The response generating module 720, for each speech request packet, generates a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
The response generating module 720 may include a speech detecting unit 720a and a de-duplicating unit 720b. The speech detecting unit 720a detects whether the speech data segment carried in the speech request packet is already received in other speech request packets. The de-duplicating unit 720b performs de-duplication processing for the speech data segment carried in the speech request packet when the speech detecting unit 720a detects that the speech data segment has been received in other speech request packets. The response generating module 720 may also include a speech sorting unit 720c, a lack judging unit 720d, a first execution unit 720e, and a second execution unit 720f. The speech sorting unit 720c, for each speech request packet, sorts the speech data segments carried in the speech request packet and other respective speech request packets according to the offset positions to obtain a speech sequence. The lack judging unit 720d determines whether the speech sequence lacks a speech data segment. The first execution unit 720e generates an expected offset position according to the offset position, in the speech stream, of the missing speech data segment, and generates an expected data length according to the data length of the missing speech data segment, when the lack judging unit 720d determines that the speech sequence lacks a speech data segment. The first execution unit 720e also generates an expected offset position according to the offset position, in the speech stream, of the first missing speech data segment in the speech sequence, and generates an expected data length according to the data length of the first missing speech data segment in the speech sequence, when the speech sequence lacks two or more speech data segments. The second execution unit 720f generates an expected offset position according to the end position of the speech sequence, and configures the expected data length as a preset value, when the lack judging unit 720d determines that the speech sequence does not lack a speech data segment.
In one possible implementation, each speech request packet may carry the request end identifier, and the request end identifier denotes whether the speech data segment carried in the speech request packet is the last segment of the speech stream.
The second execution unit 720f also adds 1 to the end position of the speech sequence to generate the expected offset position when all the received request end identifiers are False.
The response generating module 720 may also include a result adding unit 720g, which adds a processing result of the speech stream into the speech response packet when the request end identifier carried in the speech request packet is true and all speech data segments of the speech stream have been received. The result caching module 722 may be used to cache the processing result of the speech stream for a predetermined length of time. The response feedback module 730 may be used to feed back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
The terminal 800 may include a speech transmission apparatus which can be implemented by using software, hardware, or a combination thereof. The speech transmission apparatus may comprise of a request sending module 810, a response receiving module 820, a request determination module 830, a response detecting module 840, a request resending module 850, and an identifier adding module 860.
The request sending module 810 sequentially sends the server at least one speech request packet. Each speech request packet carries one speech data segment of the speech stream, the expected offset position of the speech data segment in the speech stream, and the data length of the speech data segment.
The response receiving module 820 receives the speech response packet that is sent by the server and corresponds to the speech request packet. The speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server. The expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server.
The request determination module 830 makes a determination for sending the next speech request packet according to the speech response packet. The request determination module 830 may include a length detecting unit 830a and a data processing unit 830b. The length detecting unit 830a may be used to detect whether the expected data length carried in the speech response packet is equal to a predetermined value. The data processing unit 830b may be used to carry all or part of the speech data segment that has the expected offset position and expected data length in the next speech request packet for transmitting, when the length detecting unit 830a detects that the expected data length carried in the speech response packet does not equal a predetermined value.
The request determination module 830 may also include a data determining unit 830c, a first determining unit 830d, and a second determining unit 830e. The data determining unit 830c may be used to determine whether the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent when the length detecting unit. 830a detects that the expected data length carried in the speech response packet equals to the predetermined value. The first judging unit 830d may be used to carry the speech data segment in the next speech request packet for transmitting, when the data determining unit 830c judges that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent, and there is still a speech data segment of the speech stream that has not been sent. The second determining unit 830e may be used to carry the speech data segment with the expected offset position in the next speech request packet for transmitting, when the data judging unit 830c determines that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has not been sent yet and a speech data segment with the expected offset position exists in the speech stream.
The response detecting module 840 may be used to, for each speech request packet, detect whether a speech response packet corresponding to the received speech request packet is received within a predetermined time interval after the speech request packet is sent. The request resending module 850 may be used to resend the speech request packet when the response detecting module 840 detects that the speech response packet corresponding to the speech request packet has not been received yet. The identifier adding module 860 may be used to carry a request end identifier in each speech request packet, and the request end identifier indicates whether the speech data segment carried in the speech request packet is the last segment of the speech stream.
As mentioned above, according to the speech service system provided in the various embodiments, for each speech request packet received from the terminal, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length; and the server feeds back the terminal a speech response packet so that the terminal may make a determination about the next speech request packet to be sent according to this speech response packet. According to the above methods, the issue of inaccurate processing results of the speech stream or the inability of the speech stream to be processed during the speech transmission due to packet loss, duplication, and disorder can be solved and avoided in the future.
Further, in the case that the packet loss and disorder occur in a plurality of speech request packets, after obtaining the speech sequence, regardless of the number of the missing speech data segments of the speech sequence, the server generates only the expected offset position of the first missing speech data segment in the speech stream and its expected length data. This can speed up the response speed of the server, and the server will also focus on making the calculation and processing of the speech data segments, thereby saving server overhead. Various embodiments provide an improved speech transmission mechanism, wherein within a predetermined time interval after the terminal sends a speech request packet, the terminal detects whether the speech response packet corresponding to the speech request packet has been received, and resends the speech request packet when it detects that the speech response packet corresponding to the speech request packet has not been received, thereby avoiding the influence due to the occurrence of the speech response packet loss.
In the speech transmission apparatus and speech service system described in the above embodiments, the functional modules illustrated are only examples for providing the speech service. In practical applications, the functions of the speech transmission apparatus and speech service system may be implemented by different functional modules to complete all or part of the functions described above. In addition, the speech transmission apparatus and method and the speech service system provided in the above-described embodiments have the same conception, and the various implementation process are described in detail in the method embodiments, thus, no further explanation is given.
The basic input/output system 1006 includes a display device 1008 for displaying information and an input device 1009 for users to input information, such as a mouse, a keyboard, or the like. The display device 1008 and the input device 1009 are coupled to the central processing unit 1001 by being respectively connected to an input/output controller 1010 of the system bus 1005. The basic input/output system 1006 may also include an input/output controller 1010 for receiving and processing the input information from many other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input/output controller 1010 may also output information to a display, a printer, or other type of output devices.
The mass storage device 1007 is coupled to the central processing unit 1001 by being connected to a mass storage controller (not shown) of the system bus 1005. The mass storage device 1007 and its associated computer-readable media provide non-volatile storage for the server 1000. That is, the mass storage device 1007 may include a computer readable medium (not shown), such as a hard disk or CD-ROM drive or the like.
The computer-readable media may include computer storage media and communication media. The computer storage media includes volatile and nonvolatile, removable and non-removable media realized by any method or technology which stores the information, such as computer readable instructions, data structures, program modules, or other data. The computer storage media includes RAM, ROM, EPROM, EEPROM, flash memory or other solid state memory and its technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, and magnetic disk storage or other magnetic storage devices. It will be known to those skilled in the art that the computer storage medium is not limited to the above-descried medium. The above system memory 1004 and the mass storage device 1007 may be collectively referred to as memory.
According to various embodiments, the server 1000 can also be connected via a network, such as the Internet, to a remote computer on the network for running. That is, the server 1000 can be coupled to the network 1012 by being connected to the network interface unit 1011 of the system bus 1005, or coupled to other types of networks or a remote computer system (not shown) via the network interface unit 1011.
The memory may further comprise one or more programs, the one or more programs are stored in a memory, and are configured to be executed by one or more central processing units 1001. The one or more programs include the instructions for performing the following.
The instructions perform receiving at least one speech request packet sequentially sent by a terminal. Each speech request packet carries one speech data segment of the speech stream, the offset position of the speech data segment in the speech stream, and the data length of the speech data segment. For each speech request packet, generate a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server; and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server. Feed back the speech response packet to the terminal, so that the terminal can make a determination for sending the next speech request packet according to the speech response packet.
Considering that the above implementation is a first possible implementation, it is possible to provide a second possible implementation on the basis of the first implementation, and in the second possible implementation, the memory of the server may further comprise instructions to perform the following operations: for each speech request packet, sort the speech data segments carried in the speech request packet and other respective speech request packets according to the offset positions to obtain a speech sequence, determine whether the speech sequence lacks a speech data segment, generate an expected offset position according to the offset position, in the speech stream, of the missing speech data segment, and generate an expected data length according to the data length of the missing speech data segment, when it is determined that the speech sequence lacks a speech data segment; generate an expected offset position according to the end position of the speech sequence, and configures the expected data length as a preset value, when it is determined that the speech sequence does not lack a speech data segment.
It is possible to provide a third possible implementation on the basis of the second implementation, and in the third possible implementation, the memory of the server may further comprise instructions to perform the following operations. Generate an expected offset position according to the offset position, in the speech stream, of the first missing speech data segment in the speech sequence, and generate an expected data length according to the data length of the first missing speech data segment in the speech sequence, when the speech sequence lacks two or more speech data segments.
It is possible to provide a fourth possible implementation on the basis of the second implementation, and in the fourth possible implementation, the memory of the server may further comprise of instructions to perform the following operations. Add 1 to the end position of the speech sequence to generate the expected offset position when all the received request end identifiers are False.
It is possible to provide a fifth possible implementation on the basis of the first implementation, and in the fifth possible implementation, the memory of the server may further comprise of instructions to perform the following operations. Detect whether the speech data segment carried in the speech request packet is already received in other speech request packets and, when it detects that the speech data segment carried in the speech request packet is already received, a deduplication process will be performed for the speech data segment carried in the speech request packet.
It is possible to provide a sixth possible implementation on the basis of any one of the first to fifth implementations, and in the sixth possible implementation, the memory of the server may further comprise instructions to perform the following operations.
The process, for each speech request packet generating a speech response packet corresponding to the speech request packet according to the speech request packet and other received speech request packets corresponding to the speech stream may comprise of the following. Adding a processing result of the speech stream into the speech response packet when the request end identifier carried in the speech request packet is True and all speech data segments of the speech stream have been received.
It is possible to provide a seventh possible implementation on the basis of the sixth implementations, and in the seventh possible implementation, the memory of the server may further comprise of instructions to perform the operation to cache the processing result of the speech stream for a predetermined length of time.
As mentioned above, according to the server provided in the various embodiments, for each speech request packet received from the terminal, the server generates a speech response packet corresponding to the speech request packet based on the speech request packet and other received speech request packets corresponding to the speech stream, wherein the speech response packet carries the expected offset position and the expected data length, and the server feeds back the terminal a speech response packet so that the terminal may make a determination about the next speech request packet to be sent according to this speech response packet. According to the above methods, the issue of the inaccurate processing results of the speech stream or the inability of the speech stream to be processed during the speech transmission due to packet loss, duplication, and disorder can be solved and avoided in the future.
Further, in the case that packet loss and disorder occur in a plurality of speech request packets, after obtaining the speech sequence, regardless of the number of the missing speech data segments of the speech sequence, the server generates only the expected offset position of the first missing speech data segment in the speech stream and its expected length data. This can speed up the response speed of the server and the server will also focus on making the calculation and processing of the speech data segments, thereby saving server overhead.
In some applications, the terminal 1100 may further include a communication unit 110, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a Wi-Fi (wireless fidelity) module 170, and a power supply 190, etc. Those skilled in the art will understand that the structure of the terminal illustrated in
The communication unit 110 may send and receive messages or receive and transmit signals during a call. The communication unit 110 may be a network communication device, such as RF (Radio Frequency) circuit, a router, or a modem. Particularly, when the communication unit 110 is an RF circuit, after receiving the downlink message from the base station, the communication unit 110 sends the downlink message to one or more processors 180 for processing and further sends the uplink data to the base station. Typically, the RF circuit used as the communication unit includes, but is not limited to, a communication antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, LNA (Low Noise Amplifier), a duplexer, etc. In addition, the communication unit 110 may also communicate with the network and other devices by wireless communication. The wireless communication may use any communication standard or protocol including, but not limited to, GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), WCDMA (wideband Code Division Multiple Access), LTE (long Term Evolution), e-mail, SMS (short messaging service), and so on.
The input unit 130 may receive entered numbers or character information and generates keyboard, mouse, joystick, optical, or trackball signal input related to the user settings and control functions. The input unit 130 may include an image input device 131 and other input devices 132. The image input device 131 may be a camera or a photoelectric scanning device. In addition to the image input device 131, the input unit 130 may also include other input devices 132. In particular, other input devices 132 may include, but are not limited to, one or more of physical keyboard, function keys (such as volume control buttons, switching keys, etc.), track balls, mouse, joystick, and the like.
The display unit 140 may be used to display information input by the user or information provided to the user and a variety of graphical user interfaces of the terminal 1100, wherein the graphical user interfaces may be formed of graphics, text, icons, video, and any combination thereof. The display unit 140 may include a display panel 141, which may be formed by using LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.
The sensor 150 may be a light sensor, a motion sensor, or other sensors. The optical sensor may include an ambient light sensor and proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the light and shade of the ambient light, and the proximity sensor may close the display panel 141/or backlight when the terminal 1100 moves to ears. As a motion sensor, a gravity sensor can detect the acceleration in all directions (usually triaxle), and can detect the magnitude and direction of gravity when it stands still and, thus can be used in applications (such as horizontal and vertical screen switch, the relevant Games, calibrated magnetometer attitude) for recognizing the phone posture, in functions (such as pedometer, percussion) related to vibration identification and the like. The terminal 1100 can also be configured with a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and other sensors and the detailed description will not be given further herein.
The audio circuit 160, speaker 161, and microphone 162 may provide audio interfaces between the user and the terminal 1100. The audio circuit 160 may transmit the electric signals converted from the received audio data to the speaker 161 and the speaker 161 converts the electric signals to sound signals to output. Additionally, the microphone 162 converts the collected sound signals into electric signals, the audio circuit 160 receives the electric signals and converts them into audio data, and then outputs the audio data to the processor 180 for processing. The processed audio data may be transmitted to another terminal via the RF circuit 110 or may be output to the memory 120 for further processing. The audio circuit 160 may also include a headset jack to provide communications between peripherals headset and terminal 1100.
Wi-Fi is a kind of short-range wireless transmission technology, and the terminal 1100 can help users send and receive e-mail, browse the Web, and access streaming media, etc. through the Wi-Fi module 170. It provides users with wireless broadband Internet access. Although Figurel 1 shows the Wi-Fi module 170, it will be understood that it is not the necessary part of the terminal 1100, and may be omitted as necessary within the scope of the disclosure.
The processor 180 is a control center of the terminal 1100, uses a variety of interfaces and lines to connect the various parts of the entire mobile phone there between, performs various functions and data processing for the terminal 1100 by running or executing the software program and/or modules stored in the data memory 120 and invoking the data stored in the memory 120, and thus, monitors the overall mobile phone. Among them, the processor 180 may include one or more processing cores. In addition, the processor 180 may be integrated with application processors and modem processors, wherein the application processors process the operating system, user interfaces, and applications, and the modem processors process wireless communications. The above modem processors may not be integrated into the processor 180.
The power supply 190 (such as a battery) supplies the various components, and can be logically connected to the processor 180 through a power management system to manage the charging, discharging, and power consumption by the power management system. The power supply 190 may also include any one of the one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, power status indicators, etc.
Although not shown, the terminal 1100 may also include a Bluetooth module, which will not be described further herein. In various embodiments, the terminal further includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be performed by one or more processors. The one or more programs contain instructions for performing the following operations. Sequentially send the server at least one speech request packet, each speech request packet carries one speech data segment of the speech stream, the expected offset position of the speech data segment in the speech stream and the data length of the speech data segment. Receive the speech response packet that is sent by the server and corresponds to the speech request packet. The speech response packet is generated by the server according to the speech request packet and other received speech request packets corresponding to the speech stream. The speech response packet carries the expected offset position and the expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server and makes a determination for sending the next speech request packet according to the speech response packet.
Considering the above implementation is a first possible implementation, it is possible to provide a second possible implementation on the basis of the first implementation, and in the second possible implementation, the memory of the terminal may further comprise instructions to perform the following operations. Detect whether the expected data length carried in the speech response packet is equal to a predetermined value and carry all or part of the speech data segment that has the expected offset position and expected data length in the next speech request packet for transmitting, when it is detected that the expected data length carried in the speech response packet does not equal to a predetermined value.
It is possible to provide a third possible implementation on the basis of the second implementation, and in the third possible implementation, the memory of the terminal may further comprise instructions to perform the following operations. Determine whether the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent, when it is detected that the expected data length carried in the speech response packet equals to the predetermined value. Carry the speech data segment in the next speech request packet for transmitting, when it is determined that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent, and there is still a speech data segment of the speech stream that has not been sent. Carry the speech data segment with the expected offset position in the next speech request packet for transmitting when it is determined that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has not been sent yet and a speech data segment with the expected offset position exists in the speech stream.
It is possible to provide a fourth possible implementation on the basis of the first implementation, and in the fourth possible implementation, the memory of the terminal may further comprise instructions to perform the following operations. For each speech request packet, detect whether a speech response packet corresponding to the received speech request packet is received, within a predetermined time interval after the speech request packet is sent. Resend the speech request packet when it is detected that the speech response packet corresponding to the speech request packet has not yet been received.
It is possible to provide a fifth possible implementation on the basis of any one of the first to fourth implementations, and in the fifth possible implementation, the memory of the terminal may further comprise instructions to perform the following operations. Carry a request end identifier in each speech request packet and the request end identifier indicates whether the speech data segment carried in the speech request packet is the last segment of the speech stream.
As mentioned above, according to the terminal provided in the various embodiments, the terminal sends the server a speech request packet carrying the speech data segment, the offset position of the speech request packet in the speech stream and the data length of the speech data segment, and after received, the speech response packet that fed back by the server and carries the expected offset position and the expected data length, the terminal makes a determination for sending the next speech request packet according to this speech response packet. According to the above methods, the issue of the inaccurate processing results of the speech stream or the inability of the speech stream to be processed during the speech transmission due to packet loss, duplication, and disorder can be solved and avoided in the future.
Further, various embodiments provide a less flawed speech transmission mechanism, wherein within a predetermined time interval after the terminal sends a speech request packet, the terminal detects whether the speech response packet corresponding to the speech request packet has been received, and resends the speech request packet when it detects that the speech response packet corresponding to the speech request packet has not been received, thereby avoiding the influence due to the occurrence of the speech response packet loss.
It should be understood that, as used herein, unless the context clearly supports exceptions, the singular forms “a” (“a”, “an”, “the”) are intended to include the plural forms. It should also be understood that, “and/or” used herein is intended to include any and all possible combinations of one or more of the associated listed items
The embodiments of the present disclosure are only used for description, and do not represent the merits of the implementations.
Those with ordinary skilled in the art will understand that all or part of the steps to achieve the above-described embodiments may be accomplished by hardware, or by programs instructing relevant hardware, and the programs may be stored in a computer readable storage medium, the storage medium may be a read-only memory, magnetic or optical disk, etc.
The foregoing are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modification, equivalent substitution, and improvement made within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.
One skilled in the art may be aware that some exemplary instances of intelligent terminals and operating systems thereof are listed above in detail; however, such listing is merely provided for description, and is not intended to limit the protection scope of the implementation manners of the present disclosure.
The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.
The foregoing description is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. The broad teachings of the disclosure can be implemented in a variety of forms. Therefore, while this disclosure includes particular examples, the true scope of the disclosure should not be so limited since other modifications will become apparent upon a study of the drawings, the specification, and the following claims. It should be understood that one or more steps within a method may be executed in different order (or concurrently) without altering the principles of the present disclosure. Further, although each of the embodiments is described above as having certain features, any one or more of those features described with respect to any embodiment of the disclosure can be implemented in and/or combined with features of any of the other embodiments, even if that combination is not explicitly described. In other words, the described embodiments are not mutually exclusive, and permutations of one or more embodiments with one another remain within the scope of this disclosure.
Spatial and functional relationships between elements (for example, between modules, circuit elements, semiconductor layers, etc.) are described using various terms, including “connected,” “engaged,” “coupled,” “adjacent,” “next to,” “on top of,” “above,” “below,” and “disposed.” Unless explicitly described as being “direct,” when a relationship between first and second elements is described in the above disclosure, that relationship can be a direct relationship where no other intervening elements are present between the first and second elements, but can also be an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements. As used herein, the phrase at least one of A, B, and C should be construed to mean a logical (A OR B OR C), using a non-exclusive logical OR, and should not be construed to mean “at least one of A, at least one of B, and at least one of C.”
In the figures, the direction of an arrow, as indicated by the arrowhead, generally demonstrates the flow of information (such as data or instructions) that is of interest to the illustration. For example, when element A and element B exchange a variety of information but information transmitted from element A to element B is relevant to the illustration, the arrow may point from element A to element B. This unidirectional arrow does not imply that no other information is transmitted from element B to element A. Further, for information sent from element A to element B, element B may send requests for, or receipt acknowledgements of, the information to element A.
In this application, including the definitions below, the term “module” or the term “controller” may be replaced with the term “circuit.” The term “module” may refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); a digital, analog, or mixed analog/digital discrete circuit; a digital, analog, or mixed analog/digital integrated circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor circuit (shared, dedicated, or group) that executes code; a memory circuit (shared, dedicated, or group) that stores code executed by the processor circuit; other suitable hardware components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip.
The module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that are connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any given module of the present disclosure may be distributed among multiple modules that are connected via interface circuits. For example, multiple modules may allow load balancing. In a further example, a server (also known as remote, or cloud) module may accomplish some functionality on behalf of a client module.
The term code, as used above, may include software, firmware, and/or microcode, and may refer to programs, routines, functions, classes, data structures, and/or objects. The term shared processor circuit encompasses a single processor circuit that executes some or all code from multiple modules. The term group processor circuit encompasses a processor circuit that, in combination with additional processor circuits, executes some or all code from one or more modules. References to multiple processor circuits encompass multiple processor circuits on discrete dies, multiple processor circuits on a single die, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or a combination of the above. The term shared memory circuit encompasses a single memory circuit that stores some or all code from multiple modules. The term group memory circuit encompasses a memory circuit that, in combination with additional memories, stores some or all code from one or more modules.
The term memory circuit is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium may therefore be considered tangible and non-transitory. Non-limiting examples of a non-transitory, tangible computer-readable medium are nonvolatile memory circuits (such as a flash memory circuit, an erasable programmable read-only memory circuit, or a mask read-only memory circuit), volatile memory circuits (such as a static random access memory circuit or a dynamic random access memory circuit), magnetic storage media (such as an analog or digital magnetic tape or a hard disk drive), and optical storage media (such as a CD, a DVD, or a Blu-ray Disc).
The apparatuses and methods described in this application may be partially or fully implemented by a special purpose computer created by configuring a general purpose computer to execute one or more particular functions embodied in computer programs. The functional blocks and flowchart elements described above serve as software specifications, which can be translated into the computer programs by the routine work of a skilled technician or programmer.
The computer programs include processor-executable instructions that are stored on at least one non-transitory, tangible computer-readable medium. The computer programs may also include or rely on stored data. The computer programs may encompass a basic input/output system (BIOS) that interacts with hardware of the special purpose computer, device drivers that interact with particular devices of the special purpose computer, one or more operating systems, user applications, background services, background applications, etc.
The computer programs may include: (i) descriptive text to be parsed, such as HTML (hypertext markup language) or XML (extensible markup language), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. As examples only, source code may be written using syntax from languages including C, C++, C#, Objective-C, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK, and Python®.
None of the elements recited in the claims are intended to be a means-plus-function element within the meaning of 35 U.S.C. § 112(f) unless an element is expressly recited using the phrase “means for,” or in the case of a method claim using the phrases “operation for” or “step for.”
Claims
1. A speech transmission method performed by a server, the method comprising:
- receiving a first speech request packet of a plurality of speech request packets sequentially sent by a terminal, wherein each of the speech request packets carries (i) one speech data segment of a speech stream, (ii) an offset position of the speech data segment in the speech stream, and (iii) a data length of the speech data segment, and different speech request packets carry different identifiers added by the terminal when the terminal sends the speech request packets;
- generating a first speech response packet according to the first speech request packet, wherein the first speech response packet carries (i) an identifier corresponding to an identifier of the first speech request packet, (ii) an expected offset position and (iii) an expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server;
- feeding back the first speech response packet to the terminal;
- receiving from the terminal a second speech request packet of the plurality of speech request packets;
- determining, according to the offset positions and the identifiers of the first speech packet request and the second speech packet request, that there is a missing speech data segment;
- responsive to the determining: generating a first expected offset position according to the offset position, in the speech stream, of the missing data speech segment; generating a first expected data length according to the data length of the missing data segment; and sending to the terminal a second speech response packet that includes the first expected offset position and the first expected data length of the missing data speech segment, thereby causing the terminal to resend the missing data speech segment;
- receiving from the terminal a third speech request packet of the plurality of speech request packets;
- determining, based on the offset position of the speech data segment of the third speech request packet, that the third speech request packet does not correspond to the missing data speech segment;
- in accordance with the determining: performing a sorting action to obtain a speech sequence for the received speech request packets; re-generating the first expected offset position and the first expected data length; and feeding back to the terminal a third speech response packet that includes the first offset position and the first data length.
2. The speech transmission method of claim 1, further comprising:
- when it is determined that the speech sequence does not lack a speech data segment, generating an expected offset position according to an end position of the speech sequence and configuring the expected data length as a preset value.
3. The speech transmission method of claim 2, wherein each speech request packet carries a request end identifier, and the request end identifier denotes whether the speech data segment carried in the speech request packet is the last segment of the speech stream, the method further comprising;
- adding 1 to the end position of the speech sequence to generate the expected offset position when all the received request end identifiers are False.
4. The speech transmission method of claim 1, further comprising:
- detecting whether the speech data segment carried in the speech request packet is already received in other speech request packets; and
- responsive to a determination that the speech data segment carried in the speech request packet is already received, performing a de-duplication process for the speech data segment carried in the speech request packet.
5. The speech transmission method of claim 1, each speech request packet carries a request end identifier, and the request end identifier denotes whether the speech data segment carried in the speech request packet is the last segment of the speech stream, the method further comprising;
- adding a processing result of the speech stream into the speech response packet when the request end identifier carried in the speech request packet is True and all speech data segments of the speech stream have been received.
6. The speech transmission method of a claim 5, further comprising:
- caching the processing result of the speech stream for a predetermined length of time.
7. A speech transmission method performed by a terminal, comprising:
- sequentially sending to a server a first speech request packet of a plurality of speech request packet, wherein each of the speech request packets carries (i) one speech data segment of the speech stream, (ii) an expected offset position of the speech data segment in the speech stream and (iii) a data length of the speech data segment, and adding different identifiers to different speech request packets when sending the speech request packets;
- receiving a first speech response packet from the server, wherein the first speech response packet corresponds to the first speech request packet and wherein the first speech response packet is generated by the server according to the first speech request packet and other received speech request packets corresponding to the speech stream; the first speech response packet carries (i) an identifier corresponding to an identifier of the speech request packet, (ii) an expected offset position and (iii) an expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server; and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server;
- sending to the server a second speech request packet and a third speech request packet of the plurality of speech request packets, wherein the server is configured to: based on the offset position of the speech data segment of the third speech request packet, determine that the second speech request packet is missing; perform a sorting action to obtain a speech sequence for the received speech request packets; generating a first expected offset position and the first expected data length corresponding to the second speech request packet; and feed back to the terminal a second speech response packet that includes the first offset position and the first data length
- receiving from the server the second speech response packet that includes the first expected offset position and the first expected data length;
- determining, based on the first expected position and the first expected data length, that the second speech response packet is missing from the server; and
- responsive to the determining, re-sending to the server the second speech response packet.
8. The speech transmission method of claim 7, further comprising:
- detecting whether the expected data length carried in the speech response packet equals to a predetermined value; and
- when it is detected that the expected data length carried in the speech response packet does not equal to the predetermined value, carrying all or part of the speech data segment that has the expected offset position and expected data length in the next speech request packet for transmitting.
9. The speech transmission method of claim 8, wherein the expected data length carried in the speech response packet equals to a predetermined value, the method further comprising:
- when it is detected that the expected data length carried in the speech response packet equals to the predetermined value, judging whether the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent;
- when it is judged that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has been sent, and there is still a speech data segment of the speech stream that has not been sent, carrying the speech data segment in the next speech request packet for transmitting; and
- when it is judged that the speech data segment that has the expected offset position and expected data length carried in the speech response packet has not been sent yet and a speech data segment with the expected offset position exists in the speech stream, carrying the speech data segment with the expected offset position in the next speech request packet for transmitting.
10. The speech transmission method of claim 7, further comprising:
- for each speech request packet, detecting whether a speech response packet corresponding to the received speech request packet is received, within a predetermined time interval after the speech request packet is sent; and
- resending the speech request packet when it is detected that the speech response packet corresponding to the speech request packet has not been received yet.
11. The speech transmission method of claim 7, further comprising:
- carrying a request end identifier in each speech request packet, wherein the request end identifier indicates whether the speech data segment carried in the speech request packet is the last segment of the speech stream.
12. A speech transmission apparatus, comprising: a processor and memory in communication with the processor, the memory storing one or more instructions which, when executed by the processor, cause the speech transmission apparatus to perform operations comprising:
- receiving a first speech request packet of a plurality of speech request packets sequentially sent by a terminal, wherein each of the speech request packets carries (i) one speech data segment of a speech stream, (ii) an offset position of the speech data segment in the speech stream, and (iii) a data length of the speech data segment, and different speech request packets carry different identifiers added by the terminal when the terminal sends the speech request packets;
- generating a first speech response packet according to the first speech request packet, wherein the first speech response packet carries (i) an identifier corresponding to an identifier of the first speech request packet, (ii) an expected offset position and (iii) an expected data length, wherein the expected offset position is the expected offset position, in the speech stream, of a speech data segment in a next speech request packet that is received by the server and the expected data length is the expected length of the data of the speech data segment in the next speech request packet that is received by the server;
- feeding back the first speech response packet to the terminal;
- receiving from the terminal a second speech request packet of the plurality of speech request packets;
- determining, according to the offset positions and the identifiers of the first speech packet request and the second speech packet request, that there is a missing speech data segment;
- responsive to the determining: generating a first expected offset position according to the offset position, in the speech stream, of the missing data speech segment; generating a first expected data length according to the data length of the missing data segment; and sending to the terminal a second speech response packet that includes the expected offset position and the expected data length of the missing data speech segment, thereby causing the terminal to resend the missing data speech segment;
- receiving from the terminal a third speech request packet of the plurality of speech request packets;
- determining, based on the offset position of the speech data segment of the third speech request packet, that the third speech request packet does not correspond to the missing data speech segment;
- in accordance with the determining: performing a sorting action to obtain a speech sequence for the received speech request packets; re-generating the first expected offset position and the first expected data length; and feeding back to the terminal a third speech response packet that includes the first offset position and the first data length.
13. The speech transmission apparatus of claim 12, the one or more instructions further comprising:
- when it is determined that the speech sequence does not lack a speech data segment, generating an expected offset position according to an end position of the speech sequence and configuring the expected data length as a preset value.
14. The speech transmission apparatus of claim 13, wherein each speech request packet carries a request end identifier, and the request end identifier denotes whether the speech data segment carried in the speech request packet is the last segment of the speech stream, the one or more instructions further comprising:
- adding 1 to the end position of the speech sequence to generate the expected offset position when all the received request end identifiers are False.
15. The speech transmission apparatus of claim 12, the one or more instructions further comprising:
- detecting whether the speech data segment carried in the speech request packet is already received in other speech request packets; and
- responsive to a determination that the speech data segment carried in the speech request packet is already received, performing a de-duplication process for the speech data segment carried in the speech request packet.
16. The speech transmission apparatus of claim 12, the one or more instructions further comprising:
- adding a processing result of the speech stream into the speech response packet when the request end identifier carried in the speech request packet is True and all speech data segments of the speech stream have been received.
17. The speech transmission apparatus of claim 16, the one or more instructions further comprising:
- caching the processing result of the speech stream for a predetermined length of time.
| 5805153 | September 8, 1998 | Nielsen |
| 20020102999 | August 1, 2002 | Maggenti |
| 20020181506 | December 5, 2002 | Loguinov |
| 20050102371 | May 12, 2005 | Aksu |
| 20090171659 | July 2, 2009 | Pearce |
| 20090259763 | October 15, 2009 | Karlsson |
| 20100111110 | May 6, 2010 | Wang |
| 20120272281 | October 25, 2012 | Ha et al. |
| 20140064119 | March 6, 2014 | Iizuka |
| 1513252 | July 2004 | CN |
| 1902865 | January 2007 | CN |
| 101272519 | September 2008 | CN |
| 101699840 | April 2010 | CN |
| 101911064 | December 2010 | CN |
- International Search Report for App. No. PCT/CN2015/072378 dated May 6, 2015.
- Taiwan Office Action for App. No. 1052056174001 dated Aug. 10, 2016, and an English concise explanation of relevance thereof.
- Yang, Weiru “Chapter 10 of TCP Agreement”, http://dns2.asia.edu.tw/˜wzyang/slides/info_net/info_B/CH1OTCP.pdf.
- International Preliminary Report for App. No. PCPCN2015/072378 dated Jun. 23, 2016.
Type: Grant
Filed: Jun 9, 2016
Date of Patent: Dec 24, 2019
Patent Publication Number: 20160294717
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED (Shenzhen)
Inventor: Qiuge Liu (Shenzhen)
Primary Examiner: Adnan Baig
Application Number: 15/177,478
International Classification: H04L 12/801 (20130101); H04L 29/06 (20060101);
