Patents Assigned to Agora Lab, Inc.
  • Patent number: 12568049
    Abstract: Apparatus, method and non-transitory computer readable medium for traffic policing detection and rate limit estimation for a network, including selecting from the network sampled packets having a loss rate greater than a first threshold, the selected sampled packets being added to a sliding window for traffic policing detection; in response to the number of sampled packets in the sliding window reaching a predetermined number, determining a correlation coefficient based on a sending rate and the loss rate of at least one sampled packet in the sliding window; in response to the correlation coefficient exceeding a second threshold, determining whether traffic policing is occurring in the network based on the sending rate and a transformed loss rate of the at least one sampled packet; and in response to traffic policing occurring, determining an estimated rate limit for traffic policing based on an acknowledgement rate of the at least one sampled packet.
    Type: Grant
    Filed: January 4, 2024
    Date of Patent: March 3, 2026
    Assignee: Agora Lab, Inc.
    Inventors: Yichen Qian, Zesen Zhuang
  • Patent number: 12501037
    Abstract: A method for allocating bitrate in video communications is disclosed. The method includes encoding, via an encoder, a video stream that is partitioned into tiles associated with a viewing angle of the video stream, whereby encoding the video stream includes allocating each of the tiles a bitrate. The method also includes determining an image quality score associated with rendering of the tiles at a first iteration, whereby rendering of the tiles includes decoding and displaying the tiles at the first iteration in the viewing angle of the video stream with the bitrate. The method further includes, responsive to determining the image quality score, modifying the bitrate allocated to at least some of the tiles such that rendering of the tiles at a second iteration that is after the first iteration includes decoding and displaying the tiles with the modified bitrate.
    Type: Grant
    Filed: January 4, 2024
    Date of Patent: December 16, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Mingyue Hao, Zesen Zhuang, Sheng Zhong
  • Patent number: 12439042
    Abstract: A method includes maintaining a phase-out list that includes tiles of a current viewport of a wide-angle image that are predicted to no longer be in the viewport after a time duration d from a current time t. Tiles predicted to be viewed at a time that is later than the current time t are obtained. Those of the predicted tiles that are not in a receiver list are added to the receiver list. Tiles that form a current viewing area are rendered using the receiver list. Tiles of the receiver list and a phase-out list are associated with respective display times. Rendering the tiles includes decoding and displaying tiles of the receiver list having display times that are not larger than the current time t and stopping decoding and displaying tiles of the phase-out list associated with the display times that are not larger than the current time t.
    Type: Grant
    Filed: January 5, 2022
    Date of Patent: October 7, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Sheng Zhong, Zhonghou Zheng, Zesen Zhuang
  • Patent number: 12341701
    Abstract: Apparatus, method and non-transitory computer readable medium for bandwidth estimation for a network using traffic shaping detection are provided, including selecting periodically sampled packets from the network, bandwidth samples each having a delay jitter greater than a jitter threshold, the delay jitter of a packet determined as the difference between a delay for the packet in the network and a minimum delay for all packets sampled within a delay statistics window for the packet; updating a probability distribution of sampled bandwidth based on the selected bandwidth samples; determining whether traffic shaping is occurring in the network based on a density of the updated probability distribution of sampled bandwidth; and based on a determination that traffic shaping is occurring in the network, adjusting an upper limit of estimated bandwidth for the network based on an estimated traffic shaping rate, the estimated traffic shaping rate determined based on the updated probability distribution.
    Type: Grant
    Filed: January 19, 2024
    Date of Patent: June 24, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Huaming Zhang, Zesen Zhuang
  • Patent number: 12314862
    Abstract: A method for image generation based on a Generative AI Network. The Generative AI Network includes a generator and an encoder. The method includes determining, by the encoder, a first encoding E(Y) of a target image Y; generating, by the generator, a generated image G(Z) corresponding to the target image Y, wherein the generated image G(Z) is located in a close vicinity of a target neighborhood of the target image Y, and outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the Generative AI Network; and generating, by the encoder, a second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map the target image Y and the generated image G(Z) to the latent space.
    Type: Grant
    Filed: April 26, 2024
    Date of Patent: May 27, 2025
    Assignee: Agora Lab, Inc.
    Inventor: Sheng Zhong
  • Patent number: 12207075
    Abstract: A real-time communication software application for generating spatial audio with uniform reverberation in a real-time communication session is performed by an electronic communication device. The application is adapted to remove the reverberation of recorded speech signals from far-end participants by the dereverberation approach, render the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generate reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, and combine direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Grant
    Filed: March 1, 2024
    Date of Patent: January 21, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 12204686
    Abstract: A method, an apparatus, and a system for video communications include: transmitting, from a first apparatus using a network, a first video stream of a first user to a second apparatus of a second user, wherein the first user is in video communication with the second user; receiving, from the second apparatus using the network, a second video stream of the second user; determining, by a processor, a reaction of the second user to an area of interest in the first video stream using the second video stream; and updating, in response to the reaction of the second user to the area of interest in the first video stream, a parameter for encoding the area of interest in the first video stream at the first apparatus.
    Type: Grant
    Filed: March 25, 2021
    Date of Patent: January 21, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Sheng Zhong, Yue Feng
  • Patent number: 12206737
    Abstract: An apparatus for communication between a sending application and a receiving application of a receiving apparatus includes a processor that is configured to establish a stream for transmitting data between the sending application and the receiving application; receive a first request from the sending application to transmit metadata to the receiving application; receive a second request from the sending application to transmit application data to the receiving application; responsive to a determination that a frame that includes the application data and the metadata has a size that is smaller than or equal to a maximum frame size, construct the frame to include the application data and the metadata; and transmit the frame in a packet to the receiving apparatus.
    Type: Grant
    Filed: June 22, 2021
    Date of Patent: January 21, 2025
    Assignee: Agora Lab, Inc.
    Inventors: Tian Xia, Liu Yong
  • Patent number: 12154586
    Abstract: A computer-implemented method for suppressing noise from audio signal uses both statistical noise estimation and neural network noise estimation to achieve more desirable noise reduction. The method is performed by a noise suppression computer software application running on an electronic device. The noise suppression computer software application first transforms the speech signal in time domain into frequency domain before determining a statistical noise estimate and a neural network noise estimate. The noise suppression computer software application merges the two noise estimates to derive a final noise estimate, and determines and refines a noise suppression filter. The filter is applied to the speech signal in frequency domain to obtain an enhanced signal. The enhanced signal is transformed back into time domain.
    Type: Grant
    Filed: May 24, 2022
    Date of Patent: November 26, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Jimeng Zheng, Bo Wu, Xiaohan Zhao, Liangliang Wang, Ruofei Chen
  • Patent number: 12101458
    Abstract: A system and method for determining the performance of a video codec for real-time communication applications. A set of network conditions is used to simulate certain typical real-world network conditions. A coded video stream is transmitted from the transmitting end to the receiving end under these conditions. The end-to-end latency and received video fluency are measured along with a set of existing video quality measures by a video codec performance evaluation system including a network model and a video codec quality analyzer. The decodable frame ratio, latency and video fluency are used as the performance metrics for real-time communication video quality evaluation. The video codec performance evaluation system does not send any video data over a network when it determines the real-time communication quality of the video codec.
    Type: Grant
    Filed: January 5, 2023
    Date of Patent: September 24, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Wei Dai, Sheng Zhong, Tao Duan, Zesen Zhuang, Xiajun Gu
  • Patent number: 12101370
    Abstract: A system and method for determining the performance of a real-time communication application, such as a video codec and transmission of the coded data. The method factors in burst data packet loss in determining a performance measure. The method uses a Hidden Markov Model on a set of data packets of a unit of data to determine the performance measure in the form a probability that all the data packets within the set are received. The measure is used to fine tune the system settings. The method causes the real-time communication application to adjust parameters of the application for improved transmission of data packets. The adjustment of a parameter can be an increase or a decrease.
    Type: Grant
    Filed: May 11, 2023
    Date of Patent: September 24, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Sheng Zhong, Jiajie Jian, Tao Duan
  • Patent number: 12001956
    Abstract: A method for image generation based on a Generative Adversarial Network (GAN) including a generator, a discriminator, and an encoder, wherein outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the GAN, the method including generating, by the encoder, a first encoding E(Y) of a target image Y and a second encoding E(G(Z)) of a generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map Y and G(Z) to the latent space having a lower dimension than dimensionality of Y and G(Z), wherein the encoder is trained to minimize the differences between the first and second encodings E(Y) and E(G(Z)), and the generator is trained by using the first and second encodings E(Y) and E(G(Z)) as part of a loss function.
    Type: Grant
    Filed: May 17, 2023
    Date of Patent: June 4, 2024
    Assignee: Agora Lab, Inc.
    Inventor: Sheng Zhong
  • Patent number: 11973930
    Abstract: A new real-time video communication system includes a set of electronic devices. Each device runs a specialized real-time video communication software application including a video quality module. The video quality module retrieves network connection statistic data from a network connection module, and video encoder statistic data from a video quality module. The video quality module uses the network connection statistic data and the video encoder statistic data to determine a first objective video quality measure without extensive mathematical operations. The video quality module also uses the first objective video quality measure, the network connection statistic data and the video encoder statistic data to determine a second objective quality measure without extensive mathematical operations.
    Type: Grant
    Filed: February 21, 2022
    Date of Patent: April 30, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Xiaoran Wu, Tianbo Chen, Kai Wang, Lin Feng
  • Patent number: 11950088
    Abstract: A computer-implemented method for generating spatial audio with uniform reverberation in a real-time communication session is performed by a real-time communication software application running on an electronic communication device. The method includes removing the reverberation of recorded speech signals from far-end participants by the dereverberation approach, rendering the direct sound parts by filtering the output signals by head-related transfer functions of desired directions, generating reverberant sound parts by convolving the output signals from with uniform room impulse responses or an artificial reverberator, combining direct and reverberant sound components to generate spatialized speech signals. When speakers and listeners are located in two virtual conference rooms, the reverberation of the two rooms are coupled. The reverberant sound parts are then generated by convolving the output signals and coupled RIRs from the two rooms.
    Type: Grant
    Filed: July 7, 2022
    Date of Patent: April 2, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Song Li, Jianyuan Feng, Bo Wu, Jimeng Zheng
  • Patent number: 11949588
    Abstract: A method, an apparatus for real-time multimedia communications using a software-defined network (SDN) are provided. The method includes receiving, in a periodic manner, a path metric associated with a first service node in the SDN and a second service node in the SDN, wherein the path metric comprises at least one of: a load status of at least one of the first service node or the second service node, or a transmission metric between the first service node and the second service node; and in response to receiving the path metric, updating a cascade network topology comprising an optimal path for transmitting multimedia data between a first edge node and a second edge node.
    Type: Grant
    Filed: March 17, 2021
    Date of Patent: April 2, 2024
    Assignee: Agora Lab, Inc.
    Inventors: Yong Liu, Yurun Sun, Bin Zhao
  • Patent number: 11811877
    Abstract: A transport framework for heterogeneous data streams includes session management module and a connection management module. The session management module is configured to receive a request to establish a first stream that is used for transmitting or receiving data, where the request includes an express indication as to whether the first stream is reliable or unreliable; construct a first data frame based on application data; handoff the first data frame to the connection management module; and maintain a record for the first data frame that includes whether the first data frame is successfully transmitted to the receiver. The connection management module is configured to receive the first data frame of the first stream from the session management module; receive a second frame from the session management module; encapsulate the first data frame and the second frame in a packet; and transmit the packet to the receiver using an unreliable protocol.
    Type: Grant
    Filed: May 13, 2021
    Date of Patent: November 7, 2023
    Assignee: Agora Lab, Inc.
    Inventors: Tian Xia, Yong Liu
  • Patent number: 11803462
    Abstract: An automatic SDK testing case generation system includes a computer software application for automatically generating optimized test cases. The automatic SDK testing case generation system categorizes candidate APIs presented by an SDK to be tested. The categories conform to an API call category sequence. The system iterates through API calling paths, which conform to the API call category sequence, to generate optimized subsets of test cases. The combination of the optimized subsets of test cases forms the final set of test cases for being used to test the SDK.
    Type: Grant
    Filed: April 27, 2022
    Date of Patent: October 31, 2023
    Assignee: Agora Lab, Inc.
    Inventors: Jiyu Bai, HuaMing Zhang, Xuewen Xu, Sheng Zhong
  • Patent number: 11800423
    Abstract: A wireless access point software application, including multiple software modules, dynamically determines the interference level of each channel. The application determines that a new channel is better suited to serve an RTC/RTE application over the present channel based on the interference levels and a set of interference measure thresholds. The application automatically creates a new basic service set on the new channel and associates the RTC/RTE device running the RTC/RTE application with the new basic service set. The application further refuses to associate the RTC/RTE device with the previous channel when the interference level on the new channel remains low based on a second set of interference measure thresholds. In addition, the application allows additional RTC/RTE devices to join the new basic service set with the first RTC/RTE device when the interference level on the new channel remains low based on the second set of interference measure thresholds.
    Type: Grant
    Filed: September 1, 2022
    Date of Patent: October 24, 2023
    Assignee: Agora Lab, Inc.
    Inventor: Yue Cao
  • Patent number: 11757967
    Abstract: Respective bandwidth information of respective connected devices of active media sessions are obtained. At least one additional media session is detected to have become active. A first total bandwidth of the at least one additional media session is determined. At least one scaling factor is determined using the respective bandwidth information and the first total bandwidth. The at least one scaling factor is transmitted to at least one connected device of the at least one of the respective connected devices. The at least one scaling factor is determined so as to keep a second total bandwidth of the active media sessions and the at least one additional media session below or equal an upper bandwidth limit of the communications network.
    Type: Grant
    Filed: January 31, 2022
    Date of Patent: September 12, 2023
    Assignee: Agora Lab, Inc.
    Inventor: Sheng Zhong
  • Patent number: 11706456
    Abstract: A reference-order AL-FEC system for recovering network video data packet loss during real-time video communication includes a packetizer, a reference-order AL-FEC encoder, a reference-order AL-FEC decoder and a depacketizer. The packetizer constructs source symbols from source packets of a current frame. The encoder generates a repair symbol from the source symbols of the current frame and other reference frames based on the reference-order, not time-order, between the frames within an encoding window. The encoder also generates a repair packet based on the repair symbol. The decoder recovers a lost source symbol based on the source symbols of the frames of the encoding window and the repair symbol by decoding the repair packet. The decoding is achieved by solving a linear system of the repair symbol.
    Type: Grant
    Filed: August 15, 2022
    Date of Patent: July 18, 2023
    Assignee: Agora Lab, Inc.
    Inventors: Rui Wang, Liang Si, Bifeng He