Abstract: The present invention relates to systems and methods for network labeling in order to enhance real time data transfers. A network for a real time data transfer is identified and predictive models for network performance are compared against to determine if the network is suitable for the data transfer. If the network is predicted to be unsuitable for transmission an alternate means for connection may be suggested. The alternate suggestion may include delaying the data transfer until the network is expected to be in better conditions, connecting to another access point in the network, or switching to another network entirely. During the data transfer, the quality of the network is monitored in order to update the predictive models for the network's quality.

Abstract: A method of training a generator G of a Generative Adversarial Network (GAN) includes generating a real contextual data set {x1, . . . , xN} for a high resolution image Y; generating a generated contextual data set {g1, . . . , gN} for a generated high resolution image G(Z); calculating a perceptual loss Lpcept value using the real contextual data set {x1, . . . , xN} and the generated contextual data set {g1, . . . , gN}; and training the generator G using the perceptual loss Lpcept value. The generated high resolution image G(Z) is generated by the generator G of the GAN in response to receiving an input Z, where the input Z is a random sample that corresponds to the high resolution image Y.

Abstract: A method of training a generator G of a Generative Adversarial Network (GAN) includes receiving, by an encoder E, a target data Y; receiving, by the encoder E, an output G(Z) of the generator G, where the generator G generates the output G(Z) in response to receiving a random sample Z that is a noisy sample, and where a discriminator D of the GAN is trained to distinguish which of the G(Z) and the target data Y is real data; training the encoder E to minimize a difference between a first latent space representation E(G(Z)) of the output G(Z) and a second latent space representation E(Y) of the target data Y, where the output G(Z) and the target data Y are input to the encoder E; and using the first latent space representation E(G(Z)) and the second latent space representation E(Y) to constrain the training of the generator G.

Abstract: A method, an apparatus, and a system for tagging a video for video communications include: transmitting, from a first apparatus using a network, a first video stream of a first user to a second apparatus of a second user that is in video communication with the first user; receiving, from the second apparatus using the network, a second video stream of the second user; determining, using the second video stream by a processor, a reaction of the second user to the first video stream; and associating the reaction of the second user with the first video stream. The apparatus includes a processor and a memory coupled to the processor. The memory is configured to store instructions which when executed by the processor become operational with the processor to perform the method.

Abstract: A method, an apparatus, and a system for real-time multimedia communications using a software-defined network (SDN) are provided. The method includes receiving, by a processor, a path metric indicative of transmission capacity between directly-connected service nodes in the SDN, determining, by the processor based on the path metric, a cascade network topology comprising an optimal path between a first edge node and a second edge node, wherein the optimal path has the lowest transmission latency among data transmission paths in the SDN between the first edge node and the second edge node, and based on a determination that multimedia data is to be transmitted between the first edge node and the second edge node, transmitting the multimedia data between the first edge node and the second edge node in accordance with the optimal path.

Abstract: Content moderation in real-time communication includes receiving, by a sending device of a sending user, a first portion of a media stream from the sending user; determining, by the sending device, that the first portion of the media stream is classified as uncertain for transmission to a receiving user; transmitting, by the sending device to a centralized content moderation device, the first portion of the media stream, where the centralized content moderation device determines a centralized determination of whether the first portion is appropriate; in response to the centralized determination indicating that the first portion is appropriate, transmitting, by the sending device to the receiving user, the first portion; and, in response to the centralized determination indicating that the first portion is inappropriate, transmitting, by the sending device to the receiving user, a blurred first portion of the first portion.

Abstract: A method, an apparatus, and a system for error concealment of video communications include determining, by a processor in an encoding process, a current block of a current slice of a current picture of a video sequence for error concealment, determining recovery data of the current block, wherein the recovery data comprises at least one of a coefficient of a discrete transform performed to the current block and similarity data associated with a first block similar to the current block, and the first block is in a first slice different from the current slice, and transmitting the current block and the recovery data using a network, wherein the current block is transmitted as in-band data and the recovery data is transmitted as out-of-band data.

Abstract: The present invention relates to systems and methods for improving transmission of voice packets over a network are provided. The systems and methods include a few central internet data centers (IDCs) which include a routing controller and an access controller. The system also includes a number of edge IDCs. Each edge IDC includes a last mile optimizer and a relay server. The last mile optimizer operates along with an application located on the users' devices and the access controller in the central IDCs to identify the best edge server for the particular device to connect to. The edge servers continually monitor pathway performance once the call is in progress. If an error is detected, then the server may automatically transition to back-up pathways rapidly to minimize call performance disruption.

Abstract: System and methods of managing reference frame adaptively for video communication under a network are provided. The system adaptively decides the interval between each I frame, the bitrate of the encoded bit steams, and the resolution of the encoded video to achieve quick connection. The system may set the appropriate reference frame(s) for the current frame, manage the reference frame(s) in the DPB to be long/short term reference frame(s) or mark them as unused for reference; mark the current frame to be a long term reference frame, a short term reference frame, unused for reference or skip encoding the current frame; adaptively change the ratio of long term reference frame(s) with respect to the short term reference frame(s) in the DPB; adaptively change the frequency ratio of marking the current frame as long term reference frame and marking the current frame as short term reference frame.

Abstract: A method for encoding multiple descriptions for a media stream includes: determining, for a current block of the media stream, a first number of duplicate blocks, determining, for the current block, a plurality of complementary forward gain factors, and determining a first number of descriptions for the current block by applying the plurality of complementary forward gain factors to the first number of duplicate blocks. A method for decoding multiple descriptions for a media stream includes: determining, for a current block of the media stream, a first number of descriptions from the media stream, in which each description comprises a scaled block corresponding to the current block, determining backward gain factors for the first number of scaled blocks, and determining the current block by applying the backward gain factors to the first number of scaled blocks.

Abstract: A method for decoding multiple descriptions for a media stream includes decoding at least one of a first description and a second description from the media bitstream, wherein the first description and the second description are associated with a current sample of the media bitstream and independently decodable; based on a determination that the second description is missing, reconstructing, by a decoder, the current sample by performing a third shifting operation on the first description, wherein the third shifting operation on the first description comprises left shifting the first description by one bit; and based on a determination that both the first description and the second description have been received, reconstructing, by the decoder, the current sample by summing the first description and the second description. An encoding method is also provided.

Abstract: A decoding method includes: decoding from the video bitstream a prediction status parameter indicative of whether SOP is enabled; if the SOP is enabled, determining a SOP type indicative of a specific operation for predicting a current picture of the video bitstream; and determining a predicted picture by applying the specific operation to at least one of a reference picture for the current picture and the current picture, in which the specific operation includes at least one of rotation and zoom. An encoding method includes: if SOP is enabled, determining SOP data indicative of a specific operation for predicting a current picture of a video sequence; determining a predicted picture by applying the specific operation to at least one of a reference picture for the current picture and the current picture, in which the specific operation includes at least one of rotation and zoom; and encoding the SOP data.

Abstract: Systems and methods for improving quality of a call over network (CON) are provided. Call quality may be improved via buffer length modulation based upon the call scenario type. Scenario detection may be based upon who speaks, and the duration of the speaking, as well as contextual analysis. Further, the call over network quality may further be improved by deploying modules over the network. The modules are intermediary vehicles between each communicator and backend servers. The modules intercept audio packets from the communicator to detect packet loss, and perform recovery of lost packets, thereby accelerating real-time audio conversations.

Abstract: A real-time communication monitoring and diagnosing system is provided on a real-time communication monitoring and diagnosing computer system. The monitoring and diagnosing system includes time line, a video downstream bitrate axis extending upward from reference line, an audio downstream bitrate axis extending downward from the reference line, a video downstream bitrate curve in a first color above the reference line, an audio downstream bitrate curve in a second color below the reference line, a set of choppy video marks in a third color above and along the reference line, a set of blurry video marks in a fourth color as a dotted line along the video downstream bitrate curve, and a set of audio marks in a fifth color below and along the reference line.

Abstract: A method for encoding a video stream, including encoding, by a processor, a first slice of a first picture of the video stream; receiving, from a decoder, a feedback message indicative of having received the first slice of the first picture; and in response to receiving the feedback message indicative of having received the first slice of the first picture, updating a dynamic reference picture (DRR) with the first slice of the first picture, wherein the first slice of the first picture replaces the first slice of a second picture in the dynamic reference picture, and the first slice of the first picture and the first slice of the second picture are co-located.

Abstract: A method for decoding a compressed video stream is disclosed. The method includes decoding, by a processor from the compressed video stream, a first picture and a second picture, each including multiple slices in the same pattern, in which the first picture includes a first I-slice decoded using intra-prediction, the second picture includes a second I-slice decoded using intra-prediction, and the first I-slice and the second I-slice are not co-located with each other; and decoding, using inter-prediction, a first P-slice of the current picture referencing the first I-slice, and a second P-slice of the current picture referencing the second I-slice, in which the first P-slice of the current picture is co-located with the first I-slice, and the second P-slice of the current picture is co-located with the second I-slice.

Abstract: A method for encoding a video stream, including encoding, by a processor, a first slice of a first picture of the video stream; receiving, from a decoder, a feedback message indicative of having received the first slice of the first picture; and in response to receiving the feedback message indicative of having received the first slice of the first picture, setting the first slice of the first picture as a reference slice for encoding a second picture of the video stream, wherein the second picture is encoded after receiving the feedback message.

Abstract: The present invention relates to systems and methods for network labeling in order to enhance real time data transfers. A network for a real time data transfer is identified and predictive models for network performance are compared against to determine if the network is suitable for the data transfer. If so, then the real time data transfer may be completed as expected. However, if the network is predicted to be unsuitable for transmission an alternate means for connection may be suggested. The alternate suggestion may include delaying the data transfer until the network is expected to be in better conditions, connecting to another access point in the network, or switching to another network entirely. During the data transfer, the quality of the network is monitored in order to update the predictive models for the network's quality. Identifiers for the network may be utilized to keep track of the networks.

Abstract: The present invention relates to systems and methods for best server selection for real-time data transmission are provided. The systems and methods include an access controller, and an application which receives information of a selection of several best servers with which to transmit real-time packets to. The application sends a request to the access controller for server information before call initiation. The access controller identifies servers with the same ISP, and then selects a regional server within this group of same ISP servers. Subsequently the geo optimizer identifies servers closest to the application from among the regional servers. The respective loads of each of the closest servers are analyzed to select only servers that are likely to provide satisfactory quality of service. A monitor within the central IDC is able to measure load conditions of each server and provide this load information to the access controller.

Abstract: The present invention relates to systems and methods for last mile optimization for the transmission of real-time data. Pseudo-packets are sent to a last mile optimizer located at each of several ‘best’ servers. The last mile optimizers provide feedback regarding jitter, latency and packet loss of the pseudo-packets to determine quality of service for each of the servers. The server with the best quality of service is selected as the ‘single best’ server. A plurality of transporter algorithms may then be applied to pseudo-packets that are then sent to the single best server, and the quality of service for each transporter algorithm is subsequently measured for effectiveness. The transporter algorithms that are shown to be effective are combined together for the actual transmission of real-time data. Continuous monitoring of transmission quality allows for utilizing backup pathways upon detection of a problem, and negotiation of preferred transporter algorithms.