Abstract: A server may be configured to receive first pose data from a virtual reality device, generate second pose data, including a pose prediction, based at least on the first pose data, render a map representation based at least on the pose prediction, determine regions of the map representation based on a likelihood of view of each of the regions, perform foveation in each of the regions and generate an encoded frame based on the regions and a network parameter, and send the encoded frame and second pose data to the device. The encoding may associate each of the regions with a quality level and base the foveation on each region's associated quality level. The foveation may use different quantization parameters for each of the regions and/or use rendering foveation for each of regions. Network conditions such as latency, bandwidth, and/or jitter may be considered for bit allocation in the foveation.

Abstract: Methods and devices for encoding and decoding data streams are disclosed. In some aspects, the data streams are multimedia data streams. One method disclosed includes obtaining, by a client device, a first multimedia data stream and a second multimedia data stream, the second multimedia data stream being a lower fidelity version of the first multimedia data stream, generating, by the client device, a third multimedia data stream based on differences between the first and second multimedia data streams, compressing, by the client device, the second multimedia data stream to generate a first compressed multimedia data stream, compressing, by the client device, the third multimedia data stream to generate a second compressed multimedia data stream; and transmitting, by the client device, the first and second compressed multimedia data steams to the server.

Abstract: A server computing device for inducing latency on target input streams is provided. The server computing device includes a processor configured to receive a plurality of input streams from a respective plurality of client computing devices. Each input stream includes a plurality of inputs controlling actions of respective characters in a multiplayer online software program. The processor is further configured to determine a latency of each of the input streams, identify a higher latency input stream and a lower latency input stream among the plurality of input streams, and induce a higher latency in the lower latency input stream to narrow a difference in latency between the higher latency input stream and the lower latency input stream.

Abstract: A server may be configured to receive first pose data from a virtual reality device, generate second pose data, including a pose prediction, based at least on the first pose data, render a map representation based at least on the pose prediction, determine regions of the map representation based on a likelihood of view of each of the regions, perform foveation in each of the regions and generate an encoded frame based on the regions and a network parameter, and send the encoded frame and second pose data to the device. The encoding may associate each of the regions with a quality level and base the foveation on each region's associated quality level. The foveation may use different quantization parameters for each of the regions and/or use rendering foveation for each of regions. Network conditions such as latency, bandwidth, and/or jitter may be considered for bit allocation in the foveation.

Abstract: A server computing device for inducing latency on target input streams is provided. The server computing device includes a processor configured to receive a plurality of input streams from a respective plurality of client computing devices. Each input stream includes a plurality of inputs controlling actions of respective characters in a multiplayer online software program. The processor is further configured to determine a latency of each of the input streams, identify a higher latency input stream and a lower latency input stream among the plurality of input streams, and induce a higher latency in the lower latency input stream to narrow a difference in latency between the higher latency input stream and the lower latency input stream.

Abstract: A server may be configured to receive first pose data from a virtual reality device, generate second pose data, including a pose prediction, based at least on the first pose data, render a map representation based at least on the pose prediction, determine regions of the map representation based on a likelihood of view of each of the regions, perform foveation in each of the regions and generate an encoded frame based on the regions and a network parameter, and send the encoded frame and second pose data to the device. The encoding may associate each of the regions with a quality level and base the foveation on each region's associated quality level. The foveation may use different quantization parameters for each of the regions and/or use rendering foveation for each of regions. Network conditions such as latency, bandwidth, and/or jitter may be considered for bit allocation in the foveation.

Abstract: A virtual reality system including a head mounted display (HMD), a server, and a beam steering apparatus utilizes optical data transmission from the server to the HMD. The data rates/bandwidths provided by optical data transmission allow the amount of compression needed for data transfer (and the latency associated with compression) to the HMD to be reduced. The efficient offloading of processing tasks from the HMD to the server, reduces HMD power consumption, and enables the delivery of video having high resolution, framerate, and quality. As the user of the HMD moves, and the pose (position and orientation) of the HMD changes, the pose is provided to the server and beam steering apparatus. Based on the pose, the server renders image frames for transmission to the HMD and the beam steering apparatus directs an optical beam to the HMD to enable transmission of the image frames from the server to the HMD.

Abstract: Methods and devices for encoding and decoding data streams are disclosed. In some aspects, the data streams are multimedia data streams. One method disclosed includes obtaining, by a client device, a first multimedia data stream and a second multimedia data stream, the second multimedia data stream being a lower fidelity version of the first multimedia data stream, generating, by the client device, a third multimedia data stream based on differences between the first and second multimedia data streams, compressing, by the client device, the second multimedia data stream to generate a first compressed multimedia data stream, compressing, by the client device, the third multimedia data stream to generate a second compressed multimedia data stream; and transmitting, by the client device, the first and second compressed multimedia data steams to the server.

Abstract: A virtual reality system including a head mounted display (HMD), a server, and a beam steering apparatus utilizes optical data transmission from the server to the HMD. The data rates/bandwidths provided by optical data transmission allow the amount of compression needed for data transfer (and the latency associated with compression) to the HMD to be reduced. The efficient offloading of processing tasks from the HMD to the server, reduces HMD power consumption, and enables the delivery of video having high resolution, framerate, and quality. As the user of the HMD moves, and the pose (position and orientation) of the HMD changes, the pose is provided to the server and beam steering apparatus. Based on the pose, the server renders image frames for transmission to the HMD and the beam steering apparatus directs an optical beam to the HMD to enable transmission of the image frames from the server to the HMD.

Abstract: Virtual reality is provided by a mobile VR device. VR scenes are displayed by serving an application's image requests from a local cache of pre-rendered image frames. As the device user moves, image frames that match a VR scene viewed from a current pose (3D position and 3D orientation of the device translated to the virtual environment) are retrieved from the cache and displayed to the user as a VR scene. The cache may include a static cache including static image frames and a dynamic object cache including dynamic object image frames. The static cache may be indexed by the device current pose. The dynamic object cache may be indexed by the dynamic object's animation stage, orientation, and relative distance from the device user's current pose for a given time or movement-based trigger. Static image frames and dynamic object image frames may be composited, and displayed as a scene.

Abstract: Nodes of a computing cluster can be selected to run new computing jobs while providing acceptable performance of jobs running on the nodes. Respective performance metrics of respective workloads on respective computing nodes can be determined. Each workload can include a new computing job and the performance metrics can be determined based at least in part on respective measured performance data of the ones of the computing nodes and information of the new computing job. Candidate ones of the computing nodes can be determined based at least in part on the respective performance metrics. One of the candidate computing nodes can be selected based at least in part on the information of the new computing job. In some examples, identification of the new computing job can be transmitted to the selected node. In some examples, state data of the nodes can be updated based on the performance data.

Abstract: A server may be configured to receive first pose data from a virtual reality device, generate second pose data, including a pose prediction, based at least on the first pose data, render a map representation based at least on the pose prediction, determine regions of the map representation based on a likelihood of view of each of the regions, perform foveation in each of the regions and generate an encoded frame based on the regions and a network parameter, and send the encoded frame and second pose data to the device. The encoding may associate each of the regions with a quality level and base the foveation on each region's associated quality level. The foveation may use different quantization parameters for each of the regions and/or use rendering foveation for each of regions. Network conditions such as latency, bandwidth, and/or jitter may be considered for bit allocation in the foveation.

Abstract: The claimed subject matter includes techniques for live migration of a graphics processing unit (GPU) state. An example method includes receiving recorded GPU commands from a relay at a destination GPU. The method also includes replaying the recorded GPU commands at the destination GPU. The method also includes detecting a downtime for the GPU commands. The method further includes establishing a connection between the destination GPU and the client during the detected downtime.

Abstract: Virtual reality is provided by a mobile VR device. VR scenes are displayed by serving an application's image requests from a local cache of pre-rendered image frames. As the device user moves, image frames that match a VR scene viewed from a current pose (3D position and 3D orientation of the device translated to the virtual environment) are retrieved from the cache and displayed to the user as a VR scene. The cache may include a static cache including static image frames and a dynamic object cache including dynamic object image frames. The static cache may be indexed by the device current pose. The dynamic object cache may be indexed by the dynamic object's animation stage, orientation, and relative distance from the device user's current pose for a given time or movement-based trigger. Static image frames and dynamic object image frames may be composited, and displayed as a scene.

Abstract: A system is provided that allows a spectator to spectate video gameplay over a network. Recorded information associated with a selected recording of a player playing a video game that includes a game engine is received over the network. The recorded information includes game commands that were input to the game engine as the player was playing the game, and inputs the player made to the game to control it. The recorded information is replayed to the game at the timing in which this information was originally recorded, where this replay includes replaying the game commands to the game engine. The replay generates a playback of the selected recording which is displayed on the display screen of the spectator's computer. Upon receiving a request from the spectator to view the inputs the player made to the game they are highlighted on the screen during the playback of the selected recording.

Abstract: Nodes of a computing cluster can be selected to run new computing jobs while providing acceptable performance of jobs running on the nodes. Respective performance metrics of respective workloads on respective computing nodes can be determined. Each workload can include a new computing job and the performance metrics can be determined based at least in part on respective measured performance data of the ones of the computing nodes and information of the new computing job. Candidate ones of the computing nodes can be determined based at least in part on the respective performance metrics. One of the candidate computing nodes can be selected based at least in part on the information of the new computing job. In some examples, identification of the new computing job can be transmitted to the selected node. In some examples, state data of the nodes can be updated based on the performance data.

Abstract: The claimed subject matter includes techniques for live migration of a graphics processing unit (GPU) state. An example method includes receiving recorded GPU commands from a relay at a destination GPU. The method also includes replaying the recorded GPU commands at the destination GPU. The method also includes detecting a downtime for the GPU commands. The method further includes establishing a connection between the destination GPU and the client during the detected downtime.

Abstract: A cloud-based connectivity system is disclosed. The system includes a plurality of wireless devices and a computing device connected to at least one of the wireless devices. The computing device includes a control unit to receive context information from each of the plurality of wireless devices, to determine future consumption patterns for each of the plurality of wireless devices by using the received context information, to create a connectivity representation for the plurality of wireless devices, and to determine an optimal network topology for the plurality of wireless devices.

Abstract: Systems and methods of scheduling data in background services on mobile devices are disclosed. An example method includes identifying data consumption patterns on a mobile device. The method also includes determining sensitivity of data arriving at the mobile device based on the data consumption patterns. The method also includes aggregating network access by background services on the mobile device according to a schedule based on the sensitivity of the data arriving at the mobile device.