Abstract: A remote support system provides support to an autonomous drive system of a vehicle in response to a support request. The remote support may include guiding the autonomous drive system of the vehicle through a hazardous environment. A call to the remote support system may be initiated, for example, via a user interface control button or by scanning a bar code, QR code, or RFID associated with the vehicle on a device that sends the code with vehicle information to the remote support system. The remote support system may determine an urgency of the request to assess whether immediate support should be initiated or whether to first initiate a communication connection with a passenger. Furthermore, an emergency brake system of the vehicle may be triggered by the remote support system or in response to the vehicle losing a connection to the remote support system during a support request.

Abstract: A vehicle remote support system includes a communication system that operates over a plurality of parallel wireless network connections to provide low-latency video from vehicle to a remote support server that provides remote support to the vehicle dependent on real-time video. The vehicle includes a source that encodes multiple versions of the original video segments (e.g., one per wireless network connection) and transmits the multiple versions of the segments to a sink at the remote support server over the respective wireless connections. This redundant multi-path communication system rationally allocates network resources to the managed video streams and balances bandwidth against latency in order to avoid network congestion and safety issues associated with single-path transmissions. In other embodiments, a similar communication system that transmits video or other real-time messages between a source and a sink may be utilized in cloud robotics applications.

Abstract: A system provides on-demand remote support services including teleoperation of vehicles. In response to a remote support request, the system assesses environmental conditions, historical data, and/or parameters of a request for remote support to rank available operators capable of fulfilling the request, and selects an optimal operator for the remote support request based on desired criteria.

Abstract: A communication system facilities low-latency, high-availability multipath streaming between terminals (e.g., mobile terminals) and a server platform. In an example application, a remote support service operating on the server platform provides remote teleoperation, monitoring, or data processing services to a mobile terminal embodied as a vehicle or robot utilizing a low latency communication link. The low latency link enables a remote operator to receive video or telemetry feeds, and timely monitor and respond to hazards in substantially real-time. The low latency communication link may be achieved even when the data streams are transmitted over public networks incorporating at least one wireless leg, and where individual connections have varying quality of service in terms of delivery latency due to congestion or stochastic packet losses. Assignment of data streams to particular communication channels may be made on an optimization model derived from a machine-learning process or simulation.

Abstract: A vehicle policy server maintains a set of policies for constraining operations of one or more remote vehicles. The policies may specify areas, locations, or routes that specified vehicles are restricted from accessing based on a set of acquired information. An application programming interface (API) enables programmatic updates of the policies or related information. Policies may be enforced by transmitting control signals fully or in part to onboard vehicle computers or to a teleoperation support module providing remote support to the vehicles using human teleoperators and/or artificial intelligence agents. The control signals may directly control the vehicles or teleoperation, or may cause a navigation system to present known restrictions in a suitable fashion such as generating an augmented reality display or mapping overlays.

Abstract: In a vehicle system that can receive remote support from a remote support server (e.g., interfacing with a human or computer teleoperator), a local normalization engine locally normalizes operation of the vehicle based on locally available sensor data that may not be accessible to the remote support server. The local normalization engine applies transformations to control commands received from the remote support server to transform the command to compensate for conditions that are locally sensed and may be unknown to the remote support server. Alternatively, or in addition, the local normalization engine controls auxiliary functions of the vehicle (e.g., by activating one or more auxiliary actuators) that may not be under direct control of the remote support server.

Abstract: A remote support system facilitates assignment of vehicles to remote support agents for providing teleoperation or other remote support services. The remote support system may generate assignments based on a mapping function that optimizes various parameters based on sensed data associated with the vehicle, a requested service mode of the vehicle, or other factors. In some situations, the remote support server assigns a redundant set of remote support agents to a vehicle that provide similar command streams. The vehicle selects between the command streams to minimize latency or another performance parameter. Alternatively, the remote support server assigns multiple diverse remote support agents to a vehicle that generate diverse command streams. A proxy agent then generates a consensus command stream for providing to the vehicle.

Abstract: In a connected vehicle environment, network connection parameters such as a network congestion window and bit rate are automatically adjusted dependent on a location of a vehicle in order to optimize network performance. A geospatial database stores learned relationships between network performance of a connected vehicle at different physical locations when configured in accordance with different network parameters. The vehicle can then adjust its network parameters dynamically dependent on its location. A vehicle may maintain multiple connections to different networks concurrently for transmitting duplicate data of a data stream, with the vehicle independently adjusting parameters associated with different networks to optimize performance.

Abstract: In a vehicle teleoperation session, a speed limit is determined for which the vehicle can be safely teleoperated. A safety system senses data relating to the vehicle environment and generates a depth map from which obstacles in the vicinity of the vehicle can be identified. Based on the detected obstacles and a motion state of the vehicle, a speed limit is determined at which the teleoperator is predicted to be able to utilize an emergency braking command to avoid a collision. The speed limit may be automatically applied to the vehicle or may be provided to the teleoperator to enable the teleoperator to adjust the vehicle speed.

Abstract: A smart container yard includes systems for intelligently controlling operations of vehicles in the container yard using teleoperation and/or autonomous operations. A remote support server controls remote support sessions associated with vehicles in the container yard to provide teleoperation support for loading and unloading operations. Aerial drones may be utilized to maintain positions above a teleoperated vehicle and act as signal re-transmitters. An augmented reality view may be provided at a teleoperator workstation to enable a teleoperator to control vehicle operations in the smart container yard.

Abstract: A vehicle routing evaluation system makes predictions about network performance data along road segments relevant to navigation of a vehicle and generates routing instructions or recommendations based in part on the network performance data. The vehicle routing evaluation system may send control signals to automatically control navigation of the vehicle to follow the routing generated routing or may present the navigation instructions to a teleoperator that controls the vehicle. Alternatively, the vehicle routing evaluation system generates a display with a navigational map overlay providing visual cues indicative of predicted wireless performance along various road segments to enable the teleoperator to select between possible routes.

Abstract: A control platform generates commands for coordinating use of network resources between a plurality of vehicles within a geographic region. In an embodiment, game-theoretical modelling is employed to determine allocation of resources in a manner that provides an optimal solution for a given allocation strategy. This model may reward controllers of vehicles that comply with a coordination policy while penalizing controllers of vehicles that defect from compliance.

Abstract: A video encoder compresses video for real-time transmission to a video decoder of a remote teleoperator system that provides teleoperator support to the vehicle based on the real-time video. The video encoder recognizes one or more generic objects in captured video that can be removed from the video without affecting the ability of the teleoperator to control the vehicle. The video encoder removes regions of the video corresponding to the generic objects to compress the video, and generates a metadata stream encoding information about the removed objects. The video decoder generates replacement objects for the objects removed the compressed video. The video decoder inserts the rendered replacement objects into relevant regions of the compressed video to reconstruct the scene.

Abstract: A virtual support element may be activated when it is beneficial to assist an autonomous drive system of the vehicle through a physical environment having one or more hazards. A remote support system generates the virtual support element (e.g., a virtual vehicle) that can be controlled through a simulated environment that corresponds to a physical environment of a vehicle. The autonomous drive system of the vehicle follows a path through the physical environment corresponding to the path of the virtual support element through the simulated environment corresponding to the physical environment.

Abstract: A vehicle remote support system includes a communication system that operates over a plurality of parallel wireless network connections to provide low-latency video from vehicle to a remote support server that provides remote support to the vehicle dependent on real-time video. The vehicle includes a source that encodes multiple versions of the original video segments (e.g., one per wireless network connection) and transmits the multiple versions of the segments to a sink at the remote support server over the respective wireless connections. This redundant multi-path communication system rationally allocates network resources to the managed video streams and balances bandwidth against latency in order to avoid network congestion and safety issues associated with single-path transmissions. In other embodiments, a similar communication system that transmits video or other real-time messages between a source and a sink may be utilized in cloud robotics applications.

Abstract: A communication system facilities low-latency, high-availability multipath streaming between terminals (e.g., mobile terminals) and a server platform. In an example application, a remote support service operating on the server platform provides remote teleoperation, monitoring, or data processing services to a mobile terminal embodied as a vehicle or robot utilizing a low latency communication link. The low latency link enables a remote operator to receive video or telemetry feeds, and timely monitor and respond to hazards in substantially real-time. The low latency communication link may be achieved even when the data streams are transmitted over public networks incorporating at least one wireless leg, and where individual connections have varying quality of service in terms of delivery latency due to congestion or stochastic packet losses. Assignment of data streams to particular communication channels may be made on an optimization model derived from a machine-learning process or simulation.

Abstract: In a vehicle system that can receive remote support from a remote support server (e.g., interfacing with a human or computer teleoperator), a local normalization engine locally normalizes operation of the vehicle based on locally available sensor data that may not be accessible to the remote support server. The local normalization engine applies transformations to control commands received from the remote support server to transform the command to compensate for conditions that are locally sensed and may be unknown to the remote support server. Alternatively, or in addition, the local normalization engine controls auxiliary functions of the vehicle (e.g., by activating one or more auxiliary actuators) that may not be under direct control of the remote support server.

Abstract: A system provides on-demand remote support services including teleoperation of vehicles. In response to a remote support request, the system assesses environmental conditions, historical data, and/or parameters of a request for remote support to rank available operators capable of fulfilling the request, and selects an optimal operator for the remote support request based on desired criteria.

Abstract: A vehicle policy server maintains a set of policies for constraining operations of one or more remote vehicles. The policies may specify areas, locations, or routes that specified vehicles are restricted from accessing based on a set of acquired information. An application programming interface (API) enables programmatic updates of the policies or related information. Policies may be enforced by transmitting control signals fully or in part to onboard vehicle computers or to a teleoperation support module providing remote support to the vehicles using human teleoperators and/or artificial intelligence agents. The control signals may directly control the vehicles or teleoperation, or may cause a navigation system to present known restrictions in a suitable fashion such as generating an augmented reality display or mapping overlays.

Abstract: In a connected vehicle environment, network connection parameters such as a network congestion window and bit rate are automatically adjusted dependent on a location of a vehicle in order to optimize network performance. A geospatial database stores learned relationships between network performance of a connected vehicle at different physical locations when configured in accordance with different network parameters. The vehicle can then adjust its network parameters dynamically dependent on its location. A vehicle may maintain multiple connections to different networks concurrently for transmitting duplicate data of a data stream, with the vehicle independently adjusting parameters associated with different networks to optimize performance.