Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.

Abstract: Example passenger validation systems and methods are described. In one implementation, a method receives, at a vehicle, a transport request indicating a passenger and a pick-up location. The vehicle drives to the pick-up location and authenticates the passenger at the pick-up location. If the passenger is successfully authenticated, the method unlocks the vehicle doors to allow access to the vehicle, determines a number of people entering the vehicle, and confirms that the number of people entering the vehicle matches a number of passengers associated with the transport request.

Abstract: A device for detecting adverse events in a fire hydrant, the device comprising a housing configured to mount to an exterior of the fire hydrant, the housing having a surface in register with the exterior of the fire hydrant, the surface having an aperture therein, a cover enclosing the housing and mounted to the fire hydrant at the surface. The housing further includes a controller, a transceiver, and a water sensor.

Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media, for training a value neural network that is configured to receive an observation characterizing a state of an environment being interacted with by an agent and to process the observation in accordance with parameters of the value neural network to generate a value score. One of the systems performs operations that include training a supervised learning policy neural network; initializing initial values of parameters of a reinforcement learning policy neural network having a same architecture as the supervised learning policy network to the trained values of the parameters of the supervised learning policy neural network; training the reinforcement learning policy neural network on second training data; and training the value neural network to generate a value score for the state of the environment that represents a predicted long-term reward resulting from the environment being in the state.

Abstract: Methods, systems and apparatus, including computer programs encoded on computer storage media, for training a value neural network that is configured to receive an observation characterizing a state of an environment being interacted with by an agent and to process the observation in accordance with parameters of the value neural network to generate a value score. One of the systems performs operations that include training a supervised learning policy neural network; initializing initial values of parameters of a reinforcement learning policy neural network having a same architecture as the supervised learning policy network to the trained values of the parameters of the supervised learning policy neural network; training the reinforcement learning policy neural network on second training data; and training the value neural network to generate a value score for the state of the environment that represents a predicted long-term reward resulting from the environment being in the state.

Abstract: A device for transferring adverse event information from a plurality of remote hydrants to a municipal monitoring server comprises detecting an adverse event in a hydrant that relates to an adverse hydrant condition; transferring data representative of the adverse condition to a host server by routing the data along a predefined hopping path; and transferring the data from the host monitor to the municipal monitoring server. A system for detecting adverse events at a hydrant and event information to a municipal monitoring sever is also disclosed.

Abstract: A device for detecting adverse events in a fire hydrant, the device comprising a housing configured to mount to an exterior of the fire hydrant, the housing having a surface in register with the exterior of the fire hydrant, the surface having an aperture therein, a cover enclosing the housing and mounted to the fire hydrant at the surface. The housing further includes a controller, a transceiver, and a water sensor.

Abstract: A customer's server datagram request and/or customer field unit's datagram is wrapped with a header and footer that contains a defined hopping path with direction control. An IP Host stores predefined hopping paths to all of the customer's field units, and looks at the address of the customer's request packet to determine the correct hopping path. A communication device receives the customer's field unit's datagrams and adds a defined hopping path to send the datagram to the IP Host which removes the hopping path and sends the datagram to the customer server.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network used to select actions performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes maintaining a replay memory, where the replay memory stores pieces of experience data generated as a result of the reinforcement learning agent interacting with the environment. Each piece of experience data is associated with a respective expected learning progress measure that is a measure of an expected amount of progress made in the training of the neural network if the neural network is trained on the piece of experience data. The method further includes selecting a piece of experience data from the replay memory by prioritizing for selection pieces of experience data having relatively higher expected learning progress measures and training the neural network on the selected piece of experience data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for asynchronous deep reinforcement learning. One of the systems includes a plurality of workers, wherein each worker is configured to operate independently of each other worker, and wherein each worker is associated with a respective actor that interacts with a respective replica of the environment during the training of the deep neural network.

Abstract: A device for transferring adverse event information from a plurality of remote hydrants to a municipal monitoring server comprises detecting an adverse event in a hydrant that relates to an adverse hydrant condition; transferring data representative of the adverse condition to a host server by routing the data along a predefined hopping path; and transferring the data from the host monitor to the municipal monitoring server. A system for detecting adverse events at a hydrant and event information to a municipal monitoring sever is also disclosed.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an actor neural network used to select actions to be performed by an agent interacting with an environment. One of the methods includes obtaining a minibatch of experience tuples; and updating current values of the parameters of the actor neural network, comprising: for each experience tuple in the minibatch: processing the training observation and the training action in the experience tuple using a critic neural network to determine a neural network output for the experience tuple, and determining a target neural network output for the experience tuple; updating current values of the parameters of the critic neural network using errors between the target neural network outputs and the neural network outputs; and updating the current values of the parameters of the actor neural network using the critic neural network.

Abstract: This invention provides a method of treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome comprising administering to the subject laquinimod as an add-on therapy to or in combination with flupirtine. This invention also provides a package and a pharmaceutical composition comprising laquinimod and flupirtine for treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome. This invention also provides laquinimod for use as an add-on therapy or in combination with flupirtine in treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome. This invention further provides use of laquinimod and flupirtine in the preparation of a combination for treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome.

Abstract: A tank level monitoring system with wireless transmission capability. The monitoring system includes a wireless tank monitor for level sensing and connected to one or more float level switches. The system regularly measures the level of fluid in a tank, but may change the measurement rate based upon the float switch. This system actively monitors conditions of a tank and alerts a user when conditions exceed a predetermined parameter.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reinforcement learning using goals and observations. One of the methods includes receiving an observation characterizing a current state of the environment; receiving a goal characterizing a target state from a set of target states of the environment; processing the observation using an observation neural network to generate a numeric representation of the observation; processing the goal using a goal neural network to generate a numeric representation of the goal; combining the numeric representation of the observation and the numeric representation of the goal to generate a combined representation; processing the combined representation using an action score neural network to generate a respective score for each action in the predetermined set of actions; and selecting the action to be performed using the respective scores for the actions in the predetermined set of actions.

Abstract: An autonomous vehicle may be configured to receive, using a computer system, a plurality of remission signals from a portion of a lane of travel in an environment in response to at least one sensor of the vehicle sensing the portion of the lane of travel. A given remission signal of the plurality of remission signals may include a remission value indicative of a level of reflectiveness for the portion of the lane of travel. The vehicle may also be configured to compare the plurality of remission signals to a known remission value indicative of a level of reflectiveness for a lane marker in the lane of travel. Based on the comparison, the vehicle may additionally be configured to determine whether the portion of the lane of travel in the environment is indicative of a presence of the lane marker.

Abstract: The present disclosure is directed towards systems and methods for evaluating a seller entity in an electronic marketplace offering products related to a brand, owner entity comprising obtaining market information from one or more electronic marketplaces. The market information comprises market listing information, seller information and marketplace information. Market data is derived from one or more of the market listing information, the seller information and the marketplace information and stored in a non-transient memory. The authenticity of the market data is determined in view of the one or more authenticated branded products and behavioral data is updated in response to an authentication of the market data. At least, one reputational score value is generated, wherein said generating comprises combining the behavioral data and the market data.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed training of reinforcement learning systems. One of the methods includes receiving, by a learner, current values of the parameters of the Q network from a parameter server, wherein each learner maintains a respective learner Q network replica and a respective target Q network replica; updating, by the learner, the parameters of the learner Q network replica maintained by the learner using the current values; selecting, by the learner, an experience tuple from a respective replay memory; computing, by the learner, a gradient from the experience tuple using the learner Q network replica maintained by the learner and the target Q network replica maintained by the learner; and providing, by the learner, the computed gradient to the parameter server.

Abstract: This invention provides a method of treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome comprising administering to the subject laquinimod as an add-on therapy to or in combination with flupirtine. This invention also provides a package and a pharmaceutical composition comprising laquinimod and flupirtine for treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome. This invention also provides laquinimod for use as an add-on therapy or in combination with flupirtine in treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome. This invention further provides use of laquinimod and flupirtine in the preparation of a combination for treating a subject afflicted with multiple sclerosis or presenting a clinically isolated syndrome.

Abstract: In one embodiment, an indication that a decision has been requested, selected, or applied with respect to one or more users may be obtained. After the indication that a decision that has been requested, selected, or applied is obtained, a value function may be updated, where the value function approximates an expected reward associated with the one or more users over time since the decision has been requested, selected, or applied with respect to the one or more users. The value function may be updated by performing or providing one or more updates to the value function, where a time at which each of the one or more updates is performed or provided is independent of activity of the one or more users.