Abstract: Training large neural network models by providing training input to model training machines organized as multiple replicas that asynchronously update a shared model via a global parameter server is described herein. In at least one embodiment, a system including a model module storing a portion of a model and a deep learning training module that communicates with the model module are configured for asynchronously sending updates to shared parameters associated with the model. The techniques herein describe receiving and processing a batch of data items to calculate updates. Replicas of training machines communicate asynchronously with a global parameter server to provide updates to a shared model and return updated weight values. The model may be modified to reflect the updated weight values. The techniques described herein include computation and communication optimizations that improve system efficiency and scaling of large neural networks.

Abstract: The invention includes a system and method for logging network server data such as data relating to client requests. In accordance with the invention, end users of a server program can create one or more logging modules, each having a predefined interface that is defined by the server program. In response to client requests, the server program calls logging modules that have been designated by a system administrator, and passes potential log data to the logging modules. In response to receiving the potential log data, each logging module makes its own decision regarding (a) whether to make a log entry, (b) which data should be included in the log entry, and (c) the format that is used for recording the log data. In this way, end users are not constrained to any given logging format or set of logging criteria.