Abstract: Implementations described herein relate to methods, systems, and computer-readable media to write data records. In some implementations, a method may include calculating a data rate of a data stream that includes a plurality of data records and determining if the data rate of the data stream is less than an ingest threshold. The method may further include, if the data rate of the data stream is less than the ingest threshold, calculating a number of write requests per time unit based on the data stream; determining a storage capacity per storage bucket; determining a read interval for the data stream; based on the number of write requests per time unit, the storage capacity, and the read interval, selecting a size of time window per storage bucket; and writing the plurality of data records to a particular storage bucket.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to obtain snapshots used for analysis of streaming data. In some implementations, a computer-implemented method includes receiving initial data that includes a plurality of identifiers and corresponding timestamps, generating and storing a snapshot based on the initial data, wherein the snapshot includes the identifiers and a corresponding status, receiving a data stream that includes a subset of the identifiers, activity information for each identifier in the subset, and corresponding timestamps. The method further includes periodically analyzing the data stream to obtain a delta that includes an updated status for each identifier in the subset, storing the delta separate from the snapshot. The method further includes receiving a request for identifiers that are active in a particular time period, and based on the particular time period, retrieving active identifiers from the data stream, the delta, or the snapshot.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to automatically compute features from a data stream. In some implementations, a method includes receiving a data stream comprising a plurality of data fields. The method further includes storing at least three copies of data in the data stream in a database, wherein each copy is stored on a corresponding hardware storage device. The method further includes performing, one or more rollup operations on a first copy of the data to compute one or more aggregate features and storing the one or more aggregate features on each of the hardware storage devices. The method further includes receiving a request for a particular aggregate feature and providing the particular aggregate feature in response to the request.

Abstract: In some implementations, a computer-implemented method includes receiving a state update command. The method further includes, in response to the state update command, sending a command to perform a state update in a first server of a plurality of servers and queueing incoming requests for access to the plurality of servers. The method further includes receiving confirmation from the first server that the state update was successful. The method further includes, after receiving the confirmation, sending a respective command to the plurality of servers to perform the state update and a scheduled update time associated with the state update. The method further includes receiving a respective state update message from a subset of the plurality of servers that the state update was successful and after receiving the state update message, transmitting the queued incoming requests to the subset of the plurality of servers.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to obtain snapshots used for analysis of streaming data. In some implementations, a computer-implemented method includes receiving initial data that includes a plurality of identifiers and corresponding timestamps, generating and storing a snapshot based on the initial data, wherein the snapshot includes the identifiers and a corresponding status, receiving a data stream that includes a subset of the identifiers, activity information for each identifier in the subset, and corresponding timestamps. The method further includes periodically analyzing the data stream to obtain a delta that includes an updated status for each identifier in the subset, storing the delta separate from the snapshot. The method further includes receiving a request for identifiers that are active in a particular time period, and based on the particular time period, retrieving active identifiers from the data stream, the delta, or the snapshot.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to write data records. In some implementations, a method may include calculating a data rate of a data stream that includes a plurality of data records and determining if the data rate of the data stream is less than an ingest threshold. The method may further include, if the data rate of the data stream is less than the ingest threshold, calculating a number of write requests per time unit based on the data stream; determining a storage capacity per storage bucket; determining a read interval for the data stream; based on the number of write requests per time unit, the storage capacity, and the read interval, selecting a size of time window per storage bucket; and writing the plurality of data records to a particular storage bucket.

Abstract: In some implementations, a computer-implemented method includes receiving a state update command. The method further includes, in response to the state update command, sending a command to perform a state update in a first server of a plurality of servers and queueing incoming requests for access to the plurality of servers. The method further includes receiving confirmation from the first server that the state update was successful. The method further includes, after receiving the confirmation, sending a respective command to the plurality of servers to perform the state update and a scheduled update time associated with the state update. The method further includes receiving a respective state update message from a subset of the plurality of servers that the state update was successful and after receiving the state update message, transmitting the queued incoming requests to the subset of the plurality of servers.

Abstract: Some implementations relate to storage of data in a storage device with a plurality of chips. In some implementations, a computer-implemented method includes identifying a plurality of software applications that are configured to access data from the storage device, determining a data access pattern for each of the plurality of software applications, and based on the data access pattern, assigning a respective subset of the plurality of storage chips to each software application such that each storage chip is configured for access by a specific software application.

Abstract: Technologies are described to perform workload based device access. An input-output (IO) request received from an application. An application profile for the application is determined. Based on the application profile, one or more IO parameter values to access a device are set. The device is accessed based on the one or more IO parameter values to fulfill the IO request.

Abstract: Technologies are described to perform workload based device access. An input-output (IO) request received from an application. An application profile for the application is determined. Based on the application profile, one or more IO parameter values to access a device are set. The device is accessed based on the one or more IO parameter values to fulfill the IO request.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to write data records. In some implementations, a method may include calculating a data rate of a data stream that includes a plurality of data records and determining if the data rate of the data stream is less than an ingest threshold. The method may further include, if the data rate of the data stream is less than the ingest threshold, calculating a number of write requests per time unit based on the data stream; determining a storage capacity per storage bucket; determining a read interval for the data stream; based on the number of write requests per time unit, the storage capacity, and the read interval, selecting a size of time window per storage bucket; and writing the plurality of data records to a particular storage bucket.

Abstract: Some implementations relate to storage of data in a storage device with a plurality of chips. In some implementations, a computer-implemented method includes identifying a plurality of software applications that are configured to access data from the storage device, determining a data access pattern for each of the plurality of software applications, and based on the data access pattern, assigning a respective subset of the plurality of storage chips to each software application such that each storage chip is configured for access by a specific software application.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to automatically compute features from a data stream. In some implementations, a method includes receiving a data stream comprising a plurality of data fields. The method further includes storing at least three copies of data in the data stream in a database, wherein each copy is stored on a corresponding hardware storage device. The method further includes performing, one or more rollup operations on a first copy of the data to compute one or more aggregate features and storing the one or more aggregate features on each of the hardware storage devices. The method further includes receiving a request for a particular aggregate feature and providing the particular aggregate feature in response to the request.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to obtain snapshots used for analysis of streaming data. In some implementations, a computer-implemented method includes receiving initial data that includes a plurality of identifiers and corresponding timestamps, generating and storing a snapshot based on the initial data, wherein the snapshot includes the identifiers and a corresponding status, receiving a data stream that includes a subset of the identifiers, activity information for each identifier in the subset, and corresponding timestamps. The method further includes periodically analyzing the data stream to obtain a delta that includes an updated status for each identifier in the subset, storing the delta separate from the snapshot. The method further includes receiving a request for identifiers that are active in a particular time period, and based on the particular time period, retrieving active identifiers from the data stream, the delta, or the snapshot.

Abstract: Some implementations relate to storage of data in a storage device with a plurality of chips. In some implementations, a computer-implemented method includes identifying a plurality of software applications that are configured to access data from the storage device, determining a data access pattern for each of the plurality of software applications, and based on the data access pattern, assigning a respective subset of the plurality of storage chips to each software application such that each storage chip is configured for access by a specific software application.

Abstract: In some implementations, a computer-implemented method includes receiving a state update command. The method further includes, in response to the state update command, sending a command to perform a state update in a first server of a plurality of servers and queueing incoming requests for access to the plurality of servers. The method further includes receiving confirmation from the first server that the state update was successful. The method further includes, after receiving the confirmation, sending a respective command to the plurality of servers to perform the state update and a scheduled update time associated with the state update. The method further includes receiving a respective state update message from a subset of the plurality of servers that the state update was successful and after receiving the state update message, transmitting the queued incoming requests to the subset of the plurality of servers.

Abstract: Technologies are described to perform workload based device access. An input-output (IO) request received from an application. An application profile for the application is determined. Based on the application profile, one or more IO parameter values to access a device are set. The device is accessed based on the one or more IO parameter values to fulfill the IO request.

Abstract: Some implementations relate to storage of data in a storage device with a plurality of chips.