Abstract: In examples, streaming data is received from a data source (e.g., by an edge device associated with a data platform) and is queued and aggregated. Batch data may similarly be received from a data source. The batch data and the aggregated streaming data may be processed to generate metadata accordingly. The data and metadata may be provided to the data platform, where the metadata may be used to update an index and the data may be stored in association with the index. In some instances, the data may be stored in chunks to facilitate subsequent retrieval of the data. In response to a request for the data, the index may be used to identify relevant data, which may include data associated with batch and/or streaming data formats, thereby enabling the client device to access data having any of a variety of formats from the data platform.

Abstract: In examples, streaming data is received from a data source (e.g., by an edge device associated with a data platform) and is queued and aggregated. Batch data may similarly be received from a data source. The batch data and the aggregated streaming data may be processed to generate metadata accordingly. The data and metadata may be provided to the data platform, where the metadata may be used to update an index and the data may be stored in association with the index. In some instances, the data may be stored in chunks to facilitate subsequent retrieval of the data. In response to a request for the data, the index may be used to identify relevant data, which may include data associated with batch and/or streaming data formats, thereby enabling the client device to access data having any of a variety of formats from the data platform.